Python Executing Shell Commands

Introduction

Python is a widely used programming language that provides a rich set of libraries and modules to simplify the development process. In some cases, you may need to execute shell commands in a Python program to implement some operating system-level functions. This article details how to execute shell commands using Python, providing example code and results.

Background

In some scenarios, you may need to execute shell commands within a Python program. These scenarios include, but are not limited to:

• Automated deployment: When deploying an application, you may need to execute a series of shell commands, such as creating folders and copying files.
• System management: When managing servers or operating systems, you may need to execute shell commands to obtain system information and monitor status.
• Data processing: Sometimes, you may need to call shell commands to process data, such as using commands like grep and awk for text processing.

Python provides multiple methods for executing shell commands, which we will introduce in the following sections.

Using the os Module to Execute Shell Commands

Python’s built-in module os provides an interface for executing shell commands. The os.system() function allows us to execute simple shell commands and obtain their exit status.

The following is an example code using os.system() to execute a shell command:

import os

# Execute the shell command
exit_code = os.system(‘ls -l’)

# Get the command’s exit status
print(f’Exit code: {exit_code}’)

Running the above code will output Exit code: 0, indicating that the ls -l command executed successfully. If the command fails, os.system() will return a non-zero exit status.

Note that after executing a command, the os.system() function prints the command’s output directly to the terminal and does not return it to the caller. If we need to obtain the output of a command, we can use the os.popen() function.

The following is an example code using os.popen() to execute a shell command and obtain its output:

import os

# Execute a shell command and obtain its output
output = os.popen('ls -l').read()

# Print the command execution result
print(output)

The above code prints the output of the ls -l command.

Using os.system() and os.popen() to execute shell commands is simple and easy to use, making it suitable for executing simple, one-time commands. However, for complex commands or when you need to obtain command output in real time, you can use the subprocess module.

Using the subprocess Module to Execute Shell Commands

subprocess is a powerful Python tool for creating and interacting with subprocesses. The subprocess module provides greater flexibility in executing shell commands and capturing their output.

The following is an example code snippet using the subprocess module to execute shell commands:

import subprocess

# Execute shell command
result = subprocess.run(['ls', '-l'], capture_output=True, text=True)

# Print the command execution result
print(result.stdout)

The above code uses the subprocess.run() function to execute the ls -l command and uses the capture_output=True parameter to capture the command’s output. Finally, we access the command’s standard output through result.stdout.

The subprocess.run() function also provides many other parameters, such as the cwd parameter for specifying the working directory for the command execution and the timeout parameter for setting the command timeout. For more details, please refer to the official documentation.

In addition to the subprocess.run() function, the subprocess module also provides other call methods, such as subprocess.check_output(), which executes a command and returns its output.

The following is an example code using subprocess.check_output() to execute a shell command:

import subprocess

# Execute a shell command and get the output
output = subprocess.check_output('ls -l', shell=True, text=True)

# Print the command execution result
print(output)

Similar to subprocess.run(), the subprocess.check_output() function provides many other parameters, such as the timeout parameter for setting the command timeout and the input parameter for passing command input.

Using the sh Module to Execute Shell Commands

sh is a third-party module that provides a concise, Pythonic interface for executing shell commands. The sh module allows us to call shell commands just like Python functions.

First, we need to install the sh module. You can install the sh module from the command line using the following command:

pip install sh

Once the installation is complete, we can use the sh module in Python programs.

The following is an example code snippet of using the sh module to execute shell commands:

import sh

# Executes a shell command
output = sh.ls(‘-l’)

# Prints the command output
print(output.stdout)

The above code uses the sh.ls() function to execute the ls -l command and uses output.stdout to retrieve the command’s standard output.

sh also provides many other methods for executing shell commands in different ways. For example, we can use the sh module to perform pipe operations, redirect input and output, and more.

The following is an example code using the sh module to perform pipe operations:

import sh

# Execute a shell command and use pipe operations
output = sh.grep('example', sh.cat('file.txt'))

# Print the command execution results
print(output.stdout)

The above code uses the sh.cat() function to read the contents of the file.txt file, then uses the sh.grep() function to filter out the lines containing example, and finally prints the results.

The sh module provides a shell-like syntax, allowing for very flexible shell command execution. However, it’s important to note that due to the extensive metaprogramming techniques used within the sh module, it may be slightly slower than other methods.

Summary

This article introduced various methods for executing shell commands in Python programs. We can use the os.system() and os.popen() functions of the os built-in module, the subprocess.run() and subprocess.check_output() functions of the subprocess module, or the third-party sh module to execute shell commands. Each method has its own advantages and disadvantages, and you should choose the one that best suits your needs.