Python 3 modules
Python 3 Module
In the previous chapters, we used the Python interpreter to program our scripts. If you exit and then re-enter the Python interpreter, all the methods and variables you defined will disappear.
Python provides a way to store these definitions in files for use in scripts or interactive interpreter instances. These files are called modules.
A module is a file with the suffix .py that contains all the functions and variables you define. Modules can be imported by other programs to use the functions and other functionality within them. This is also how you use the Python standard library.
The following example uses a module from the Python standard library.
Example (Python 3.0+)
#!/usr/bin/python3
# File name: using_sys.py
import sys
print('Command line parameters are as follows:')
for i in sys.argv:
print(i)
print('The nn Python path is:', sys.path, 'n')
The execution result is as follows:
$ python using_sys.py Parameter 1 Parameter 2
Command line parameters are as follows:
using_sys.py
Parameter 1
Parameter 2
The Python path is: ['/root', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
- 1. `import sys` imports the `sys.py` module from the Python standard library; this is how you import a module.
- 2. `sys.argv` is a list containing command-line arguments.
- 3. `sys.path` contains a list of paths where the Python interpreter automatically searches for required modules.
Import Statement
To use a Python source file, simply execute the import statement in another source file. The syntax is as follows:
import module1[, module2[,... moduleN]
When the interpreter encounters an import statement, the module is imported if it is in the current search path.
The search path is a list of directories that the interpreter will search first. To import the support module, place the command at the top of your script:
support.py File Code
#!/usr/bin/python3
# Filename: support.py
def print_func( par ):
print ("Hello : ", par)
return
test.py Imports the support module:
test.py File Code
#!/usr/bin/python3
# Filename: test.py
# Import module
import support
# Now you can call the functions contained in the module
support.print_func("Geekdoc")
The above example outputs:
$ python3 test.py
Hello : Geekdoc
A module is only imported once, no matter how many times you execute the import statement. This prevents the module from being imported over and over again.
When we use the import statement, how does the Python interpreter find the corresponding file?
This involves Python’s search path. The search path consists of a series of directory names, and the Python interpreter searches these directories in turn for the imported module.
This looks very similar to an environment variable. In fact, you can also define the search path by defining an environment variable.
The search path is determined when Python is compiled or installed, and it should also be modified when installing new libraries. The search path is stored in the path variable in the sys module. To perform a simple experiment, enter the following code in the interactive interpreter:
>>> import sys
>>> sys.path
['', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
>>>
sys.path The output is a list, the first item of which is an empty string. This represents the current directory (printing this from a script makes it easier to tell which directory it is), which is the directory where the Python interpreter is executed (or, for a script, the directory where the script is running).
Therefore, if, as I did, a file with the same name as the module you want to import exists in the current directory, the module will be blocked.
Now that you understand the concept of the search path, you can modify sys.path in your script to import modules that are not in the search path.
Now, create a file called fibo.py in the current directory of the interpreter or in a directory in sys.path. The code is as follows:
Example
# Fibonacci sequence module
def fib(n): # Define the Fibonacci sequence up to n
a, b = 0, 1
while b < n:
print(b, end=' ')
a, b = b, a+b
print()
def fib2(n): # Returns the Fibonacci number at n
result = []
a, b = 0, 1
while b < n:
result.append(b)
a, b = b, a+b
return result
Then enter the Python interpreter and use the following command to import this module:
>>> import fibo
This does not write the function names defined directly in fibo into the current symbol table; it only writes the name of the module fibo there.
You can access the function using the module name:
Example
>>>fibo.fib(1000)
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
>>> fibo.fib2(100)
[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
>>> fibo.__name__
'fibo'
If you plan to use a function often, you can assign it to a local name:
>>> fib = fibo.fib
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377
from … import statement
Python’s from The `from modname` statement allows you to import a specific part from a module into the current namespace. The syntax is as follows:
from modname import name1[, name2[, ... nameN]]
For example, to import the `fib` function from the `fibo` module, use the following statement:
>>> from fibo import fib, fib2
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377
This statement does not import the entire `fibo` module into the current namespace; it only imports the `fib` function from `fibo`.
from … import * Statement
It is also possible to import all the contents of a module into the current namespace using the following statement:
from modname import *
This provides a simple way to import all items from a module. However, this statement should not be used too often.
In-Depth Modules
In addition to method definitions, modules can also contain executable code. This code is typically used to initialize the module. This code is only executed when it is first imported.
Each module has its own independent symbol table, which serves as the global symbol table for all functions within the module.
Therefore, module authors can safely use these global variables within the module without worrying about confusing other users’ global variables.
On the other hand, if you really know what you are doing, you can also use modname.itemname This notation is used to access functions within a module.
Modules can import other modules. Using import at the beginning of a module (or script, or elsewhere) to import a module is a convention, not mandatory. The name of the imported module will be placed in the symbol table of the currently operating module.
There is another way to import. You can use import to directly import the names of functions and variables within the module into the currently operating module. For example:
>>> from fibo import fib, fib2
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377
This import method does not place the name of the imported module in the current symbol table (so in this example, fibo This name is not defined).
There is another way to import all the names (functions, variables) in a module into the current module’s character table at once:
>>> from fibo import *
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377
This will import all names, except those that begin with a single underscore (_). In most cases, Python programmers don’t use this method because imported names from other sources are likely to overwrite existing definitions.
__name__ Attribute
When a module is first imported by another program, its main program will run. If we want a block of code within the module not to execute when the module is imported, we can use the __name__ attribute to prevent that block from executing only when the module itself is run.
#!/usr/bin/python3
# Filename: using_name.py
if __name__ == '__main__':
print('The program itself is running')
else:
print('I came from another module')
The output of running it is as follows:
$ python using_name.py
The program itself is running
$ python
>>> import using_name
I come from another module
>>>
Description:
Each module has a __name__ attribute. When its value is ‘__main__’, it indicates that the module is running on its own; otherwise, it has been imported.
Description: __name__ and __main__ are followed by double underscores. _ _ removes the space between them.
dir() Function
The built-in function dir() finds all names defined within a module. It returns a list of strings:
>>> import fibo, sys
>>> dir(fibo)
['__name__', 'fib', 'fib2']
>>> dir(sys)
['__displayhook__', '__doc__', '__excepthook__', '__loader__', '__name__',
'__package__', '__stderr__', '__stdin__', '__stdout__',
'_clear_type_cache', '_current_frames', '_debugmallocstats', '_getframe',
'_home', '_mercurial', '_xoptions', 'abiflags', 'api_version', 'argv',
'base_exec_prefix', 'base_prefix', 'builtin_module_names', 'byteorder',
'call_tracing', 'callstats', 'copyright', 'displayhook',
'dont_write_bytecode', 'exc_info', 'excepthook', 'exec_prefix',
'executable', 'exit', 'flags', 'float_info', 'float_repr_style',
'getcheckinterval', 'getdefaultencoding', 'getdlopenflags',
'getfilesystemencoding', 'getobjects', 'getprofile', 'getrecursionlimit',
'getrefcount', 'getsizeof', 'getswitchinterval', 'gettotalrefcount',
'gettrace', 'hash_info', 'hexversion', 'implementation', 'int_info',
'intern', 'maxsize', 'maxunicode', 'meta_path', 'modules', 'path',
'path_hooks', 'path_importer_cache', 'platform', 'prefix', 'ps1',
'setcheckinterval', 'setdlopenflags', 'setprofile', 'setrecursionlimit',
'setswitchinterval', 'settrace', 'stderr', 'stdin', 'stdout',
'thread_info', 'version', 'version_info', 'warnoptions']
If no arguments are given, the dir() function lists all currently defined names:
>>> a = [1, 2, 3, 4, 5]
>>> import fibo
>>> fib = fibo.fib
>>> dir() # Get a list of attributes defined in the current module
['__builtins__', '__name__', 'a', 'fib', 'fibo', 'sys']
>>> a = 5 # Create a new variable 'a'
>>> dir()
['__builtins__', '__doc__', '__name__', 'a', 'sys']
>>>
>>> del a # Delete the variable name a
>>>
>>> dir()
['__builtins__', '__doc__', '__name__', 'sys']
>>>
Standard Modules
Python comes with several standard module libraries, which are described in the Python Library Reference (see “Library Reference” below).
Some modules are built directly into the interpreter. Although these aren’t built-in language features, they can be used efficiently and even with system-level calls.
These components are configured differently depending on the operating system, such as The winreg module is only available on Windows systems.
Note that there is a special module, sys, built into every Python interpreter. The variables sys.ps1 and sys.ps2 define the strings corresponding to the primary and secondary prompts:
>>> import sys
>>> sys.ps1
'>>> '
>>> sys.ps2
'... '
>>> sys.ps1 = 'C> '
C> print('Geekdoc!')
Geekdoc!
C>
Packages
A package is a way of managing the Python module namespace, using “dot module names.”
For example, if a module is named A.B, it represents a submodule B within package A. Just as you don’t have to worry about global variables affecting each other when using modules, using dotted module names also eliminates the need to worry about duplicate module names across different libraries. This way, different authors can contribute NumPy modules, such as the Python graphics library. Suppose you want to design a unified module (or “package”) for processing sound files and data. There are many different audio file formats (generally distinguished by their file extensions, for example: .wav, :file:.aiff, :file:.au, etc.), so you need an ever-growing set of modules to convert between these formats.
And there are many different operations you can perform on this audio data (such as mixing, adding echo, adding equalizer functionality, and creating artificial stereo effects), so you need an endless set of modules to handle these operations.
Here’s a possible package structure (in a hierarchical file system):
sound/ Top-level package
__init__.py Initializes the sound package
formats/ File format conversion subpackage
__init__.py
wavread.py
wavwrite.py
aiffread.py
aiffwrite.py
auread.py
auwrite.py
...
effects/ Sound effects subpackage
__init__.py
echo.py
surround.py
reverse.py
...
filters/ Filters subpackage
__init__.py
equalizer.py
vocoder.py
karaoke.py
...
When importing a package, Python searches the directories in sys.path for the subdirectories within that package.
A directory is considered a package only if it contains a file named __init__.py. This is to prevent uncommon names (such as string) from accidentally affecting valid modules in the search path.
In the simplest case, an empty :file:__init__.py file will suffice. This file can also contain initialization code or assign a value to the __all__ variable (described later).
You can also import specific modules within a package at a time, for example:
import sound.effects.echo
This will import the submodule: sound.effects.echo. It must be accessed using its full name:
sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)
Another way to import a submodule is:
from sound.effects import echo
This also imports the submodule: echo, but without the long prefix, so it can be used like this:
echo.echofilter(input, output, delay=0.7, atten=4)
Another variation is to import a function or variable directly:
from sound.effects.echo import echofilter
Similarly, this method imports the submodule: echo, and you can use its echofilter() function directly. Function:
echofilter(input, output, delay=0.7, atten=4)
Note that when using the form from package import item, item can be a submodule (subpackage) within the package, or other names defined within the package, such as functions, classes, or variables.
The import syntax first treats item as a package name. If that name is not found, it attempts to import it as a module. If that is still not found, an :exc:ImportError exception is raised.
Conversely, when using the form import item.subitem.subsubitem, all but the last item must be a package, which can be a module or package, but not a class, function, or variable name.
Importing * from a Package
Imagine if we use from What happens if you call sound.effects import *?
Python will go to the file system, find all the submodules of the package, and import them one by one.
Unfortunately, this approach doesn’t work very well on Windows, which is a case-insensitive system.
On such platforms, there’s no guarantee that a file named ECHO.py will be imported as the module echo, Echo, or even ECHO.
(For example, Windows 95 annoyingly capitalizes the first letter of every file.) And DOS’s 8+3 naming convention for long module names complicates matters even further.
To fix this, the package author must provide an accurate package index.
The import statement follows this rule: If the package definition file __init__.py contains a list variable named __all__, then using from package import * , all the names in this list will be imported as package contents.
As a package author, don’t forget to update __all__ after updating the package. You might say you don’t want to do that, and you won’t use import *? Okay, no problem, you’re the boss after all. Here’s an example: file:sounds/effects/__init__.py contains the following code:
__all__ = ["echo", "surround", "reverse"]
This means that when you use from sound.effects import *, only these three submodules within the package will be imported.
If __all__ is not defined, then using the syntax from sound.effects import * will not import the package sound.effects. It simply imports the sound.effects package and all its definitions (possibly running initialization code defined in __init__.py).
This imports all names defined in __init__.py. It also does not clobber any explicitly imported modules before this statement. Consider this code:
import sound.effects.echo
import sound.effects.surround
from sound.effects import *
In this example, both the echo and surround modules from the sound.effects package are imported into the current namespace before the from…import statement is executed. (Of course, this would be even more correct if __all__ were defined.)
Usually, using * is not recommended. This method of importing modules often makes the code less readable. However, it does save you a lot of typing, and some modules are designed to be imported only in certain ways.
Remember, it’s never wrong to use from Package import specific_submodule . In fact, it’s the recommended method, unless the submodule you’re importing has a duplicate name in another package.
If a package is a subpackage in the structure (such as the sound package in this example), and you want to import a sibling package (a package at the same level), you must use an absolute import path. For example, if the module sound.filters.vocoder needs to use the module echo from the package sound.effects , you would write from sound.effects import echo.
from . import echo
from .. import formats
from ..filters import equalizer
Relative imports, both implicit and explicit, begin at the current module. The main module is always named “__main__” and should always be referenced using an absolute path.
Packages also provide an additional attribute, __path__. This is a list of directories, each containing a __init__.py file for that package. You must define this attribute before any other __init__.py files are executed. This variable can be modified to affect modules and subpackages within the package.
This feature is rarely used and is typically used to extend the modules within a package.