Python

The Python interpreter is installed on all WebFaction machines. For your convenience, several different Python versions are available. All of our machines have the following versions installed:

  • Python 2.4
  • Python 2.5
  • Python 2.6
  • Python 2.7
  • Python 3.0
  • Python 3.1
  • Python 3.2
  • Python 3.3

Some machines also have Python 2.3 installed. For a listing of Python installations on your machine, run ls /usr/local/bin/ | grep python in an SSH session. To identify the default Python version, run python -V.

To specify a Python version, use pythonX.Y where X.Y is the Python version number. For example, to use Python 2.6, enter python2.6.

Python Search Path

To make working with Python and applications easier, the Python search path has been configured with two additions:

  • $HOME/lib/pythonX.Y is added to the Python search path for each version of Python.
  • $HOME/webapps/app/lib/pythonX.Y is added to the Python search path when the current working directory is $HOME/webapps/app/ or its children, where app is the name of an application as it appears in the control panel.

To see the contents of your Python search path, enter pythonX.Y -c "import sys; print sys.path" and press Enter.

See also

/usr/local/lib/pythonX.Y/sitecustomize.py and /usr/lib/pythonX.Y/sitecustomize.py contain the complete implementation of the Python search path modifications.

Creating a python Alias

On WebFaction’s CentOS 6 servers (web300 and greater or dweb98 and greater), the default python is Python 2.6. On all other servers, the default python command line program is Python 2.4. You can create an alias for python to refer to another version during interactive SSH sessions. To create a python alias:

  1. Open an SSH session.
  2. Open $HOME/.bash_profile in a text editor.
  3. On a new line, add alias python=pythonX.Y where X.Y is the Python version number. For example, to use Python 2.7, add a new line containing alias python=python2.7.
  4. Save and close the file.
  5. Reload $HOME/.bash_profile. Enter source $HOME/.bash_profile and press Enter.

Now all future interactive SSH sessions will use the specified Python version at the command line.

Note

The python alias will not be available in shell scripts or other situations where the alias is not defined. In such cases, the explicit Python version should be called instead (for example, python2.7).

Installing Python Packages

Python packages and modules are typically distributed

  • automatically with easy_install,
  • as a Python Egg,
  • as an archive containing a setup.py file,
  • or as a collection of source (.py) files.

You can install Python packages and modules distributed in each of these ways in your WebFaction account.

Installing Packages with easy_install

Many common Python packages can be automatically installed with easy_install. easy_install will look up information about the module from the Python Package Index (also known as the “Cheeseshop”), download, and install the module automatically.

To use easy_install to install a package to your home directory:

  1. Make sure the destination directory exists. Enter mkdir -p $HOME/lib/pythonX.Y, where X.Y is the Python version, and press Enter.
  2. Install the package. Enter easy_install-X.Y package, where package is the name of the package to install, and press Enter.

To use easy_install to install a package to a specific web application which makes use of the Python interpreter, enter PYTHONPATH=$HOME/webapps/app_name/lib/pythonX.Y easy_install-X.Y --install-dir=$HOME/webapps/app_name/lib/pythonX.Y --script-dir=$HOME/webapps/app_name/bin module_name, where app_name is the name of the web application as it appears in the control panel and press Enter.

Installing Eggs

Some packages are distributed as Python Eggs. These are single files which contain an entire Python package and typically have the .egg file extension. Eggs are installed using easy_install.

To install an egg to your home directory:

  1. Make sure the destination directory exists. Enter mkdir -p $HOME/lib/pythonX.Y, where X.Y is the Python version, and press Enter.
  2. Install enter easy_install-X.Y egg_path, where egg_path is the path to the egg to install, and press Enter.

To install an egg to a specific web application which makes use of the Python interpreter, enter PYTHONPATH=$HOME/webapps/app_name/lib/pythonX.Y easy_install-X.Y --install-dir=$HOME/webapps/app_name/lib/pythonX.Y --script-dir=$HOME/webapps/app_name/bin egg_path, where app_name is the name of the web application as it appears in the control panel and press Enter.

Installing Packages with setup.py

Many packages are distributed with a setup.py file created with the help of setuptools.

To install a package with setup.py to your home directory:

  1. Make sure the destination directory exists. Enter mkdir -p $HOME/lib/pythonX.Y, where X.Y is the Python version, and press Enter.
  2. Switch to the directory of setup.py. Enter cd path, where path is the path to the directory which contains setup.py, and press Enter.
  3. Run setup.py. Enter PYTHONPATH=$HOME/lib/pythonX.Y pythonX.Y setup.py install --install-lib=$HOME/lib/pythonX.Y --install-scripts=$HOME/bin --install-data=$HOME/lib/pythonX.Y and press Enter.

To install a package with setup.py to a Python-based application:

  1. Switch to the directory of setup.py. Enter cd path, where path is the path to the directory which contains setup.py, and press Enter.
  2. Run setup.py. Enter PYTHONPATH=$HOME/webapps/app_name/lib/pythonX.Y pythonX.Y setup.py install --install-lib=$HOME/webapps/app_name/lib/pythonX.Y --install-scripts=$HOME/webapps/app_name/bin --install-data=$HOME/webapps/app_name/lib/pythonX.Y, where app_name is the name of the application and X.Y is the Python version, and press Enter.

Note

To install a package with setup.py and the --install-lib and`–install-scripts` options, the --install-lib directory must also be in Python’s search path.

Installing Packages Manually

To install a Python package manually:

  1. Make sure the destination directory exists. Enter mkdir -p $HOME/lib/pythonX.Y, where X.Y is the Python version, and press Enter.
  2. Copy the source files to $HOME/lib/pythonX.Y. Alternatively, you may symbolically link to those files.

Alternatively, you can install packages directly to a web application which makes use of the Python interpreter by copying files to $HOME/webapps/app_name/lib/pythonX.Y, where app_name is the name of the web application as it appears in the control panel and press Enter.

Installing Packages with Pip

Pip, like easy_install, installs Python packages. Most packages that can be installed with easy_install can be installed with pip.

Before you can use pip to install packages, you must install pip in your home directory. To install pip:

  1. Make sure the destination directory exists. Enter mkdir -p $HOME/lib/pythonX.Y, where X.Y is the Python version, and press Enter.
  2. Install pip. Enter easy_install-X.Y pip, where X.Y is the version of Python you wish to use, and press Enter.

To use pip to install packages in your home directory, enter pipX.Y install --user package, where package is the name of the package, and press Enter. To upgrade an existing package, enter pipX.Y install --user --upgrade package and press Enter.

To use pip to install to a specific application:

  1. Switch to the application directory. Enter cd $HOME/webapps/app and press Enter.

  2. Install the package with pip.

    To install a new package, enter pipX.Y install --target=$PWD/lib/pythonX.Y --install-option="--install-scripts=$PWD/bin" package and press Enter.

    To upgrade an existing package, enter pipX.Y install --upgrade --target=$PWD/lib/pythonX.Y --install-option="--install-scripts=$PWD/bin" package and press Enter.

Troubleshooting

Fixing ImportError Exceptions

Python will raise an ImportError exception whenever it cannot find a particular package or module as requested by an import statement. This is caused by the named module not appearing in any of the module search path directories. The Python search path, represented in Python as the list sys.path, is constructed from Python defaults, the PYTHONPATH environment variable, and direct manipulations to sys.path.

To see the contents of sys.path:

  1. Run Python interactively. Enter pythonX.Y and press Enter.
  2. At the prompt, enter import sys and press Enter.
  3. At the prompt, enter print sys.path and press Enter. The contents of sys.path will be displayed on screen.

By default, Python searches in this order:

  1. the directory where Python is executed;
  2. if applicable, the $HOME/webapps/app_name/lib/pythonX.Y directory where app_name is the name of an application and X and Y are the Python major and minor version numbers, respectively;
  3. the $HOME/lib/pythonX.Y directory;
  4. directories specified by PYTHONPATH;
  5. the Standard Library;
  6. built-in components of Python.

There are several possible fixes for this problem:

  • add the path of the module to PYTHONPATH in your $HOME/.bash_profile file,
  • add the path to the module to PYTHONPATH from the command line before executing the Python program which raises the exception,
  • add the path to the module to sys.path before the import statement which raises the exception within the offending program,
  • add the path to the module to httpd.conf for applications using mod_python,
  • add the path to the module to myproject.wsgi for applications using WSGI,
  • or, install offending packages directly to a path where Python will find it.

See also

For more information on how Python loads modules, please see The Python Tutorial » 6. Modules.

Adding to PYTHONPATH from the $HOME/.bash_profile File

Adding an entry to the end of the PYTHONPATH environment variable is an easy and persistent way to avoid ImportError exceptions. Unfortunately, this approach can cause complications while debugging Python programs, since the change to the PYTHONPATH environment variable happens silently, before you enter any commands at all. This is especially apparent when using two or more versions of the same module.

To add the path to a Python module to the PYTHONPATH environment variable from the $HOME/.bash_profile file:

  1. Open $HOME/.bash_profile with a text editor.
  2. Add export PYTHONPATH="path_to_module:$PYTHONPATH" to the end of the file.
  3. Save and close the file.

Log out and log in for the changes to take effect.

Note

Changes to $HOME/.bash_profile only change sys.path for Python programs executed from a shell session. This approach would not resolve ImportError exceptions associated with web applications and other programs run with the help of cron.

Adding to PYTHONPATH from the Command Line

You can add to the PYTHONPATH environment variable with each run of a Python program. This method is not persistent, which makes it easy to switch between versions of a particular module. Unfortunately, it requires that you include the PYTHONPATH changes on each run of the program. You can also use this method in conjunction with the .bash_profile method.

To add the path to a Python module to the PYTHONPATH environment variable from the command line, run the program like this:

PYTHONPATH=path_to_module:$PYTHONPATH python program_name

Adding to sys.path within Python and WSGI Scripts

You can add to a script’s Python module search path programmatically by adding entries to the sys.path list. It requires that you make changes to each script (like .py files) or WSGI file (like .wsgi files) for which you want to change the module search path.

To add to sys.path within a script, add the follow statements before the import statement which raises the ImportError exception:

import sys
sys.path.insert(0, "path_to_module")

where path_to_module is the path you wish to add to the module search path.

An equivalent method is to add to sys.path with the + operator like this:

import sys
sys.path = ["path_to_module1", "path_to_module2"] + sys.path

That method is common for for WSGI-based applications, where sys.path can be modified in the myproject.wsgi file found in /home/username/webapps/appname/. Typically, a myproject.wsgi file contains a line similar to this:

sys.path = ['/home/username/webapps/djangoapp/myproject', '/home/username/webapps/djangoapp/lib/python2.5'] + sys.path

Just add additional paths to the list to include them in the script’s search path.

Adding to sys.path from httpd.conf

For web applications running on mod_wsgi, the Python module search path can be amended in the httpd.conf file, found in /home/username/webapps/appname/apache2/conf/. The file contains a line like this:

WSGIDaemonProcess app_name processes=5 python-path=/home/username/webapps/app_name:/home/username/webapps/app_name/lib/python2.7 threads=1

The WSGIDaemonProcess directive accepts a parameter python-path and a colon-separated list of directories to set the Python module search path.To add a directory to the search path, add a directory path and (if applicable) a colon after python-path=.

Note

If you’re using virtualenv, then include the full path to the virtualenv’s lib/python2.X and lib/python2.X/site-packages directories in the python-path list.

See also

See the mod_wsgi documentation on Configuration Directives for details on WSGIDaemonProcess‘s parameters and how python-path is handled.

Moving or Installing Packages Directly to a Path in sys.path

Finally, for any application which makes use of the Python interpreter, you can also install or copy modules and packages directly to a directory which is included in Python’s search path.

For example, you could install additional modules to the $HOME/webapps/app_name/lib/pythonX.Y directory, which are included in the search path for the application by default.

For more information about installing Python packages and modules, see Installing Python Packages.

Error: Permission denied During Package Installation

Some Python package installations, like lxml, attempt to execute files within /tmp, which is not permitted on a WebFaction server. To install such packages, you must create and specify your own temporary directory before installing the package. To install a package using your own temporary directory:

  1. Open an SSH session to your account.
  2. Create the temporary directory. Enter mkdir -p $HOME/tmp and press Enter. A new directory, tmp, is created in your home directory.
  3. Configure the shell to use the new temporary directory. Enter export TEMP=$HOME/tmp and press Enter.
  4. Install the package.