When I install a python deb package, say python-numpy the files are plain python files. How do I tell to the package manager -- that I actually would rather use pyc or even better -- pyo files?


You already have them and you're already using them

Well-packaged Python applications are being compiled to .pyc files in a script run after installation of the files has taken place. This is needed according to the packaging guidelines to be able to adopt to the Python installation you're using at that moment. Do remember that .pyc files are very specific to your system (Python version and dependencies).

All .pyo and .pyc files are therefore specifically excluded in packages and tagged as errors by Lintian:

Compiled python source files must not be included in the package. These files should be removed from the package and created at package installation time in the postinst.

Refer to Debian Python Policy section 2.6 (Modules Byte-Compilation) for details.

Severity: serious, Certainty: certain

In the case of python-numpy this post-install byte-compilation is handled by the debhelper hook of pycentral. After installation it looks like this:

ls -l /usr/lib/python2.7/dist-packages/numpy
lrwxrwxrwx  1 root root     47 Mar 20  2012 add_newdocs.py -> ../../../../share/pyshared/numpy/add_newdocs.py
-rw-r--r--  1 root root 251912 Aug  6 22:06 add_newdocs.pyc

Some more background information

The reason for why these .pyc/.pyo files aren't compiled at runtime during the first application startup as you expected is the following.

The Python files are installed in a system-wide directory, available for all users on the system. Whenever a user starts the application, the Python interpreter can read the .py files, but it can't write to the directories (e.g. /usr/lib/python2.7/dist-packages/). This is part of general security in all Linux systems; users shouldn't write to /usr, only with root rights this should be possible. For this reason, the APT hooks will compile the Python files for you at installation time; firstly to minimize the package size, secondly to be able to retrigger the hooks once something changes on your system regarding Python, because it's required to recompile when they get incompatible during upgrades for example.

This won't, however, affect regular user-owned python files being compiled at runtime.

  • So I should make /usr/share/pyshared/ writable to myself -- in order to enable python to compile files on the first run.
    – Adobe
    Jan 16 '13 at 11:21
  • 4
    @Adobe What? no! Never make these system directories writeable for users. Your package management already compiled them for you (as per what I was trying to say in my answer). Now updated to be more clear about it.
    – gertvdijk
    Jan 16 '13 at 11:24
  • You now -- that I've just look at the /usr/lib/python2.7/dist-packages/numpy -- there are pyc files. I remember I looked there on seveal installations -- and there were just plain py files. So I sort of "can't reproduce the bug".
    – Adobe
    Jan 16 '13 at 14:10
  • The only reason for why the .pyc wouldn't be there yet is that the package wasn't installed completely. After unpacking there are other steps to take. IIRC the python-central hook is run only after all packages are installed. So, in a running or interrupted APT session installing the package the observation you described can be made probably.
    – gertvdijk
    Jan 16 '13 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.