Py2Exe and SQLAlchemy

I recently had the task of delivering an application written in Python to a large customer.  This would be the first application developed with Python that my company has distributed.  We chose to use Py2Exe for distribution of this program.  Py2Exe is a Python distutils extension that takes your Python code as input and outputs Windows executables that can run without a local Python installation.  Read about it at www.py2exe.org.  I have blogged about it before as well, you can read that here.

At times Py2Exe will not be able to determine that you are using some Python modules and packages.  In this case you will have to manually include them or your resultant binary file will raise an ImportError and stop running.  This is the case when using SQLAlchemy. 

SQLAlchemy is an awesome Python based SQL toolkit and ORM package.  Read all about it at www.sqlalchemy.org.  SQLAlchemy was completely new to me.  I wanted to learn about it so I forced myself to use it on this project.  I have had a very good experience with SQLAlchemy and will use it again whenever I can.  Below is my setup.py file that I used for my application with names and descriptions replaced with meaningless names. 

from distutils.core import setup
import py2exe #@UnusedImport
import platform
 
# 1. List of python modules to exclude from the distribution
mod_excludes = [
    "Tkinter",
    "doctest",
    "unittest",
    "pydoc",
    "pygments",
    "pdb",
    "email"
]
 
# 2. List of dll's (and apparently exe's) to exclude from the distribution
#    if any Windows system dll appears in the dist folder, add it to this
#    list.
dll_excludes = [
    "API-MS-Win-Core-LocalRegistry-L1-1-0.dll",
    "POWRPROF.dll",
    "w9xpopen.exe"
]
 
# 3. List of python modules that are to be manually included.
mod_includes = [
    "Cheetah.DummyTransaction",
]
 
# 4. List of python packages that are to be manually included.
package_includes = [
    "sqlite3",
    "sqlalchemy.dialects.sqlite"
]
 
# 5. determine the distribution folder.
arch = platform.architecture()
if '32bit' in arch:
    dist_dir = "dist-x86"
elif '64bit' in arch:
    dist_dir = "dist-x64"
else:
    raise RuntimeError("Unsupported architecture!")
 
# 6. Dictionary of options to pass to py2exe
py2exe_options = {
    "optimize": 2, # 0 (None), 1 (-O), 2 (-OO)
    "includes": mod_includes,
    "excludes": mod_excludes,
    "dll_excludes": dll_excludes,
    "packages": package_includes,
    "xref": False,
    # bundle_files: 1|2|3
    #    1: executable and library.zip
    #    2: executable, Python DLL, library.zip
    #    3: executable, Python DLL, other DLLs and PYDs, library.zip
    "bundle_files": 3,
    "dist_dir": dist_dir
}
 
# 7. call setup to create the service and the console app
setup(service=[{'modules': 'myservice',
                'icon_resources': [(1, '..\\my.ico')]
               }],
      console=[{'script': '..\\myexe.py',
                'icon_resources': [(1, '..\\my.ico')]
              }],
      version='1.0',
      description='My Service',
      long_description="My service verbose description.",
      author='Jon Anglin',
      author_email='jonanglin@somewhere.com',
      url='http://japrogbits.blogspot.com',
      options={"py2exe": py2exe_options}
     )

To make your executable run correctly, you need to tell Py2exe to include your database DB-API package and the SQLAlchemy database dialect package that you have used.  You can see this in the code above at 4.  I have indicated that Py2exe should include the sqlite3 and the sqlalchemy.dialects.sqlite packages in the final output.  You can also indicate that Py2exe should include a certain Python module.  I have done this as well (above at 3) as I was using the Cheetah template library for formatting my web reports (read about Cheetah).

Also at times Py2exe may include in your output DLLs or Python package and modules that should not be included.  When a Windows system DLL ends up in my distribution folder I know for sure that I need to exclude it in my setup file.  You can see this at 2 above.  Python modules are not so easy to determine though.  You may exclude a module and suddenly find that you executable will no longer run.  If you are certain that a Python package or module is not used in your application you may exclude it as shown in 1 above.

I also build 32 and 64 bit versions of the software.  This is accomplished simply by running setup.py first with the 32 bit python.exe and then again with the 64 bit python.exe.  I use the currently running platform.architecture() string to create separate distribution folders for each build of the software.  You can see that at 5 above.