Need help with data files and setup.py

Posted on by

I’m working on a package that includes some files that are meant to be copied and edited by people using the package.

My project is named “pitz” and it is a bugtracker. Instead of using a config file to set the options for a project, I want to use python files.

When somebody installs pitz, I want to save some .py files somewhere so that when they run my pitz-setup script, I can go find those .py files and copy them into their working directory.

I have two questions:

  1. Do I need to write my setup.py file to specify that the .py files in particular directory need to be treated like data, not code? For example, I don’t want the installer to hide those files inside an egg.
  2. How can I find those .py files later and copy them?

Here’s my setup.py so far:

from setuptools import setup, find_packages
version = '0.1'
setup(name='pitz',
version=version,
description="Python to-do tracker inspired by ditz (ditz.rubyforge.org)",

long_description="""\
ditz (http://ditz.rubyforge.org) is the best distributed ticketing
system that I know of. There's a few things I want to change, so I
started pitz.""",

classifiers=[],
keywords='ditz',
author='Matt Wilson',
author_email='[email protected]',
url='http://tplus1.com',
license='',
packages=find_packages(exclude=['ez_setup', 'examples', 'tests']),

include_package_data=True,
package_dir={'pitz':'pitz'},

data_files=[('share/pitz',
[
'pitz/pitztypes/agilepitz.py.sample',
'pitz/pitztypes/tracpitz.py.sample',
])],

zip_safe=False,
install_requires=[
# 'PyYAML',
# 'sphinx',
# 'nose',
# 'jinja2',
# -*- Extra requirements: -*-
],

# I know about the much fancier entry points, but I prefer this
# solution. Why does everything have to be zany?
scripts = ['scripts/pitz-shell'],

test_suite = 'nose.collector',
)

When I run python setup.py install, I do get those .sample files copied, but they get copied into a folder way inside of my pitz install:

$ cd ~/virtualenvs/scratch/lib/
$ find -type f -name '*.sample'
./python2.6/site-packages/pitz-0.1dev-py2.6.egg/share/pitz/tracpitz.py.sample
./python2.6/site-packages/pitz-0.1dev-py2.6.egg/share/pitz/agilepitz.py.sample

I don’t know how I can write a script to copy those tracpitz.py.sample files out. Maybe I can ask pitz what its version is, and then build a tring and use os.path.join, but that doesn’t look like any fun at all.

So, what should I do instead?

19 thoughts on “Need help with data files and setup.py”

  2. Yeah, that's what I'm working with. My problem is that the files that I describe in data_files get installed to a bizarre place:

    $ cd ~/virtualenvs/scratch/lib/
    $ find -type f -name '*.sample'
    ./python2.6/site-packages/pitz-0.1dev-py2.6.egg/share/pitz/tracpitz.py.sample
    ./python2.6/site-packages/pitz-0.1dev-py2.6.egg/share/pitz/agilepitz.py.sample

    So it isn't clear how I can find those files from a script later. pitz-0.1dev is not likely to be a fixed location. And same with py2.6.egg.

  4. OK, now I think I get it. To pull the stuff out, I gotta use pkg_resources. Then that library finds the data, regardless of where it might be.

    Thanks for the help.

  5. is installing in “pitz-0.1dev-py2.6.egg” as oposed to just “pitz” the default??

    I thought only easy_install does it like that?

    pip.py OTOH shouldn't do it.

  6. Also consider providing an installer for your application instead. It may be overkill for command-line app with only a couple files which the user needs to customize … but distutils and setuptools are focused on just managing Python packagaes. If you want to install a python-based application it's nice to have a proper installer. Buildout makes a nice cross-platform install tool – although again that maybe overkill, and you could have a simple 'install-pitz-instance.py' script or such that handles copying the files into a user-editable location.

    Also, using entry points is more preferable than using the distutils 'scripts' metadata. The advantage of entry points is that you can write-out the script at install time such that it has a shebang that is hard-coded to a particular Python install and location of the pitz package files. Where as with 'scripts' you need to use #!/usr/bin/env python — which requires that the user's environment conform to the application (e.g. on some terminals on my computer this might point to a Python 3 install, and on another terminal it's supporting an old Python 2.3 based app). It also requires that you install the library files into a specific Python's site-packages, and that also has downsides.

  7. Thanks for a great post, was an insight into the global variable utility, I am new to programming and have been learning everyday.
    Nice i discovered your post…

  8. My knowledge in this scenario is still neophyte to be considered as I'm still in my first stage of knowing this thing. I'm a designer too but still grasping for more info. I find you so experts that I may somehow follow the concept.

  14. Where as with 'scripts' you need to use #!/usr/bin/env python — which requires that the user's environment conform to the application (e.g. on some terminals on my computer this might point to a Python 3 install, and on another terminal it's supporting an old Python 2.3 based app). It also requires that you install the library files into a specific Python's site-packages, and that also has downsides.

Comments are closed.