The –quick option

Normally, running pitz-add-task will prompt for a title and then open $EDITOR so I can write a description. After that, pitz will ask for choices for milestone, owner, status, estimate, and tags.

Sometimes I want to make a quick task without getting prompted for all this stuff.

I already had a --no-description option that would tell pitz-add-task to not open $EDITOR for a description. And I already had a --use-defaults option to just choose the default values for milestone, owner, status, estimate, and tags.

But when I just want to make a quick to-do task as a placeholder, writing out all this stuff:

$ pitz-add-task --no-description --use-defaults -t "Fix fibityfoo"

is kind of a drag. So I made a --quick option (also available as -q) that does the same thing as --no-description --use-defaults.

The -1 alias for –one-line-view

This is the typical way that a to-do list looks:

$ pitz-my-todo -n 3
==============================
slice from To-do list for matt
==============================

(3 task entities, ordered by ['milestone', 'status', 'pscore'])

Write new CLI scripts (witz!) to talk to pitz-webapp           9f1c76
matt | paused | difficult | 1.0 | 1
webapp, CLI
Starting up and loading all the pitz data at the beginning of ever...

Experiment with different task summarized views                0f6fee
matt | unstarted | straightforward | 1.0 | 1
CLI
Right now, the summarized view of a task looks a little like this:...

Add more supported URLs to pitz-webapp                         295b5f
matt | unstarted | straightforward | 1.0 | 0
webapp
I want to allow these actions through the webapp:  *   Insert a ne...

Incidentally, notice the -n 3 option limits the output to the first three tasks.

Tasks also have a one-line view:

$ pitz-my-todo -n 3 --one-line-view
==============================
slice from To-do list for matt
==============================

(3 task entities, ordered by ['milestone', 'status', 'pscore'])

Write new CLI scripts (witz!) to talk to pitz-webapp           9f1c76
Experiment with different task summarized views                0f6fee
Add more supported URLs to pitz-webapp                         295b5f

Typing out --one-line-view is tedious, so now, -1 is an alias that works as well:

$ pitz-my-todo -n 3 -1
==============================
slice from To-do list for matt
==============================

(3 task entities, ordered by ['milestone', 'status', 'pscore'])

Write new CLI scripts (witz!) to talk to pitz-webapp           9f1c76
Experiment with different task summarized views                0f6fee
Add more supported URLs to pitz-webapp                         295b5f

pretty colors

It took me a while to figure out how to include the jinja2 template files with the package, but you can now download pitz from PyPi here.

$ pitz-help
    pitz-abandon-task          Abandon a task
    pitz-add-task              Walks through the setup of a new Task.
    pitz-components            All components in the project
    pitz-estimates             All estimates in the project
    pitz-everything            No description
    pitz-finish-task           Finish a task
    pitz-milestones            No description
    pitz-my-tasks              List my tasks
    pitz-people                All people in the project
    pitz-prioritize-above      Put one task in front of another task
    pitz-prioritize-below      Put one task behind another task
    pitz-recent-activity       10 recent activities
    pitz-setup                 No description
    pitz-shell                 Start an ipython session after loading in a ...
    pitz-show                  Show detailed view of one entity
    pitz-start-task            Begin a task
    pitz-statuses              All statuses in the project
    pitz-tasks                 All tasks in the project
    pitz-todo                  List every unstarted and started task in the...
    pitz-unassign-task         Take this task off somebody's list of stuff ...

def pitz_estimate_task():

    p = optparse.OptionParser()

    p.add_option('--version', action='store_true',
        help='Print the version and exit')

    # This script requires these arguments.
    p.set_usage("%prog task [estimate]")
    options, args = p.parse_args()
   
    if options.version:
        print_version()
        return

    # This is unique to this script.
    if not args:
        p.print_usage()
        return

    # And now we're back to boring generic stuff.
    pitzdir = Project.find_pitzdir(options.pitzdir)
    proj = Project.from_pitzdir(pitzdir)
    proj.find_me()

    # This section is specific to this script.
    t = proj[args[0]]
   
    if len(args) == 2:
        est = proj[args[1]]
   
    else:   
        est = Estimate.choose_from_already_instantiated()
   
    t['estimate'] = est
    # That was the last thing that was specific to just this script.

    # Save the project (generic).
    proj.save_entities_to_yaml_files()

That script does some “generic” stuff to build an object p, then adds on some extra tweaks to p, and uses p to build an options object and an args object.

Then the script does some generic stuff to build a proj object based on the data in the options.pitzdir object, and does some various method calls on the proj object.

And here’s another script:

def pitz_attach_file():

    p = optparse.OptionParser()

    p.add_option('--version', action='store_true',
        help='Print the version and exit')

    # Notice this line is different than the one in pitz_estimate_task.
    p.set_usage("%prog entity file-to-attach")
    options, args = p.parse_args()

    if options.version:
        print_version()
        return

    # This section is different too.
    if len(args) != 2:
        p.print_usage()
        return

    # Back to the generic code to build the project.
    pitzdir = Project.find_pitzdir(options.pitzdir)
    proj = Project.from_pitzdir(pitzdir)
    proj.find_me()

    # Some interesting stuff that is specific just for this script.
    e, filepath = proj[args[0]], args[1]
    e.save_attachment(filepath)

    # Save the project. (Generic).
    proj.save_entities_to_yaml_files()

So, the pattern in every script is: generic code, specific code, generic code, specific code, generic code. And each step depends on the previous step.

I know I could do stuff like wrap all the generic stuff into functions, but I’m not really a fan of that approach. I’m looking for an interesting way to reduce all repetition, but keep the legibility. I’m thinking some nested context managers or decorators might be the way to go. I like to hear ideas from other people, so, please, let me hear them.

By the way, all this code is from the command-line module of pitz, available here. That’s where you can see all the different variations on the same theme.

My project is named “pitz” and it is a bugtracker. Instead of using a config file to set the options for a project, I want to use python files.

When somebody installs pitz, I want to save some .py files somewhere so that when they run my pitz-setup script, I can go find those .py files and copy them into their working directory.

I have two questions:

1. Do I need to write my setup.py file to specify that the .py files in particular directory need to be treated like data, not code? For example, I don’t want the installer to hide those files inside an egg.
2. How can I find those .py files later and copy them?

Here’s my setup.py so far:

from setuptools import setup, find_packages
version = '0.1'
setup(name='pitz',
    version=version,
    description="Python to-do tracker inspired by ditz (ditz.rubyforge.org)",

    long_description="""
ditz (http://ditz.rubyforge.org) is the best distributed ticketing
system that I know of.  There's a few things I want to change, so I
started pitz.""",

    classifiers=[],
    keywords='ditz',
    author='Matt Wilson',
    author_email='matt@tplus1.com',
    url='http://tplus1.com',
    license='',
    packages=find_packages(exclude=['ez_setup', 'examples', 'tests']),

    include_package_data=True,
    package_dir={'pitz':'pitz'},

    data_files=[('share/pitz',
        [   
            'pitz/pitztypes/agilepitz.py.sample',
            'pitz/pitztypes/tracpitz.py.sample',
        ])],

    zip_safe=False,
    install_requires=[
        # 'PyYAML',
        # 'sphinx',
        # 'nose',
        # 'jinja2',
          # -*- Extra requirements: -*-
    ],

    # I know about the much fancier entry points, but I prefer this
    # solution.  Why does everything have to be zany?
    scripts = ['scripts/pitz-shell'],

    test_suite = 'nose.collector',
)

When I run python setup.py install, I do get those .sample files copied, but they get copied into a folder way inside of my pitz install:

$ cd ~/virtualenvs/scratch/lib/
$ find -type f -name '*.sample'
./python2.6/site-packages/pitz-0.1dev-py2.6.egg/share/pitz/tracpitz.py.sample
./python2.6/site-packages/pitz-0.1dev-py2.6.egg/share/pitz/agilepitz.py.sample

I don’t know how I can write a script to copy those tracpitz.py.sample files out. Maybe I can ask pitz what its version is, and then build a tring and use os.path.join, but that doesn’t look like any fun at all.

So, what should I do instead?

I’m storing the list of stuff to do for pitz in the pitz source code. Here’s how to see the unfinished stuff in pitz.

Get a copy of the code

$ git clone git://github.com/mw44118/pitz.git
Initialized empty Git repository in /home/matt/pitz/.git/
remote: Counting objects: 621, done.
remote: Compressing objects: 100% (604/604), done.
remote: Total 621 (delta 383), reused 0 (delta 0)
Receiving objects: 100% (621/621), 98.52 KiB | 135 KiB/s, done.
Resolving deltas: 100% (383/383), done.

Install it, probably in a virtualenv

$ source ~/virtualenvs/pitz/bin/activate
$ python setup.py develop

Fire up pitz-shell

I have written one command-line tool so far: pitz-shell. Use it to start a python interpreter loaded with any pitz project. Here’s how to start a session for pitz itself:

$ pitz-shell pitz/pitzfiles/project-99c58812-5c1c-4fec-874c-c998933ba88b.yaml
/home/matt/virtualenvs/pitz/lib/python2.6/site-packages/ipython-0.9.1-py2.6.egg/IPython/Magic.py:38: DeprecationWarning: the sets module is deprecated
  from sets import Set

pitz-shell imports a bunch of classes and makes an object named p (p stands for project). p has all the information about the project described in the yaml file passed in as an argument to pitz-shell. The __repr__ method on p gives some summarized data:

In [1]: p
Out[1]: <pitz.PitzProject 'Pitz' (31 task entities, 1 person entities, 1 milestone entities) sorted by by_whatever(['type', 'status', 'created time'])>

In [4]: p.todo

Out[4]: <pitz.Bag 'Stuff to do' (23 task entities) sorted by by_whatever(['type', 'status', 'created time'])>

In [5]: print(p.todo)
===========
Stuff to do
===========

(23 task entities)
------------------

   0: Add support for something like 'ditz grep' (unknown status)
   1: Update entities by loading a CSV file (unknown status)
   2: Figure out why some tasks are not converting pointers to objects (unknown status)
   3: Support intersection, union, and other set operations on bags (unknown status)
   4: Demonstrate really simple tasks and priorities workflow (unknown status)
   5: Support a .pitz config file with all pitz scripts (unknown status)
   6: Add a todo property on project (or maybe bag) (unknown status)
   7: Write code to use strings as keys (unknown status)
   8: Prompt to save work at the end of an interactive pitz session (unknown status)
   9: Make it possible to support a filter like attribute!=value (unknown status)
  10: Write code to support sorting by anything (unknown status)
  11: Support hooks (unknown status)
  12: Write an attributes property on a bag that lists count of each attribute in any entities (unknown status)
  13: Allow two bags to be compared for equality by using their entities (unknown status)
  14: Make it easy to list each employee's tasks (unknown status)
  15: Support a $PITZDIR env var to tell where yaml files live (unknown status)
  16: Demonstrate release -< iteration -< user story -< task workflow. (unknown status)
  17: Load new entities from a CSV file (unknown status)
  18: Support grep on entities (unknown status)
  19: write data to yaml in order (unknown status)
  20: Support entity subclasses like releases, iterations, user stories, and tasks (unknown status)
  21: A bag should dump to a single CSV file (unknown status)
  22: Support using substring of name as name (unknown status)

That’s how you see the to-do list for pitz!

In a future post, I’ll show how to make new tasks and how to update tasks.

I also need to explain how Pitz lets you come up with whatever wacky workflow you want. When you set up a pitz project, you can use the classes I came up with, or subclass Entity into your own weird types. In a future post, I’ll show I’m using pitz to model an agile development system using releases, iterations, checkpoints, user stories, tasks, and people.

There are two classes in pitz: entities and bags. Everything else are subclasses of these two.

Entities

Making them

Every entity is an object like a dictionary. You can make an entity like this:

>>> from pitz import Entity
>>> e = Entity(title="example entity",
...            creator="Matt",
...            importance="not very")

You can also load an entity from a yaml file, but I’ll explain that later.

You can look up a value for any attribute like this:

>>> e['title']
'example entity'
>>> e.keys() #doctest: +NORMALIZE_WHITESPACE
['name', 'creator', 'importance', 'title', 'modified_time',
'created_time', 'type']
>>> e['type']
'entity'

Viewing them

Entities have a summarized view useful when you want to see a list of entities, and a detailed view that shows all the boring detail:

>>> e.summarized_view
'example entity (entity)'

>>> print(e.detailed_view) #doctest: +SKIP
example entity (entity)
-----------------------

name:
entity-bdd31951-cff0-42a5-92b4-97ef966a6f6f

creator:
Matt

importance:
not very

title:
example entity

modified_time:
2009-04-04 07:47:09.456068

created_time:
2009-04-04 07:47:09.456068

type:
entity

By the way, you can ignore the #doctest: +SKIP comment. That is there so the doctests will skip trying to running this example, which will generate unpredictable values.

Saving and loading them

Entities have an instance method named to_yaml_file and a from_yaml_file classmethod. Here’s how to use them:

>>> outfile = e.to_yaml_file('.') # Writes file to this directory.
>>> e2 = Entity.from_yaml_file(outfile)

Bags

Making them

While entities are based on dictionaries, bags are based on lists. You can give a bag instance a title, which is nice for remembering what it is you want it for. Bags make it easy to organize a bunch of entities.

>>> from pitz import Bag
>>> b = Bag(title="Stuff that is not very important")
>>> b.append(e)

Viewing them

Converting a bag to a string prints the summarized view of all the entities inside:

>>> print(b) #doctest: +SKIP
================================
Stuff that is not very important
================================

1 entity entities
-----------------

   0: example entity (entity)

>>> e == b[0]
True

Querying them

Bags have a matches_dict method that accepts a bunch of key-value pairs and then returns a new bag that contains all the entities in the first bag that match all those key-value pairs.

First, I’ll make a few more entities:

>>> e1 = Entity(title="example #1", creator="Matt",
...             importance="Really important")
>>> e2 = Entity(title="example #2", creator="Matt",
...             importance="not very")

>>> b = Bag('Everything')
>>> b.append(e1)
>>> b.append(e2)
>>> print(b)
==========
Everything
==========
<BLANKLINE>
2 entity entities
-----------------
<BLANKLINE>
   0: example #1 (entity)
   1: example #2 (entity)
<BLANKLINE>

>>> not_very_important = b.matches_dict(importance="not very")
>>> len(not_very_important) == 1
True
>>> not_very_important[0] == e2
True

>>> not_very_important = b(importance="not very")

Saving and loading them

Bags can send all contained entities to yaml files with to_yaml_files, and bags can load a bunch of entities from yaml files with from_yaml_files.

Right now, there is no way for a bag to save itself to yaml.

The Special Project Bag

After I finished bags and entities, I thought I was done, but then I ran into a few frustrations:

• When I made a bunch of entities, but didn’t append them all into one bag, then I couldn’t run filters across all of them.
• At the end of a session, it wasn’t easy for me to make sure that all of the entities got saved out to yaml.
• I couldn’t figure out an elegant way to store one entity as a value for another entity’s attribute.

So I made a “special” Bag subclass called Project. The idea here is that every entity should be a member of the project bag. Also, every entity should have a reference back to the project.

Using a project is easy. Just pass it in as the first argument when you make an entity. Imagine I want to link some tasks to Matt and some other tasks to Lindsey. First I make a project:

>>> from pitz import Project
>>> weekend_chores = Project(title="Weekend chores")

>>> matt = Entity(weekend_chores, title="Matt")
>>> lindsey = Entity(weekend_chores, title="Lindsey")
>>> t1 = Entity(weekend_chores, title="Mow the yard", assigned_to=matt)
>>> t2 = Entity(weekend_chores, title="Buy some groceries",
...             assigned_to=lindsey)

>>> chores_for_matt = weekend_chores(assigned_to=matt)
>>> mow_the_yard = chores_for_matt[0]
>>> mow_the_yard['assigned_to'] == matt
True

Pointers

There’s a problem in that last example: when I send this mow_the_yard entity out to a YAML file, what will I store as the value for the “assigned_to” attribute?

In SQL, this is what foreign keys are good for. In my chores table, I would store a reference to a particular row in the people table.

I wanted the same functionality in pitz, so I came up with pointers. This is dry stuff, so here’s an example:

>>> class Chore(Entity):
...     pointers = dict(assigned_to='person')
...
>>> class Person(Entity):
...     pass
>>> matt = Person(weekend_chores, title="Matt")
>>> lindsey = Person(weekend_chores, title="Lindsey")
>>> ch1 = Chore(weekend_chores, title="Mow the yard", assigned_to=matt)
>>> ch2 = Chore(weekend_chores, title="Buy some groceries",
...             assigned_to=lindsey)

>>> ch1['assigned_to']
<pitz.Person 'Matt'>
>>> matt['name'] # doctest: +SKIP
'person-530ad3cc-14f1-491a-bdb6-ed1dd65afe46'
>>> ch1.replace_objects_with_pointers()
>>> ch1['assigned_to'] # doctest: +SKIP
'person-530ad3cc-14f1-491a-bdb6-ed1dd65afe46'

After running the replace_objects_with_pointers method, I don’t have a reference to the matt object. Instead, I have matt’s name now.

Now I can send this data out to a yaml file. And when I load it back in from yaml, I can then reverse this action, and go look up an entity with the same name:

>>> mn = matt.name
>>> matt == weekend_chores.by_name(mn)
True

That’s the end of the data model documentation. I hope that shines enough light so that it is obvious if pitz would be useful to you or not.

I’m working on a separate article where I show some real-world workflows modeled in pitz, but that will be next week’s post.

Here’s some code that you can use to load some ditz issues into a python interpreter. You need to install my pitz project first though.

>>> from pitz.junkyard.ditzloader import *
>>> from glob import glob
>>> issue_file_path = glob('../.ditz/issue-*.yaml')[0]
>>> import yaml
>>> issue = yaml.load(open(issue_file_path))
>>> issue.title
'Distribute reports by email'
>>> print issue.desc
Somehow allow people to sign up for report subscriptions.

So when new reports come out, they get updated.  Maybe I can use
<b>RSS</b> feeds to hold the reports.
>>> print issue.log_events
[[datetime.datetime(2008, 9, 2, 17, 47, 44, 549355), 'Matthew Wilson <mw@staffknex.com>', 'created', ''], [datetime.datetime(2008, 9, 2, 19, 18, 3, 286902), 'Matthew Wilson <mw@staffknex.com>', 'assigned to release 3.5.1 from unassigned', ''], [datetime.datetime(2008, 9, 4, 18, 27, 19, 571991), 'Matthew Wilson <mw@staffknex.com>', 'unassigned from release 3.5.1', '']]

That above just showed how to read the ditz issue. I’m having trouble updating the issue and then saving it in a format that ditz can still read. Updating the ditz issue and saving it out again is easy:

>>> issue.title = 'Distribute reports by email or RSS'
>>> open(issue_file_path, 'w').write(yaml.dump(issue))

But when I try to load it with ditz, I get this error:

$ ditz show 1209b
/home/matt/checkouts/ditz/lib/ditz/model-objects.rb:124:in `sort_by': comparison of String with Time failed (ArgumentError)
        from /home/matt/checkouts/ditz/lib/ditz/model-objects.rb:124:in `assign_issue_names!'
        from /home/matt/checkouts/ditz/lib/ditz/model-objects.rb:51:in `issues='
        from /home/matt/checkouts/ditz/lib/ditz/file-storage.rb:21:in `load'
        from /home/matt/checkouts/ditz/bin/ditz:165

I compared my dumped file to the original ruby file and found that python wrote dates out like this:

>>> print yaml.dump(issue.creation_time)
2008-09-02 17:47:43.268059

But in the ruby yaml files, the dates look like this:

$ grep creation_time ../../../.ditz/issue-1209b17b64335383a710ccadf10b74c3401dbcb2.yaml 
creation_time: 2008-09-02 17:47:43.268059 Z

That trailing Z seems important.

Also, python and ruby seem to write out lists differently. Here’s how python dumped a list of lists:

>>> print yaml.dump(issue.log_events)
- [!!timestamp '2008-09-02 17:47:44.549355', Matthew Wilson <mw@staffknex.com>, created,
  '']
- [!!timestamp '2008-09-02 19:18:03.286902', Matthew Wilson <mw@staffknex.com>, assigned
    to release 3.5.1 from unassigned, '']
- [!!timestamp '2008-09-04 18:27:19.571991', Matthew Wilson <mw@staffknex.com>, unassigned
    from release 3.5.1, '']

But the same data dumped by ruby looks like:

log_events:
- - 2008-09-02 17:47:44.549355 Z
  - Matthew Wilson <mw@staffknex.com>
  - created
  - ""
- - 2008-09-02 19:18:03.286902 Z
  - Matthew Wilson <mw@staffknex.com>
  - assigned to release 3.5.1 from unassigned
  - ""
- - 2008-09-04 18:27:19.571991 Z
  - Matthew Wilson <mw@staffknex.com>
  - unassigned from release 3.5.1
  - ""

So, there’s clearly some more work for me (or you, this is an open-source project) to do.

This post describes a little of what I’ve gotten done so far. I haven’t written any interface yet other than from within an interactive python session. Deal with it.

Getting started

You have to make a bag to keep all the tasks and then you make tasks with any keywords you can dream up.

>>> from pitz import Bag, Task
>>> b = Bag('.pitz')
>>> b.append(Task(title='Wash the dishes', creator='Matt', importance='Not very'))
>>> b.append(Task(title='Clean the cat box', creator='Matt', importance='Not very'))

You can add tasks to the bag by using the append method on the bag, or by passing in the bag as the first argument to the task:

>>> Task(b, title='Set new high score on Sushi-go-round', creator='Matt', importance='critical')

Printing a bag really prints the title of each task in the bag:

>>> print(b)
Clean the cat box
Set new high score on Sushi-go-round
Wash the dishes

Running queries

I’m proud of this one. This is the main reason I’m working on ditz. Bag instances have a feature called matching_pairs that can filter the tasks down to a smaller new bag.

You can filter by a single pair like this:

>>> critical_tasks = b.matching_pairs([('importance', 'critical')])
>>> print(critical_tasks)
Set new high score on Sushi-go-round

Or you can filter by multiple pairs and the filtered tasks must satisfy ALL the pairs.

>>> print(b.matching_pairs([('creator', 'Matt'), ('importance', 'Not very')]))
Clean the cat box
Wash the dishes

Dumping and loading

Use the bag that holds all the tasks to quickly write all the tasks out to your hard drive like this:

>>> b.to_yaml_files()
['.pitz/task-07e1af97-0ac6-4904-9187-0c2fd61692b6.yaml', '.pitz/task-6a7af07c-d0fb-4a77-9347-8dc78ef490fe.yaml', '.pitz/task-5ce725dc-c1db-4eca-a74c-55cd0e910786.yaml']

The returned stuff is a list of files that pitz just wrote.

Loading from the hard drive is pretty simple too. Just tell the bag where to load from.

>>> b2 = Bag('.pitz')
>>> print(b2)
Clean the cat box
Set new high score on Sushi-go-round
Wash the dishes

Task details

Printing a task by itself gives all the details on the task.

>>> t = Task.from_yaml_file ('.pitz/task-4d9c1db2-fef3-4b50-8095-b2339384e118.yaml')
>>> print(t)
Do the 2008 taxes
-----------------

            type: Task
            name: task-4d9c1db2-fef3-4b50-8095-b2339384e118
           title: Do the 2008 taxes
    created date: 2009-03-01 22:29:58.242512
   modified date: 2009-03-01 22:29:58.242512
         creator: Matt
last modified by: Matt

     description: 
Do the 2008 taxes

Other stuff

You can use bags as iterators to go through the tasks one-by-one:

for task in b:
    print(t['title'])

Also notice that tasks are really just subclassed dictionaries (UserDict, actually) with some extra methods bolted on.

Instead of just banging out code, I decided to write the documentation and the list of supported features FIRST.

Once I have my feature set established, then I’ll write the tests for those features, and finally, I’ll write the code.

I’m looking for feedback on the feature set. What would an ideal bugtracker look like?