Monthly Archives: October 2009

Help me rewrite some repetitive scripts

I have about a dozen functions (that are run as scripts) that have very similar sections interleaved with specialized code. I copied two of the scripts below. Here’s the first:

def pitz_estimate_task():

    p = optparse.OptionParser()

    p.add_option('--version', action='store_true',
        help='Print the version and exit')

    # This script requires these arguments.
    p.set_usage("%prog task [estimate]")
    options, args = p.parse_args()
   
    if options.version:
        print_version()
        return

    # This is unique to this script.
    if not args:
        p.print_usage()
        return

    # And now we're back to boring generic stuff.
    pitzdir = Project.find_pitzdir(options.pitzdir)
    proj = Project.from_pitzdir(pitzdir)
    proj.find_me()

    # This section is specific to this script.
    t = proj[args[0]]
   
    if len(args) == 2:
        est = proj[args[1]]
   
    else:   
        est = Estimate.choose_from_already_instantiated()
   
    t['estimate'] = est
    # That was the last thing that was specific to just this script.

    # Save the project (generic).
    proj.save_entities_to_yaml_files()

That script does some “generic” stuff to build an object p, then adds on some extra tweaks to p, and uses p to build an options object and an args object.

Then the script does some generic stuff to build a proj object based on the data in the options.pitzdir object, and does some various method calls on the proj object.

And here’s another script:

def pitz_attach_file():

    p = optparse.OptionParser()

    p.add_option('--version', action='store_true',
        help='Print the version and exit')

    # Notice this line is different than the one in pitz_estimate_task.
    p.set_usage("%prog entity file-to-attach")
    options, args = p.parse_args()

    if options.version:
        print_version()
        return

    # This section is different too.
    if len(args) != 2:
        p.print_usage()
        return

    # Back to the generic code to build the project.
    pitzdir = Project.find_pitzdir(options.pitzdir)
    proj = Project.from_pitzdir(pitzdir)
    proj.find_me()

    # Some interesting stuff that is specific just for this script.
    e, filepath = proj[args[0]], args[1]
    e.save_attachment(filepath)

    # Save the project. (Generic).
    proj.save_entities_to_yaml_files()

So, the pattern in every script is: generic code, specific code, generic code, specific code, generic code. And each step depends on the previous step.

I know I could do stuff like wrap all the generic stuff into functions, but I’m not really a fan of that approach. I’m looking for an interesting way to reduce all repetition, but keep the legibility. I’m thinking some nested context managers or decorators might be the way to go. I like to hear ideas from other people, so, please, let me hear them.

By the way, all this code is from the command-line module of pitz, available here. That’s where you can see all the different variations on the same theme.

Why I’m hooked on parameters

I have this need to make everything into a parameter. Here’s some code I wrote recently:

import subprocess, tempfile

def edit_with_editor(s=None):
    """
    Returns the text typed in the editor, after running strip().
    """
    with tempfile.NamedTemporaryFile() as t:
        if s:
            t.write(str(s))
            t.seek(0)

        subprocess.call([os.environ.get('EDITOR', 'vi'), t.name])
        return t.read().strip()

There’s a voice in my head that says I shouldn’t be importing the subprocess and tempfile modules outside my function and then referring to them from within. And, the voice goes on, it wouldn’t be OK to move the import within my function either. Instead, if my function needs to use NamedTemporaryFile, I should pass that function in as a parameter.

I think it all starts with the fact that I studied economics in college. I had a lot of lectures that started with a professor drawing something like this on the chalk board:

labor supply = f(...)

And then during the lecture, she would slowly replace the ellipses with parameters. By the end of class, the function might look like:

labor supply = f(income, wealth)

Then I’d also have a few pages of notes explaining the nature of the relationship between how hard somebody is willing to work, the wage they can earn, and the wealth they already have. The thing that sunk in deep is the idea that parameters (and only parameters) are what drives the dependent variable on the left hand site. Anything that drives labor supply is listed as a parameter. If it ain’t on the right hand side, then it is not relevant.

By the way, the war between the wealth effect and the income effect is one of the areas of economics that really does explain our behavior pretty dang well, and there’s neat charts involved, so go read about it.

Open letter to all candidates for Cleveland Heights City Council

I just sent the email below to lots of the candidates running for city council in my little burb. I’m hoping a few will reply.

Hi,

I’m your neighbor; I live at XXXXXXX. Anyhow, I’m emailing as many city council candidates as I can, looking for their thoughts on a few topics.

I would like to copy and paste anything you send me in my blog. I get very little traffic from locals; most of my posts deal with fairly obscure computer science issues.

So, here’s the topics:

  1. Cleveland has red-light cameras. Should Cleveland Heights (CH) have red-light cameras?
  2. Cleveland now allows people to keep chickens in their backyards. Should CH?
  3. Should we ban or regulate gas leaf blowers?
  4. If you couldn’t vote for yourself, what other candidate would you pick?

And here’s my opinions on those topics:

  • I have nothing against red-light cameras in theory. But when I lived in DC, the city put them at intersections where lights were poorly timed and traffic flows were confusing. So it turned into a way for the city skim huge amounts of cash from drivers that believed they were following the law.

    I wouldn’t mind cameras as long as we clearly pointed them out way in advance of intersections. Also I believe we should be generous and give some number of warnings for each driver before the tickets count for anything.

  • Yup. More generally, the city should do anything it can to encourage backyard and community vegetable gardens.
  • On nice days when I work from home with all my windows open, everything is great until some landscaping crews show up. I don’t like banning things, but I would like to create incentives for landscaping firms to explore less noisy options.
  • No opinions here. I just started doing research.

Thanks for your time, and good luck!

Matt

How to download photos from phone over bluetooth with ubuntu

I have a samsung sync phone. It is about two years old now. It isn’t fancy, but it has a camera and a bluetooth device.

I wrote a python script to copy photos from my phone to my laptop. Any time I take a bunch of pictures, I just run
$ getpics.py
and if my phone is anywhere nearby, my computer will pull all the photos off my phone.

Bluetooth is a great little tool and I’m surprised it hasn’t caught on more with people like us. I’ve been able to access my phone from my laptop when I’m on the second floor and the phone is across the house and downstairs, still in my work bag. And bluetooth USB dongles are cheap (like around $9). There’s no need for carrying around USB keys or plugging in iPods. Everything should use bluetooth.

The getpics.py script requires that the phone and the laptop have already been bonded with each other. That’s not much work. Put your phone in to discoverable mode, then search for devices with your computer. When you find your phone, punch the same 4-digit PIN into the phone and your computer.

Anyway, here’s the script. I hope it helps somebody out.

#! /usr/bin/env python

# W. Matthew Wilson <matt@tplus1.com> wrote this script and he released
# it into the public domain.

# vim: set expandtab ts=4 sw=4 filetype=python:

"""
Pull all the images off my phone.
"""

import commands, logging, os

from xml.etree import ElementTree
from xml.parsers.expat import ExpatError

logging.basicConfig(level=logging.DEBUG)

def get_list_of_pictures():
    
    """
    Yield a list of strings, where each string is the file name of a
    picture.
    """
    
    # You'll need to change the 00:1D... stuff to your phone's
    # MAC address.
    # You might need to change /Graphics/ to where your phone
    # stores its photos.
    status, output = commands.getstatusoutput(
        "obexftp -b 00:1D:F6:55:7E:D9 -l /Graphics/")
    
    logging.debug("status of obexftp: %s" % status)
    
    logging.info("Finished running obexftp...")
    
    # Unfortunately, the output from obexftp includes some goofy status
    # crap and then some XML data.  So I parse each line separately,
    # and skip the lines where parsing fails.

    for line in output.split('n'):
        
        logging.debug("About to parse line %s." % line.strip())
        
        try:
            x = ElementTree.XML(line)
            # Skip lines without a name attribute.
            if 'name' not in x.keys(): continue

            # Extract and yield the name
            photofilename = x.get('name')

            logging.debug("About to yield %s." % photofilename)

            yield photofilename

            logging.debug("Finished yielding %s." % photofilename)

        except ExpatError:

            logging.debug("This line is not valid XML: %s."
                % line.strip())

def grab_pic(photofilename):

    """
    Pull the photo named photofilename and save it locally.
    """

    # You probably want to change this.
    os.chdir("/home/matt/Documents/Photos/phonepics")

    status, output = commands.getstatusoutput(
        "obexftp -b 00:1D:F6:55:7E:D9 -g /Graphics/%s" % photofilename)

    logging.info("Finished storing %s locally." % photofilename)

if __name__ == "__main__":

    logging.info("Starting...")

    for photo in get_list_of_pictures():
        grab_pic(photo)

    logging.info("All done!")