Working with em-based widths can be baffling

These four paragraphs have the same number of characters, but because the letter M is fat and the letter I is skinny, different amounts of characters appear in each line:

M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M

I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I

This is the exact same text, but now I’m using a monospace font:

M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M

I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I

Franz Kafka would like this

I got a subscription to Backyard Poultry magazine earlier this year. It’s awesome. This month’s issue led me to to the City Chicken website. And that led me to this hilarious video that chronicles one man’s quest to find out if he can raise chickens in Chicago:

Update: Cleveland Heights has an ordnance against keeping chickens. Grr. Maybe I can team with the libertarian dude down the street and make a run for city council to get this thrown out.

Use bazaar and subversion together

Subversion is great and I have been using it for years. But I keep running into the same two problems over and over:

  • I did a bunch of work and I want to commit it, but I don’t have network access.
  • I spent a day playing around with a crazy idea, and I want to store my half-finished and buggy work. I don’t want to put it into the trunk because it will break a bunch of other people’s work. I don’t want to make a branch because I find branching and merging again to be tedious.

Both problems could be solved if I could make commits that were local to my hard drive and visible to me only.

Today I spent a few hours on IRC and I found out that bazaar plays really nicely with subversion, and now I have a solution.

  1. First, I loaded my subversion repository into bazaar. Here’s the command I ran:

    bzr branch svn+http://my-svn-repository.example.com/myproj

    My subversion repository has 990 revisions. This command took about 3 minutes to finish.

  2. Then I went into the local myproj directory and edited some code. I used bzr status and bzr diff to see what I changed while I worked. They operate nearly identically to their svn counterparts, but bzr diff has no --summarize option.
  3. Committing locally is really simple:

    bzr commit -m "This is a local commit and svn won't see it!"

  4. Then I did some more edits and committed those:

    bzr commit -m "OK, now maybe I have something that I want to put into subversion."

  5. Before committing to the subversion repository, I pulled down other people’s updates to the subversion repository like this:

    bzr pull svn+http://my-svn-repository.example.com/myproj

    This is a lot like running svn update.

  6. After making sure my stuff plays nice with other people’s updates, I pushed my stuff into subversion:

    bzr push svn+http://my-svn-repository.example.com/myproj

  7. Now when anyone runs svn update in their subversion working copy, they will get my work along with my commit comment.

So far, I’m really happy with this set up. The Bazaar-Subversion FAQ had a lot of really useful tips. Finally, there’s still a lot of stuff I haven’t tried yet, like renaming or copying files with svn.

My zeal for McCain just evaporated

Yeesh. This is akin to a programmer mixing up javascript and java:

Questioned by reporters, McCain continued, adding that it is “common knowledge and has been reported in the media that al-Qaida is going back into Iran and receiving training and are coming back into Iraq from Iran, that’s well known. And it’s unfortunate.”

Then, Senator Joseph Lieberman, who, like McCain is a staunch supporter of the war in Iraq, stepped forward and whispered in McCain’s ear.

McCain then said, “I’m sorry, the Iranians are training extremists, not al-Qaida,” the Washington Post reported.

This article has the bloody details.

Python’s hash lookup is insanely good

As near as I can tell, looking up a string inside a set is effectively free.

I made a 9000-element set, where each element is foo_0, or foo_1, foo_2, … , on up to foo_8999. Then I measured the time cost of testing whether an element belongs to that set:

$ python -m timeit -s 's = set(["foo_%s" % i for i in range(9000) ])' '"foo_4500" in s'
1000000 loops, best of 3: 0.447 usec per loop

Next I measured a few scans across a list of the same size:

$ python -m timeit -s 's = ["foo_%s" % i for i in range(9000) ]' '"foo_0" in s'
1000000 loops, best of 3: 0.447 usec per loop
$ python -m timeit -s 's = ["foo_%s" % i for i in range(9000) ]' '"foo_1" in s'
1000000 loops, best of 3: 0.659 usec per loop
$ python -m timeit -s 's = ["foo_%s" % i for i in range(9000) ]' '"foo_900" in s'
10000 loops, best of 3: 130 usec per loop
$ python -m timeit -s 's = ["foo_%s" % i for i in range(9000) ]' '"foo_4500" in s'
1000 loops, best of 3: 631 usec per loop

It takes more time to do to two string comparisons than it does to hash the string and look it up in the set.

A few rules I try to follow with TurboGears

These are a few of the rules I try to follow in my design. So far, they’ve helped me out.

I aim to finish all interaction with the database before I get to the template layer.

This is non-trivial because it is so easy to forget that a method or an attribute will evaluate into a query. I use this rule because it lets me be certain about the number of interactions each page will have with the database.

I avoid branching (if-else clause) in my templates as much as possible.

I have a really hard time detangling code when I find a bunch of nested if statements. For all but the most trivial instances, I prefer to have a bunch of similar templates and then choose the best one. For example, instead of handling both a successful login and a failed login in a single template, I’ll make two different files and then choose the right one in my controller.

In practice, I have some really similar templates. But then I go back and strip out as much of the common code as possible and put those into widgets.

Any time I find a select() call in my controller, I consider making a new method in my model.

When I write something like this in a controller:

bluebirds = model.Bird.select(Bird.q.color == 'blue')

I usually come back later and put in something like this into the Bird class:

class Bird(SQLObject):
color = UnicodeCol()

@classmethod
def by_color(cls, color)
return cls.select(cls.q.color == color)

Now I have something that I can reuse. If I’m feeling whimsical I’ll use functools.partial to do something like this:

class Bird(SQLObject):
color = UnicodeCol()

def by_color(self, color):
return self.select(self.q.color == color)

redbirds = classmethod(partial(by_color, color='red'))
bluebirds = classmethod(partial(by_color, color='blue'))

Sidenote: I couldn’t figure out how to use the @classmethod decorator in the second version of by_color because partial complained. Appararently, callable(some_class_method) returns False, and partial requires the first argument to be a callable.

Maybe a reader can explain to me what’s going on there…

A few half-formed thoughts on SQLObject

I love SQLObject, but this is a rant about the tiny frustrations I face with it.

First, this is a minor point. I don’t really care about database independence that much. Postgres has a lot of wonderful features: I never have to worry about choosing the table engine that will enforce foreign key constraints, I like creating indexes with function inside:

create unique index nodup_parent on category (org_id, parent_cat, lower(name));

and I really like how easy it is to write stored procedures. Anyway, since I know I’m going to use postgresql, I don’t want to be restricted to only the features that exist or can be emulated in every platform. I know all about sqlmeta and createSQL and use it plenty. But I don’t like how when I set a default value, sometimes it is set in the database table, and other times, it isn’t.

Anyway, in practice, the most dangerous part of using SQLObject is that it hypnotizes you into forgetting about the queries behind everything. Imagine you have employees, departments, and a join table between them. You can set this up in SQLObject like this:

class Employee(SQLobject):
name = UnicodeCol(alternateID=True)
departments = RelatedJoin('Department')

class Department(SQLObject):
name = UnicodeCol(alternateID=True)
employees = RelatedJoin('Employee')

You want to draw a grid that indicates whether each user is a member in every group, so you might dash off some code like this:

for emp in Employee.select():
for d in Department.select():
if d in emp.departments:
print "yes!"
else:
print "no!"

In an ideal scenario, you can do this with three simple queries:

  • You need a list of employees
  • You need a list of departments
  • You need the list of employee-department of associations.

People that talk about how you can use outer joins to cram all that into one query will be dropped into a bottomless pit. Besides, I profiled it, and three separate queries is often much cheaper.

Anyway, back to the point. SQLObject will only run a single query to get the employees and a separate single query to get all the departments. So that’s good.

However, the place where all hell breaks loose is that if clause in the middle. If we have three employees and four departments, this statement

if d in emp.departments:

executes a dozen times. That’s unavoidable. The problem is that each time it executes, SQLObject runs a query like:

select department_id from department_employee where employee_id = (whatever);

Every time you say “is this particular department in this employee’s list of departments?” SQLObject grabs the full list of departments for that employee. So, if you ask about 10 different departments, you will run the exact same query ten times. Sure, the database is likely to cache the results of the query for you, but it is still very wasteful.

With just a few employees and a few departments, that’s not so bad. Eventually, though, as the number of employees and departments grow, the cost of that code grows at N2, which is just geek slang for sucky.

So, in conclusion, this may sound like a rant, but it really isnt. SQLObject is great. But it isn’t magic. It’s a great scaffolding system. But now I find that I’m rewriting a fair portion of code in order to reduce the database costs.

Aside: when I started paying attention to the queries generated by SQLObject, I found it really useful to edit postgresql.conf and enable log_min_duration_statement. Then every query and its cost will be logged for you. This is really useful stuff. It’s helped me to relax about doing a lot of things that I used to think were really bad.

jQuery for the win

I needed to toggle some text between “show help” and “hide help”.

First, I wrote this and thought I was good:


Show HelpHide Help

That flips the text from “Show Help” to “Hide Help”. But the code can be made even shorter by calling toggle on both spans at once:


I heart jQuery.