Mark Ramm makes a lot of interesting points about Django in this talk. Really good stuff.
Category Archives: Programming
Now I have a reason to use staticmethod
The python builtin staticmethod has been one of those language features that I understood, but couldn’t figure out why I would ever use it. When I did my PyOhio talk on decorators, I asked if anyone in the room could explain why to use it. People came up with these ideas:
- Put associated functions inside the relevant class, so we don’t pollute the module namespace
- Make java programmers feel less homesick
Both of these are valid, I guess, but in my mind, it just confirmed that I would never use staticmethod.
Then I watched this neat video. You don’t have to go watch that video before you read the next section. I just wanted to point to where I picked up this trick.
Remember that any function bound to a class gets called with an extra parameter inserted at the beginning. That parameter is a reference to the instance of the class. So, you can’t do this:
>>> def f(a, b):
... return a - b
...
>>> class C(object):
... pass
...
>>> C.f = f
>>> c = C()
>>> c.f(3, 1)
------------------------------------------------------------
Traceback (most recent call last):
File "
TypeError: f() takes exactly 2 arguments (3 given)
The call to f blew up because f only takes two parameters, but it got three parameters instead because python automatically adds on the self parameter at the beginning. So f really got called like this: f(self, 3, 1)
Now the reason for staticmethod becomes obvious. If I want to allow instances of my class C to call function f, I have to make f a staticmethod on C, so that the python plumbing won’t insert that extra first argument at the beginning.
So, I’ll overwrite the first C.f method with a static method, and then call c.f again:
>>> C.f = staticmethod(f)
>>> c.f(3, 1) # This is the same already-instantiated c from earlier.
2
As a side effect of watching that video, I think about dependency injection (DI) differently now. In the video, Thomas Woulters says he uses staticmethod for dependency injection. This kind of DI is a different approach than what I’ve seen before. Most everybody that talks about DI emphasizes passing in lots and lots of crap into the __init__ method or in any subsequent methods, so every component can be mocked out.
But it makes just as much sense to graft on dependencies to the class outside of the __init__ method too, since the language supports that. The end goal is still reached. Code can still tweak the dependencies and provide alternative objects, and at the same time, I don’t end up with ridiculously long function signatures.
I need a list of skills that you wish designers had
I’m team-teaching a course at Tri-C in the visual communications department this Fall. The class is for graphic designers, and we go through the experience of meeting with a client, building a prototype web site, revising it, then releasing it on the world, then going back and fixing any post-release issues.
The students all have excellent graphic design skills, but nothing in previous courses covers anything programming-related. The other instructor has the graphic design chops, and that’s really the meat of the course, but they do want to punch up the level of instruction beyond building static HTML into building simple web apps.
I need a short list of skills you wish all graphic designers have. Here’s what I plan to cover so far:
- How to use subversion. Probably we’ll use google code.
- How to use tools like Fiddler and Firebug to look at page weight and watch the requests go back and forth.
- How to check for valid HTML.
- How to do some simple AJAX (by hand at first and then later with jQuery).
- What’s the terminal in OS X for?
- HTTP gets versus posts and what the heck does caching mean?
- PHP fundamentals.
I think I’m going to use PHP rather than anything else, but I’d like to hear arguments against that.
Also, the class lab has a bunch of expensive Mac machines, so if there are really good tools out there, I’d love to hear about them.
How to style an HTML link like a button?
I have some HTML links and I want to display them as buttons. I spent a few hours appying CSS styles, but never got something that appears identical to a button.
So this is what I’m doing now. It is obvious to read, and seems to work.
The W3 validator approved it in HTML 4 strict. So, all signs suggest that this is the way to go, but I don’t follow the semantic web conversation very closely.
So, for those of you that do pay attention to all that stuff, what’s bad about this approach?
Toward a horrifying new workflow system
Offline version control is nice. Reviewing logs, viewing diffs, and applying merges is dramatically faster when everything is local.
It would be nice to be able to use my ticketing system offline also. I don’t like the context switch of leaving my editor to go over to my web-based ticket system when I want to make a note of something to do later.
Completely unrelated to offline access is the idea that most of the time, my ticketing system and my source control systems are barely aware of eachother.
For example, when I write a “fix this low-priority bug” ticket, it isn’t obvious what revision and branch of my code I’m talking about. Sure, I can add that information manually, and maybe my ticket system will even require me to do that, but I can very easily put incorrect information in there.
I’m thinking about starting an open-source project to make this happen. Please leave a comment with your thoughts (if you have any).
Goals
- Allow offline access to reading and writing tickets.
- Unify source control with ticketing so that tasks, requested features, and bug reports are linked with the relevant code.
What fields are in a ticket?
I’ve used bugzilla, various homemade ticketing systems, trac, and redmine. They all have at least these fields in every ticket:
- Globally unique ID
- Title
- Description
- Comments
- Deadline
- Status
- Priority
- Assigned To
A decent ticketing system usually also supports category keywords, lists of interested people that want to be included in discussions, links to external files, estimated time cost, etc.
Also, a decent ticketing system has lots of nice aggregate views of tickets.
I don’t want to talk about building a *decent* distributed system. I’ll settle for a toy one that just barely works.
Text file approach
Put each ticket in a separate file. Name the files like “make-login-box-green.txt”. Use some markup to divide the file into fields.
Append comments and follow-up notes to the end of the ticket.
This is easy and intuitive. I can use vim to write the tickets. The source control system can track changes in the text file itself to notice changes. So, the source control system reveals if I renamed the title.
The disadvantage is that doing diffs on different versions of a text file just notices text differences.
For example, if I rename a ticket’s title from one line to two lines, and then also change the deadline, all I can see in my report of what changes is the information about what characters on what lines are different.
Local client approach
Use something like the text file approach, with a few bells and whistles.
Instead of just firing up vim/emacs/notepad/textmate and starting typing, I want to use a simple local app like this:
$ cd ~/projects/myproj1/trunk/tickets
$ dist-ticket new --title "Change color of login buttons from green to pink"
Then the app would open $EDITOR with a new file that includes the template of a new ticket, populated with the command-line options passed in.
This local client could create the new file with a file name that includes a UUID function to generate globally unique file names.
The app would also be able to compare diffs between revisions of tickets, like this:
$ dist-ticket diff -r 1:2 ticket-123.txt
Title renamed from "Change color of login buttons from green to pink"
to "Change login buttons style"
Deadline extended by six months.
Changed priority from High to Low.
Of course, the underlying source control system would still be able to show the text differences between the two files. But the dist-ticket script would apply a little bit more intelligence.
Stuff I’m pretty sure about
I want this thing to be agnostic to the underlying source control system. People using centralized source control should still be able to use it and people using some homemade crap should all be able to use it.
Initially it might not be possible to have dist-ticket go and talk to the VCS and get revisions. Instead of the dist-ticket diff thing above, we might need to do this at first:
$ bzr cat -r 1 ticket-123.txt > /tmp/rev1.txt
$ bzr cat -r 2 ticket-123.txt > /tmp/rev2.txt
$ dist-ticket diff /tmp/rev1.txt /tmp/rev2.txt
So after the interface is defined, I can build the implementations for the VCS that I care about. The first two lines could be built into dist-ticket.
Associating tickets, branches, and revisions
I know I want to link my source control system somehow to my ticketing system. I see a couple of ways to do it:
- Underneath each branch, make a top-level directory called tickets. Then tickets would be seen as related to that branch. One ticket would be linked to exactly one branch.
OR
- Keep tickets in some place completely outside the branches directory.
Make some other intermediate data structure that holds a many-to-many relationsip between tickets and branches.
I like the second approach a lot more than the first approach. I would love to be able to look backward and see how I started working on feature X95321 as of revision 123, and then marked it complete as of revision 148. It would be nice to easily see which files are relevant and which ones are not relevant.
Maybe I would need to hack the VCS commit process somehow track relevant tickets per each commit. Maybe I could just store extra text in the comment part of the commit.
I finally have a project that justifies learning prolog
Sometimes when I’m feeling batty,I’ll put the Three Laws of Robotics in my source code. Usually, this is an veiled insult aimed at myself; the comment is my script is gotten so morbidly complex that it threatens to wake up and kill me.
On a completely unrelated note, I’ve been picking at the edges of prolog for the last couple of years. I’ve worked my way through a free Prolog textbook, and now I’m very slowly working my way through Language, Proof and Logic in order to learn me some predicate calculus.
Now I thought of a project that combines these two. I’m gonna build daydream about defining an ontology suitable for making robots comply with those three laws of robotics. Once I finished, it would work like this:
> Eat baby
Violates law #1!
> Mop floor
OK
> Burn down abandoned house
OK
You get the idea.
PyOhio was a smashing success
The Columbus Metro Library offered a fantastic location for us. Wireless internet, multiple meeting rooms, one room with about 30 workstations, etc. Really great location. A $15 donation makes you a “friend” of the library, and gets you a 15% discount at the coffee booth.
Catherine Devlin led the charge of organizing this conference, and she did it amazingly well.
The slides from my decorator talk are available here. I’ll be breaking them down into a series of blog posts with a lot more commentary, so stay tuned.
Notes from clerb meeting on Thursday, July 17th
DimpleDough provided a great location for this month’s Cleveland Ruby Users Group and they even shelled out for dinner. We heard a really good talk about about ruby and F#.
The ruby material covered some neat corners of the language like the method_missing method, which operates like python’s getattr. Here’s a toy example of how it can be used:
irb(main):005:0> class C
irb(main):006:1> def foo
irb(main):007:2> 1
irb(main):008:2> end
irb(main):009:1> end
=> nil
irb(main):010:0> c = C.new
=> #
irb(main):011:0> c.foo
=> 1
irb(main):012:0> class C
irb(main):013:1> def method_missing(m, *args)
irb(main):014:2> puts "you tried to call a method #{m}"
irb(main):015:2> end
irb(main):016:1> end
=> nil
irb(main):017:0> c.baz
you tried to call a method baz
=> nil
irb(main):018:0>
Incidentally, note how I added a new method to class C after I originally defined it. That’s a cute trick in ruby. I can imagine a lot of nasty misuses of that, but I think the “we’re all consenting adults” rule should apply. And when a class has dozens of methods, it might be helpful to divide them across different files.
We talked about currying as well, in the context of F#. I tend to use currying in this scenario:
- I recognize that two separate functions could be refactored to be a single function with a whole bunch more parameters;
- I remake the original functions as curried versions of the new super function.
In other words, if I already have two methods, like paint_it_red(it) and paint_it_green(it), it’s trivial to realize I could write a paint_it_some_color(it, color) and then replace the original paint_it_red with a curried version.
I found this really useful when it isn’t just a single parameter I’m fixing to a constant value, but maybe a whole bunch.
Apparently, Ruby will add currying support in 1.9. I tried to see if I could “fake it” in the irb interpreter, but I just made a mess:
irb(main):036:0> def f(a, b)
irb(main):037:1> a + b
irb(main):038:1> end
=> nil
Nothing interesting so far. f adds its two parameters. So now I’m going to try to make a new function that returns a version of function f with the first parameter a set to fixed value:
irb(main):046:0> def curried_f(a)
irb(main):047:1> def g(b)
irb(main):048:2> a+b
irb(main):049:2> end
irb(main):050:1> return g
irb(main):051:1> end
irb(main):053:0> curried_f(1)
ArgumentError: wrong number of arguments (0 for 1)
from (irb):50:in `g’
from (irb):50:in `curried_f’
from (irb):53
from :0
The problem (I think) stems from how in Ruby, if I just type the name of the function, the function gets called. So in line 50, when I’m trying to return a reference to the new function I just created, Ruby evaluates the result of calling g without giving it any parameters.
I bet I’m doing something very un-ruby-tastic with this approach. I’m probably supposed to leverage those anonymous blocks instead.
F# looks really interesting. It supports all those weird prolog/erlang/haskell-style features like single assignment, pattern matching, and optimal tail-call recursion, with the benefit of having access to the .NET libraries as well.
One of the best professors I studied under made a remark that in COBOL, you think for five minutes and then type for two hours, but in prolog, you think for two hours, and then type for five minutes. I agree. I have learned a lot of languages, but I haven’t gotten any smarter. I’m just learning how to map my thoughts into notation much more quickly.
I would love to have the time and reason to do a project with F#. I think I’ll start by installing mono and messing around.
I need to write faster tests
This is not ideal:
----------------------------------------------------------------------
Ran 84 tests in 370.741s
OK
My tests take so long for two reasons. First of all, most of them use twill to simulate a browser walking through a version of the web app running on localhost. Second, my test code reads like a novel. Here’s an example, slightly embellished to make a point:
setup: connect to the database and find or create a hospital and an employee named “Nurse Ratched.” Find or create a bunch of open shifts in the emergency department. Find or create another nurse named Lunchlady Doris*.
test: Nurse Ratched wants to see what shifts are available to be picked up. So she logs into the app. Then she navigates to the “open shifts” screen, and then filters down to shifts in the emergency department over the next seven days. Then she wants to sign up for the shift starting at midnight on Saturday night. So, she clicks the “sign up” icon. The system verifies that this shift + her already-scheduled hours won’t push her into overtime, and she has no other flags on her account, so she is automatically scheduled.
Then the system sends her a confirmation message, which according to her preferences, is sent to her email address. Then the system queues an SMS message to be delivered an hour before the shift starts in order to remind her (also according to her preferences).
Finally, the test verifies that the shift is now not listed as available by simulating Lunchlady Doris logging in and checking that same “open shifts” screen.
If everything checks out, print a dot, and move on to the next chapter.
teardown: Unassign Nurse Ratched from the shift she picked up.
I think twill in itself is fine. Marching through a series of pages is problematic. I do this to set up conditions for testing later on. As a side benefit, I verify everything checks out along the way.
On the plus side, I’m confident that the integration of all these components do in fact play nice together. I don’t think it’s safe to abandon end-to-end testing like this, but I would like not to depend it every time I want to make some slight change to a component. It would be nice to run these right before a commit, but only run some super-fast tests after each save.
[*]People that understand this reference should reevaluate their priorities in life. back
I heart Python doctests
I wrote the doctests for the function below and then wrote the code to satisfy them in a total of about 30 seconds. As an extra plus, these doctests immediately clarify behavior in corner cases.
def has_no(s):
"""
Return False if string s doesn't have the word 'no' inside.
>>> has_no('no problem')
True
>>> has_no('not really')
False
>>> has_no('no')
True
>>> has_no('oh nothing')
False
"""
if s.lower() == 'no': return True
if s.lower().startswith('no '): return True
if s.lower().endswith(' no'): return True
if ' no ' in s.lower(): return True
return False
Writing tests in any other testing framework would have taken me much longer. Compared to writing these tests with nose, writing this:
assert not has_no('oh nothing')
wouldn’t take me any more time than
>>> has_no('oh nothing')
False
But that’s not all there is to it. With nose, I’d need to open a new test_blah.py file, then import my original blah.py module, then I would have to decide between putting each assert in a separate test function or just writing a single function with all my asserts.
That’s how a 30-second task turns into a 5-minute task.
Anyhow, I’m surprised doctests don’t get a lot more attention. They’re beautiful. Adding tests to an existing code base couldn’t be any simpler. Just load functions into an interpreter and then play around with it (ipython has a %doctest_mode, by the way).
For a lot of simple functions (like the one above) it is easy to just write out the expected results manually rather than record from a session.
It is also possible to store doctests in external text files. The Django developers use this trick frequently.
Finally, I don’t try to solve every testing problem with doctests. I avoid doctests when I need elaborate test fixtures or mock objects. Most of my modules have a mix of functions with doctests and nose tests somewhere else to exercise the weird or composite stuff.
Incidentally, this post is where Tim Peters introduced the doctests module.