Skip to content

My favorite software quality metric is the income statement

I’ve been working on an open-source project for the better part of a year. It is very complex, very fancy, and has no users except for me. But I love it. It sharpens my skills and gives me a chance to discover puzzles and solve them in the most elegant solution possible.

But in the business world, this approach is downright silly. Nobody writes blank checks to vendors. Nothing in nature works like this either.

When a shark chases down and eats an unlucky spring breaker, the shark burns some calories during the swim, and then gets some calories back during the meal.

On average, the shark has to extract at least many calories while eating as it burned during the chase.

So on to the income statement. The shark makes a good metaphor for a software business. The hunt is the process of acquiring customers. The calories are the firm’s revenues and expenses. A lot of the quality of a software team can be measured in the income statement.

The perfect product is written once and it solves all problems and never needs any updates or extensions or bug fixes. The worst product has to be rewritten from scratch for every new customer and requires lots of bug fixes. Really well-written software falls somewhere between. It supports extensions, but they are quick and safe to do. Bugs can be quickly patched.

It wasn’t obvious to me initially, but this aspect is easy to measure in the cost of goods sold. If a firm lands a deal for $60k, but burns 160 hours rushing some tweaks through, then those hours drive up the cost of goods sold and drive down the firm’s margins.

You can look at the firm’s margins over time and watch whether the app is getting better or worse. Sure, there are mitigating factors, but in the long run, you can’t maintain an attractive income statement and a shitty code base.

Posted in Programming, startup life, work.

By matt
June 28, 2009

Comments

How to write a resume for an internship

I’ve been on both sides of internships. I’ve puffed up a skinny resume to look more interesting, and I’ve scanned through resumes looking for a candidate. Here’s some tips:

  • Emphasize coursework. Don’t just list your major. Talk about what you’ve done in various courses and what you liked. Describe interesting stuff you studied.
  • Don’t mention your crappy high school jobs. I don’t care if you managed a Taco Bell. In fact, I will hold it against you because Taco Bell is disgusting.
  • Mention any leadership positions in volunteer organizations. Coaching little league, running an Amnesty International chapter, or anything like that all show you know how to interact politely.
  • Help me figure out what the hell to do with you. Do you want to publish an article? Do you want to go on lots of sales calls? Do you want to learn how to program? If you don’t tell me what you want to do, then I have to guess. Or maybe I won’t guess, and I’ll give you a bunch of tedious work instead.

Posted in work.

By matt
June 26, 2009

Comments

Is this pylint error message valid or bogus?

The vast majority of pythoners use None, but I prefer descriptive strings for default values, like this:

from datetime import datetime, timedelta

def midnight_next_day(initial_time="use today's date"):

    if initial_time == "use today's date":
        initial_time = datetime.now()

    return initial_time.date() + timedelta(days=1)

Pylint doesn’t like it though:

$ pylint -e f.py 
No config file found, using default configuration
************* Module f
E: 10:midnight_next_day: Instance of 'str' has no 'date' member (but some types could not be inferred)

I changed from using a string as the default to None, and then pylint
didn’t mind:

$ cat f.py
from datetime import datetime
def midnight_next_day(initial_time=None):

if initial_time is None:
    initial_time = datetime.now()

return initial_time.date() + timedelta(days=1)

$ pylint -e f.py
No config file found, using default configuration

Checking where a variable is None is probably much faster than doing string comparison, but I find the string more informative.

Looking for comments. I already posted this to comp.lang.python here and got some good feedback.

Posted in Programming, Python.

By matt
June 19, 2009

Comments

Which code layout do you like more?

I can lay out the same function in two ways. Which one is better?

One return statement and a mutable temporary variable


def f(a, b):
    temp = g(a)
    if b:
        temp = h(temp)
    return temp

Two return statements and everything uses single assignment


def f(a, b):
    if b:
        return h(g(a)
    else:
        return g(a)

I prefer the second approach. Ever since I learned how prolog requires single assignment, I avoid updating variables. I’m not adamant about it, but in general, I try not to. It’s just a style thing though. I’ve met a lot of programmers (older ones, usually) that like only having one return statement in a function. I don’t know why they like that, but I’ve heard it from numerous people.

Posted in Programming, Python, Uncategorized.

By matt
June 18, 2009

Comments

Don’t negotiate on your estimates

My dad made a remark once that if you want to be good in sales, you can’t just believe that the glass is half full. You have to say with a straight face that a half-full glass is better than a completely full glass.

Anyhow, my shop has generally good relationships between the technical team and the sales team. There’s one thing that drives me nuts though: when we discuss priority and they try to negotiate with me on my estimates.

It goes a little like this:

me: I saw your note about creating a new permission-level between employee and administrator, where you want to let the shift supervisor do admin work on just their direct reports. In other words, two different shift supervisors will see two different lists of employees even though their permissions are identical. I figure that’s an 8-point project.

Billy Mays: No way! You just need an entity code to track what department that supervisor is attached to and then only show the people in that department.

Me: Yeah.

the shamwow guy: I don’t see how that is a big change. It’s just one extra column in the database. Just add that entity code into your queries.

Me: It is straightforward. I’m not confused about how to do it. It’s going to take a long time. There are lot of places that I need to alter to use this new approach.

Ron Popeil: are we using an object-oriented approach in the code? I hear you say that you’re going to have to change a lot of places in the code. At ${A MUCH BIGGER SOFTWARE COMPANY}, we used single objects so that we didn’t have to change a lot to alter behavior.

The minute when I start justifying my estimate or explaining why I have to change a lot of things, I’ve lost. They go into “close the deal” mode. There is a primal instinct at work here that outclasses higher brain functions. If sales people didn’t have a highly evolved chase instinct, they would suck at sales. When they see me trying to explain why it’s going to take that long, they will just nag nag nag until I give in.

And that’s what we need them for — that’s how sales works.

So I have found that the way out of this situation IS NOT to go technical and start explaining how code works. They’re NEVER going to say gee, now that you gave me a quick tutorial in web application design, and then you explained how you’ll need to rewrite the permissions decorators to use a new state factory, I guess your estimate makes sense.

Instead, this approach works OK for me:

Me: I think it is a great idea and it won’t require a lot of research and development, but it’s going to take a while. So based on that estimate, should we put into the queue or not?

I don’t address the remark about OOD and I don’t explain why. From a salesman’s perspective, you don’t close deals by acknowledging why the prospect doesn’t need what you’re selling. You close deals by overcoming objections. The salesman’s natural instinct is to see an obstacle and wear it down. So the key is not to feed that. Instead, stonewall them. Over time, they’ll respect you more because of it.

Here’s the bottom line: sales people are not like us. And that is OK. We need them. And we can work together as long as we all follow some rules. Don’t take away their bowl while they eat and never justify anything.

Posted in Programming, Venture Brothers, work.

By matt
June 7, 2009

Comments

Guest post: 25 things my wife won’t miss about her intern year

My wife wrote this on her facebook page. I loved it.

Awhile back, the “25 Things About Me” meme was pretty popular on Facebook. I’ll admit, I did it- it’s fun to broadcast all these bits and pieces of myself into cyberspace. But it was pretty self-serving. Do any of you really care about my first job, first car, or my most embarrassing moments?

But now there’s something about me that I really do feel like sharing. I’m sounding my barbaric yawp over the roofs of the world.

That’s right, folks. I am no longer an intern.

(Well, technically I am. I still have to go in early tomorrow for bullshit cross cover on 10C. But my months of doing random intern rotations are over the moment I walk out the building tomorrow. I’m spending the last month in radiology; and as far as I’m concerned, it’s a brave new world down there.)

I was an intern on internal medicine, general surgery, the ED, trauma surgery, the PICU, peds urgent care, pathology, and a few other random things like ortho and rheum. I think it’s fair to say I’ve been on almost every service in the hospital. I’ve taken care of neonates in the NICU, and I’ve done autopsies. Seen far too many people die this year, but just one being born (well, I was doing the birthing). Some months were much more emotionally and physically difficult than others. And honestly, there were a lot of fun times. I got lucky and had a pretty nice year overall and met some very cool people. But it still sucked the big one most of the time.

So, fellow interns, soon-to-be interns, people who were interns once, and the curious other few who might peruse this, I give you…

25 Things I Won’t Miss About Intern Year

1. The Goldenrod Form: An ugly yellow piece of paper you have to fill out when patients go to a nursing home. It’s so ugly it hurts my eyes; it’s like pee that sat in a bedpan too long and had half its free water content evaporate off. The space to write meds is horribly small. I’m sure some nurse somewhere couldn’t read my crappy handwriting and gave a patient 400 mg of Lasix qD.

2. Waking up at 4:30 (or earlier, on gen surg) every morning. Falling asleep at 9:30 pm. Falling asleep in the middle of a conversation. Feeling like people in Target/at preschool/my neighbors must think I’m drunk or stoned because my eyes are bloodshot and I can barely get a sentence out.

3. Environmental services staff who buff floors and vacuum rugs right next to us during rounds. You guys do a great job, really, but do you have to literally vacuum under the chair I’m sitting in when I’m post-call and fumbling my way through a presentation?

4. The Trauma Bomb: 6776. I will remember that number forever.

5. Tele alarms. I hear you in my sleep.

6. Watching five fat, breathless hospital cops try to wrestle my demented 78 year-old crackhead patient back from the elevators. Didn’t work. Actually, I might miss this, cause it was so damn funny to watch.

7. Bouncebacks.

8. Short calls.

9. Swallowers.

10. Drunk members of a certain family (well-known to 10C) who threaten me. You know who you are. Next time I’m calling security. I mean it.

11. Kids kept alive, born at 23 weeks, now trached and PEGed and living in an LTAC in between PICU admissions. Sucks.

12. Smelly trachs. It must suck to have a trach, and I don’t mean to make fun of anyone who has one, but sometimes the smell they emit makes me gag. And the stuff that gets sucked out? Forget it. I just threw up.

13. Patients who insist on repeatedly calling me honey, sweetie, lady, or nurse, even after I remind them I’m their doctor. It’s not hard. I call you Mr. Smith. You call me Dr. Wilson. Get it? Apparently not.

14. The ED. I’m sorry, Deb and John, it’s just not for me. No part of it. But please come see me often in the rads reading room. I’ll be the one in the rocker, eating popcorn in the dark with my ipod on.

15. The smell of horrific gingivitis and that weird yellow gunk that accumulates around the teeth of those who don’t floss.

16. Patients who want me to check out “this spider bite on my ass/nuts/labia.”

17. Tripping and dropping 3 pagers down a stairwell, all of which are going off (true story, happened on rainbow surg call one night)

18. Diabetic nails, gnarly toes, scaly skin. Get thee to a podiatrist.

19. The plastic pillows in the call rooms. Covered with microbes due to years of resident sweat and drool accumulating on them.

20. Digital rectal exams. I feel dirty just seeing a package of Surgilube. I apologize to anyone whose behind I violated. I tried to be gentle, really.

21. The Metrohealth Cafe’s weird obsession with fish. Face it, you can’t do a damn thing to make your cod, scrod, tilapia or salmon palatable to me.

22. The smell of poop that permeates 5A/5C SICU. It’s killer.

23. My stethoscope. Like a noose around my neck. Only I don’t wear it around my neck, I keep it in my pocket. Cause I’m cool like that.

24. Calling bullshit consults cause someone else made me do it. Usually to Ortho or Neurosurg. The call is inevitably returned by some snarling dude who talks to me like I’m an idiot.

And finally…

25. Those two dudes who seem to be preying on the unlucky Clevelanders just mindin’ their own business, sittin’ on the porch, or walkin’ home from church. At 3 am.

Actually, I’d like to thank those two dudes. You bring my hospital a lot of business. (Not that we’re getting paid for it, but whatever).

Posted in Uncategorized.

By matt
May 30, 2009

Comments

Puppet vs. Robot

puppet vs. robot from shopcart on Vimeo.

Posted in Uncategorized. Tagged with .

By matt
May 24, 2009

Comments

Some notes on nosetests and coverage

I use Nose to run my automated tests. It can report on code coverage like this:


$ nosetests --quiet --with-coverage --cover-package pitz
Name                           Stmts   Exec  Cover   Missing
------------------------------------------------------------
pitz                              29     29   100%   
pitz.bag                         108    107    99%   150
pitz.cmdline                      50     12    24%   23-54, 62-79, 92-93, 96, 109-114, 119-130
pitz.entity                      105    105   100%   
pitz.exceptions                    1      1   100%   
pitz.junkyard                      0      0   100%   
pitz.junkyard.ditzloader          22     15    68%   31-37, 45-47
pitz.pitzfiles                     0      0   100%   
pitz.project                      52     52   100%   
pitz.projecttypes                  0      0   100%   
pitz.projecttypes.agilepitz       55     54    98%   66
pitz.projecttypes.simplepitz      66     61    92%   84-90
------------------------------------------------------------
TOTAL                            488    436    89%   
----------------------------------------------------------------------
Ran 69 tests in 6.350s

OK (SKIP=6)

Most of the uncovered code is in the cmdline module, which does a lot of work with the filesystem and starting up IPython, and I’m having trouble writing tests there. You could help, you know :) .

I’m keenly aware that running with coverage makes tests much slower. Normally, my pitz tests run in about half a second:


$ nosetests --quiet
----------------------------------------------------------------------
Ran 69 tests in 0.504s

OK (SKIP=6)

Fortunately, I don”t have to rerun all your tests to see what lines are uncovered. I can query the .coverage file created by nose afterward to get details. It is really easy to get coverage for just one module:


$ coverage -r -m pitz/cmdline.py
Name           Stmts   Exec  Cover   Missing
--------------------------------------------
pitz/cmdline      50     12    24%   23-54, 62-79, 92-93, 96, 109-114, 119-130

And getting coverage on multiple modules is straightforward, but kind of tedious:

$ coverage -r -m pitz/cmdline.py pitz/__init__.py pitz/entity.py
Name            Stmts   Exec  Cover   Missing
---------------------------------------------
pitz/__init__      29     29   100%   
pitz/cmdline       50     12    24%   23-54, 62-79, 92-93, 96, 109-114, 119-130
pitz/entity       105    105   100%   
---------------------------------------------
TOTAL             184    146    79%  

I don’t know of an elegant solution to do what nosetests does, where it shows me all coverage for a package. Running coverage -r without any module lists dumps out everything, which is never what I want:

$ coverage -r
Name                                                                                                               Stmts   Exec  Cover
--------------------------------------------------------------------------------------------------------------------------------------
/home/matt/checkouts/myfiles/bin/__init__                                                                              0      0   100%
/home/matt/virtualenvs/pitz/lib/python2.6/site-packages/Jinja2-2.1.1-py2.6-linux-i686.egg/jinja2/__init__             12     11    91%
/home/matt/virtualenvs/pitz/lib/python2.6/site-packages/Jinja2-2.1.1-py2.6-linux-i686.egg/jinja2/bccache             111     36    32%
Traceback (most recent call last):
KeyboardInterrupt

A while back on twitter I posted that I wanted my editor to read my .coverage file and highlight the lines that aren’t covered. I don’t know anything about how to control syntax highlighting, but I run this little deal from within vim to get the same facts:


:! coverage -r -m %

My edit buffer disappears and I see this instead:

Name      Stmts   Exec  Cover   Missing
---------------------------------------
cmdline      50     12    24%   23-54, 62-79, 92-93, 96, 109-114, 119-130

Press ENTER or type command to continue

In vim, :! is how you run a command in a shell. Incidentally, it is also possible to pull the results from the command into vim and write data from vim to the command, but that’s not what I want right now.

Vim replaces the % symbol with the file I’m currently editing. Of course, this command only works when I start vim in the same directory as the .coverage file. If I ain’t in that directory, then I have to specify how to find the .coverage file by setting the environmental variable COVERAGE_FILE like this:


:! COVERAGE_FILE=../.coverage coverage -r -m %

Setting it that way means it doesn’t last beyond that one shell. If I want vim to remember the setting, I could set COVERAGE_FILE when I start vim like this:

$ COVERAGE_FILE=../.coverage vi cmdline.py

Or I could export it like this:

$ export COVERAGE_FILE=../.coverage
$ vi cmdline.py

In summary, coverage is a neat tool, but it is silly to think that 100% test coverage guarantees anything. Coverage won’t warn you when your calculator returns 3 for 1 + 1.

Posted in Programming, Python.

By matt
May 13, 2009

Comments

Need help with data files and setup.py

I’m working on a package that includes some files that are meant to be copied and edited by people using the package.

My project is named “pitz” and it is a bugtracker. Instead of using a config file to set the options for a project, I want to use python files.

When somebody installs pitz, I want to save some .py files somewhere so that when they run my pitz-setup script, I can go find those .py files and copy them into their working directory.

I have two questions:

  1. Do I need to write my setup.py file to specify that the .py files in particular directory need to be treated like data, not code? For example, I don’t want the installer to hide those files inside an egg.
  2. How can I find those .py files later and copy them?

Here’s my setup.py so far:


from setuptools import setup, find_packages
version = '0.1'
setup(name='pitz',
    version=version,
    description="Python to-do tracker inspired by ditz (ditz.rubyforge.org)",

    long_description="""
ditz (http://ditz.rubyforge.org) is the best distributed ticketing
system that I know of.  There's a few things I want to change, so I
started pitz.""",

    classifiers=[],
    keywords='ditz',
    author='Matt Wilson',
    author_email='matt@tplus1.com',
    url='http://tplus1.com',
    license='',
    packages=find_packages(exclude=['ez_setup', 'examples', 'tests']),

    include_package_data=True,
    package_dir={'pitz':'pitz'},

    data_files=[('share/pitz',
        [   
            'pitz/pitztypes/agilepitz.py.sample',
            'pitz/pitztypes/tracpitz.py.sample',
        ])],

    zip_safe=False,
    install_requires=[
        # 'PyYAML',
        # 'sphinx',
        # 'nose',
        # 'jinja2',
          # -*- Extra requirements: -*-
    ],

    # I know about the much fancier entry points, but I prefer this
    # solution.  Why does everything have to be zany?
    scripts = ['scripts/pitz-shell'],

    test_suite = 'nose.collector',
)

When I run python setup.py install, I do get those .sample files copied, but they get copied into a folder way inside of my pitz install:


$ cd ~/virtualenvs/scratch/lib/
$ find -type f -name '*.sample'
./python2.6/site-packages/pitz-0.1dev-py2.6.egg/share/pitz/tracpitz.py.sample
./python2.6/site-packages/pitz-0.1dev-py2.6.egg/share/pitz/agilepitz.py.sample

I don’t know how I can write a script to copy those tracpitz.py.sample files out. Maybe I can ask pitz what its version is, and then build a tring and use os.path.join, but that doesn’t look like any fun at all.

So, what should I do instead?

Posted in Programming, Python, TurboGears, mlp, pitz.

By matt
April 21, 2009

Comments

I submitted my proposal for PyOhio 2009

Here it is: Clever uses for metaclasses

SUMMARY

This talk introduces metaclasses and attempts to “defang” them, showing what they are good for, and when they are silly.

Metaclasses can reduce redundancy in code but can be very confusing, so in this talk, I will walk through several examples of how to use metaclasses to solve problems.

I plan to cover (at least) these examples:

  • Add extra methods and attributes to a class after its definition.
  • Verify a class correctly implements a specification.
  • Give a subclass its own class-variables rather than sharing them with the parent class.
  • Implement the basics of an object-relational mapper (ORM).

I published an article in the November 2008 issue of Python Magazine with the same title (Clever Uses for Metaclasses) and I’ll use some code from that article.

I want to give this talk in a friendly, informal manner, so that people that feel intimidated by metaclasses realize that there’s nothing to be scared of.

EXPERTISE LEVEL

This talk is aimed at the intermediate-level programmer, already familiar with object-oriented concepts and is really comfortable with Python.

I do not expect people in the audience to know ANYTHING about metaclasses before this talk.

AREAS OF PYTHON

  • object-oriented programming
  • metaprogramming
  • dynamic language tomfoolery
  • introspection

Posted in Programming, Python.

By matt
April 20, 2009

Comments