Category Archives: work

A few web app ideas

I read somewhere that having a good idea for a business is like finding your sneakers before running a marathon. I’m convinced that focusing on coming up with an awesome business idea is a waste of time. In other words, it isn’t the quality of the idea that separates successful entrepeneurs from all the failures out there. If you can learn to program, you’re smart enough to come up with an idea for a web application that is good enough. The hard part is building the prototype, selling it, training your customers, adding new employees, etc.

Anyhow, since I don’t believe ideas are all that precious, I’m sharing a few side projects I’m thinking about. Feel free to take them. In fact, I’d be flattered.

Just to be clear, these ideas are released into the public domain. When I have some free time I may do a few of these.

Idea 1: make something that helps teachers keep track of grades over the semester

I don’t mean some elaborate tool like blackboard. I mean something that tracks attendance and grades. And maybe a few other things, but not much more. I’ve been teaching for about four semesters now, and it’s a big hassle to keep track of this mundane stuff.

Idea 2: Graphical cash flow / burn rate projection tool

At work, I have to deal with scenarios like paying an extra 10% for some outsourcers to finish a product release that then might lead to increased sales. I want to be able quickly see effects of lots different possible rates of return.

At home, I wonder about stuff like paying an extra $100 toward the mortgage versus putting the same money to the car note.

I want to be able to enter in stuff like recurring payments, starting balances, interest rates, etc, and then see some pretty charts that show the results.

Last idea: workout tracker

I have a hard time remembering workouts. I know I could carry around a notebook, but that’s no fun. I would prefer that after each set, or when I want to take a break, I could send in a text from my phone to some shortcode. The message would have some details of part of my workout, maybe using some shorthand like:

dbp 12 120

meaning that I did twelve reps of dumbbell bench presses with one hundred and twenty pounds.

Then, at the next workout session, I could send in a text message and the system would figure out what I should do.

The system might adjust up weights by 3% for example, or make sure I vary exercises.

There could very easily be some facebook extension that brags to my friends with the details of my workout.

My favorite software quality metric is the income statement

I’ve been working on an open-source project for the better part of a year. It is very complex, very fancy, and has no users except for me. But I love it. It sharpens my skills and gives me a chance to discover puzzles and solve them in the most elegant solution possible.

But in the business world, this approach is downright silly. Nobody writes blank checks to vendors. Nothing in nature works like this either.

When a shark chases down and eats an unlucky spring breaker, the shark burns some calories during the swim, and then gets some calories back during the meal.

On average, the shark has to extract at least many calories while eating as it burned during the chase.

So on to the income statement. The shark makes a good metaphor for a software business. The hunt is the process of acquiring customers. The calories are the firm’s revenues and expenses. A lot of the quality of a software team can be measured in the income statement.

The perfect product is written once and it solves all problems and never needs any updates or extensions or bug fixes. The worst product has to be rewritten from scratch for every new customer and requires lots of bug fixes. Really well-written software falls somewhere between. It supports extensions, but they are quick and safe to do. Bugs can be quickly patched.

It wasn’t obvious to me initially, but this aspect is easy to measure in the cost of goods sold. If a firm lands a deal for $60k, but burns 160 hours rushing some tweaks through, then those hours drive up the cost of goods sold and drive down the firm’s margins.

You can look at the firm’s margins over time and watch whether the app is getting better or worse. Sure, there are mitigating factors, but in the long run, you can’t maintain an attractive income statement and a shitty code base.

How to write a resume for an internship

I’ve been on both sides of internships. I’ve puffed up a skinny resume to look more interesting, and I’ve scanned through resumes looking for a candidate. Here’s some tips:

  • Emphasize coursework. Don’t just list your major. Talk about what you’ve done in various courses and what you liked. Describe interesting stuff you studied.
  • Don’t mention your crappy high school jobs. I don’t care if you managed a Taco Bell. In fact, I will hold it against you because Taco Bell is disgusting.
  • Mention any leadership positions in volunteer organizations. Coaching little league, running an Amnesty International chapter, or anything like that all show you know how to interact politely.
  • Help me figure out what the hell to do with you. Do you want to publish an article? Do you want to go on lots of sales calls? Do you want to learn how to program? If you don’t tell me what you want to do, then I have to guess. Or maybe I won’t guess, and I’ll give you a bunch of tedious work instead.

Don’t negotiate on your estimates

My dad made a remark once that if you want to be good in sales, you can’t just believe that the glass is half full. You have to say with a straight face that a half-full glass is better than a completely full glass.

Anyhow, my shop has generally good relationships between the technical team and the sales team. There’s one thing that drives me nuts though: when we discuss priority and they try to negotiate with me on my estimates.

It goes a little like this:

me: I saw your note about creating a new permission-level between employee and administrator, where you want to let the shift supervisor do admin work on just their direct reports. In other words, two different shift supervisors will see two different lists of employees even though their permissions are identical. I figure that’s an 8-point project.

Billy Mays: No way! You just need an entity code to track what department that supervisor is attached to and then only show the people in that department.

Me: Yeah.

the shamwow guy: I don’t see how that is a big change. It’s just one extra column in the database. Just add that entity code into your queries.

Me: It is straightforward. I’m not confused about how to do it. It’s going to take a long time. There are lot of places that I need to alter to use this new approach.

Ron Popeil: are we using an object-oriented approach in the code? I hear you say that you’re going to have to change a lot of places in the code. At ${A MUCH BIGGER SOFTWARE COMPANY}, we used single objects so that we didn’t have to change a lot to alter behavior.

The minute when I start justifying my estimate or explaining why I have to change a lot of things, I’ve lost. They go into “close the deal” mode. There is a primal instinct at work here that outclasses higher brain functions. If sales people didn’t have a highly evolved chase instinct, they would suck at sales. When they see me trying to explain why it’s going to take that long, they will just nag nag nag until I give in.

And that’s what we need them for — that’s how sales works.

So I have found that the way out of this situation IS NOT to go technical and start explaining how code works. They’re NEVER going to say gee, now that you gave me a quick tutorial in web application design, and then you explained how you’ll need to rewrite the permissions decorators to use a new state factory, I guess your estimate makes sense.

Instead, this approach works OK for me:

Me: I think it is a great idea and it won’t require a lot of research and development, but it’s going to take a while. So based on that estimate, should we put into the queue or not?

I don’t address the remark about OOD and I don’t explain why. From a salesman’s perspective, you don’t close deals by acknowledging why the prospect doesn’t need what you’re selling. You close deals by overcoming objections. The salesman’s natural instinct is to see an obstacle and wear it down. So the key is not to feed that. Instead, stonewall them. Over time, they’ll respect you more because of it.

Here’s the bottom line: sales people are not like us. And that is OK. We need them. And we can work together as long as we all follow some rules. Don’t take away their bowl while they eat and never justify anything.

My thoughts on roles in software teams

My biz is growing. We’re going from a state where I did everything all by myself to a “real” company. I’ve been working on a mental model to define what developers do.

I’ve boiled it down to three different kinds of work: engineering, consulting, and management. In any real shop, individuals have to be able to wear all three hats depending on circumstances.

Engineering

A great engineer builds real stuff out of blueprints.

The keyword for engineers is execution. They’re the ones that actually get shit done, and do it brilliantly.

Bad engineers

An engineer that doesn’t love building things is not going to be a good engineer. A bad engineer hacks shit together then hides from peer review.

Good engineers

A good engineer spends her free time reading manuals about obscure technologies. A good engineer gets physically ill when seeing bad design. A good engineer builds beautiful stuff and obsesses about skill mastery.

When a new problem comes up, a good engineer builds a tool that can be used to solve other problems.

Consulting

This is the task of figuring out the next big thing. This takes a lot more work than just talking to customers and writing down what they say.

Good consultants

A great consultant is a lot like an investigative journalist. They know how to keep asking questions until they know exactly what really needs to be built. Then they write down what they figured out from all those questions in plain language, so the engineer can start working.

Good consultants go beyond transcribing what the customer said and figure out what the customer really needs. Great consultants are able to synthesize product ideas from unordered, generalized customer complaints.

Managers

Good engineers and good consultants free the manager from doing anything but planning for sustainable growth. A manager’s single responsibility is to keep the system running over the long run. Managers analyze performance problems and figure out what to do; usually the decision is a choice between training somebody or firing that somebody.

A manager makes sure that her team can handle a loss of a few engineers or consultants as people move around, get new jobs, get hit by buses, or win the lottery.

Good managers

A good manager is really the mom of the organization. She makes sure everyone is happy, working on interesting projects, and generally, reaching their potential. Good managers make sure that as people inevitably leave the team, the team can still function at high capacity.

Ditz versus bugs everywhere

A few months ago, I sketched out a ticket-tracking system that would be married with my source code. Then some commenters told me about bugs everywhere (be) and ditz.

I’ve looked at both, but I’ve been using ditz full-time while just watching be. Anyhow, here’s a few comparisons:

setting up a project

Here’s what it looks like when you set up a project in ditz:


$ ditz init
I wasn't able to find a configuration file ./.ditz-config.
We'll set it up right now.
Your name (enter for Matthew Wilson): 
Your email address (enter for matt@sprout.tplus1.com): 
Directory to store issues state in (enter for .ditz): 
Use your text editor for multi-line input when possible (y/n)? y
Paginate output (always/never/auto)? auto
Project name (enter for scratch): 
Issues can be tracked across the project as a whole, or the project can be
split into components, and issues tracked separately for each component.
Track issues separately for different components? (y/n): y

Current components:
None!

(A)dd component, (r)emove component, or (d)one: a
Component name: documentation

... snip ...

(A)dd component, (r)emove component, or (d)one: d
Ok, .ditz directory created successfully.

And here’s how you can create a single issue.

$ ditz add
Title: Write something justifying yet another web framework
Is this a (b)ugfix, a (f)eature, or a (t)ask? t
Choose a component:
  1) scratch
  2) documentation
  3) model code
  4) controller code
  5) view code
Component (1--5): 2
Issue creator (enter for Matthew Wilson <matt@sprout.tplus1.com>): 
Added issue documentation-1 (e8a4a43f78ee83300cc0372a13375d9534b97abb).

You can’t tell, but when I punched in the title, ditz opened my $EDITOR and I wrote a longer description in there.

Now the same thing in be:


$ be set-root
Guessing id 'matt <matt@sprout>'
No revision control detected.
Directory initialized.

$ be new 'Write something justifying yet another web framework'
Guessing id 'matt <matt@sprout>'
Guessing id 'matt <matt@sprout>'
Created bug with ID 4d4

Not quite the same experience!

Here’s what a ditz issue looks like:


$ ditz show documentation-1
Issue documentation-1
---------------------
      Title: Write something justifying yet another web framework
Description: Why not just polish any of the ones already out there?
       Type: task
     Status: unstarted
    Creator: Matthew Wilson <matt@sprout.tplus1.com>
        Age: four minutes
    Release: 
 References: 
 Identifier: e8a4a43f78ee83300cc0372a13375d9534b97abb

Event log:
- created (matt, four minutes ago)

And in be:


$ be show 4d4
Guessing id 'matt <matt@sprout>'
          ID : 4d4e6a17-2097-42bb-a3cd-3c17566ecce8
  Short name : 4d4
    Severity : minor
      Status : open
    Assigned : 
      Target : 
     Creator : matt <matt@sprout>
     Created : Mon, 22 Dec 2008 20:25 (Tue, 23 Dec 2008 01:25:04 +0000)
Write something justifying yet another web framework

Ditz issues have titles, long descriptions, types (feature, bugfix, or task), releases (optionally) and links to components (also optionally). There are ditz plugins to add support for assigning issues to people.

be has most of the same concepts, just with different names.

data serialization and storage

ditz makes a .ditz directory at the top of a project and be makes a .be directory in the top of the project.

Inside the .ditz folder, there’s one project.yaml file that lists releases (groupings of issues) and components (also groupings of issues, but cross-cutting). Then each issue lives in its own yaml file, and they look like this:


$ cat .ditz/issue-ac3177b3bf8c6757625977ef27279c1fe05df662.yaml 
--- !ditz.rubyforge.org,2008-03-06/issue 
title: Write some "WHY?" documentation
desc: Justify the existence of this project.
type: :task
component: documentation
release: 
reporter: Matthew Wilson <matt@sprout.tplus1.com>
status: :unstarted
disposition: 
creation_time: 2008-12-23 00:59:05.840956 Z
references: []

id: ac3177b3bf8c6757625977ef27279c1fe05df662
log_events: 
- - 2008-12-23 00:59:05.841349 Z
  - Matthew Wilson <matt@sprout.tplus1.com>
  - created
  - ""
- - 2008-12-23 01:08:58.605955 Z
  - Matthew Wilson <matt@sprout.tplus1.com>
  - commented
  - |-
    Yeah, if you're gonna build another web framework, this needs to be
    really good.

Meanwhile, be is fairly similar, but bugs get whole directories to themselves. be uses what seems to be a home-made plain-text format for storing bugs:

$ cat .be/bugs/4da8ee85-9353-4a92-a654-8510bb8be0d0/values 

creator=matt <matt@sprout>

severity=minor

status=open

summary=Write some "WHY?" documentation

time=Tue, 23 Dec 2008 01:12:13 +0000

There’s actually much more whitespace than that. I replaced the eight blank lines between each line of text with just two blank lines.

While ditz stores the comments inside the issue’s yaml file, be makes a directory under the issue’s directory, and then stores the text of the comments in one file and the information about who said it in a separate file.

The community

The ditz mailing list is really active with people debating ideas for new features. The be mailing list is now showing some signs of life after looking dead in August.

What ditz has that be lacks

ditz can make really pretty HTML pages for all the issues for a project. example.

yaml was a really good choice. yaml makes it easy to deserialize to higher objects than just crappy boring primitive types like arrays. Instead, you can hop all the way to your own weird home made objects by specifying a tag. Then all the stuff in the yaml file gets passed in to your object.

Ditz has lots and lots of commands that are only on the be roadmap. You can search your issues with regular expressions with ditz grep, you can claim issues for yourself, you can group issues by releases and components, etc, etc, etc.

The ditz issue data model can be extended with plugins. Like I mentioned earlier, one plugin makes it possible for people to claim issues as assigned to them.

What I like about be

It’s written in python. I hate to feed the python snobbery monster, but there are certain python niceties that I don’t like doing without. In particular, ipython is just too awesome. When I read the ditz code, I spent most of my time navigating the code to get to the part that I cared about that. With ipython, I don’t have that problem. I just hit foo?? and immediately see the source code.

And ruby’s documentation is not what I’ve grown accustomed to with python. For comparison:

I think the Python docs have more explanatory text in just the table of contents.

In addition, ditz uses a lot homemade code: there’s a homemade option parser library (trollop), a homemade hack on the way ruby stores data files so that all the HTML templates are available, and all sorts of gymnastic FP tricks to get a lot of shit done in a very small number of lines. That’s cool, but as a yellow-belt in Ruby, it is really @#$ing hard to make any contributions to this project. Here’s some code that I find a little difficult to read:


def operation method, desc, *args_spec, &options_blk
    @operations ||= {}
    @operations[method] = { :desc => desc, :args_spec => args_spec,
                              :options_blk => options_blk }
end

operation :stop, "Stop work on an issue", :started_issue do
    opt :comment, "Specify a comment", :short => 'm', :type => String
    opt :no_comment, "Skip asking for a comment", :default => false
end

def stop project, config, opts, issue
    puts "Stopping work on issue #{issue.name}: #{issue.title}."
    issue.stop_work config.user, get_comment(opts)
    puts "Recorded work stop for #{issue.name}."
end

After tracing through a few hundred lines of stuff like that, I usually get discouraged and just write a feature request rather than a patch.

In summary

I like ditz. I like reading nearly inscrutable Ruby code to see how wacky people solve problems. My experience with ditz so far has been about an A minus, which is pretty good!

Why I’m going to write my own

There have been a few times where the lack of a proper database system has bit me. Like when I renamed a release, I had to do some searching and replacing in lots and lots of files. Also, regenerating my HTML views is taking almost two minutes now that I have so many issues. Also, certain operations, like moving a handful of issues from one release to another, or searching for intersections of issue subsets, are trickier than what they should be.

Besides all that, I’m fascinated by couchdb, and I think this would be a good use.

I think my system is going to use a local couchdb server that loads in all the issues from local yaml files into the server on startup. Then after lots of work updating, I’ll write out all the issues back into yaml. So, when you update your checkout of your code, you’ll need to restart or reload your couchdb server. Then you can use the couchdb server to work with the system, and then at the end, re-serialize the data back out to JSON, and then to yaml.

ditz and be are sort of like old-school CGI web apps where each user action has to start up some the framework, do the action, then tear down. My system will instead keep all the issue data in memory and require explicit startups and shutdowns.

We’re presenting at the Jump Start Angel Fair!

The First Annual Cleveland Angel Fair picked us to present. This is fantastic news.

In other news, spent the day writing code in the house by myself. The wind is howling outside, and my fingers are shaking because I refuse to turn on the heater. I haven’t gotten so many hours of consecutive geek time in years.

This is the life. I can’t believe that I used to take a shower, put on clean clothes, and interact with humans every day.

Spreadsheets are the devil, but here is how to avoid getting burned.

Spreadsheets seem like they are adequate tools for serious analysis. And unfortunately, people are graduating from stats and OR programs without mastering any of the other alternatives. But brother, I stand before you today to tell you that spreadsheets are the devil.

When you face a modeling problem, spreadsheets tempt you with the seemingly easy way out. It all starts with how easy it is to import data. Excel’s import wizard is fast and pretty smart about automatically assigning column types. Meanwhile, your hapless colleagues are going to spend a day reading manuals just to load in that same tab-delimited text file.

Now that you’ve got the raw inputs loaded, you figure that within a few days you’ll be done building your trendlines and you’ll kill time choosing fonts for your pie charts. But what happens — invariably — is that you think you are done and then you look at your number on your final worksheet and realize it can’t be right. You must now find the error in any of the possibly hundreds of tiny formulas all chained together. Welcome to cell HE11.

Meanwhile, while you’ve got numbers that are laughably wrong, your SAS friend after a few days at least has his PROC REPORT output to show the boss, even if he did have to print it on the basement mainframe dot-matrix printer.

So, despite all that, sometimes, I find that I just have to use a spreadsheet. In that circumstance, I try to follow a set of rules. Any time I deviate from these rules, I always get burned.

  1. Put at the top of each sheet a few paragraphs that describe the model. Ideally, this text should be so clear and specific that I can rebuild the spreadsheet just based on this information. (This also helps make sure that you implemented the logic correctly.)
  2. Indicate what are the cells that the user should play with, and what cells should not be tweaked. Point out where the final answer pops out. Establish a color scheme to distinguish between input data and formulas.
  3. Emulate the IRS 1040, where there is a column of text and just a few columns of numbers, and each row is as simple as possible. There’s a main column that gets summed at the bottom, and a secondary column where complex totals are broke down further.
  4. Decompose those formulas and don’t store literal data inside of formulas! For example, in a mortgage calculator, break out the interest rate, the mortgage size, and the number of years in the mortgage into separate cells, and then show the result in another cell:

    s1

    Don’t be tempted to cram all those numbers inside a single cell like this:

    s2

    Sure, you save a few rows and it compresses the size of your sheet, but in the end, you make your sheet much less flexible, and it will be more difficult to separate data-entry errors from formula errors.

  5. Finally, Put everything in top-to-bottom order in each sheet and have a single flow. Don’t have lots of parallel panels side-by-side. It becomes too confusing.

I am certain that there are even more rules that are better than these. Enlighten me.