Python: log uncaught exceptions with sys.excepthook

You ever notice how when your script dies because of some uncaught error, you don’t get that error in your log files? This post walks through how to make sure that you log that uncaught exception.

This is a trivial script that will raise an uncaught exception (code available here):

$ cat rgl/kaboom1.py
# vim: set expandtab ts=4 sw=4 filetype=python:

import logging

def f():
return g()

def g():
return h()

def h():
return i()

def i():
1/0

if __name__ == '__main__':

logging.basicConfig(
level=logging.DEBUG,
filename='/tmp/kaboom1.log',
filemode='w')

logging.debug('About to do f().')

f()

Notice the helpful traceback:

$ python rgl/kaboom1.py
Traceback (most recent call last):
File "rgl/kaboom1.py", line 28, in
f()
File "rgl/kaboom1.py", line 9, in f
return g()
File "rgl/kaboom1.py", line 13, in g
return h()
File "rgl/kaboom1.py", line 17, in h
return i()
File "rgl/kaboom1.py", line 21, in i
1/0
ZeroDivisionError: integer division or
modulo by zero

Unfortunately, that helpful traceback does not show up in the output logs!

$ cat /tmp/kaboom1.log
DEBUG:root:About to do f().

You could wrap your code with big try / except

This diaper pattern is a popular solution::

try:
f()

except Exception as ex:
logging.exception(ex)
raise

Make sure you re-raise the exception, otherwise your program will end with a zero return code.

Sidenote: how to log an exception instance

If you do any of these, you probably won’t like what you get:

logging.error(ex)
logging.error(str(ex))

In both cases, you are just turning the exception to a string. You won’t see the traceback and you won’t see the exception type.

Instead of those, make sure you do one of these:

logging.exception(ex)

# this is exactly what logging.exception does inside
logging.error(ex, exc_info=1)

# sets a higher log level than error
logging.critical(ex, exc_info=1)

For the last two, without that exc_info=1 parameter, you won’t see the traceback in your logs. You’ll just see the message from the exception.

Or you can use sys.excepthook

Instead of nesting your code inside a try-except clause, you can customize the built-in sys.excepthook function.

The kaboom2.py script has this extra code:

def log_uncaught_exceptions(ex_cls, ex, tb):

logging.critical(''.join(traceback.format_tb(tb)))
logging.critical('{0}: {1}'.format(ex_cls, ex))

sys.excepthook = log_uncaught_exceptions

And here’s the results:

$ python rgl/kaboom2.py

$ cat /tmp/kaboom2.log
DEBUG:root:About to do f().
CRITICAL:root: File "rgl/kaboom2.py", line 39, in
f()
File "rgl/kaboom2.py", line 9, in f
return g()
File "rgl/kaboom2.py", line 13, in g
return h()
File "rgl/kaboom2.py", line 17, in h
return i()
File "rgl/kaboom2.py", line 21, in i
1/0

CRITICAL:root:: integer division or modulo by zero

Incidentally, sys.excepthook preserves the non-zero return code.
Also incidentally, you can use sys.excepthook for all sorts of fun stuff. This shows how to make it fire off pdb when stuff blows up.

First episode of the next season of Mad Men plot summary

I just watched the first episode from the next season of Mad Men.

Peggy and her beatnik friends score a bunch of LSD and they stay up for three days doing art. Peggy keeps sketching pictures of a little girl dressed just like Vicki from Small Wonder. Somehow they accidentally invent a time machine.

Then they go back to Downton Abbey, where Lady Edith is having nightmares that she is a Cylon, but nobody cares until Bates explains that his ex-wife was a Cylon and that’s why he had to kill her.

Peggy starts work on a Cylon detector.

Thomas the footman steals the time machine, travels back to 1962, seduces Don Draper, kills him, and then hops back into the time machine, but it explodes.

Back in Downton Abbey, Countess Dowager Violet Grantham looks out her window and sees a bunch of shiny silver Cylon centurions marching toward the house. She twists her cane’s handle and pops out a magic wand.

And the epsiode ends when Don Draper wakes up in a Cylon resurrection ship with a bunch of other Cylon Don Drapers, Lady Ediths, and Turtles from Entourage.

Posted in fun

This is the nerdiest and awesomest thing you will read today

Last week, while exploring the caves of chaos, we found a room with a big ruby-encrusted throne up on a raised platform (a dais) and twelve skeleton statues along the walls.

Ogus (that’s me), the brash half-orc thief, goes to pry the gems off the throne, but as soon as he touches the dais, to absolutely nobody else’s surprise, the twelve skeleton statues start moving.

Queue the Steve Erkel catch phrase.

Anyhow, all the party joins in to fight the skeletons. We destroy them handily, but Ogus and Tulo (he’s a halfling thief), get beat up pretty bad during the battle.

Then Ogus and Tulo go back to prying gems off the throne, while the rest of the party watches the hallway entrance to the room.

Maybe because of all the loud yelling during the fight with the statues of skeletons turned into magic skeletons, the party attracted more monsters — a horde of zombies appears in the hallway.

Skargan, our badass half-orc fighter, rushes in and starts hacking zombies, while our half-elf rangers, Bane Woodsoul and Althana, use bows and arrows from a distance.

Tulo and Ogus sit this one out and work on prying gems off the throne. They hear somebody yelling from behind the wall.

Tulo finds a hidden door and and then discovers a secret room!

Inside the secret room, a human is tied up, blindfolded, and gagged. His armor and weapons are scattered on the ground, and a closed trunk is against one wall.

The human seems nice enough, so Ogus unties him.

The paladin notices how Tulo is injured, and right after we get the gag off his mounth, he says “Hi — may I touch you?”

Now everybody thinks the new guy comes off creepy.

Later we learn he meant he can do healing hands once daily. Not that he has some kind of small person fetish.

Meanwhile, Decimus our newly second-level magic user does something awesome in the battle vs the zombies. He makes “pew-pew” sounds and points with his fingers and then one of the zombies explodes.

Tulo helps the paladin (Milo Maccabee) put on his armor and Ogus checks the trunk for traps.

Ogus doesn’t notice any traps on the trunk so he opens it. He disregards whatever needle pricked his finger because there’s cool stuff inside the trunk:

  • neat-o punching dagger
  • bag of gold coins
  • a ring
  • a scroll
  • a potion

Ogus drinks the potion and puts the ring on. The potion heals some wounds! Hurray! The ring seems to have no effect.

The rest of the party finish turning the zombies into paste, and search for loot. The zombies all wear cool 1970s silver medallions with their zodiac symbols on them. Well, not really, but pretty much something like that.

Once all the party comes together to talk about what to do, Ogus passes the potion flask (with two doses remaining) over to Tulo for a swig. And he hands the scroll and the ring to Decimus our elven magic-user. Everyone meets the new paladin (“My name is Milo and I’m from the first temple of heal-the-party”).

It turns out Milo is on a quest to deliver a sealed message to two halvlings that coincidentally just happen to be Tulo’s lost parents! So naturally we all agree that Milo Maccabee should join our group of adventurers.

Somebody even volunteers to let the paladin touch him.

Then suddenly Ogus falls over –poisoned from the pinprick on the trunk! It was trapped after all!

Decimus discovers the scroll is a “find famililar” scroll, so that’s no help. And the healing potion won’t cure poison. The ring has no effect either.

At this point, the party has to decide what to do. Go back to the keep and look for help to cure Ogus, or just let nature take its course? The party decides to go back to town, but also look around for a potion to reduce foolhardiness.

Fat chance that’s gonna work!

Tune in next week for more kickassery.

Posted in fun

How I like to work

I just stumbed on an old email I wrote a while ago to a prospective employer. I’m posting it because I’m curious how much it matches other people’s preferences.

> We didn’t get a chance to talk much about culture, but there’s just a few questions that I’d love to hear more from you on:

> -What kind of environment do you want to work in? What kind of environment do you work best in?

I’m happiest when I understand the big picture and can participate in the conversation about what to build. I know some programmers hate vague requests, but I thrive on talking with clients, especially when they don’t know a damn thing about how computers work.

I start with some rough sketches and then iteratively, we design the product together.

> -How do you like to be managed? How do you manage others?

The best managers I’ve had made it seem effortless. I knew my work mattered, I got a chance to learn things and work on cool projects, and when we had meetings they felt more like kids designing secret hideouts rather than a trip to the principal’s office.

How did they do that? I don’t really know, but I think it was a function of what we were working on, who we worked with, and how we did it.

When I lead teams, before each day starts, I spend a good amount of time by myself, planning what I want each person to do.

Then I’ll tell each person what I expect them to get done. That’s a starting point though — I’m happy to have a conversation and then reassign and reorganize tasks.

After that, I try to handle as much of the tedious stuff as possible, so that my team can think deeply about big problems. That means I’m happy to handle the tasks for stuff like changing labels on buttons,or debugging some non-core system, or replying to clueless customers.

I don’t think of management as a reward for paying dues. I think of running a team of developers as sort of like programming, but at a much higher, more abstract level.

The other thing I aim to do is make myself redundant. There’s nothing worse than feeling like you can never take a vacation because everything will fall apart. I encourage everyone else to share knowledge and cross-train each other, even if it means it costs us throughput in the short run.

> -Can you tell me a bit more about remote working in your experience? What’s worked well / not well?

Emailed screenshots can save a lot of time when there’s some layout bug that needs to be fixed. And I like IM for discrete questions, like “what is the URL to the testing box”.

But really, I’m a big fan of talking on the phone. I find that a 15-minute phone conversation where both people are totally focused on that conversation is often way more efficient than two people just barely paying attention.

Generally, I think regularly talking through stuff is key to keeping everyone invested and focused on the real goal.

That’s it!

Undo a fast-forward git merge

A friend has a fork of my project and sent me a pull-request. Without doing any eyeballing at all, I did this:

$ git fetch XXX
$ git merge XXX/master

Git ran a fast-forward merge and now all those commits are in my code. Then I ran some tests and KABOOM.

After spending a while digging around and seeing just how much needed to be fixed, I decided that it would be better to just send an email with the test errors back to my friend rather than try to fix them myself. I got my own work to do, after all.

So I sent the email, but now I had a git checkout with all those commits.

And since this was a fast-forward merge, git didn’t create a commit for the merge.

Incidentally, today I learned that you can force git to make an explicit commit for fast-forward merges like this:

$ git merge --no-ff XXX/master

I went to #git on irc.freenode.org, where I always go, and I explained the situation, and then was told to do these commands:

$ git reflog show master
$ git reset --keep master@{1}

And it worked! All the foreign commits are now gone.

But what just happened?

I’m not absolutely certain, but it seems like the git reflog show master command shows the changes applied to HEAD over time. This is what the top lines showed for me:

cce8252 master@{0}: merge XXX/master: Fast-forward
08c8f50 master@{1}: commit: Cleaning up the my-account pages
5526212 master@{2}: commit: Deleted a handler that was never getting used

This is different than git log. This is talking about the state of my local master branch over time.

Then, the next command git reset –keep master@{1} is telling git to reset the checkout to look like what master looked like one state in the past.

Like I said, I’m still not sure I understand this, but I plan to study it more.

The gauntlet software pattern

I find code like this really confusing:

if a and b and c:

if d or e:
return True

elif f and g:
return True

else:
return False

Furthermore, updating those rules is nasty. I’m very likely to get something wrong, especially when “if a” is really some elaborate method call with several parameters.

It get a little more obvious when I flatten it out like this:

if a and b and c and (d or e or (f and g)):
return True

else:
return False

That looks OK for this contrived example, but imagine I need to stick in another clause that says also return True when a and g are True. It ain’t so easy to arrange the code any more.

Over the years, I’ve run into this situation a lot. Usually, I’m doing something like registering a new user, and I want to rule out that this new user is already in the database. So I have to look up the user on a variety of fields, like maybe email address, their user name, their mobile number, etc.

When I can boil the action down into something like “do this, unless any of these scenarios apply” I write the code in this format:


# Beginning of the gauntlet
if not a:
return False

if not b:
return False

if not c:
return False

if not d or e:
return False
# End of the gauntlet

return True

I put the fastest reasons to disqualify the action at the beginning. The really elaborate stuff belongs at the bottom.

Now,

Notes from Cleveland code retreat

I attended the day-long Code Retreat at Lean Dog’s floating office on Sunday.

We worked in pairs for 45-minute blocks of time, building Conway’s Game of Life. In each session, we started from the beginning. We didn’t build on previous code.

After each session, teams talked about what stuff they discovered. Then we reorganized into new pairs and started working from the beginning again.

First pairing

At first I paired with a C# developer. While other teams went right into writing test programs, we spent a fair amount of time talking and sketching out ideas about how to model the game.

Then we made a grid that knew how to generate the next state just by copying itself into a new grid.

We planned to add in code to support each of the four rules of the game of life one by one.

So, we wrote a test that created a grid with just a single living cell. Then we told that grid to make a grid for the next generation, and our test verified that the single cell survived the copy.

Then we started working on the first rule: if a cell is alive and has less than two neighbors, it dies.

We burned through the reminder of time sketching out ideas on my clipboard. By the end of our time, it was clear that we only needed to pay attention to living cells and neighbors of living cells.

It was also clear that the time spent writing the first test and then writing the code to satisfy the first test did nothing to move us toward a real solution.

Second pairing

Next I worked with a Java guy. This time, we did something a little closer to TDD.

We decided to start from the point of view of an end-user firing up the game. The user would want to make a grid, add a few living cells, and then tell the grid to generate its next state.

The first code we wrote was a test and we started writing tests almost immediately. In the test, we instantiated a Grid class. That test blew up with an error, since no Grid class existed. So then we wrote a grid class. It had no methods or attributes, but that was all we needed to write in order to satisfy our test.

Next we wrote a new test that instantiated a grid and then added a single cell by calling an (undefined) addCell(…) method.

After seeing that test crash with an error, we wrote an addCell(…) method that was a no-op, because that’s all our tests required us to do.

Then we wrote a test for rule one, where a living cell with less than two neighbors dies.

In this test, we made a grid, added a single cell, and then told the grid to create a new grid that represented itself in the future. Then we used an assert to verify the new grid had no cells.

First we added a “cells” instance-level ListArray attribute to our grid class. Then we wrote a generate_next_grid method on our grid class, and all that did was instantiate and return a new grid with an empty cells.

And bingo, we had enough code again to pass our test.

At this point, I became convinced that the kind of testing we were doing was harming our productivity. We had fallen into some kind of xeno’s paradox* where we would never actually get anything done, because we could always think of some new test that applied to a trivial subset of the project, and then work on writing code to satisfy that test.

*A dude shoots an arrow at a target. Before the arrow gets to the target, it has to cross half the distance to the target. Then it can cross the other half of the distance. But now it has to cross half of the remaining distance, so it does that.

Then it has to cross half of that remaining distance, and then half of the remaining distance after that, and so on for infinity.

Since the arrow has an infinite number of partial distances to cross, the arrow will never reach the target.

Read more about it here.

I was itching to solve a real problem, rather than just watch lights blink from red to green, so I cajoled my partner into writing a test for the blinker pattern, which is when a 3×1 rectangle of cells converts to a 1×3 rectangle of cells on the second turn, and then on the third turn, converts back to a 3×1 rectangle.

So, we started talking about how to implement the rules. We decided we needed to figure out for every cell that was near a living cell, how many living neighbors it had. So we wrote a method to generate the coordinates for the eight neighboring cell to a particular cell, and wrote a test for that.

Then we were going to make a neighbor_count hashmap attribute on our grid and we were going to use that to link cells to how many neighbors each had, and then the time ran out.

At this point, I had a good feeling about the algorithms involved in solving this project.

Third pairing

Right before the third pairing, a much older fellow raised his hand and asked if anyone wanted to work together on a solution that only tracked the living cells. That’s the approach I was using, so I paired with this guy. He said he didn’t care what language we used, but he was a lousy typist. So I said I could do the typing and we could use python.

At first we excitedly talked over each other and drew pictures until we confirmed we had the same solution in mind. It turned out my partner was an old common lisp hacker, and once he realized that I understood what he meant when he talked about stuff like &rest and cadr, we got along really well.

The first bit of code I wrote was a cell class with an x and a y attribute, and a method to generate a list of eight nearby cells.

I wrote a quick test in the same file as my code for that to make sure the execution was correct.

Of course, that test failed. I had lots of syntax errors because I was writing so fast and defending against good-natured jabs about how vim was so vastly inferior to emacs.

After a few cycles of fixing typos and rerunning the tests, we got the syntax errors out of the code. Then we attacked the project of generating the next state of the grid.

I had already seen how easy it was to write a test for the blinker pattern, so I wrote a quick test for that. The test made a grid with three cells in a vertical rectangle, then told the grid to generate the next state, then it tested that in the next state, the three cells were in a horizontal rectangle.

We wrote a grid class with a dictionary attribute of living cells in the current generation and a dictionary attribute of cells that would live in the next generation.

Then we wrote the function in our grid class to generate another grid. The function did an outer loop on all the living cells, then for each living cell, it looped through all the neighbors of that living cell, and then did an inner loop of all neighbors of that neighbor, and then counted up how many were alive.

Then we implemented the four rules of the game with two if-clauses.

We ran the test again, and of course, the code was again littered with indentation errors and mismatched parentheses because of how fast I had been typing and moving stuff around.

Then the timer went off. I stayed put while everybody was gathering in the middle to talk about how everything went. Within about 30 seconds I got the last few silly typing glitches fixed, and BAM. The test passed.

I walked over to my partner and told him we solved it.

Then we listened to other teams talk. The conversation centered around naming conventions that people were coming up with, like “sustainable neighborhood” and “four horsemen”.

I raised my hand and asked if anybody had written tests for any of the patterns in game of life beyond the blinker pattern, like tests for the glider pattern, or any of the other patterns shown on the wikipedia page.

Nobody replied to that. Instead the conversation shifted to a discussion of whether or not booleans hidden behind getters and setters were a sign of a bad design.

I think it was while somebody was talking about writing tests for their (nonexistent) display code that my partner leaned over and said “This is a drunkard’s search.” Of course, I didn’t know what meant, so he explained it. Here’s the wikipedia entry:

Conducting a drunkard’s search is to look in the place that’s easiest, rather than in the place most likely to yield results. Taken from an old joke about a drunkard who loses his car keys while unlocking his car and is found looking under a streetlamp down the road because the light is better, it has been an object of consideration in the social sciences since at least 1964.

Fourth pairing

I asked if anybody wanted to see a python solution, and four of us worked together, with me doing the typing.

First I described how we would need to find the eight neighboring cells of a particular cell. So I wrote a test for that, and then wrote the code to satisfy that test.

Then once that was finished, I described what the blinker looked like, and I wrote a test for that.

Then I reused the approach that we came up with in the third pairing, and then got it working fairly quickly.

At one point I showed the code we had written to one of the people running the event, and he pointed out two things (this is roughly paraphrased, and I hope I am getting these ideas correct):

  1. I had four if-clauses in a single function, and each if-clause tested two independent variables. So that means there were dozens of paths through the function that would have to be tested.
  2. My blinker test could be satisfied by code that just hardcoded the expected results, rather than built an actual game-of-life system. My test didn’t force a complete solution.

Both points illustrate the biggest difference in the TDD mindset versus whatever it is called that I do. (design by intuition?)

The fact that I didn’t have a test for every path that I created didn’t really concern me, because I was focused on getting the blinker pattern to work. If that blinker pattern test failed, then I might go in to my code and make sure that what I wrote was really what I meant, or maybe I’d even throw it all out and start from scratch. But I wouldn’t worry about covering all corners of this version until I was confident I was on the right track.

As for the second point, I use tests to catch errors, not to tell me what to write. I wouldn’t write code that returns a hard-coded list of values just to pass a test, unless I actually thought I could solve the real problem by doing that.

Anyhow, we got everything working pretty quickly this second time. You can see the output from my fourth pairing here.

After you download that code, you can run the tests like this:

$ python life.py
..
--------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Those two dots mean that my two tests passed. I spent the remainder of the time showing how to use the interactive interpreter and how to write doctests. I’ve tested a few of the patterns from the wikipedia page, and my code seems to do the right thing, but I’m not absolutely convinced that this implementation is correct.

Fifth pairing

I worked with two guys and we used ruby and rspec. I described the algorithm that I had used in the last two times, and we decided to work on building that.

We went with the approach of not writing any more code than was required to satisfy the tests.

So, first, we wrote some tests to instantiate a cell class, then we wrote some tests to call a neighbors method on the cell class and verify that it return a list of eight things, and then we verified that the first and last and elements in the neighbors list were at the position that we expected them to be at.

Then we started work on the grid class. We wanted to make sure that two different objects with the same values for their x and y attributes would be considered equal. But none of us knew how to redefine the == operator in ruby, and while we were researching this, the time ran out.

Again, I felt like we had fallen into the trap of slicing off trivial subsets of the real problem, and solving them, rather than attacking the real problem.

Commentary

Writing tests to check for errors (which is what I do) is not test-driven development. I daydream about solutions until I find one that I like, and then I build it, and I see if it works by writing tests to cover all the use cases I can think of.

My design comes from idle thoughts and doodles on my notepad, not from the testing.

Final note: I’ve tried my best to keep any generalizations and extrapolations out of this post. I don’t mean for this to be seen as an attack on anyone’s style of work. I’m very, very grateful for everything I’ve learned about how to write tests for software from the TDD community.

how to restart a gunicorn server without leaving vim

I’ve been using the gunicorn WSGI server lately while I work on my next project. gunicorn is fantastic, except for one tiny nuisance. I have to restart gunicorn as I change my app code. At the beginning, this meant I would switch over to the terminal where gunicorn was running. Then I’d kill the parent process by hitting CTRL+C, and then start it up again, by typing:

gunicorn crazyproject.webapp:make_application

Other frameworks often have a development mode that forks off a helper process that watches the source code folder. When some file changes, the helper tells the server to reload.

I’m not using any frameworks this time, so I don’t have that feature. I never really liked the fact that every time I saved a file, I would restart the server during development. Instead, in this case, I want to easily restart gunicorn, but only when I want to, and I don’t want to leave my editor.

It turned out to be pretty easy to do.

First I read in the excellent gunicorn docs how to tell gunicorn to reload my app:

How do I reload my application in Gunicorn?

You can gracefully reload by sending HUP signal to gunicorn:

$ kill -HUP masterpid

Next I added a –pid /tmp/gunicorn.pid option to the command that I use to start gunicorn, so that gunicorn would write the parent’s process ID into /tmp/gunicorn.pid.

Now any time I want gunicorn to reload, I can do this little command in a terminal window:

$ kill -HUP `cat /tmp/gunicorn.pid`

Those backticks around cat /tmp/gunicorn.pid tell the shell to do that part of the command first, and then feed the result into the rest of the command.

You can always use ! in vim to run some command-line program. If I had to explain the difference between vim and emacs, the one difference that is most interesting to me is that vim makes it super-easy to send buffer contents to other programs or read buffer contents from other programs, while the people behind emacs seem to think that any external dependency should ultimately be moved into emacs itself. I get the feeling that vim wants to be my text editor, but emacs wants to be my OS.

Anyhow, while I’m in vim, I run that command to restart like this:

: !kill -HUP `cat /tmp/gunicorn.pid`

After using that code for a while, and feeling confident it worked right, I mapped F12 in vim to do that action for me by adding this little thing to the end of my ~/.vimrc:

:map :!kill -HUP `cat /tmp/gunicorn.pid`

Now that means I hit F12 when I want to reload gunicorn.

Some notes from when I start using git

When I switched to git from subversion at my old business, I stored notes on how to do certain tasks. I’m pasting it below. Maybe some of this will help you out.

If you know a better way to do these tricks, please let me now.

My git diary
============

.. contents::

Slowly learning how to use git.

Typical upgrade flow
--------------------

I just committed a bunch of code and I want to see what the diffs were,
so I ran this::

$ git diff HEAD^

My production server runs branch 3.5.1 of my software. I want to start
work on a scary new feature that may take a long time, so I made a new
branch called 3.5.2 in my local sandbox::

$ git checkout -b 3.5.2

Now I can commit all my intermediate stuff in here.

Somebody found a bug in the production site (running 3.5.1) so I switch
to my 3.5.1 checkout::

$ git checkout 3.5.1
$ vi # fixing the problem
$ git commit -a -m "Fixed the prod bug"
$ git push origin 3.5.1

That last line sends my local commits to my remote bare repository.
That remote bare repository is on a box with an SSH server and RAID
storage.

Then I connect to the production box and pull down the most recent
changes::

$ ssh prod
matt@prod$ cd where-the-repo-is
matt@prod$ git pull origin 3.5.1
matt@prod$ restart-everything

That restart-everything command is a homemade script that does just what
you think it does.

Now back in my dev sandbox, I want to pull the bug fix from 3.5.1 into
my 3.5.2 branch. So I do it like this::

$ git checkout 3.5.2
$ git pull . 3.5.1

And now my 3.5.2 branch has that code in it. Hurray! Understand that
the dot (.) in git pull . 3.5.1 means that git should retrieve code from
this repository, not the fancypants remote one.

Pulling stuff
-------------

When I run::

$ git pull origin 3.5.1

That means to pull from the origin's 3.5.1 branch into whatever branch I
currently have checked out.

A typical day
-------------

I have two branches, an experimental branch and a public branch.
The production server used by customer runs the public branch.

Usually I work in my development branch. Sometimes I have to do some
code into production so this is how I do it::

$ git checkout stable # switch my local copy to the stable branch.
$ git pull origin stable # update local copy, just in case.
$ vi blah.py # do the bug fixes.
$ git commit -a -m "Notes about bug fixes"
$ git push origin # This is my deploy system.
$ git checkout dev # Go back to my development branch.
$ git pull origin stable # merge in that bug fix.

So far, this works well. I did something similar with SVN, and it also
worked fine. But git is way better at safely merging and it is much
faster.

Undo changes to a single file
-----------------------------

I typically edit a dozen files and then figure out that I want to undo
some stuff. If I did::

$ git reset --hard

Then my whole working copy would be destroyed. Usually, I just want to
do something like revert one file. So this is how::

$ git checkout that/particular/file

That deletes my working-copy changes just there. Everything else is
left as-is.

How to submit patches by email
------------------------------

I cloned the ditz project and tweaked the code, and I wanted to submit a
patch, so this is what they told me to do::

For this type of thing, it's "very simple". git commit -a, add a
one-line description followed by a blank line and more commentary, and
then git format-patch HEAD^.

When I ran git format-patch HEAD^, that produced a file on my local
machine that had a pretty formatted patch. Then I emailed that patch to
the list with some text.

How to branch from a remote repository
--------------------------------------

This is what somebody in the #git room told me to do, since I use a
remote bare repository::

$ git fetch origin
$ git checkout -b newbranch origin/original

Here's another person's opinion::

$ git fetch && git checkout --track -b mynewbranchagain remotename/mynewbranch

Set the remote branch
---------------------

Normally I have to pull from my remote repo and specify the branch I
want to pull in like this::

$ git checkout experimental
$ git pull origin experimental

It is possible to associate my local branch of experimental with that
remote branch of experimental like this::

$ git config --add branch.experimental.remote origin
$ git config --add branch.experimental.merge experimental

And now I can run::

$ git pull origin

without specifying the remote branch. In fact this works too::

$ git pull

Compare one file across two local branches
------------------------------------------

I want to look at a diff of x.py in my public branch vs my experimental
branch, so this is what I did::

$ git diff public experimental -- x.py

I can see everything different between the two branches like this::

$ git diff public experimental

Break up a whole bunch of changes into different commits
--------------------------------------------------------

I'm irresponsible about committing after each conceptual unit of work.
Lots of time, I'll edit a file to fix one bug, then while I'm in there,
I'll edit some other code because I see a better way to do something
else. Then I'll maybe add a few doctests to a completely different
section just because I want to.

After a few hours, typically inside of the same file, I have edits
related to multiple separate tasks.

Before I commit my changes, I run::

$ git add -p frob.py

Which walks through all the changes in that file and asks me if I want
to stage each one. In the first pass, I stage all the hunks related to
the first issue. Then I commit those changes. Then I rerun git add -p
frob.py again and march through the file for the next the second issue.

Keep in mind that I committed my changes after the first pass, so when I
go through the file the second time, I won't get prompted for those
changes.

This is one of those git features that you just couldn't do with svn.

Find the commits in branch A that are not in branch B
-----------------------------------------------------

Sometimes I'll patch a production bug in my production branch and then I
will forget to merge it into the development branch. This is how I can
check for that::

$ git checkout production_branch
$ git cherry dev_branch

This will spit out a list of commits that are in production_branch but
not in dev_branch.

It will not return any commits made to dev_branch but not in
production_branch.

See a file as of a point in time
--------------------------------

Looking at a commit shows the changes. Sometimes I want to see the file
itself.

Here's how to look at foo.py as of two commits ago::

$ git show HEAD^^:b/foo.py

Here's how to see what foo.py looked as of a particular commit::

$ git show bf51ebdbc:b/foo.py

b/foo.py is the path to the foo.py from the top of the repository. I'm
not certain, but I don't think the current working directory matters.

Copy a file from one branch to another
--------------------------------------

I committed a file into one branch then checked out a new branch. This
is how I copied that file into the new branch WITHOUT merging it in::

$ git checkout newbranch # 1
$ git checkout otherbranch foo.py # 2

Step 1 moves me into the newbranch. Step 2 gets me a copy of the file
foo.py from otherbranch and saves it into this branch named newbranch.

See a file in a different branch
--------------------------------

After I checkout the maxhenry branch I want to see the version of
printable.kid in the mayfield branch::

$ git checkout maxhenry
$ git show mayfield:bazman/bazman/templates/printable/printable.kid

Set a file to an old version of itself
--------------------------------------

Sometimes I want to get the version of a file as of a certain commit and
check that in. There are lots of ways to do this, including using git
revert or git reset, but I've had good luck with this approach::

$ git log # Use this to find the commit you want.

$ git checkout ac778cbb5517e1aeef446c9a8a1092eef81717fa:repo-top/a/foo.py

After you run this, the index will have this old copy queued up to be
commited. You can use git diff --cached to see the changes.

Create a new branch on the remote repo
--------------------------------------

We just pushed the branch **mollusk** up to production, so now it is
time to create a new branch named **nosehair**::

$ git branch nosehair # creates the new branch
$ git checkout nosehair # switches to the new branch
$ git push origin nosehair # pushes it up to origin

Switch to a new branch that already exists on a remote repo
-----------------------------------------------------------

After somebody else already created the branch nosehair, and pushed it
up to the remote repository, here's how to switch to work on the
nosehair branch::

$ git fetch origin
$ git branch -a # make sure origin/nosehair exists!
$ git checkout -b nosehair origin/nosehair

Search through commit messages
------------------------------

This is how I found all commits that had a commit message that included
the word CCPL::

$ git log --grep=CCPL

This is how I limited it to just my (Matt's) commits for CCPL work::

$ git log --grep=CCPL --author=matt --all-match

Without the --all-match option, you'll see all commits by matt or that
have CCPL in the commit message.