defaultdict.fromkeys does not play nice.

I use defaultdicts a lot when I’m grouping elements into a dictionary of lists. Here’s a simple example:


>>> a = defaultdict(list)

>>> a['x']
[]

>>> a['y'].append('yellow')

>>> a
defaultdict(<type 'list'>, {'y': ['yellow'], 'x': []})

Now here’s where I got silly. I used defaultdict.fromkeys to prepopulate the ‘x’ and ‘y’ key right away, because I know I needed those:


>>> b = defaultdict.fromkeys(['x', 'y'], list)

>>> b
defaultdict(None, {'y': <type 'list'>, 'x': <type 'list'>})

>>> b['x']
<type 'list'>

>>> b['z']
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
KeyError: 'z'

Wowsa! b calls itself a defaultdict, but it is not a defaultdict.

I haven’t really thought this through, but this behavior is so unexpected that I would prefer that defaultdict.fromkeys raised a NotImplementedError.

Posted in Programming, Python.

By matt

June 3, 2008

Comments

Viewing 5 Comments

Georg Brandl 06/04/2008 03:14 AM 2 points
Please login to rate.

Do you already have an account? Log in and claim this comment.

With dictionaries, the second argument for fromkeys() is the value (not a value factory) that all keys from the first argument will be set to. Since defaultdict.fromkeys() has no special handling, this works the same with defaultdict. That's not a bug to me.

Of course defaultdict.fromkeys could be extended to take a factory function argument, but as you have seen you can set the factory yourself too.

All in all, I don't see the value of using fromkeys() for a defaultdict.

reply edit reblog flag

Matt Wilson 06/04/2008 10:32 AM 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

Hey Georg,

Yeah, I agree that the defaultdict.fromkeys behavior is not a bug, but it does seem a little surprising. Anyhow, I posted to comp.lang.python here about this topic and got some pretty good explanation.

reply edit reblog flag

26 /people/mw44118/ /people/mw44118/following/ http://blog.tplus1.com

Gary Bernhardt 06/03/2008 02:42 PM 2 points
Please login to rate.

Do you already have an account? Log in and claim this comment.

Ahh... good point. Clearly I didn't think it through. :)

I considered taking a stab at fixing this, but the effort involved for a first-time Python contributor is prohibitive. So laziness wins again.

reply edit reblog flag

http://blog.extracheese.org/

Matt Wilson 06/03/2008 01:33 PM 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

Hey Gary, thanks for the comment.

Setting the default_factory after I make an instance doesn't go back and rerun the keys through the default factory. So, in that sense, fromkeys is kinda useless to me. That's why I'd prefer that it just raised that NotImplementedError.

>>> b = defaultdict.fromkeys(['x', 'y'], list)
>>> b
defaultdict(None, {'y': <type 'list'="">, 'x': <type 'list'="">})
>>> b.default_factory = list
>>> b['z']
[]
>>> b['x']
<type 'list'=""></type></type></type>

reply edit reblog flag

26 /people/mw44118/ /people/mw44118/following/ http://blog.tplus1.com

Gary Bernhardt 06/03/2008 01:19 PM 2 points
Please login to rate.

Do you already have an account? Log in and claim this comment.

It looks like defaultdict.fromkeys just fails to pass the factory function along:

>>> defaultdict(list)
defaultdict(<type 'list'="">, {})
>>> defaultdict.fromkeys([], list)
defaultdict(None, {})

There's no mention of fromkeys in the collections module's source, so it's probably just inheriting it from dict. That makes sense - dict doesn't know about the default_factory. You can manually set the defaultdict's default_factory argument to fix it.</type>

reply edit reblog flag

http://blog.extracheese.org/