Category Archives: Python

Using state with FormEncode and TG’s validate decorator

Posted on by

I believe I figured out a way to reduce a few redundant lines from my controller methods. I’m looking for opinions about whether this is a wise idea.

At the top of nearly every method in my controllers, I look up the current user and the hospital this user belongs to, sort of like this:

@expose('.templates.m1')
def m1(self):
u = identity.current.user
hospital = u.hospital

Anyhow, I realized I can offload this irritating work to a validator that uses a state factory. Now my method looks like this:

@expose('.templates.m1')
@validate(validators=LookupSchema(), state_factory=my_state_factory)
def m1(self, u=None, hospital=None):

So now all my methods get called with those values already set up. I have to make u and hospital keyword parameters, because otherwise TG will try to pull their values out of the URL.

Here’s how it works. First I make my_state_factory that builds an object that has those values:

def my_state_factory():

class StateBlob(object):
pass

sb = StateBlob()
sb.u = identity.current.user
sb.hospital = u.hospital

return sb

Now the LookupSchema extracts those values out of the state blob object and adds them to the dictionary of values:

from formencode.schema import Schema, SimpleFormValidator
@SimpleFormValidator
def f(value_dict, state, validator):
value_dict['u'] = state.u
value_dict['hospital'] = state.hospital

class LookupSchema(Schema):
allow_extra_fields = True # otherwise, it fusses about self ?!?!?
chained_validators = [f]

So the benefit of all this is that some repetitive code is now just defined in a single place. Also, I’m getting more comfortable with the internals of FormEncode and the TG validate decorator.

Pretty soon, my controllers will be some really skinny methods. All the calculations of new variables based on the original parameters will happen outside the controller. The controller will just handle picking the right template.

Use FormEncode to verify one date precedes another

Posted on by

I have a form on my site that lets people choose a start date and a stop date. Then I show statistics for that date range. I wrote a FormEncode schema to verify that the start date is before the stop date.

The documentation on schema validators is fairly sparse, so I’m publishing this because it might help somebody else out.

Code


# This is in a file named formencodefun.py
from formencode import Schema
from formencode.validators import DateConverter, FancyValidator, Invalid

class DateCompare(FancyValidator):
messages = dict(invalid="Start date must be before stop date")

def validate_python(self, field_dict, state):

start_date = field_dict['start_date']
stop_date = field_dict['stop_date']

if start_date > stop_date:
msg = self.message('invalid', state)

raise Invalid(msg, field_dict, state,
error_dict=dict(stop_date=msg))

class MySchema(Schema):
start_date = DateConverter()
stop_date = DateConverter()

chained_validators = [DateCompare()]

Usage


>>> from formencodefun import MySchema
>>> s = MySchema()
>>> d1 = {'start_date':'11-02-2008', 'stop_date':'11-15-2008'}
>>> d2 = {'start_date':'11-15-2008', 'stop_date':'11-02-2008'}
>>> s.to_python(d1)
{'stop_date': datetime.date(2008, 11, 15), 'start_date': datetime.date(2008, 11, 2)}
>>> s.to_python(d2)
------------------------------------------------------------
Traceback (most recent call last):
File "", line 1, in
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/api.py", line 400, in to_python
value = tp(value, state)
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/schema.py", line 200, in _to_python
new = validator.to_python(new, state)
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/api.py", line 403, in to_python
vp(value, state)
File "formencodefun.py", line 18, in validate_python
error_dict=dict(stop_date=msg))
Invalid: Start date must be before stop date

How it works

Notice when I run s.to_python(d1), I get a dictionary back with the the values for start_date and stop_date replaced with datetime.date objects.

Then when I run my schema on d2, where the start_date is after the stop_date, my schema raises an Invalid exception. In a web framework like TurboGears, there is some exception handler that will catch that exception and take that error dictionary and redraw the form and print my error message.

Notice that the DateConverters first take my strings and turn them into datetime.date objects before the test in DateCompare. FormEncode runs the chained validators after it runs the individual validators.

In this case, I just want to make sure that the start date precedes the stop date. I have written other validators that add extra keys into the field dict or change the values, but I want to keep this example simple.

If the first DateConverters fail, then the chained validators never run:

>>> s.to_python({'start_date':'UNPARSEABLE', 'stop_date':'11-20-2008'})
------------------------------------------------------------
Traceback (most recent call last):
File "", line 1, in
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/api.py", line 400, in to_python
value = tp(value, state)
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/schema.py", line 197, in _to_python
error_dict=errors)
Invalid: start_date: Please enter the date in the form mm/dd/yyyy

When my DateCompare validator is run, I can be confident that the objects with the keys start_date and stop_date in the field_dict have already been converted to datetime.date objects.

In summary, FormEncode is awesome, but I have spent a lot of time beating my head against the wall trying to learn how to use it.

My metaclass article published

Posted on by

The November issue of Python Magazine has my article on metaclasses. I regain the rights to my text after 3 months, so I’ll be posting sections of that article here as blog posts over the next year.

In the meantime, I’d love to hear critical feedback so that I can improve the material.

Or if you want to write a message telling me how awesome I am, that’s cool too, but I would prefer you give me money. In fact, that applies to people that want to call me an idiot. You also should just give me money.

Got back from PyWorks

Posted on by

PyWorks was a lot of fun.

Mark Ramm did a talk about WSGI where in one slide he showed how to build up something akin to the full Pylons stack by just applying lots of WSGI middlewares. That really inspired me. Sure, frameworks are great, but I think I’m going to try to going to the opposite extreme for a while — building up from components, rather than using somebody else’s aggregation of components.

Based on a few hallway conversations, It turns out I’m not the only one using version control as a way to deploy code in production. My production box runs a git clone of my production branch.

jonEbird did a talk on using LDAP with Python. It was the first time I had realized that LDAP can do more than user authentication.

I liked Mike Naberezny’s talk on routes also. Good introduction. It reaffirmed my idea that it’s not going to be all that difficult to string a bunch of stuff together for my next web app.

Chris Perkins showed some neat TDD / agile programming techniques possible and made SQLAlchemy seem much friendlier than when I looked at it last year.

I learned some neat stuff at clepy last night

Posted on by

Brian Beck showed how to use metaclasses and descriptors to make DSLs with python.

I do this kind of this kind of thing every so often in my code:

def f(x):
class C(object):
y = x
return C

That function takes a parameter and makes and returns a class based on that parameter. Whoop-di-do. I was surprised to learn that you can’t do this:

class C(object):
x = 99
class D(object):
y = x + 1

I gotta explore this some more until it makes sense.

Here’s another neat trick: It isn’t possible to add two classes together:

>>> class C(object):
... pass
...
>>> C + C
------------------------------------------------------------
Traceback (most recent call last):
File "", line 1, in
TypeError: unsupported operand type(s) for +: 'type' and 'type'

But if you want to support this, the solution would be to define an __add__ method on the metaclass:

>>> type(C)

>>> class MC(type):
... def __add__(self, other):
... print 'Adding!'
... return 99
...
>>> class C(object):
... __metaclass__ = MC
...
>>> C + C
Adding!
99

Wacky, right? More realistically, I could build a new class by taking attributes of both classes together. In other words, if class C has a class attribute x, and class D has a class attribute y, then we can use a metaclass to add C and D together to get a new class E, that has both x and y as class attributes.

In this example, C has a class attribute x and D has a class attribute y. When I add the two classes, I get a new class with both of those class attributes.

>>> C.x, D.y
(99, 98)
>>> E = C + D
>>> E.x, E.y
(99, 98)

Here’s the metaclass that allows this sort of nonsense:

class MC(type):

def __add__(self, other):

class E(self):
pass

for k,v in other.__dict__.items():
if k not in ('__dict__', ):
setattr(E, k, v)

return E

Sometimes I think validate + formencode is more hassle than it is worth

Posted on by

I’m hoping somebody will read this and show me a better way.

In general, I like formencode. I like that I can do stuff like:

@validate(validator=SomeGnarlySchema())
def m(self, a, b, c, d, e=None):

And then I know that all my parameters have been converted from their original string values into whatever I want.

But I also find that I spend a lot of time getting my complex schemas to work. Like right now, I have an optional parameter e. e should either be a string representing a date, or it can be None.

I’ve got a validator with this logic in it for e:

  1. First try to return a datetime.date object from parsing e.
  2. Otherwise, look in the cookie for a key “e” and try to return that after parsing it into a datetime.date.
  3. Finally, just return today’s date.

So, the idea is that some visitor can come to page /m and always see data for today. Or, they can use a calendar widget to choose a value. On subsequent visits back to /m, I’ll keep showing them that same date they chose because I saved in it a cookie.

Here’s the problem. I have to make e an optional parameter because I don’t want to require that people hit the site with a url that contains a value for e.

However, when e is None, then my validator for e is ignored! So, as far as I know, at this point, I need to use a validator that operates on the whole set of parameters.

Which is also possible, but in my brain, it seems wrong that I have to use a schema-level validator when I really am only validating one single field.

More generally, anybody that subscribes to the formencode mailing list sees first-hand just how confusing a lot of people find formencode. It is a very powerful library, but very tricky to get right.

Here’s my question — does validate really need to use formencode? Is there some better, simpler solution? I’ve read about how django tackles this problem, and their approach does seem simpler, but I can’t say for sure until I really build something with it.

If any readers can show how to make a form.clean method that does the 1-2-3 logic I described above, I’d be really grateful.

Maybe formencode just needs a fat cookbook of solutions.