Define your validation schema inline

The TurboGears docs show how to assign validators for individual parameters in the validate decorator like this:

@validate(validators={'a':validators.Int(), 'b':validators.DateConverter()})
@error_handler()
def f(self, a, b, tg_errors=None):
# Now a is already an integer and b is already a datetime.date object,
# unless there were some validation errors.

That’s great, but there are some validations that depend on numerous parameters at the same time. For example, you might want to make sure that an employee’s hire date precedes the termination date.

I already knew how to subclass validators.Schema to do this, and then pass that instance into the validate decorator like this:

class MattSchema(validators.Schema):
a = validators.Int()
b = validators.DateConverter()
chained_validators = [blah] # pretend that blah does some compound validation.

@validate(validators=MattSchema())
def f(self, a, b)

This approach is fine, but today I discovered that it is also possible to define a Schema inline, inside the validate decorator, and specify the chained_validators right there, like this:

@expose('.templates.shiftreports.overtime')
@validate(validators=validators.Schema(
a=validators.Int(),
dt=validators.DateConverter(),
chained_validators=[blah]),
state_factory=matt_state_factory)
def f(self, a, b):

What’s the point? Well, it seems wasteful to define a class and hide it in another file if that schema is only going to be used for exactly one controller. Also, this makes it really fast for me to mix and match comound validators with controllers. I don’t need to pop open my separate validators file where all my elaborate schemas live. I can define them right here.

I’m very forgetful too, so I like to keep my code shallow so that I can instantly see what the heck something does. With all the validators right there, I can easily figure out what the system intends to do.

However, I would define a Schema subclass as soon as I see that I need the same thing twice.

I’m happy that the FormEncode authors had the foresight to support this inline approach along with the declarative style.

Using state with FormEncode and TG’s validate decorator

I believe I figured out a way to reduce a few redundant lines from my controller methods. I’m looking for opinions about whether this is a wise idea.

At the top of nearly every method in my controllers, I look up the current user and the hospital this user belongs to, sort of like this:

@expose('.templates.m1')
def m1(self):
u = identity.current.user
hospital = u.hospital

Anyhow, I realized I can offload this irritating work to a validator that uses a state factory. Now my method looks like this:

@expose('.templates.m1')
@validate(validators=LookupSchema(), state_factory=my_state_factory)
def m1(self, u=None, hospital=None):

So now all my methods get called with those values already set up. I have to make u and hospital keyword parameters, because otherwise TG will try to pull their values out of the URL.

Here’s how it works. First I make my_state_factory that builds an object that has those values:

def my_state_factory():

class StateBlob(object):
pass

sb = StateBlob()
sb.u = identity.current.user
sb.hospital = u.hospital

return sb

Now the LookupSchema extracts those values out of the state blob object and adds them to the dictionary of values:

from formencode.schema import Schema, SimpleFormValidator
@SimpleFormValidator
def f(value_dict, state, validator):
value_dict['u'] = state.u
value_dict['hospital'] = state.hospital

class LookupSchema(Schema):
allow_extra_fields = True # otherwise, it fusses about self ?!?!?
chained_validators = [f]

So the benefit of all this is that some repetitive code is now just defined in a single place. Also, I’m getting more comfortable with the internals of FormEncode and the TG validate decorator.

Pretty soon, my controllers will be some really skinny methods. All the calculations of new variables based on the original parameters will happen outside the controller. The controller will just handle picking the right template.

Use FormEncode to verify one date precedes another

I have a form on my site that lets people choose a start date and a stop date. Then I show statistics for that date range. I wrote a FormEncode schema to verify that the start date is before the stop date.

The documentation on schema validators is fairly sparse, so I’m publishing this because it might help somebody else out.

Code


# This is in a file named formencodefun.py
from formencode import Schema
from formencode.validators import DateConverter, FancyValidator, Invalid

class DateCompare(FancyValidator):
messages = dict(invalid="Start date must be before stop date")

def validate_python(self, field_dict, state):

start_date = field_dict['start_date']
stop_date = field_dict['stop_date']

if start_date > stop_date:
msg = self.message('invalid', state)

raise Invalid(msg, field_dict, state,
error_dict=dict(stop_date=msg))

class MySchema(Schema):
start_date = DateConverter()
stop_date = DateConverter()

chained_validators = [DateCompare()]

Usage


>>> from formencodefun import MySchema
>>> s = MySchema()
>>> d1 = {'start_date':'11-02-2008', 'stop_date':'11-15-2008'}
>>> d2 = {'start_date':'11-15-2008', 'stop_date':'11-02-2008'}
>>> s.to_python(d1)
{'stop_date': datetime.date(2008, 11, 15), 'start_date': datetime.date(2008, 11, 2)}
>>> s.to_python(d2)
------------------------------------------------------------
Traceback (most recent call last):
File "", line 1, in
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/api.py", line 400, in to_python
value = tp(value, state)
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/schema.py", line 200, in _to_python
new = validator.to_python(new, state)
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/api.py", line 403, in to_python
vp(value, state)
File "formencodefun.py", line 18, in validate_python
error_dict=dict(stop_date=msg))
Invalid: Start date must be before stop date

How it works

Notice when I run s.to_python(d1), I get a dictionary back with the the values for start_date and stop_date replaced with datetime.date objects.

Then when I run my schema on d2, where the start_date is after the stop_date, my schema raises an Invalid exception. In a web framework like TurboGears, there is some exception handler that will catch that exception and take that error dictionary and redraw the form and print my error message.

Notice that the DateConverters first take my strings and turn them into datetime.date objects before the test in DateCompare. FormEncode runs the chained validators after it runs the individual validators.

In this case, I just want to make sure that the start date precedes the stop date. I have written other validators that add extra keys into the field dict or change the values, but I want to keep this example simple.

If the first DateConverters fail, then the chained validators never run:

>>> s.to_python({'start_date':'UNPARSEABLE', 'stop_date':'11-20-2008'})
------------------------------------------------------------
Traceback (most recent call last):
File "", line 1, in
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/api.py", line 400, in to_python
value = tp(value, state)
File "/home/matt/virtualenvs/scratch/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg/formencode/schema.py", line 197, in _to_python
error_dict=errors)
Invalid: start_date: Please enter the date in the form mm/dd/yyyy

When my DateCompare validator is run, I can be confident that the objects with the keys start_date and stop_date in the field_dict have already been converted to datetime.date objects.

In summary, FormEncode is awesome, but I have spent a lot of time beating my head against the wall trying to learn how to use it.

Sometimes I think validate + formencode is more hassle than it is worth

I’m hoping somebody will read this and show me a better way.

In general, I like formencode. I like that I can do stuff like:

@validate(validator=SomeGnarlySchema())
def m(self, a, b, c, d, e=None):

And then I know that all my parameters have been converted from their original string values into whatever I want.

But I also find that I spend a lot of time getting my complex schemas to work. Like right now, I have an optional parameter e. e should either be a string representing a date, or it can be None.

I’ve got a validator with this logic in it for e:

  1. First try to return a datetime.date object from parsing e.
  2. Otherwise, look in the cookie for a key “e” and try to return that after parsing it into a datetime.date.
  3. Finally, just return today’s date.

So, the idea is that some visitor can come to page /m and always see data for today. Or, they can use a calendar widget to choose a value. On subsequent visits back to /m, I’ll keep showing them that same date they chose because I saved in it a cookie.

Here’s the problem. I have to make e an optional parameter because I don’t want to require that people hit the site with a url that contains a value for e.

However, when e is None, then my validator for e is ignored! So, as far as I know, at this point, I need to use a validator that operates on the whole set of parameters.

Which is also possible, but in my brain, it seems wrong that I have to use a schema-level validator when I really am only validating one single field.

More generally, anybody that subscribes to the formencode mailing list sees first-hand just how confusing a lot of people find formencode. It is a very powerful library, but very tricky to get right.

Here’s my question — does validate really need to use formencode? Is there some better, simpler solution? I’ve read about how django tackles this problem, and their approach does seem simpler, but I can’t say for sure until I really build something with it.

If any readers can show how to make a form.clean method that does the 1-2-3 logic I described above, I’d be really grateful.

Maybe formencode just needs a fat cookbook of solutions.