Sometimes I think validate + formencode is more hassle than it is worth

Posted on by

I’m hoping somebody will read this and show me a better way.

In general, I like formencode. I like that I can do stuff like:

@validate(validator=SomeGnarlySchema())
def m(self, a, b, c, d, e=None):

And then I know that all my parameters have been converted from their original string values into whatever I want.

But I also find that I spend a lot of time getting my complex schemas to work. Like right now, I have an optional parameter e. e should either be a string representing a date, or it can be None.

I’ve got a validator with this logic in it for e:

  1. First try to return a datetime.date object from parsing e.
  2. Otherwise, look in the cookie for a key “e” and try to return that after parsing it into a datetime.date.
  3. Finally, just return today’s date.

So, the idea is that some visitor can come to page /m and always see data for today. Or, they can use a calendar widget to choose a value. On subsequent visits back to /m, I’ll keep showing them that same date they chose because I saved in it a cookie.

Here’s the problem. I have to make e an optional parameter because I don’t want to require that people hit the site with a url that contains a value for e.

However, when e is None, then my validator for e is ignored! So, as far as I know, at this point, I need to use a validator that operates on the whole set of parameters.

Which is also possible, but in my brain, it seems wrong that I have to use a schema-level validator when I really am only validating one single field.

More generally, anybody that subscribes to the formencode mailing list sees first-hand just how confusing a lot of people find formencode. It is a very powerful library, but very tricky to get right.

Here’s my question — does validate really need to use formencode? Is there some better, simpler solution? I’ve read about how django tackles this problem, and their approach does seem simpler, but I can’t say for sure until I really build something with it.

If any readers can show how to make a form.clean method that does the 1-2-3 logic I described above, I’d be really grateful.

Maybe formencode just needs a fat cookbook of solutions.

15 thoughts on “Sometimes I think validate + formencode is more hassle than it is worth”

  1. The “None” thing is one of maybe a few very glaring issues which should be changed. If you look at the source code, the Schema object totally bypasses the validator if the value is None. This is an inconsistency in how the data is handled and it would be nice if Schema/FancyValidator had one single datapath for all values.

    Its also confusing to have to integrate composite form elements with variabledecode and it would be nicer for this to be more explicitly integrated.

    Beyond that I think the docs just need more real world examples.

  2. Hi Mike, thanks for the comments.

    Maybe I'll put together some restructured text documentation of all
    the formencode tricks that I know of and then see if anyone else wants
    to add their stuff to the pile.

    Matt

  3. I had a similar reaction to formencode. Every time I've picked it up to use, I've run into edge cases and surprises, especially about inheritance. And these problems had nothing to do with the problem space that I wanted to think about, and everything to do with implementation details of the library. Now, Ian is a great programmer, so I'm not knocking the quality of the implementation at all — but in my view the declarative syntax of formencode, while appealing for simple cases, doesn't scale well. So I wrote a library for my own use (“validino”) that I tend to use (I still use formencode sometimes). Validino's behavior is always transparent, although it has its own limitations and I don't make claims for it — it's a minor effort. What I'd really like to see is a well-thought out, next-gen formencode that stands on formencode's shoulders, but reduces or eliminates the DSL tax so you can use it without needing to think much about the library and its quirks. That's why we use Python and not Perl, right?

  4. Hi Jacob — that validino package looks neat. How would I do something like stick in a value for a key if that key is not defined in the initial data passed into the schema?

    For example, if my data is d = dict(a=1, c=3), then I want my schema to create a key b that points to, say, 99. How to?

  5. OK, after all of 5 minutes, I figured out to use the compose + default + to_list to make sure I got a default value.

    import validino
    d = dict(a=1, c=3) # this will be my input data
    f = validino.compose(validino.default(99), validino.to_list())
    v = dict.fromkeys('abc', f)
    s = validino.Schema(v)
    print s(d)

    So, when do you revert to formencode? What does formencode offer that this approach doesn't, or couldn't?

  6. I think what most people find confusing is for FormEncode 'parameter missing' and 'parameter empty' are two different cases. There is no 'default' but only if_missing and if_empty. Furthermore you can define what you consider an 'empty' value.

    Then FormEncode has some brain-dead parameter names like 'not_empty'. What the hell? I mean negative logic is hard to grasp. Instead of calling it 'required'.

    Another grip I have with FormEncode is that there are almost no unit tests and most validators rely on the input values being strings which is plain wrong if you use FormEncode's variabledecode.

    But some things in FormEncode are really nice like the declarative object or schema inheritance. One of my imagined projects (though low priority at the moment) is to build my own validator library on the basis of declarative which hopefully avoid the common FormEncode pitfalls.

    fs

  7. Yeah, a lot of problems could be solved with a lot of example formencode recipes, especially the weird stuff. I'm going to write up a few of my recipes and see if anyone else wants to help.

    It's a great library, but it's just not that easy for me to jump in, build a complex validator, then walk away from formencode for a month.

  8. or more concisely (with a behavior that matches your original statement of the problem more closely):

    >>> import validino as V
    >>> s=V.Schema(dict(b=(V.default(99), V.strip, V.integer())))
    >>> s({})
    {'b' : 99}

    I mostly use formencode nowadays for maintaining and extending code that is already using it. Validino was written quickly and doesn't do everything that formencode does — it handles messages a bit differently and I'm not sure I like either system much, but an alternative hasn't occurred to me. Also it doesn't handle schema introspection particularly well, which is something you'd want in some cases. But you've found what I like about the approach — that it is entirely transparent what the meaning of a particular validator is, because you build up the various parts of it via function composition.

  14. Form me FormEncode is pain. I like easy approach. Something like Web2py at data level defined easy validators: Field(xxx, requires=IS_EMPTY_OR(IS_DATE(..))).
    Does some separate library for validation other then FormEncode exist? And is FormEncode the mainstream?

Comments are closed.