Data validation Validate documents before saving them to the database.

It's common for ORM/ODMs to provide some level of validation for documents but MongoFrames doesn't (here's a short explanation why). This snippet presents a simple approach for implementing a Frame-like class that supports validation.

The ValidatedFrame class

The ValidatedFrame class is a replacement for Frame that supports validation when using the save method to insert or update a document. Validation is performed using the excellent WTForms library however if you wish to use another library it should be straightforward to switch.

# validated.py

from mongoframes import *

__all__ = [
    'InvalidDocument',
    'ValidatedFrame'
    ]


class FormData:
    """
    A class that wraps a dictionary providing a request like object that can be
    used as the `formdata` argument when initializing a `Form`.
    """

    def __init__(self, data):
        self._data = {}
        for key, value in data.items():

            if key not in self._data:
                self._data[key] = []

            if isinstance(value, list):
                self._data[key] += value
            else:
                self._data[key].append(value)

    def __iter__(self):
        return iter(self._data)

    def __len__(self):
        return len(self._data)

    def __contains__(self, name):
        return (name in self._data)

    def get(self, key, default=None):
        if key in self._data:
            return self._data[key][0]
        return default

    def getlist(self, key):
        return self._data.get(key, [])


class InvalidDocument(Exception):
    """
    An exception raised when `save` is called and the document fails validation.
    """

    def __init__(self, errors):
        super(InvalidDocument, self).__init__(str(errors))
        self.errors = errors


class ValidatedFrame(Frame):

    # The form attribute should be assigned a WTForm class
    _form = None

    def save(self, *fields):
        """Validate the document before inserting/updating it"""

        # If no form is defined then validation is skipped
        if not self._form:
            return self.upsert(*fields)

        # Build the form data
        if not fields:
            fields = self._fields

        data = {f: self[f] for f in fields if f in self}

        # Build the form to validate our data with
        form = self._form(FormData(data))

        # Reduce the form fields to match the data being saved
        for field in form:
            if field.name not in data:
                delattr(form, field.name)

        # Validate the document
        if not form.validate():
            raise InvalidDocument(form.errors)

        # Document is valid, save the changes :)
        self.upsert(*fields)

Validating on save

To demonstrate how validation works we'll create a collection/class for storing information on films:

# films.py

from wtforms import Form
from wtforms.fields import *
from wtforms.validators import *

from validated import ValidateFrame

__all__ = ['Film']


class SaveFilmForm(Form):
    title = StringField('Title', [Required(), Length(max=80)])
    release_year = IntegerField(
        'Release year',
        [Required(), NumberRange(min=1890)]
        )
    summary = StringField('Summary', [Length(max=2500)])


class Film(ValidatedFrame):
    """
    A film archive.
    """

    _fields = {
        'title',
        'release_year',
        'summary'
        }

    _form = SaveFilmForm

Along with the Film class we've also defined a SaveFilmForm that will be used to validate the document on save. We can see this in action if we attempt to insert a valid and invalid film:

from films import Film

# Valid film entry
valid_film = Film(
    title='Moneyball',
    release_year=2011,
    summary="""Oakland As general manager Billy Beanes successful attempt to
assemble a baseball team on a lean budget by employing computer-generated
analysis to acquire new players."""
    )
valid_film.save()

# Invalid film - this will fail and raise an `InvalidDocument` exception
invalid_film = Film(
    title='The Ridiculous 6',
    release_year='Who gives a s**t',
    summary="""Really if you take nothing else from this snippet let it be this
- this film is terrible."""
    )
invalid_film.save()

Why doesn't MongoFrames support validation out-of-the-box?

My reservation to applying validation at the Frame level (a.k.a Model in other ORM/ODBs) is that IMO this has a tendency to lead to duplication. For example:

• A visitor submits a contact form on our website.
• We validate the visitor's submission before attempting to save it to the database and feedback any validation errors.
• When the visitor provides a valid submission we'll validate it again at the Frame/model level before saving it to the database. This second validation is often in part or entirety a duplication of the first.

Some ORM/ODMs allow you create a form based on your Frame/model validation rules, but these don't always marry up (for example we might decide we need to add a reCAPTCHA field to our contact form to stop the bots) and form fields often accept other arguments such as label, placeholder, id, and so on which aren't relevant in the context of the Frame/model.

My approach for some time now has been to validate external input using forms (e.g submissions from a web form) and to define tests that check internal code inserts/updates documents correctly (for which I highly recommend the fantastic pytest tool).

The code within this article is available in the MongoFrames repository within the snippets directory.