.. _serialization-deserialization: Serialization and Deserialization ================================= Serialization is the act of converting application data into a form rendering. Deserialization is the act of converting data resulting from a form submission into application data. Serialization ------------- Serialization is what happens when you ask Deform to render a form given a :term:`schema`. Here's a high-level overview of what happens when you ask Deform to do this: - For each :term:`schema node` in the :term:`schema` provided by the application developer, Deform creates a :term:`field`. This happens recursively for each node in the schema. As a result, a tree of fields is created, mirroring the nodes in the schema. - Each field object created as a result of the prior step knows about its associated schema node (it has a ``field.schema`` attribute); each field also knows about an associated :term:`widget` object (it has a ``field.widget`` attribute). This widget object may be a default widget based on the schema node type or it might be overridden by the application developer for a particular rendering. - Deform passes an :term:`appstruct` to the root schema node's ``serialize`` method to obtain a :term:`cstruct`. The root schema node is responsible for consulting its children nodes during this process to serialize the entirety of the data into a single :term:`cstruct`. - Deform passes the resulting :term:`cstruct` to the root widget object's ``serialize`` method to generate an HTML form rendering. The root widget object is responsible for consulting its children nodes during this process to serialize the entirety of the data into an HTML form. If you were to attempt to produce a high-level overview diagram this process, it might look like this: .. code-block:: text appstruct -> cstruct -> form | | v v schema widget Peppercorn Structure Markers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You'll see the default deform widget "serializations" (form renderings) make use of :term:`Peppercorn` *structure markers*. Peppercorn is a library that is used by Deform. It allows Deform to treat the :term:`form controls` in an HTML form submission as a *stream* instead of a flat mapping of name to value. To do so, it uses hidden form elements to denote structure. Peppercorn structure markers come in pairs which have a begin token and an end token. For example, a given form rendering might have a part that looks like so: .. code-block:: xml :linenos: ... ... The above example shows an example of a pair of Peppercorn structure markers which begin and end a *mapping*. The example uses this pair to mean that the widget related to the *date* node in the schema will be be passed a :term:`pstruct` that is a dictionary with multiple values during deserialization. The dictionary will include the keys ``day`` , ``month``, and ``year``, and the values will be the values provided by the person interacting with the related form controls. Other uses of Peppercorn structure markers include a "confirm password" widget that can render a Peppercorn mapping with two text inputs in it, or a "mapping widget" that can serve as a substructure for a fieldset. Basically Peppercorn makes it more pleasant to deal with form submission data by pre-converting the data from a flat mapping into a set of mappings, sequences, and strings during deserialization. However, if a widget doesn't want to do anything fancy and a particular widget is completely equivalent to one form control, it doesn't need to use any Peppercorn structure markers in its rendering. .. note:: See the `Peppercorn documentation `_ for more information about using Peppercorn structure markers in HTML. Deserialization --------------- The following is a high-level overview of how "deserialization" (converting form control data resulting from a form submission to application data) works: - For each :term:`schema node` in the :term:`schema` provided by the application developer, Deform creates a :term:`field`. This happens recursively for each node in the schema. As a result, a tree of fields is created, mirroring the nodes in the schema. - Each field object created as a result of the prior step knows about its associated schema node (it has a ``field.schema`` attribute). Each field also knows about an associated :term:`widget` object (it has a ``field.widget`` attribute). This widget object may be a default widget based on the schema node type, or it might be overridden by the application developer for a particular rendering. - Deform passes a set of :term:`form controls` to the ``parse`` method of :term:`Peppercorn` in order to obtain a :term:`pstruct`. - Deform passes the resulting :term:`pstruct` to the root widget node's ``deserialize`` method in order to generate a :term:`cstruct`. - Deform passes the resulting :term:`cstruct` to the root schema node's ``deserialize`` method to generate an :term:`appstruct`. This may result in a validation error. If a validation error occurs, the form may be re-rendered with error markers in place. If you were to attempt to produce a high-level overview diagram of this process, it might look like this: .. code-block:: text formcontrols -> pstruct -> cstruct -> appstruct | | | v v v Peppercorn widget schema When a user presses the submit button on any Deform form, Deform itself runs the resulting :term:`form controls` through the ``peppercorn.parse`` method. This converts the form data into a mapping. The *structure markers* in the form data indicate the internal structure of the mapping. For example, if the form submitted had the following data: .. code-block:: xml :linenos: ... ... There would be a ``date`` key in the root of the pstruct mapping which held three keys: ``day``, ``month``, and ``year``. .. note:: See the `Peppercorn documentation `_ for more information about the result of the ``peppercorn.parse`` method and how it relates to form control data. The bits of code that are "closest" to the browser are called "widgets". A chapter about creating widgets exists in this documentation at :ref:`writing_a_widget`. A widget has a ``deserialize`` method. The deserialize method is passed a structure (a :term:`pstruct`) which is shorthand for "Peppercorn structure". A :term:`pstruct` might be a string, it might be a mapping, or it might be a sequence, depending on the output of ``peppercorn.parse`` related to its schema node against the form control data. The job of the deserialize method of a widget is to convert the pstruct it receives into a :term:`cstruct`. A :term:`cstruct` is a shorthand for "Colander structure". It is often a string, a mapping, or a sequence. An application eventually wants to deal in types less primitive than strings, such as a model instance or a datetime object. An :term:`appstruct` is the data that an application using Deform eventually wants to deal in. Therefore once a widget has turned a :term:`pstruct` into a :term:`cstruct`, the :term:`schema node` related to that widget is responsible for converting that cstruct to an :term:`appstruct`. A schema node possesses its very own ``deserialize`` method, which is responsible for accepting a :term:`cstruct` and returning an :term:`appstruct`. Raising Errors During Deserialization ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If a widget determines that a pstruct value cannot be converted successfully to a cstruct value during deserialization, it may raise an :exc:`colander.Invalid` exception. When it raises this exception, it can use the field object as a "scratchpad" to hold on to other data, but it must pass a ``value`` attribute to the exception constructor. For example: .. code-block:: python :linenos: import colander def serialize(self, field, cstruct, readonly=False): if cstruct is colander.null: cstruct = '' confirm = getattr(field, 'confirm', '') template = readonly and self.readonly_template or self.template return field.renderer(template, field=field, cstruct=cstruct, confirm=confirm, subject=self.subject, confirm_subject=self.confirm_subject, ) def deserialize(self, field, pstruct): if pstruct is colander.null: return colander.null value = pstruct.get('value') or '' confirm = pstruct.get('confirm') or '' field.confirm = confirm if value != confirm: raise Invalid(field.schema, self.mismatch_message, value) return value The schema type associated with this widget is expecting a single string as its cstruct. The ``value`` passed to the exception constructor raised during the ``deserialize`` when ``value != confirm`` is used as that ``cstruct`` value when the form is re-rendered with error markers. The ``confirm`` value is picked off the field value when the form is re-rendered at this time. Say What? --------- Q: "So Deform, Colander, and Peppercorn are pretty intertwingled?" A: "Colander and Peppercorn are unrelated; Deform is effectively something that integrates Colander and Peppercorn together."