Automatic generation of xml from models using xslt

Most of the data being transferred to a flash frontend is in xml. This is both because xml is very well supported by Flash (e4x) and because hierarchical data is easily mapped to xml. Most data used for flash sited is hierarchical in nature, because the display list -flash it’s version of html’s DOM- is hierarchical as well.

What easymode tries to do is give you a basic hierarchical xml document that mirrors your database model, which you can then transform using xslt [1].

Why Xslt?

Xslt is a functional programming language, specifically designed to transform one type of xml into another. So if we can reduce django’s template rendering process to transforming one type of xml to another, xslt would be a dead on match for the job.

In fact we can. Easymode comes with a special xml serializer. This serializer differs from the normal django serializers, in that it treats a foreign key relation as a child parent relation. So while django’s standard serializers output is flat xml, easymode’s serializer outputs hierarchical xml.

Relations must be organized as a DAG

In order for easymode to be able to do it’s work, the model tree should be organised as a DAG. if you accidently created a cycle (using ManyToManyField relations), easymode will let you know and throw an exception. Any ManyToManyField that is related to “self” will be ignored by the serializer.

Most of the time you don’t really need the cyclic relation at all. You just need to do some preprocessing of the data. You can render a piece of xml yourself, without using easymode’s serializers and pass it to the xslt, see Injecting extra data into the XSLT.

Getting xml from a model

There are several ways to obtain such a hierarchical xml tree from a django model. The first is by decorating a model with the toxml() decorator:

from easymode.tree.xml.decorators import toxml

@toxml
class Foo(models.ModelAdmin):
    title = models.CharField(max_length=255)
    content = TextField()

class Bar(models.ModelAdmin):
    # use lazy foreign keys!
    # Even in the same models module!
    foo = models.ForeignKey('Foo', related_name=bars)

    label = models.CharField(233)

The Foo model has now gained a __xml__ method on both itself as on the queryset it produces. Calling it will produce hierarchical xml, where all inverse ForeignKey [2] relations are followed (easymode’s serializer follows the managers on a related model).

The preferred method for calling the __xml__ method is by it’s function:

from easymode.tree.xml import xml

foos = Foo.objects.all()
rawxml = xml(foos)

Getting xml from several queries

The next option, which can also be used with multiple queries, is use the XmlQuerySetChain

from easymode.tree.xml import xml
from easymode.tree.xml.query import XmlQuerySetChain

foos = Foo.objects.all()
qsc = XmlQuerySetChain(foos)
rawxml = xml(qsc)

Normally you would use the XmlQuerySetChain to group some QuerySet objects together into a single xml:

from easymode.tree.xml import xml
from easymode.tree.xml.query import XmlQuerySetChain

foos = Foo.objects.all()
hads = Had.objects.all()

qsc = XmlQuerySetChain(foos, hads)
rawxml = xml(qsc)

Using xslt to transform the xml tree

Now you know how to get the xml as a tree from the models, it is time to show how xslt can be used to transform this tree into something a flash developer can use for his application.

Easymode comes with one xslt template [3] that can give good results, depending on your needs:

from easymode.xslt.response import render_to_response

foos = foobar_models.Foo.objects.all()
return render_to_response('xslt/model-to-xml.xsl', foos)

The render_to_response() helper function will take an xslt as a template and a XmlQuerySetChain or a model/queryset decorated with toxml() to produce it’s output. Additionally you can pass it a dict containing xslt parameters. You have to make sure to use prepare_string_param() on any xslt parameter that should be passed to the xslt processor as a string.

Other helpers can be found in the easymode.xslt.response module.

When the standard serializer is not enough

It could be that the raw values are not what you want from your model. For example, it could be you wanted a datetime.datetime nicely formatted or maybe you want to have the output of get_absolute_url in your xml.

Fortunately you can do that by implementing your own serialization function in your model:

@I18n('value')
class TagModel(models.Model):

    value = models.CharField(max_length=23)

    def __serialize__(self, stream):
        stream.startElement('taggy', {})
        stream.characters(self.value)
        stream.endElement('taggy')

The __serialize__ method will be called by the serializer instead of the regular handler. You get the xml stream as an argument and you’ve got to write to it by calling the methods on it. see xml.sax.saxutils.XMLGenerator for the correct api.

The same is True when you’ve got custom fields that should do something with their values before you can use it (like storing data pickled). Implementing a __serialize__ method in you custom field will make the serializer use your implementation (for example SafeHTMLField implements it’s own __serialize__ method to validate the xml before writing it to the serializer stream).

Exclude certain relations from being followed by the serializer

Sometimes you want to have a simple foreign key relation, but you don’t need the data of the related object in your xml. As a matter of fact, it would degrade the performance of your view. You could mark the ForeignKey as serialize=False but that would make the field simply disappear from the xml. You might only want to exclude it from being rendered as a child of the parent object, but on the object itself you want the foreign key rendered as usual.

You can mark any foreign key with a nofollow attribute and it will not be followed, when the parent is serialized:

class Bar(models.ModelAdmin):
    foo = models.ForeignKey('Foo', related_name=bars)
    foo.nofollow = True

    label = models.CharField(233)

The foreign key will still be serialized as it always would, but the related object will not be expanded on the parent. This means if I query Foo:

xml(Foo.objects.all())

I will not see any Bar objects as children of foo. But when querying Bar:

xml(Bar.objects.all())

You will still see that the foreign key is included in the xml.


[1]Xslt requires a python xslt package to be installed. Easymode can work with lxml , libxslt
[2]

While ForeignKey relations are followed ‘inverse’ by the managers on the related model, this is not the case for ManyToManyField. Instead they are followed ‘straight’.

Easymode does not follow ‘straight’ foreigkey relations because that would cause a cycle, instead it only takes the value of the foreignkey, which is an integer. If you do need some data from the related object in your xml, you can define the natural_key method on the related model. The output of that method will become the value of the foreignkey, instead of an integer. This way you can include data from a ‘straight’ related model, without introducing cyclic relations.

[3]The default xslt, with template path: ‘xslt/model-to-xml.xsl’ comes with easymode and looks like this:
<?xml version="1.0"?>
<!-- 
  model-to-xml.xsl

  Created by Lars van de kerkhof on 2009-07-30.
-->
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output encoding="UTF-8" indent="yes" method="xml" />

  <!-- Render an object node -->
  <xsl:template match="object">
    <xsl:element name="{substring-after(@model,'.')}">
      <xsl:apply-templates select="field"/>
      <xsl:if test="object">
        <children>
          <xsl:apply-templates select="object"/>
        </children>
      </xsl:if>

    </xsl:element>
  </xsl:template>

  <!-- render a field node -->
  <xsl:template match="field">
    <xsl:if test="@type">
      <xsl:element name="{@name}">
        <xsl:copy-of select="@type"/>
        <xsl:copy-of select="@font"/>
        <xsl:value-of select="."/>
      </xsl:element>
    </xsl:if>
  </xsl:template>

  <!-- 
    ManyToManyField is only shown if not empty.
    ForeignKey is only shown if natural_key is defined on
    the related object.
    
    see http://packages.python.org/django-easymode/xslt/index.html#id2
  -->
  <xsl:template match="field[@to]">
    <xsl:if test="natural or object">
      <xsl:element name="{@name}">
        <xsl:copy-of select="@rel"/>
        <xsl:choose>
          <xsl:when test="count(natural) > 1">
            <xsl:for-each select="natural">
              <property><xsl:value-of select="."/></property>
            </xsl:for-each>
          </xsl:when>
          <xsl:otherwise>
            <xsl:value-of select="natural"/>
          </xsl:otherwise>
        </xsl:choose>
        <xsl:if test="object">
          <xsl:apply-templates select="object"/>
        </xsl:if>
      </xsl:element>
    </xsl:if>
  </xsl:template>

  <!-- just copy unmatched nodes -->
  <!-- so we know something is wrong -->
  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <!-- Parse the root node of the serialized xml -->
  <xsl:template match="django-objects">
    <root>
      <xsl:apply-templates/>
    </root>
  </xsl:template>

</xsl:stylesheet>