some background
Diving experience in Maria la Gorda, Cuba 2015.

Dynamic Forms In Django

Intoduction

Django offers an easy way of automatically generating forms based on database models. In what follows, rendering a webpage with a form for submitting data to a server is straightforward. All one needs to do is to iterate through a form’s fields using the so-called liquid syntax.

In most of the situations, this approach is of course more than sufficient. However, imagine a situation, in which depending on choosing of a spcific parameter, we would like to dynamically generate a form, such that each time the scope of parameters accepted can be different. As long as the problem remains relatively small, we could for example:

  • Design a large(er) relational database capable of handling of each of the different cases.
  • Design one large table in a database and agree on a convetion that perhaps not all of its fields will be populated.

Obviously, the first option comes at a cost of increasing of the complexity, while the second will just make the database very “sparse”, but none of these two actually solves a problem.

How about the third possibility? We can create just one database field, to which we would commit all the parameters and values, irrespectively of their number and names. This can simply be done serializing all entries in a e.g. JSON form. In order to make it work, however, we will need dynamic forms. In this post, we will see how we can do it in Django.

Static Forms: Problem Outline

Before we move on to discussing the dynamic forms, let’s quickly recap on how to make static ones. Here, we will give a dummy example of cooking recipes and a form to accept quantities of the ingridients. If you feel like not wasting your time, you may jump directly to the dynamic forms section.

Let’s assume our model to be:

models.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

from django.db import models


class Ingridients(models.Model):
    # for hamburgers
    cheese  = models.IntegerField(default=0)
    ham     = models.IntegerField(default=0)
    onion   = models.IntegerField(default=0)
    bread   = models.IntegerField(default=0)
    ketchup = models.IntegerField(default=0)
    # for pancakes
    milk    = models.IntegerField(default=0)
    butter  = models.IntegerField(default=0)
    honey   = models.IntegerField(default=0)
    eggs    = models.IntegerField(default=0)
    # ...

class CookBook(models.Model):
    RECIPES = (
            (0, 'Hamburger'),
            (1, 'Pancake'))
    recipe_name = models.IntegerField(default=0,
            choices=RECIPES)
    ingridients = models.ForeignKey(Ingridients,
            on_delete=models.CASCADE)

Here, we create one table CookBook to be our main table, and link a second one with all possible ingridients. Each time, we choose of a particular dish, we will use a static form to populate only the relevant ingridients with respective quantities. For simplicity, we assume they can all be measured using integer numbers. We will not add any assertions either.

forms.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

from django import forms
from .models import Ingridients, CookBook

class HamburgerForm(forms.ModelForm):
    class Meta:
        model = Ingridients
        fields = ['cheese', 'ham', 'onion', 'bread', 'ketchup']

class PancakeForm(forms.ModelForm):
    class Meta:
        model = Ingridients
        fields = ['milk', 'butter', 'honey', 'eggs']

class CookBookForm(forms.ModelForm):
    class Meta:
        model = CookBook
        exclude = ['ingridients']

Next, for every recipe, we create a separate form that defines a set of dedicated fields.

views.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

## Dynamic forms demo
def nondynamic(request):
    context = {}

    ing = Ingridients.objects.last()
    if ing == None:
        ing = Ingridients.objects.create()

    ckb = CookBook.objects.last()
    if ckb == None:
        ckb = CookBook.objects.create(ingridients=ing)

    if 'recipe_name' in request.POST.keys():
        ckb.recipe_name = int(request.POST['recipe_name'])
        ckb.save()

        if request.POST['recipe_name'] == '0':
            ing_form = HamburgerForm(request.POST, instance=ing)
        elif request.POST['recipe_name'] == '1':
            ing_form = PancakeForm(request.POST, instance=ing)
        context['ingridients_form'] = ing_form
    else:
        recipe = ckb.recipe_name
        if recipe == 0:
            ing_form = HamburgerForm(request.POST, instance=ing)
        elif recipe == 1:
            ing_form = PancakeForm(request.POST, instance=ing)
        ing_form.save()
    
    context['ingridients_form'] = ing_form
    context['cookbook_form'] = CookBookForm(request.POST or None)
    return render(request, 'demo/nondynamic.html', context)

Again, for simplicity we will only rely on single entries in a database. Depending on the arguments passed by POST method, we let Django render unique forms and use them to assign only a subset of the parameters.

nondynamic.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

{% extends 'demo/base.html' %}
{% block title %}Dynamic Forms{% endblock %}
{% block dynamic_active %}active{% endblock %}
{% load widget_tweaks %}

{% block body %}
<div class="row">
  <div class="col-sm-12">
    <div class="page-header">
      <h2>Choose Recipe</h2>
      <form action="{% url 'demo:nondynamic' %}" class="form-hotizontal" method="POST">
        {% csrf_token %}
        {% for field in cookbook_form %}
        <div class="form-group row">
          <div class="col-sm-6">
            <label for="{{ field.id_for_label }}">{{ field.label }}</label>
              {{ field|add_class:'form-control' }}
          </div>
        </div>
        {% endfor %}
        <br>
        <div class="form-group row">
          <div class="col-sm-6">
            <div class="btn-group btn-group-justified" role="group">
              <div class="btn-group" role="group">
                <button class="btn btn-primary" type="submit">Choose Recipe</button>
              </div>
            </div>
          </div>
        </div>
      </form>
    </div>
    <h3>Specify Ingridients</h3>
    <form action="{% url 'demo:nondynamic' %}" class="form-hotizontal" method="POST">
      {% csrf_token %}
      {% for field in ingridients_form %}
      <div class="form-group row">
        <div class="col-sm-6">
          <label for="{{ field.id_for_label }}">{{ field.label }}</label>
            {{ field|add_class:'form-control' }}
        </div>
      </div>
      {% endfor %}
      <br>
      <div class="form-group row">
        <div class="col-sm-6">
          <div class="btn-group btn-group-justified" role="group">
            <div class="btn-group" role="group">
              <button class="btn btn-primary" type="submit">Specify Ingridients</button>
            </div>
          </div>
        </div>
      </div>
    </form>
  </div>
</div>
{% endblock %}

Finally, this is an exmaple of a template that makes things work. Most of it is Boostrap-based with a little extra bit python-widget-tweaks that helps to render things a bit nicer.

Depending on the selection of recipes, we have now two different forms for ingridients.

static forms example
Figure 1. Exmaple of static forms, whose rendering depends upon choice.

Now, imagine a situation, in which you would like to integrate more and more recipes. How much more complicated will models and views be? Do you see the problem?

This problem can, however, be circumvented combining two things:

  • Serializing all the ingridients into just a single field within CookBook.
  • Creating a flexible form that we can extend to capture all (and only) relevant parameters.

The first trick will simplify our database structure greatly. It will, however, require the second one to accept a variable number or parameters just such that our interface can always adapt to the situation.

Dynamic Forms

Before we begin, let’s redefine our database to a simplified form. We will leave our CookBook table, but instead of linking it to another, Ingridients, we simply store of all its key-value pairs in one field.

models.py

1
2
3
4
5
6
7
8
9
10

from django.db import models


class CookBook(models.Model):
    RECIPES = (
            (0, 'Hamburger'),
            (1, 'Pancake'))
    recipe_name = models.IntegerField(default=0,
            choices=RECIPES)
    ingridients = models.CharField(max_length=1024)

Having no dedicated table for Ingridients, we also simplify our forms.

forms.py

1
2
3
4
5
6
7
8
9
10
11

from django import forms
from .models import CookBook


class CookBookForm(forms.ModelForm):
    class Meta:
        model = CookBook
        exclude = ['ingridients']

class IngridientsForm(forms.Form):
    pass

Lines 5-8 are obvious, but there is a new thing. Lines 10-11 is exactly where the fun starts. Here, we create an “empty class” to be our container for the dynamic form. Note that while CookBookForm inherits from the forms.Model class, IngridientsForm inherits from a more generic forms.Form. This way we know it will have all the attributes and methods necessary for the forms to work, but nothing more. …and nothing more is necessary either. All we need to do now is to dynamically define the fields. Note, for this work we can reuse our nondynamic.html template as long as we change the URLs form url 'demo:nondynamic' to url 'demo:dynamic'.

views.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

from django.shortcuts import render
from .forms import CookBookForm, IngridientsForm
from .models import CookBook
import json

# ...

def dynamic(request):
    context = {}
    content = {}

    ckb = CookBook.objects.last()
    if ckb == None:
        ckb = CookBook.objects.create()

    if request.method == 'POST':
        if 'recipe_name' in request.POST:
            ckb.recipe_name = int(request.POST['recipe_name'])
            ckb.save()
            try:
                content = json.loads(ckb.ingridients)
            except json.JSONDecodeError:
                content = {}
        else:
            for key in request.POST.keys():
                if key != 'csrfmiddlewaretoken':
                    content[key] = request.POST[key]
            ckb.ingridients = json.dumps(content)
            ckb.save()
    
    if ckb.recipe_name == 0:
        new_fields = {
            'cheese': forms.IntegerField(),
            'ham'   : forms.IntegerField(),
            'onion' : forms.IntegerField(),
            'bread' : forms.IntegerField(),
            'ketchup': forms.IntegerField()}
    else:
        new_fields = {
            'milk'  : forms.IntegerField(),
            'butter': forms.IntegerField(),
            'honey' : forms.IntegerField(),
            'eggs'  : forms.IntegerField()}

    DynamicIngridientsForm = type('DynamicIngridientsForm',
            (IngridientsForm,),
            new_fields)

    IngForm = DynamicIngridientsForm(content)
    context['ingridients_form'] = IngForm
    context['cookbook_form']    = CookBookForm(request.POST or None)
    return render(request, "demo/dynamic.html", context)

This dynamic(...) function is rather long so “in production” it would certainly be plasuible to split it into smaller ones. Here, we are however only interested in the demonstration.

Two things are achieved by this function. First of all, we use json.loads() and json.dumps() methods for serializing and un-serializing the relevant content coming form the Ingridients form. It is as simple as that.

The key thing is, however, what is represented in lines 45-47. Here, we use type function in Python, which in addition to verifying of what is the type of a variable can be used to construct objects at a runtime. It is a powerful thing, since we can use it to generate a new class object on the go. The first argument is the name of the class itself. The second is a source. Finally, the third is for passing additional parameters. This is actually the whole secret. Knowing the case (ckb.recipe_name == ???) we can define a dictionary of field objects that we pass to the new class, which results in our dynamic definitions of fields.

Now, it may look like we overcomplicate things a bit. Why do we have this if-else statement in the first place? Shall we extend it every time a new recipe is added? Absolutely not. This is just to show the “freedom of creation”. With this approach, we can simply pass a dictionary of fields as function argument or, and this is what I would do, store each recipe interface in a separate field and dynamically import it into the views using importlib module.

With this approach, we create an extendable system, getting complitely rid of stiff forms and obtain the freedom for the fields to be altered in numerous ways, without having to access them as class’ attributes.