Chandelier in Khor Virap, Armenia 2014.

# Django and Celery - demo application, part I: building.

## Introduction

While Django is one of the most popular web-development frameworks available for python language, as any application is may at some point grow in size and reach a stage, when a simple request-reponse strategy no longer remains sufficient. Celery has been known for offering a great flexibility when it comes to asynchroneous task scheduling, making it possible to process key tasks in the background, leaving the main application quick and responsive.

In this series of three posts:

we are going to build a simple application that demonstrates how Django and Celery can be used together for monitoring progress of a task, with incrementing of a counter given as an exmaple. Naturally, such task can be replaced with any other, perhaps more elaborate one, allowing to construct more sophisticated front-end monitoring systems. In addition, we will make it the way a user can retain freedom to move across the application without loosing track of its progress and abort it if necessary.

For consistency reasons, we will begin with creating of a Django project from scratch. This way, we will ensure that all dependencies and paths are correctly resolved for the sake of this work. This post can hence be considered as optional. However, if you are new to either Django or Celery (or both!), it is recommened that you take a look at it first. Otherwise, you may proceed to part II directly. In case you would like to focus on deployment (part III), it shows how to use this project in production environment, featuring:

• Gunicorn for running of the application,
• Supervisor for monitoring of the process,
• Nginx as a proxy server,
• RabbitMQ as a message broker between Django and Celery,

and all that using Ubuntu operating system.

You can also use the finished work on github.

## Creating the project

Before proceeding with the actual work, let us first define a virtual environment for the application to ensure that all packages are installed in a systematic manner. The name and place (path) to the virtual environment directory is arbitrary, but we will call it celery-env here and define it directly in our home folder.

It will allow us to install all the packages using pip. If you don’t have pip, have a look here (pip) and here (venv).

Next, we must activate the environment using:

An immediate sign of success is our prompt being prefixed with (celery-env), where simple python --version command should output Python 3.5.x, instead of e.g. 2.7. To deactivate the environment, just type deactivate.

Once activated, we use pip (not pip3) to install the relevant packages:

The project can be initialized in the following way:

The project is localized under /home/username/hello_celery/..., which we will call project root directory. Again, the exact location and name is arbitrary for as long as we are consistent.

### Basic project configuration

In order to run, we need to specify a few things.

First of all, we need to add a list of domains and/or IP numbers to the ALLOWED_HOSTS, inside hello_celery/hello_celery/settings.py. For development purposes, it is enough that we add '127.0.0.1', to run the project on the local machine.

Next, we should define STATIC_ROOT = os.path.join(BASE_DIR, 'static/') to be the last line in the same settings.py file, allowing django to collect static files when in “production”.

### Dry run

Before running, we need to perform database migration. This step we will frequently repeat, as soon as we modify of the application.

Note that being able to execute the commands using ./manage.py rather than python3 manage.py is a consequence of having activated virtual environment (as required).

Another thing is defining a project’s superuser, making it possible to access admin panel through http://127.0.0.1:8000/admin.

Finally, we can test our settings using runserver command.

It will run django development server using 127.0.0.1:8000, unless specified differently.

## Building the application

Now that we have the basics set up, we can begin with constructing of the actual application. As mentioned earlier, we will define a very basic task of incrementing a counter and display changing numbers in the front-end, with possibility to abort the task at any given time and freedom to move across the application without loosing track of its progress.

### Organizing the project

Here, the word “application” refers to django-specific concept of one or more separated services that belong to a project. Within our hello_celery project, we will define just one application and call it demo, for simplicity. Assuming, we still operate under virtual environment, we create the application using:

The application itself is operated using URLs that must be registered in urls.py files and defined in views.py. Most of our URLs will be specific to demo, which makes sense to register them under hello_celery/demo/urls.py. However, in order for them to be recognized within the whole project, we need to include them in the hello_celery/hello_celery/urls.py file. This file should look the following:

Note that the regular expression: r'^' will result in URL definitions of demo being directly ‘attached’ to the site domain name or IP address (e.g. http://localhost:8000/start). Although not the best practice, it should simplify the demonstration. Line #6 includes all the defninition from hello_celery/demo/urls.py, and this is exactly, where we are going to register them.

The file is not automatically generated. We need to create it first. Let’s do it and type in the following content:

In this file, we will soon define URL patterns for manipulating of the application.

Finally, we must register this application in the main settings.py file. We do this by adding 'demo' to the INSTALLED_APPS in this file.

As templates are usually specific to an applicaiton, their exact form is of less importance to the project. Here, we will define just two pages:

• test.html for demonstrating of functionality of Celery, and
• index.html to simulate the “rest” of the application.

In addition, we will include base.html with common content, such as navigation bar, and add basic Bootstrap theme to improve the visual quality. To integrate templates, let’s create the following directories and files:

Then, under hello_celery/demo/templates/demo, will define the three .html files, with the following content:

#### test.html

The templates are more or less defined. Before we proceed with integrating Celery into the applicaiton, we should make sure that the basic navigation between pages works fine. For that, we should define two URLs that will allow us to switch between the pages.

Let’s begin with defining the respective functions under hello_celery/demo/views.py.

#### views.py

Then, in hello_celery/demo/urls.py (we will simply refer to it as urls.py from now on), we define two respective URL patterns: