Continuous integration of Django projects with Jenkins

Check out the new site at https://rkblog.dev.

Jenkins is a tool for watching or managing jobs. Either external like cronjobs or executing "build and test" jobs of your Django projects. There is even more, but in this article I'll focus on a basic Jenkins setup - building and testing a Django project. For Django there is django-jenkins that allows an easy integration with Jenkins tools like nice visualization of code coverage, pep8/pylint/pyflakes code violations.

In this article I'll setup a local Jenkins configuration that will look on a folder with the code. In real world Jenkins would watch a repository and execute a job when a new changeset shows up.

Jenkins

On Jenkins web page we can find packages for various OS. We can also download a WAR file for local execution (you will need a JAVA JRE and maybe few other dependencies).

  • Download the WAR file.
  • From a terminal launch it with
    java -jar jenkins.war
  • Jenkins will start and will be available at http://localhost:8080/
Jenkins after star

Jenkins after start with my first job configuration

The main part of the page are jobs (none at first run). On the left we have a menu with few handy elements. "New" can be used to add new jobs. "Manage Jenkins" allows configuring Jenkins and also installing plugins - and we need few for django-jenkins.
Plugin instalation in Jenkins

Plugin instalation in Jenkins

For django-jenkins we need two plugins: Violations and Cobertura. For local tests also File System SCM plugin will be useful. After installation you may need to restart Jenkins for them to appear as installed. If the plugin list is empty try to download it by going through tabs or hitting the install button.

Creating a job for a Django project

Click on "New" and create new job. Enter a name and choose "Build a free-style software project". If you already have a similar job you can copy it too.
Adding a new job in Jenkins

Adding a new job in Jenkins

Next we configure it:
Enter a path to a folder with a Django project

Enter a path to a folder with a Django project

Usually Jenkins would look on a repository for changes, but using File System SCM we can use a plain local folder with a Django project. Good for playing with Jenkins.
Build triggers configuration

Build triggers configuration

Poll SCM will make Jenkins check the given repository for any changes since last build. If there will be new changes it will launch the job. The */30 * * * * means that it will check every 30 minutes.
Build instructions

Build instructions

Usually for build instructions we would give dependency installation and then run all tests we need. In this case I have everything installed on the local system so I just launch tests. manage.py jenkins is a part of django-jenkins (more on that later). It will launch tests and create reports files used by Jenkins.
Code coverage reports configuration

Code coverage reports configuration

Cobertura plugin can generate code coverage reports. Set reports/coverage.xml as the path for report. Code executions statistics are configured like so:
Test execution configuration

Test execution configuration

Just set reports/junit.xml as the path. Violations pluging will allow yo visualize pep8, pylint/pyflakes violations:
Violations plugin configuration

Violations plugin configuration

Jenkins is configured. We just need to add django-jenkins to that project and it will work - Jenkins will build the applications and create reports.

Django-Jenkins

We can install it the usual way:
pip install django-jenkins
Then we add it to INSTALLED_APPS:
django_jenkins
Next we define a list of tasks for django-jenkins. The base ones are:
JENKINS_TASKS = (
        'django_jenkins.tasks.with_coverage',
        'django_jenkins.tasks.django_tests',
        'django_jenkins.tasks.run_pep8',
        'django_jenkins.tasks.run_pyflakes',
    )
We also need to install other python packages: pep8, pylint and pyflakes. After that you can test everything executing manage.py jenkins. If it succeeds then the project is configured and ready for action with Jenkins.

By default tests of Django and everything in INSTALLED_APPS will be launched. That's not the best thing for the reports. We can define PROJECT_APPS that will hold a list of our apps for for testing. For example:

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.admin',
    'javascript_settings',
    'django_jenkins'
)
PROJECT_APPS = (
    'jsdemo',
    'testapp',
)
INSTALLED_APPS += PROJECT_APPS

What Jenkins can tell us?

A list of builds of a Jenkins job

A list of builds of a Jenkins job

In our test configuration Jenkins will check every 30 minutes for changes in the project folder. If there will be any it will launch a build. From a build we can get a lot of useful informations. The main information - did the build succeed. We can also check:
  • Console: log from the executed tasks. Sometimes it may fail at downloading packages from pypi. You can retrigger a task to repeat it when pypi is online again.
  • Coverage Report: reports about code coverage with visualization of branches/conditionals coverage.
  • Violations: This will cover pep8, pylint/pyflakes violations.
Pylint/Pyflakes violation - not used import
Some pep8 violations - hover over the exclamation icon to get more info

Code coverage report

Build log

You will find more info on django-jenkins Tutorial and Continuous Integration with Jenkins.

RkBlog

Django web framework tutorials, 23 July 2012


Check out the new site at https://rkblog.dev.
Comment article
Comment article RkBlog main page Search RSS Contact