Multiple Django and Flask Sites with Nginx and uWSGI Emperor

Posted May 16, 2012

I like Python, a lot. I'm one of those people who would use it all the time if they could, especially for web development. Which is convenient, since I'm currently working on my own company.

One of the drawbacks of using Python, though, has always been getting it to play nicely with web servers. I've never been completely happy with the options out there: mod_python, mod_wsgi, FastCGI, and uWSGI. They've all had some instability, performance issue, or other problem.

But this past weekend, while doing some server upgrades, I came across a relatively new feature of uWSGI that seems to be the ideal solution for hosting Python sites: Emperor mode.

When you start uWSGI in Emperor mode, you give it a directory to watch that contains all of your uWSGI config files. If a new file is added, the emperor process reads it and spins up loyal worker processes to handle requests. If a config file is touched or modified, the emperor will gracefully restart the associated worker processes. And if a config file is removed, the emperor will kill the relevant workers.

Installation

First, you'll need to install the latest versions of Nginx and uWSGI. I'll assume an Ubuntu install, but you should be able to modify this if needed.

$ sudo apt-get install nginx uwsgi

If you're on a recent version of Ubuntu, this should get you Nginx 1.2 and uWSGI 1.04. If not, you might need to add the PPAs:

$ sudo add-apt-repository ppa:nginx/stable
$ sudo add-apt-repository ppa:uwsgi/release
$ sudo apt-get update
$ sudo apt-get install nginx uwsgi

Upstart and uWSGI

Next, you'll need an Upstart script for uWSGI:

/etc/init/uwsgi.conf

description "uWSGI"
start on runlevel [2345]
stop on runlevel [06]
respawn

env UWSGI=/usr/bin/uwsgi
env LOGTO=/var/log/uwsgi/emperor.log

exec $UWSGI --master --emperor /etc/uwsgi/apps-enabled --die-on-term --uid www-data --gid www-data --logto $LOGTO

You may need to create the appropriate directories:

$ sudo mkdir -p /var/log/uwsgi
$ sudo mkdir -p /etc/uwsgi/apps-available
$ sudo mkdir -p /etc/uwsgi/apps-enabled

Now you should be able to get uWSGI started:

sudo service uwsgi start

The emperor process's logs will go to /var/log/uwsgi/emperor.log, so if you run into any errors, check there.

Getting Flask Running

Now we need to set up the configuration for one of our sites. First we'll do a Flask site, since it covers most of what we need to know for Django, as well.

Let's assume the directory structure looks something like this:

/
  var/
    www/
      flaskapp/
        static/
        templates/
        venv/
        app.py
        models.py
        settings.py

(Where venv/ is the virtualenv for this app.)

Let's start with the uWSGI config.

/etc/uwsgi/apps-available/flaskapp.ini

[uwsgi]
# Variables
base = /var/www/flaskapp
app = app
# Generic Config
plugins = http,python
home = %(base)/venv
pythonpath = %(base)
socket = /var/www/run/%n.sock
module = %(app)
logto = /var/log/uwsgi/%n.log

Conveniently, uWSGI allows for variables in its config files. In this case, we're using three variables, %(base), %(app), and %n. We define base to be the root directory of our app, app to be the module name where the WSGI application is defined (uWSGI will expect it to be called application. If it's called something else you should specify callable = foo in the config.), and %n is already defined for us as the basename of the config file, in this case, flaskapp.

We'll enable that config in a bit, but before we do that, let's get the Nginx config ready.

/etc/nginx/sites-available/flaskapp

server {
    listen 80;
    server_name mygreatstartup.com;
    server_name www.mygreatstartup.com;
    root /var/www/flaskapp;

    location /static/ {
        alias /var/www/flaskapp/static/;
        include /etc/nginx/gzip.conf;
        expires 30d;
        access_log off;
    }

    location / {
        include uwsgi_params;
        uwsgi_pass unix:/var/www/run/flaskapp.sock;
    }
}

That should be all we need to get set up. uWSGI and Nginx are already running, so let's enable the app and the site.

$ sudo ln -s /etc/uwsgi/apps-available/flaskapp.ini /etc/uwsgi/apps-enabled/flaskapp.ini

The uWSGI Emperor will see the new file (a symlink to the real config file), spin up new processes, and point it at that config.

Next, we enable the Nginx site.

$ sudo ln -s /etc/nginx/sites-available/flaskapp /etc/nginx/sites-enabled/flaskapp
$ sudo service nginx restart

(Nginx requires a reload to pick up config changes.)

If the DNS entries are properly configured, we should start serving our new site.

Getting Django Running

Fortunately, getting Django to run under WSGI is very easy, so we just need a couple of tweaks. First, the uWSGI config.

/etc/uwsgi/apps-available/djangoapp.ini

[uwsgi]
# Variables
base = /var/www/djangoapp
app = wsgi
# Generic Config
plugins = http,python
home = %(base)/venv
pythonpath = %(base)
socket = /var/www/run/%n.sock
module = %(app)
logto = /var/log/uwsgi/%n.log

You'll notice that the only real change is that we've called our app wsgi. This is just a convention that seems to be prevalent with Django apps; the WSGI app is put in a file called wsgi.py inside of the project directory. A simple version would look like this.

/var/www/djangoapp/wsgi.py

import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Aside from replacing flaskapp with djangoapp in the rest of the config files above, everything else should work as specified.

Restarting Apps

If you ever need to restart a single WSGI app (Flask, Django, or anything else), which is usually necessary after any changes to the code, just run the following:

$ sudo touch /etc/uwsgi/apps-enabled/flaskapp

The emperor process will notice that the file has been touched and gracefully restart the process serving flaskapp.

That's it!

Comments are starting over at Hacker News

Update: Reader Jason emailed with a recipe for nested WSGI sites. In case you want to set something like that up, he's posted it on his blog.