Setting up Django Project Production Ready
Deploy Django Application with PostgreSQL, Nginx, Gunicorn, Virtualenv and Supervisor
Deploy Django Application project for Production Environment
Deploy Django Application on Linux Machine
Video Tutorial
Django is a powerful python web framework build for rapid development. It takes care of much of problems of web development, so you can only focus on writing your app rather than reinventing the wheel. The most interesting thing about Django is that it is free and open source.
So lets begin with the deployment procedure for any Django based application. The recommended configuration for running a Django application is a bit complex. It includes tools like: PostgreSQL, Nginx, Gunicorn, Virtualenv, Supervisord. This are advanced tools that makes the Django application more efficient.
There are some prerequisites for this tutorial, you must be ready with:
- A Debian based Linux machine with sudo privileges.
- A domain name server that will point to your server’s IP.
Lets start the deployment process.
1. First start with the updating packages that are already available on server. So that server will be up to date. For this:
$ sudo apt-get update
$ sudo apt-get upgrade
2. Now, install postgreSQL and its required package.
$ sudo apt-get install postgresql postgresql-contrib
Lets create a database user and database for our Django application. For this first login as a postgres user, as by default only this user has privileges to create database in postgreSQL.
$ sudo su - postgres
Now, you are logged in as postgres user. lets create database user and assign necessary privileges to it.
postgres@ubuntu:~$ createuser --interactive -P
Enter name of role to add: db_user
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n
postgres@ubuntu:~$
In above example, database user name is db_user. You can give the name as per your wish. Create database now. Give a appropraite name to database as well, as per your Django application.
postgres@ubuntu:~$ createdb --owner db_user django_db
here db_user is a databse user and django_db is a database name. Now logout from postgres user.
postgres@ubuntu:~$ logout
$
3. Till now our database setup is done. Now create a virtual environment for our Django application. For this we need to install, virtualenv package.
$ sudo apt-get install python-virtualenv
Now create a virtual environment.
Note: I am going to keep my django application in default folder that is /home/ubuntu/, So i will create my virtual environment here. If you want to keep your application in some other directory like /var/www/ then change your directory accordingly.
$ virtualenv -p python3 django_env
New python executable in django_env/bin/python
Installing distribute..............done.
Installing pip.....................done.
Here djano_env is the virtual environment name. Change it as per you, if required. To activate this environment, run
$ source django_env/bin/activate
(django_env) $
Now, install all the requirements of your project here, including django. Like:
(django_env) $ pip install django
Downloading/unpacking django
(...)
Installing collected packages: django
(...)
Successfully installed django
Cleaning up...
4. Now, clone your project directory, if you have already created project. Here I am creating a sample project for demo.
(django_env) $ django-admin.py startproject sample_project
Here project name is sample_project. Now check whether all your project requirement is setup or not by running it on development server. For this first navigate into your project and run the server.
(django_env) $ cd sample_project
(django_env) $ python manage.py runserver 0.0.0.0:8000
Validating models...
0 errors found
Django version 1.10.1, using settings 'hello.settings'
Jan 09, 2017 - 06:12:00
Development server is running at 0.0.0.0:8000/
Quit the server with CONTROL-C.
Our project is running on Server’s public IP with port 8000. Now quit the server by pressing CONTROL-C.
5. Now, we need to configure postgreSQL so that it can communicate with our Django application. For this, install psycopg2 database adapter. But this adapter have some package dependencies, so first install them.
(django_env) $ sudo apt-get install libpq-dev python3-dev
Now install,
(django_env) $ pip install psycopg2
Now, configure database section in your project’s settings.py file.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'django_db',
'USER': 'db_user',
'PASSWORD': '<password you entered when creating db_user>',
'HOST': 'localhost',
'PORT': '', # Set to empty string for default.
}
}
Also add, your server’s IP or domain name in settings.py as follow:
ALLOWED_HOSTS = ['<your server's IP or domain name>']
Without above setting, Django will not be accessible from your domain name or IP address. This is the security feature of Django.
Now, synchronise the database
(django_env) $ python manage.py migrate
Now our database setup is fully done.
6. Now we will setup our app server that is Gunicorn rather than using Django’s by default single threaded development server.
(django_env) $ pip install gunicorn
Downloading/unpacking gunicorn
Downloading gunicorn-0.17.4.tar.gz (372Kb): 372Kb downloaded
Running setup.py egg_info for package gunicorn
Installing collected packages: gunicorn
Running setup.py install for gunicorn
Installing gunicorn_paster script to /webapps/hello_django/bin
Installing gunicorn script to /webapps/hello_django/bin
Successfully installed gunicorn
Installing gunicorn_django script to /webapps/hello_django/bin
Cleaning up...
Now you have gunicorn working, test it by running
(django_env) $ gunicorn sample_project.wsgi:application --bind 0.0.0.0:8001
Now, you can access gunicorn from your server’s public IP with port 8001. Now to make gunicorn more useful for our django application, we need to configure its option. For this make a bash script called gunicorn.bash. You can change file name as per your choice.
(django_env) $ vim gunicorn_start.bash
now add following configurations into file
#!/bin/bash
NAME="django_app" # Name of the application
DJANGODIR=/home/ubuntu/sample_project # Django project directory
SOCKFILE=/home/ubuntu/django_env/run/gunicorn.sock # we will communicte using this unix socket
USER=ubuntu # the user to run as
GROUP=ubuntu # the group to run as
NUM_WORKERS=3 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=sample_project.settings # which settings file should Django use
DJANGO_WSGI_MODULE=sample_project.wsgi # WSGI module name
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source /home/ubuntu/django_env/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec gunicorn ${DJANGO_WSGI_MODULE}:application
--name $NAME
--workers $NUM_WORKERS
--user=$USER --group=$GROUP
--bind=unix:$SOCKFILE
--log-level=debug
--log-file=-
Here, all the variables are well explained in their comment part respectively. Change the variables value according to your configuration. Like NAME variable will define how you application will be identified in programs like ps, top etc. The recommended way to define no of workers is equals to 2*CPUs+1. For example, for a single CPU machine should be set with 3 workers.
Now make this script executable.
$ sudo chmod u+x gunicorn_start.bash
Lets test this script by running it.
(django_env) $ ./gunicorn_start.bash
Starting hello_app as hello
2013-06-09 14:21:45 [10724] [INFO] Starting gunicorn 18.0
2013-06-09 14:21:45 [10724] [DEBUG] Arbiter booted
2013-06-09 14:21:45 [10724] [INFO] Listening at: unix:/webapps/hello_django/run/gunicorn.sock (10724)
2013-06-09 14:21:45 [10724] [INFO] Using worker: sync
2013-06-09 14:21:45 [10735] [INFO] Booting worker with pid: 10735
2013-06-09 14:21:45 [10736] [INFO] Booting worker with pid: 10736
2013-06-09 14:21:45 [10737] [INFO] Booting worker with pid: 10737
^C (CONTROL-C to kill Gunicorn)
2013-06-09 14:21:48 [10736] [INFO] Worker exiting (pid: 10736)
2013-06-09 14:21:48 [10735] [INFO] Worker exiting (pid: 10735)
2013-06-09 14:21:48 [10724] [INFO] Handling signal: int
2013-06-09 14:21:48 [10737] [INFO] Worker exiting (pid: 10737)
2013-06-09 14:21:48 [10724] [INFO] Shutting down: Master
$ exit
7. Now, its time to setup supervisor so that it can supervise our application. If system reboots or application quits unexpectedly, supervisor will take care of its restart. For this, first install it.
$ sudo apt-get install supervisor
To supervise any program through supervisor, you need to create configuration file for that program inside /etc/supervisor/conf.d/ directory. For our Django application that is sample_project, we will create sample_project.conf
$ sudo vim /etc/supervisor/conf.d/sample_project.conf
Now, write following content into the opened file.
[program:sample_project]
command = /home/ubuntu/gunicorn_start.bash ; Command to start app
user = ubuntu ; User to run as
stdout_logfile = /home/ubuntu/logs/gunicorn_supervisor.log ; Where to write log messages
redirect_stderr = true ; Save stderr in the same log
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8 ; Set UTF-8 as default encoding
Change the above configuration value according to your setup. As we mentioned in above file that the logs will be stored at /home/ubuntu/logs/gunicorn_supervisor.log, we need to make this directory and file.
(django_env) $ mkdir -p /home/ubuntu/logs/
(django_env) $ touch /home/ubuntu/logs/gunicorn_supervisor.log
After this done, we will ask supervisor to reread configuration files and update it so the our newly configuration file get add.
For Ubuntu 14.04:
$ sudo supervisorctl reread
sample_project: available
$ sudo supervisorctl update
sample_project: added process group
As you can see, our sample_project configuration file get added to supervisor process group. Now, start our app through it. For this
$ sudo supervisorctl start sample_project
sample_project: started
For Ubuntu 16.04:
$ sudo systemctl restart supervisor
$ sudo systemctl enable supervisor
To check status:
$ sudo supervisorctl status sample_project
sample_project RUNNING pid 24768, uptime 0:00:10
To stop:
$ sudo supervisorctl stop sample_project
sample_project: stopped
To restart:
$ sudo supervisorctl restart sample_project
sample_project: stopped
sample_project: started
Now, our application will get automatically restart after system gets boot up or our application gets crashed.
8. Now, we need to setup last thing that is Nginx. So lets get start. Nginx will act as a server for our application. So first install it.
$ sudo apt-get install nginx
Now, we will need to create configuration file for our application inside /etc/nginx/sites-available/ directory. After this we will need to create symbolic link for it, in the /etc/nginx/sites-enabled directory. Lets do it one by one. First create configuration file.
$ sudo vim /etc/nginx/sites-available/sample_project.conf
Now, put following content into the opened file.
upstream sample_project_server {
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response (in case the Unicorn master nukes a
# single worker for timing out).
server unix:/home/ubuntu/django_env/run/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name <your domain name>;
client_max_body_size 4G;
access_log /home/ubuntu/logs/nginx-access.log;
error_log /home/ubuntu/logs/nginx-error.log;
location /static/ {
alias /home/ubuntu/static/;
}
location /media/ {
alias /home/ubuntu/media/;
}
location / {
# an HTTP header important enough to have its own Wikipedia entry:
# http://en.wikipedia.org/wiki/X-Forwarded-For
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# enable this if and only if you use HTTPS, this helps Rack
# set the proper protocol for doing redirects:
# proxy_set_header X-Forwarded-Proto https;
# pass the Host: header from the client right along so redirects
# can be set properly within the Rack application
proxy_set_header Host $http_host;
# we don't want nginx trying to do something clever with
# redirects, we set the Host: header above already.
proxy_redirect off;
# set "proxy_buffering off" *only* for Rainbows! when doing
# Comet/long-poll stuff. It's also safe to set if you're
# using only serving fast clients with Unicorn + nginx.
# Otherwise you _want_ nginx to buffer responses to slow
# clients, really.
# proxy_buffering off;
# Try to serve static files from nginx, no point in making an
# *application* server like Unicorn/Rainbows! serve static files.
if (!-f $request_filename) {
proxy_pass http://sample_project_server;
break;
}
}
# Error pages
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/ubuntu/static/;
}
}
Change the configuration value according to your setup. Now create the symbolic link for it.
$ sudo ln -s /etc/nginx/sites-available/sample_project.conf /etc/nginx/sites-enabled/sample_project.conf
Now, start the Nginx.
$ sudo service nginx start
Now, if you browse your server’s domain name into browser, you will get your site running. Congrats, your production ready django application is set. 🙂
When i am running the command $ supervisorctl restart sample_project
its giving me error http://localhost:9001 refused connection
I am not getting whats wrong with this
Hello Saurabh,
This error basically arises when the specified port is not open or being used by some other process. Try to check it by issuing following command:
sudo netstat -nltp
This command will show on which port, which process is running. If the port is free then try to check status of your app that whether it is running or not. You can check it with:
sudo supervisorctl status sample_project
If it is showing, "running" then try to check, log file which is stored in ~/logs/gunicorn_supervisor.log, it will tell you why it is happening.
For more insights about the whole procedure, you can use video tutorial as well!
Let us know, if your problem still persist after applying above solutions.
Gunicorn Permission Denied error while running the bash script.
Hey, you probably have not given appropriate permission to file. Run following command mentioned in post:
sudo chmod u+x gunicorn_start.bash
Still, if you are getting same error then run:
sudo chmod ugo+x gunicorn_start.bash
This will solve your error. Let us know, if problem still persist.
Application not running… showing this error in NGINX Logs.
2017/04/04 13:07:25 [error] 2850#2850: *1 connect() failed (111: Connection refused) while connecting to upstream..
How to solve this error.
Complete Error in Log:
2017/04/04 13:19:30 [crit] 3121#3121: *1 connect() to unix:/home/ubuntu/django/djangoenvironment/run/gunicorn.sock failed (2: No such file or directory) while connecting to upstream, client: 192.168.0.100, server: <your, request: "GET / / HTTP/1.1", upstream: "http://unix:/home/ubuntu/django/djangoenvironment/run/gunicorn.sock:/"
still not working after applying above commands…my ubuntu user is mahesh and group is set to www-data..
But if i remove user and group it works fine.
Try to run following command:
sudo chown mahesh gunicorn_start.bash
then:
sudo chmod ugo+x gunicorn_start.bash
It should solve your problem.
Can you please give me the output of "/home/ubuntu/logs/gunicorn_supervisor.log"?
Thanks the problem has been solved..I forgot to restart the supervisor after making some changes.
Nice! Thanks for using this post. Keep learning, keep sharing.
I have a problem with supervisor: 404
(env) root@vz281558:~# systemctl start supervisor
(env) root@vz281558:~# supervisorctl tail -f sample_project
==> Press Ctrl-C to exit <==
unix:///var/run/supervisor.sock/logtail/sample_project/stdout Cannot read, status code 404
My files:
/etc/supervisor/conf.d/sample_project.conf
[program:sample_project]
command=/home/env/bin/gunicorn_start
user=root
stdout_logfile=/home/env/logs/gunicorn_supervisor.log
redirect_stderr=true
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8
/home/env/bin/gunicorn_start (I made this file as executable: chmod u+x gunicorn_start)
#!/bin/bash
NAME="myproject"
DJANGODIR=/home/env/myproject/
SOCKFILE=/home/env/myproject/run/gunicorn.sock
USER=root
GROUP=root
NUM_WORKERS=3
DJANGO_SETTINGS_MODULE=myproject.settings
DJANGO_WSGI_MODULE=myproject.wsgi
echo "Starting $NAME as `whoami`"
cd $DJANGODIR
source ../bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
exec /home/env/bin/gunicorn ${DJANGO_WSGI_MODULE}:application –name $NAME –workers $NUM_WORKERS –user=$USER –group=$GROUP –bind=unix:$SOCKFILE –log-level=debug –log-file=-
But when I run gunicorn_start directly: ./gunicorn_start I'm getting this:
https://pastebin.com/embed_iframe/FUb83yMd
Could you help me ?
Its look like that your `gunicorn_start` script is running fine. I wonder why you are running command `supervisorctl tail -f sample_project`.
Can you please provide the output of the file `/home/env/logs/gunicorn_supervisor.log`, It will help me look into your supervisor related issue.
Also try to run:
systemctl restart supervisor
then:
supervisorctl status sample_project
And let me know the what it gives the output.
PROBLEM SOLVED !!!
THANK YOU SO MUCH FOR THIS TUTORIAL !!! <3
Thanks for tut. But I was not able to load static files even after nginx config.
Same problem here…..
work perfect nice step by step explanation 🙂