PG12 (aa) - a blog : a post

A Debain/NGNIX/Gunicorn setup for web.py

May 22, 2024 | web, python, programming

This post is part of a series of posts where I share my experiences developing, testing, and debugging the new implementation of my website, including this blog and my home page.

In this post, I will describe how I did the setup of a Debian Linux computer for my web.py web applications served by NGINX and Gunicorn. The same setup is used both on my live site hosted on Linode and on my OrbStack virtual developer machine.

The setup description is a step-by-step guide including all things you do from a freshly created Debian 12 Linux computer to a live website. The only steps done before we start this setup are creating the (virtual) Debian 12 server, configuring SSH access to this server, and, optionally, setting up the domain name service for the website pointing to this server. I will, in this setup, use two different accounts on the server: one user account for running the web applications called webuser and the superuser account root.

Initial configuration

Debian tends to complain if the LANG and LC_ALL environment variables are not set properly. To avoid this, as root, insert the following into the /etc/environment file:

LANG=en_US.UTF-8
LC_ALL=en_US.UTF-8

I prefer setting the time zone to my local time zone (Norway). To achieve this, do the following commands as root:

ln -fs /usr/share/zoneinfo/Europe/Oslo /etc/localtime
dpkg-reconfigure -f noninteractive tzdata

Install the software needed

I will run the site on a minimalistic Python-based web platform. The virtual Debian machine we have created comes with a lot of the necessary software (including Python 3), but we have to install a few other things not part of the default distribution. First, as root, ensure that the software on the computer is updated:

apt-get -y update
apt-get -y full-upgrade

Then, as root, install the necessary software:

apt-get -y install zip unzip
apt-get -y install nginx
apt-get -y install gunicorn
apt-get -y install python3-pip
apt-get -y install python3-venv
apt-get -y install uwsgi
apt-get -y install mariadb-server
apt-get -y install libmariadb-dev

We will use the NGINX open-source web server with the Gunicorn Python WSGI HTTP Server serving the Python-based web-platform. We need pip to install the Python modules we use and venv to run the site in a virtual (independent) Python environment. The SQL database we use is MariaDB.

Create a user account for the web applications

The next step is to create a user for web applications. The web applications will run in the user environment of this user and not as root to limit the privileges of this web application code. As root, create the user webuser, set a password for the user, add the user to the super-user group, and change the user's default shell to bash:

useradd -m webuser
passwd webuser
adduser webuser sudo
chsh --shell /bin/bash webuser

When this is done, we can start to do things on the Debian computer as the user webuser. When logged in as root, you can become the user webuser like this:

su - webuser

The commands following this will be done as the user webuser. The command logout (or Ctrl-D) will log out the current user, and you will be back as the super-user root. I recommend setting up a public-key-based (passwordless) SSH login for this user from your local (developer) computer. Then you can log in directly as the user webuser on the server from your local computer.

Prepare a virtual Python environment for the web applications

The next step is to prepare for the site's web applications. They will run in a virtual Python environment in the folder web of the user webuser. To prepare the virtual Python environment for the web applications, do the following commands as the user webuser from the home folder /home/webuser:

mkdir web
mkdir web/templates
cd web
python3 -m venv myweb-py
source myweb-py/bin/activate
pip3 install --upgrade pip
pip3 install web.py mariadb gunicorn …
deactivate

These commands create folders for the web applications and the templates. The virtual Python environment for the web applications and its folder myweb-py is created in the web applications folder web. The virtual Python environment is then activated to install the Python modules the web applications need (the modules listed here are just examples). In the final step, we deactivate the virtual Python environment. It will be reactivated by the running web applications.

Configure the web server

We have to create static web folders for images, CSS files, JavaScript code, and so on. For my site, as root I make the following folders:

mkdir -p /var/www/static
mkdir /var/www/static/css
mkdir /var/www/static/js
mkdir /var/www/static/icons
mkdir /var/www/images
mkdir /var/www/dist
chmod a+w /var/www/images

The dist folder is where I distribute other stuff part of the content of the site, including software and documents (PDF). The image folder is set to be writable by all since this is the folder where all images I publish are uploaded. In my case, images are uploaded to the server using MarsEdit when I publish or update posts with images.

A service file for each web application has to be installed. The service file specifies the Gunicorn instance serving a specific web application. The following file home.service, is an example:

[Unit]
Description=Gunicorn instance to serve my home page
After=network.target
[Service]
User=webuser
Group=www-data
WorkingDirectory=/home/webuser/web
Environment="PATH=/home/webuser/web/myweb-py/bin"
ExecStart=/home/webuser/web/myweb-py/bin/gunicorn --workers 3 --bind unix:home.sock -m 007 wsgihome:myhome
[Install]
WantedBy=multi-user.target

In this example, home from the wsgihome.py file represents the implementation of the specific web application (see the last part of the ExecStart line). We will below discuss this in more detail. The service file home.service should be copied to this path:

/etc/systemd/system/home.service

If I want to have another web application, for example, a blog, I will create a similar file blog.service in the same folder where all references to home are replaced by blog.

We will also create a web server setup for each web application. The home page, for example, should be accessed with the www prefix to the domain name:

server {
    server_name www.your-domain.something;
    location / {
        include proxy_params;
        proxy_pass http://unix:/home/webuser/web/home.sock;
    }
    location /static {
        alias /var/www/static;
        autoindex off;
    }
    location /images {
        alias /var/www/images;
        autoindex off;
    }
    location /dist {
        alias /var/www/dist;
        autoindex on;
    }
}

You should, of course, replaces your-domain.something with your real domain. When testing my site in OrbStack, I use test-debian.orb.local for the domain name. This configuration file, named home, should be placed at the following path:

/etc/nginx/sites-available/home

To enable it, we create a soft link to it from the sites-enabled folder:

ln -s /etc/nginx/sites-available/home /etc/nginx/sites-enabled/home

If you have more web applications, create a new NGINX server setup file for each. If we want a web application blog served with the prefix blog to the domain name, we create (and enable) a similar setup file for this web application. In this file, the www prefix will be replaced by blog and all references to home will be replaced by blog.

To make all this take effect, the services must be restarted. As root, perform the following commands:

systemctl restart nginx
systemctl daemon-reload
systemctl restart home

If you have other web applications than home (for example blog), just append the restart commands for them to this list of commands.

Install the web application files

Now, we are ready to install the actual web application files on the Debian server. In this minimal example, we will install three files:

home.py is the actual implementation of the web application.
home.html is the HTML-template for the web page.
wsgihome.py is the web server gateway for the web application.

The two files home.py and home.html are the actual implementation of the web application. The web server gateway implements the Web Server Gateway Interface (WSGI) for the application. The two Python files home.py and wsgihome.py should be installed in the working directory web of the user webuser. The HTML template should be installed in the template directory web/templates of the user webuser. To install the files from your developer computer using SSH, you can do this:

scp home.py wsgihome.py webuser@www.your-domain.something:web/
scp home.html webuser@www.your-domain.something:web/templates/

In a minimal example of a web.py web application, the content of the home.py could be the following:

import web
urls = (
    "/", "Home",
)
home = web.application(urls, globals())
render = web.template.render("templates")
class Home:
    def GET(self):
        return render.home(
            title="My home page",
            content="<p>A home page</p>")

This is an example of an HTML template for this web page using the Templetor template language:

$def with (title, content)
<h1>$title</h1>
$:content

The final piece in the puzzle is the web server gateway wsgihome.py for the web application:

from home import home
myhome = home.wsgifunc()

If you look back at the service file home.service, you will find how all this is connected (in particular, the ExecStart line). Now you should be able to access the web page in your browser:

http://www.your-domain.something/

However, one important thing is still missing. We should enable HTTPS (SSL/TLS) on our website.

Enable HTTPS (SSL/TLS) on the website

Let’s Encrypt is a free, automated, and open certificate authority (CA) run for the public’s benefit. To enable this for our site, we use Certbot, a free, open-source software tool from EFF for automatically using Let’s Encrypt certificates on websites. On the Certbot web page, we find very good instructions on how to enable this. It is recommended to use snap when you install the Certbot tool. To summarise, do the following commands as root on the server:

apt install snapd
snap install core
snap install --classic certbot
ln -s /snap/bin/certbot /usr/bin/certbot
certbot --nginx

The last command starts an interactive program prompting you for the following questions, just with much more text (explanations). I have also included my suggested answers in bold text and some comments at the end.

Enter email address: my@email.address (insert your email address)
Do you agree with the Terms of Service? Y (you have to agree on this to use it)
Would you be willing to share your email address with EFF? N (or Y, if you are OK with that)
Which names would you like to activate HTTPS for? 1,2 (choose all your domains; I had two)

The Certbot tool will now do all the setup for enabling HTTPS for the chosen domains. It will also set up a scheduled task to automatically renew the certificates in the background. The text presented by the Certbot tool when you are interacting with it explains this in much more detail. Please read all the text from the Certbot tool carefully.

A few other recommendations

Remove default from sites-enabled

When NGINX is installed on the server, a default home is created and enabled. There is no reason to continue serving this from your server. You should, as root, remove the soft link to this default web page in the sites-enabled folder:

rm /etc/nginx/sites-enabled/default

If the server is accessible at a public IP address, I recommend setting up public-key-based (passwordless) SSH login and turning off password-based login on your server. Immediately when a server is available on the internet (with a public IP address), it is overwhelmed with SSH password-based login attempts. If you have set up public-key-based SSH login and tested that it works (you can log in to your server from your local developer computer without a password), you can disable password login through the following two steps:

In the file /etc/ssh/sshd_config change the following options to this:

ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

Reload SSH with the new configuration:
```
systemctl reload ssh
```

Last updated: June 22, 2024