All code referenced on this django-tenant-schemas tutorial can be found here.

Introduction

Recently I was tasked with converting an existing Single Tenant API to a multi-tenant one. The API was created using the Django Rest Framework package and used the Postgres Database as a persistence layer, consequently, I settled on django-tenant-schemas after a few minutes of scouring the web for a solution.

django-tenant-schemas is a django based package that will enable you to perform the creation of client-specific schemas in a single Postgres DB instance, in addition, it’ll also aid in request routing ensuring data isolation is maintained, in other words, every client will only access data associated with their account. In short, it will help you convert the initially single-tenant API to a multi-tenant one with as minimal changes to your existing codebase as possible.

Glossary

  • Single Tenant – A software architecture in which a single instance of the software runs on a server and serves a single tenant.
  • Multi-Tenancy – A software architecture in which a single instance of the software runs on a server and serves multiple tenants.

Let’s get started

Prerequisites

  • Basic knowledge of how REST APIs work
  • Knowledge of Django and Django Rest Framework
  • Understanding of the basic functionality of Git and GitHub.
  • A running Postgres DB instance

Through this walk-through, you will be using an existing single-tenant API as the ‘test subject’. It’s a popular Django beginner tutorial app called the polls app, however, you will be using the API (REST) rendition of the app and include multi-tenancy on top of existing functionality. Find the final code in this GitHub repository.

Setting up PostgreSQL

Download and install Postgres software to your local machine, after that proceed to set it up as the default persistence layer for the API as opposed to the Django default SQLite.
Proceed to add below code to your setttings.py file.

Ignore the ENGINE value at the moment an explanation for it will be provided in due time. However, the remaining keys and their values should be familiar and self-explanatory. After that, you should add the database values to an env file as a security precaution. As such you should have a .env file on your root folder that resembles:

Django Tenant Schemas

Like explained before django-tenant-schemas is a django-based python package that will do most of the heavy lifting for us when it comes to restructuring our database architecture from single-tenant based to multi-tenant based. Install it by running:

pip install django-tenant-schemas

Most of the steps that follow have already been included in the django-tenant-schema documentation therefore we will not delve into details, feel free to peruse the documentation at your leisure. Make the following edits to the settings.py file:

DATABASE_ENGINE

Alter the DATABASE_ENGINE to tenant_schemas.postgresql_backend to ensure that django-tenant-schemas can automatically create schemas in our database for each tenant.

DATABASE_ROUTERS

You will also need to alter the DATABASE_ROUTERS tuple in the settings.py. Therefore, add the code below:

DATABASE_ROUTERS = (
    'tenant_schemas.routers.TenantSyncRouter',
)

MIDDLEWARE_CLASSES

In the default django-tenant-schemas implementation you should also add the tenant_schemas.middleware.TenantMiddleware to the top of the MIDDLEWARE_CLASSES list, so that each request can be set to use the correct schema. Thus your MIDDLEWARE_CLASSES list should resemble:

SHARED_APPS/TENANT_APPS/INSTALLED_APPS

It is important that you outline what apps are to be accessed publicly (SHARED_APPS) and ones that will be specific to tenants (TENANT_APPS). This will make it possible for django-tenant-schemas to save data accessed via said apps accordingly. INSTALLED_APPS is the default Django apps list and should remain as is but for one change tenant_schemas should be the placed at the top of the list:

SHARED_APPS

SHARED_APPS is a new tuple/list defined to indicate to django-tenant-schemas the apps intended for public use. The tuple/list should resemble:

TENANT_APPS

TENANT_APPS finally will be a tuple/list that indicates which apps are accessible only to tenants, thus individuals without a defined schema will not be able to access apps on this tuple/list. Existing tenants while accessing these apps will have their data routed and saved to their respective schemas. The tuple/list should resemble:

Tenant App

Create the tenant app by running the following code from your pollsapi root folder:

django-admin startapp tenant

Proceed to the models.py file in the newly created tenant app and add the following code:

The above represents the blueprint that will define our tenant. By default, django-tenant-schemas uses subdomains to detect the tenant and route the request accordingly, but in our case, since we only want to have a fixed URL for all our tenants we will be using a unique identifier (UUID) in our request headers to perform the task. Therefore, you will now proceed to add the custom middleware that will be responsible for routing our HTTP requests to the right tenant.

Custom Middleware

In the tenant app create a file called middleware.py and add the following code:

The above class inherits from django-tenant-schemas’s BaseTenantMiddleware subsequently overriding the get_tenant method where we add our own custom methodology for retrieving a tenant, we check the request headers for a unique UUID which we have attached to a field called X-Request-ID, if one is found we query the tenant model DB against it to identify the tenant if no tenant is found we return the public tenant giving the user the liberty to access apps that are allowed to the public.

We need to make changes to the settings.py file and point the tenant schema middleware to use our custom defined middleware as opposed to the default. We also need to define a new variable that points to our tenant model as a django-tenant-schemas prerequisite. In your settings.py add/alter the following parts, add the variable below. It points to your tenant model

TENANT_MODEL = 'tenant.Client'

Change the default tenant middleware from the MIDDLEWARE list to point to your newly created custom middleware, change:

'tenant_schemas.middleware.TenantMiddleware',

to

'pollsapi.apps.tenant.middleware.RequestIDTenantMiddleware',

We also include the newly created tenant app to the SHARED_APPS tuple/list as well as the INSTALLED_APPS lists:

Running migrations

We can now run migrations on our API to populate our database with the required tables. Run:

python manage.py makemigrations

The terminal response should resemble:

WARNINGS:
?: (tenant_schemas.W003) Your default storage engine is not tenant aware.
	HINT: Set settings.DEFAULT_FILE_STORAGE to 'tenant_schemas.storage.TenantFileSystemStorage'
Migrations for 'tenant':
  pollsapi/tenant/migrations/0001_initial.py
    - Create model Client

You can ignore the default storage warning, we will not be covering that and it should not affect our desired functionality. You can now proceed to run the migrate command. django-tenant-schemas has modified the migrate command in order to ensure that it runs on the correct schemas as specified in the settings.py file. Thus instead of running the usual migrate command, you should run:

NOTE: Never use migrate as it would sync all your apps to public!

python manage.py migrate_schemas

Since we are yet to create a tenant the command above should create the public schema only. We shall create a tenant next

Creating a tenant

We will create a custom Django commandline command in order to make the process of creating a tenant less hectic. Create a python package folder in the tenant app and label it management inside the folder create another one labelled commands and in it create a file called client.py and add below code:

The above code will enable us to create a tenant in a similar fashion we would a django superuser. Lets create our first tenant, run below command and populate the fields as prompted:

python manage.py client

It should look something similar to this:

Tenant name: Pollsmaster
Paid until: 2020-05-30
Schema name: pollsmaster
On trial: False

If the command is successful a new tenant, as well as their schema, should be created and the migrate command for the schema run successfully.

Finalizing on the tenant app

You will need to add a few things in the tenant app, before you can test out the functionality. In the tenant app create a serializer.py file and add below code:

In the views.py file add the following:

add a urls.py and add code below:

Testing everything out

Start up the server and head over to your favorite API testing tool. I’ll be using Insomnia.

Go to the endpoint localhost:8000/client to retrieve your tenant UUID. If successful you should receive a response of the tenant name and their UUID. Your request should look something similar to this:

Copy the tenant_uuid , we’ll use it to make requests to the polls app.

Head over to the create polls endpoint localhost:8000/polls/ add the tenant_uuid to the headers and assign it to X-Request-ID :

Calling the request above should create a new poll. To have a comparative demonstration you can create a new tenant and create multiple polls and observe the data isolation in practice.

Learning Strategy

While doing research for this project, I started off with trying to understand what exactly multitenancy was. On getting a high-level understanding of what multi-tenancy entailed the next step was deciding the methodology to consider in implementing multitenancy to the existing API. In that regard this article from Microsoft on the different implementations of multitenancy proved invaluable. Settling on a methodology entailed weighing the pros and cons which partially included, maintainability of the database, restructuring of the existing codebase, and, most importantly, how the API would scale with an increase in tenants.

In retrospect….

In depth research before writing the first line of code was the biggest take from the entire process. It was important to have a prior understanding of how extensive the codebase would have to be altered as well as whether the use of a third party package was necessary. It was also important to have an implementation plan, that is, a literal step by step plan to be undertaken in bringing the full implementation into fruition.

In conclusion

This particular project took longer than I had predicted. Having run a conservative estimate of 10 hours it took twice the time.

I hope this short example excites the reader into delving deeper into multi-tenancy architecture. Exploring the inner workings of Postgres was exciting and eye-opening. The example above can be extended to include tenant-specific authentication and authorization.

You can also find my other blogs, on pytest, on Apache and on SendGrid.

Tools for further study

Microsoft multi-tenancy article

PostgreSQL schemas documentation

Django Tenant Schemas documentation

Citations

The Featured Image is courtesy of wikimedia commons and can be found here

Get the code from GitHub on this link