In the last article, Fuel Consumption API using database was created. All business requirements are met, now it’s time to go on the production! To be on time with current trends, Fuel Consumption Api will run in Docker container.

Docker

„Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications, whether on laptops, data center VMs, or the cloud.” Docker – the buzzword of current it world. Containerization has become a standard. The list of products, which used containers is very long. Examples of the widely recognized companies:

• PayPal
• Business Insider
• Splunk
• Groupon
• Uber
• Spotify

In comparison to the VMs:

• Containers are smaller – Docker images are small, e.g. ubuntu image size is 64MB
• Fast startup – containers starting is really quick. In most cases it takes seconds.
• Environment consistency – each container build on the base of the same Dockerfile, will work exactly the same on each machine. Docker is providing infrastructure as a code.
• Less resource consumption – Docker containers are sharing the same kernel, so there is no need to virtualize whole system and all components.
• Public containers registry – docker has public registry of containers. It’s easy to find base container with all required dependencies to speed up development.

It’s time to check how containerization looks in practice.

Prerequisite

1. Installed Python 3.x
2. Code editor of your choice (e.g. PyCharm, Visual Studio Code)
3. Installed Docker

Basic application

As the base for process of creating docker containers Fuel Consumption Api created in previous articles will be used.

Code

from flask import Flask
from flask_restful import Resource, Api, abort, reqparse
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+mysqlconnector://root:test@localhost:3306/fuel'
db = SQLAlchemy(app)
api = Api(app)

API_VERSION = 'v1.0'
URL_PREFIX = '/fuelConsumption/api/' + API_VERSION


class FuelConsumptionRecord(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    odometer = db.Column(db.Float, nullable=False)
    fuelQuantity = db.Column(db.Float, nullable=False)

    def serialize(self):
        return {
            'id': self.id,
            'odometer': self.odometer,
            'fuelQuantity': self.fuelQuantity
        }


parser = reqparse.RequestParser(bundle_errors=True)
parser.add_argument('odometer', type=float, required=True, help="odometer is required parameter!")
parser.add_argument('fuelQuantity', type=float, required=True, help="fuelQuantity is required parameter!")


def calculate_consumption(fuel_quantity, distance):
    return fuel_quantity / distance * 100


def calculate_distance(start_distance, end_distance):
    return end_distance - start_distance


def get_record_by_order_desc(records_limit, record_order):
    return db.session.query(FuelConsumptionRecord).order_by(FuelConsumptionRecord.id.desc()).limit(records_limit)[record_order]


class FuelConsumptionList(Resource):
    def get(self):
        records = FuelConsumptionRecord.query.all()
        return [FuelConsumptionRecord.serialize(record) for record in records]

    def post(self):
        args = parser.parse_args()
        fuel_consumption_record = FuelConsumptionRecord(odometer=args['odometer'], fuelQuantity=args['fuelQuantity'])
        db.session.add(fuel_consumption_record)
        db.session.commit()
        return FuelConsumptionRecord.serialize(fuel_consumption_record), 201


class FuelConsumption(Resource):
    def get(self, record_id):
        return FuelConsumptionRecord.serialize(
            FuelConsumptionRecord.query.filter_by(id=record_id)
                .first_or_404(description='Record with id={} is not available'.format(record_id)))

    def delete(self, record_id):
        record = FuelConsumptionRecord.query.filter_by(id=record_id)\
            .first_or_404(description='Record with id={} is not available'.format(record_id))
        db.session.delete(record)
        db.session.commit()
        return '', 204

    def put(self, record_id):
        args = parser.parse_args()
        record = FuelConsumptionRecord.query.filter_by(id=record_id)\
            .first_or_404(description='Record with id={} is not available'.format(record_id))
        record.odometer = args['odometer']
        record.fuelQuantity = args['fuelQuantity']
        db.session.commit()
        return FuelConsumptionRecord.serialize(record), 201


class LastFuelConsumption(Resource):
    def get(self):
        last_record = get_record_by_order_desc(1, 0)
        second_last_record = get_record_by_order_desc(2, 1)
        distance = calculate_distance(second_last_record.odometer, last_record.odometer)
        consumption = round(calculate_consumption(last_record.fuelQuantity, distance), 2)
        return {'lastFuelConsumption': consumption}


class AverageFuelConsumption(Resource):
    def get(self):
        records_count = FuelConsumptionRecord.query.count()
        sum_of_consumptions = 0
        for i in reversed(range(0, records_count)):
            start_record = get_record_by_order_desc(records_count, i-1)
            end_record = get_record_by_order_desc(records_count, i-2)
            distance = calculate_distance(start_record.odometer, end_record.odometer)
            sum_of_consumptions += calculate_consumption(end_record.fuelQuantity, distance)
        avg_consumption = round(sum_of_consumptions / (records_count - 1), 2)
        return {"avgFuelConsumption": avg_consumption}


api.add_resource(FuelConsumptionList, URL_PREFIX + '/recordList',
                                      URL_PREFIX + '/')
api.add_resource(FuelConsumption, URL_PREFIX + '/record/<record_id>')
api.add_resource(LastFuelConsumption, URL_PREFIX + '/calculateLastConsumption')
api.add_resource(AverageFuelConsumption, URL_PREFIX + '/calculateAverageConsumption')

if __name__ == '__main__':
    app.run(host='0.0.0.0')

Code description

The application code is almost the same as in the previous article. Mainly configuration changes are introduced to the application.

1. Prefix to endpoint URL:

URL:API_VERSION = 'v1.0' URL_PREFIX = '/fuelConsumption/api/' + API_VERSION ... api.add_resource(FuelConsumptionList, URL_PREFIX + '/recordList', URL_PREFIX + '/') api.add_resource(FuelConsumption, URL_PREFIX + '/record/<record_id>') api.add_resource(LastFuelConsumption, URL_PREFIX + '/calculateLastConsumption') api.add_resource(AverageFuelConsumption, URL_PREFIX + '/calculateAverageConsumption') 

To make maintenance of the API easier, URL prefix with api version is added. It’s a good practice to add it, as multiple versions of the api can be hosted and operated simultaneously.

2. Application host change python if __name__ == '__main__': app.run(host='0.0.0.0') To make Flask application visible externally (outside localhost) application host needs to be changed to '0.0.0.0′ In case of running application inside container, it will be not possible to access application.

Dockerizing of the Fuel Consumption Api

Code changes are applied to code, application is ready to be dockerized.

Create project structure:

flask-restful-docker
     source-code/
       flask-restful-docker
    source-code/
        docker-api/
            fuel-consumption-api/
                fuel-consumption-api.py
                requirements.txt
            Dockerfile
        docker-db/
            sql_scripts/
                create_db_with_table.sql
                insert_data.sql
            Dockerfile
    docker-compose.yml

Dockerfile for Flask application

source-code/docker-api/Dockerfile

FROM python:3

COPY ./fuel-consumption-api /app
WORKDIR /app

RUN pip3 install --no-cache-dir -r requirements.txt
ENV FLASK_APP fuel-consumption-api

ENTRYPOINT ["flask"]
CMD ["run", "--host=0.0.0.0"]

EXPOSE 5000

1. FROM python:3 – As the base container python:3 is used – it’s image with latest version of python 3.x
2. COPY ./fuel-consumption-api /app – fuel-consumption-api directory is copied to the container under /app directory
3. WORKDIR /app – Main work directory is set to /app as it’s the directory, where all coments are executed
4. RUN pip3 install --no-cache-dir -r requirements.txt – Install all dependencies from requirements.txt file
5. ENV FLASK_APP fuel-consumption-api – Set FLASK_APP environment variable to fuel-consumptio-api
6. ENTRYPOINT ["flask"] – Entrypoint set to flask – configures container to run as an executable
7. CMD ["run", "--host=0.0.0.0"] – provides defaults for an executing container
8. EXPOSE 5000 – expose port 5000 – port 5000 can be mapped to be achievable from outside container

Build flask application container

docker build . --tag j‑labs/fuel-consumption-app:1.0

Run flask application container

docker run -d -p 5001:5000 --name fuel-consumption j‑labs/fuel-consumption-app:1.0

Dockerfile for database with DB initialization

source-code/docker-db/Dockerfile

FROM mariadb:latest

COPY ./sql_scripts/ /docker-entrypoint-initdb.d/

In the above Dockerfile, standard MariaDB database image is expanded with copy of sql_scripts to directory inside container, from which all scripts will be executed on container run. Scripts will be executed in alphabetical order. MariaDB is a database on GPLv2 license. It’s a fork of MySQL. Both databases are mostly compatible with each other.

source-code/docker-db/sql_scripts/create_db_with_table.sql

create database fuel;
use fuel;

CREATE TABLE fuel_consumption_record
(
id INTEGER AUTO_INCREMENT,
odometer FLOAT,
fuelQuantity FLOAT,
PRIMARY KEY (id)
) COMMENT='table for recording fuel consumption records';

source-code/docker-db/sql_scripts/insert_data.sql

INSERT INTO fuel.fuel_consumption_record (odometer, fuelQuantity) VALUES (0.0, 0.0);
INSERT INTO fuel.fuel_consumption_record (odometer, fuelQuantity) VALUES (100.0, 12.5);

Build db container with initialized database and test data

docker build -t mysql-initialized-fuel:1.0 .

Run container with database

Run container from customized image with preloaded startup sql scripts.

docker run --name mysql-fuel -e MYSQL_ROOT_PASSWORD=test\ -d -p 3306:3306 -v /var/container_data/mysql:/var/lib/mysql  mysql-initialized-fuel:1.0 

There is also an option to run standard MariaDB container with parameter to create database.

Step with build container is not required. In that case there is a need to create table structure and insert test data either manually or with use with SQL Alchemy tools.

docker run --name mysql-fuel -e MYSQL_ROOT_PASSWORD=test -e MYSQL_DATABASE=fuel \ -d -p 3306:3306 -v /var/container_data/mysql:/var/lib/mysql  mariadb:latest 

PhpMyAdmin

PhpMyAdmin is an administration tool of MySQL database over the Web. With use of it, it’s possible to investigate database structure and data.

5. Run phpmyadmin and link to the db

docker run --name phpmyadmin -d --link mysql-fuel:db -p 8081:80 phpmyadmin/phpmyadmin

PhpMyAdmin application will be available under http://localhost:8081 url. To login use login: root and password: test. Due to use of parameter --link mysql-fuel:db there is no need to perform any additional configuration.

Maintainable Flask application project structure

Dockerized application is working as expected.

Fuel consumption application is an one file app. Taking under consideration maintenance and further development it’s not the most comfortable solution. It’s perfect time for refactor of project structure.

New project structure

fuelapp
    __init__.py
    app.py                          # app and routes
    requirements.txt                # required external python modules
    common/
        __init__.py
        calculation_util.py         # methods for calculating fuel consumption metrics
        retrieve_record_util.py     # methods for parsing record requests data
    models/
        __init__.py
        fuel_consumption_record.py  # declaration of data model
    resources/ 
        __init__.py
        avg_fuel_consumption.py     # contains logic for average fuel consumption endpoint /calculateAverageConsumption
        fuel_consumption.py         # contains logic for fuel consumption endpoint /record
        fuel_consumption_list.py    # contains logic for fuel consumption list endpoint /recordList
        last_fuel_consumption.py    # contains logic for last fuel consumption endpoint /calculateLastConsumption

app.py – contains application and routes

from flask import Flask
from flask_restful import Api

from fuelapp.resources.fuel_consumption import FuelConsumption
from fuelapp.resources.fuel_consumption_list import FuelConsumptionList
from fuelapp.resources.avg_fuel_consumption import AverageFuelConsumption
from fuelapp.resources.last_fuel_consumption import LastFuelConsumption
from fuelapp.models.fuel_consumption_record import db

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+mysqlconnector://root:test@database/fuel'

api = Api(app)
db.init_app(app)
API_VERSION = 'v1.0'
URL_PREFIX = '/fuelConsumption/api/' + API_VERSION


api.add_resource(FuelConsumptionList, URL_PREFIX + '/recordList',
                 URL_PREFIX + '/')
api.add_resource(FuelConsumption, URL_PREFIX + '/record/<record_id>')
api.add_resource(LastFuelConsumption, URL_PREFIX + '/calculateLastConsumption')
api.add_resource(AverageFuelConsumption, URL_PREFIX + '/calculateAverageConsumption')

if __name__ == '__main__':
    app.run(host='0.0.0.0')

Below line is changed due to mapping configured in docker-compose.yml file. More information in chapter Docker-compose for Fuel Consumption Application with database and PhpMyAdmin.

app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+mysqlconnector://root:test@database/fuel'

common/calculation_util.py – helper methods for calculating consumption

def calculate_consumption(fuel_quantity, distance):
    return fuel_quantity / distance * 100


def calculate_distance(start_distance, end_distance):
    return end_distance - start_distance

common/retrieve_record_util.py – helper method for getting record from db with given order

from fuelapp.models.fuel_consumption_record import db
from fuelapp.models.fuel_consumption_record import FuelConsumptionRecord

def get_record_by_order_desc(records_limit, record_order):
    return db.session.query(FuelConsumptionRecord).order_by(FuelConsumptionRecord.id.desc()).limit(records_limit)[record_order]

models/fuel_consumption_record.py – declaration of FuelConsumptionRecord data model

from flask_sqlalchemy import SQLAlchemy

db = SQLAlchemy()

class FuelConsumptionRecord(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    odometer = db.Column(db.Float, nullable=False)
    fuelQuantity = db.Column(db.Float, nullable=False)

    def serialize(self):
        return {
            'id': self.id,
            'odometer': self.odometer,
            'fuelQuantity': self.fuelQuantity
        }

Creation of SQLAlchemy object is moved to the fuelconsumptionrecord.py. It’s done to avoid circular imports issue. Initialization of db is still done in app.py.

resources/avg_fuel_consumption.py – contains logic for AverageFuelConsumption endpoint

from flask_restful import Resource
from fuelapp.models.fuel_consumption_record import db, FuelConsumptionRecord
from fuelapp.common.retrive_record_util import get_record_by_order_desc
from fuelapp.common.calculation_util import calculate_consumption, calculate_distance


class AverageFuelConsumption(Resource):
    def get(self):
        records_count = FuelConsumptionRecord.query.count()
        sum_of_consumptions = 0
        for i in reversed(range(0, records_count)):
            start_record = get_record_by_order_desc(records_count, i-1)
            end_record = get_record_by_order_desc(records_count, i-2)
            distance = calculate_distance(start_record.odometer, end_record.odometer)
            sum_of_consumptions += calculate_consumption(end_record.fuelQuantity, distance)
        avg_consumption = round(sum_of_consumptions / (records_count - 1), 2)
        return {"avgFuelConsumption": avg_consumption}

resources/fuel_consumption.py – contains logic for FuelConsumption endpoint

from flask_restful import Resource, reqparse
from fuelapp.models.fuel_consumption_record import db, FuelConsumptionRecord

parser = reqparse.RequestParser(bundle_errors=True)
parser.add_argument('odometer', type=float, required=True, help="odometer is required parameter!")
parser.add_argument('fuelQuantity', type=float, required=True, help="fuelQuantity is required parameter!")


class FuelConsumption(Resource):
    def get(self, record_id):
        return FuelConsumptionRecord.serialize(
            FuelConsumptionRecord.query.filter_by(id=record_id)
                .first_or_404(description='Record with id={} is not available'.format(record_id)))

    def delete(self, record_id):
        record = FuelConsumptionRecord.query.filter_by(id=record_id)\
            .first_or_404(description='Record with id={} is not available'.format(record_id))
        db.session.delete(record)
        db.session.commit()
        return '', 204

    def put(self, record_id):
        args = parser.parse_args()
        record = FuelConsumptionRecord.query.filter_by(id=record_id)\
            .first_or_404(description='Record with id={} is not available'.format(record_id))
        record.odometer = args['odometer']
        record.fuelQuantity = args['fuelQuantity']
        db.session.commit()
        return FuelConsumptionRecord.serialize(record), 201

resources/fuel_consumption_list.py – – contains logic for FuelConsumptionList endpoint

from flask_restful import Resource, reqparse
from fuelapp.models.fuel_consumption_record import db, FuelConsumptionRecord

parser = reqparse.RequestParser(bundle_errors=True)
parser.add_argument('odometer', type=float, required=True, help="odometer is required parameter!")
parser.add_argument('fuelQuantity', type=float, required=True, help="fuelQuantity is required parameter!")


class FuelConsumptionList(Resource):
    def get(self):
        records = FuelConsumptionRecord.query.all()
        return [FuelConsumptionRecord.serialize(record) for record in records]

    def post(self):
        args = parser.parse_args()
        fuel_consumption_record = FuelConsumptionRecord(odometer=args['odometer'], fuelQuantity=args['fuelQuantity'])
        db.session.add(fuel_consumption_record)
        db.session.commit()
        return FuelConsumptionRecord.serialize(fuel_consumption_record), 201

resources/last_fuel_consumption.py – contains logic for LastFuelConsumption endpoint

from flask_restful import Resource
from fuelapp.common.retrive_record_util import get_record_by_order_desc
from fuelapp.common.calculation_util import calculate_consumption, calculate_distance


class LastFuelConsumption(Resource):
    def get(self):
        last_record = get_record_by_order_desc(1, 0)
        second_last_record = get_record_by_order_desc(2, 1)
        distance = calculate_distance(second_last_record.odometer, last_record.odometer)
        consumption = round(calculate_consumption(last_record.fuelQuantity, distance), 2)
        return {'lastFuelConsumption': consumption}

Docker-compose for Fuel Consumption Application with database and PhpMyAdmin

Compose is a tool to define and run multi-container Docker applications. Application containers are defined in yaml file. With correctly created docker-compose.yaml only one command is needed to run all related containers.

Code – docker-compose.yml

version: '3'
services:
  fuel-app:
    image: j‑labs/fuel-consumption-app:1.0
    build:
      context: docker-api/.
    ports:
      - "5001:5000"
    links:
      - maria-db:database
    depends_on:
      - maria-db

  maria-db:
    image: mysql-initialized-fuel:1.0
    build:
      context: docker-db/.
    ports:
      - "32000:3306"
    environment:
      - MYSQL_ROOT_PASSWORD=test
    volumes:
        - data-volume:/var/lib/mysql

  myAdmin:
    image: phpmyadmin/phpmyadmin
    ports:
      - "8081:80"
    links:
      - maria-db:db
    depends_on:
      - maria-db
volumes:
  data-volume:

Code description

1. version:3 – version of docker compose file
2. services: – defines services – containers used in application
3. fuel-app: – name of flask application container
4. image: j‑labs/fuel-consumption-app:1.0 – docker image for fuel-app container
5. build:\ context: docker-api/. – if specified image will be not available, build Dockerfile under docker-api directory
6. ports:\ - "5001:5000" – expose docker container port 5000 on 5001 port
7. links:\ -maria-db:database – maria-db container will be availalbe from inside of my-app container under database alias
8. depends_on:\ -maria-db – fuel-app container will be built after maria-db container will be operating

In maria-db service new elements description:

9. environment:\ - MYSQL_ROOT_PASSWORD=test – sets environment variable MYSQLROOTPASSWORD
10. volumes:\ -data-volume:/var/lib/mysql – mount data-volume under /var/lib/mysql directory inside container to make data persistent

Run docker-compose

To run docker-compose use command:

docker-compose up -d

Due to use of parameter -d Containers will be run in detached mode.

Stop docker-compose

To stop docker-compose use command:

docker-compose down

If you want to stop docker-compose and remove all volumes (in our case volume is used by database to make data persistent) type:

docker-compose down -v

Final notes

Built-in Flask web server is designed for development purposes. While running on production, use a production WSGI server like nginx or Python based Waitress to ensure performance, stability and security of your application.

Summary

Docker is a great solution for running applications. Infrastructure as a service, portability, low resource consumption, great documentation and community makes Docker strong player in virtualization market.

Sources

• https://medium.com/better-programming/customize-your-mysql-database-in-docker-723ffd59d8fb
• https://mariadb.com/resources/blog/mariadb-and-docker-use-cases-part-1/
• https://docs.docker.com/compose/gettingstarted/
• https://docs.docker.com/storage/volumes/
• https://docs.docker.com/engine/reference/builder/
• https://hub.docker.com/_/mysql/
• https://flask.palletsprojects.com/en/1.1.x/quickstart/#a-minimal-application
• https://flask.palletsprojects.com/en/1.1.x/quickstart/
• https://github.com/jacekzygiel/FuelConsumptionMonitoringApi
• https://www.phpmyadmin.net/
• https://flask-restful.readthedocs.io/en/0.3.5/intermediate-usage.html