In this blog post, we'll dive into building a simple URL shortener application using Flask and Docker. This project serves as an excellent opportunity to explore some fundamental Docker concepts like volumes and networks, which are crucial for anyone looking to understand containerized environments.

Blog Article Overview

The blog article can cover the following points:

Project Overview: A brief introduction to the project, mentioning its use-case, significance in learning Docker, and relevance to DevOps practices.
Code Walkthrough: Provide an overview of the key parts of the code, including Flask routes, database interaction, and NGINX configuration.
Docker Volumes: Explain how Docker Volumes are used to persist the SQLite database across container restarts.
Docker Networks: Describe the role of Docker Networks in facilitating communication between the Flask app container and the NGINX container.
Deployment: A brief guide on how to deploy this setup on AWS using EC2.

Project Overview

The URL shortener project consists of a Flask web application that takes a long URL as input and returns a shortened version. This shortened URL redirects users to the original link. We'll containerize this application using Docker and deploy it with the following components:

Flask Application: Handles URL shortening and redirects.
SQLite Database: Stores the mappings between original URLs and their shortened counterparts.
Nginx: Acts as a reverse proxy, routing incoming requests to the Flask application.

Brief Code Explanation: LinkShrinker

Let’s take a look at the core components and structure of our application:

Project Structure Overview for URL Shortener Application

Here’s the directory structure for your URL shortener application:

url_shortener/
│
├── app/
│   ├── app.py
│   ├── templates/
│   │   └── index.html
│   └── statics/
│       └── style.css
│
├── nginx/
│   └── nginx.conf
│
├── docker-compose.yml
├── Dockerfile
├── requirements.txt
└── README.md

Let’s break down each component:

`app/` Directory

This directory contains the core application code and static assets.

`app.py` file :

This is the main Flask application file where the URL shortening logic is implemented.

Initialization

The code starts by importing the necessary modules:

from flask import Flask, request, redirect, render_template
import sqlite3
import os

It then initializes the Flask application and sets the database file path:

app = Flask(__name__)
DATABASE = '/data/urls.db'

Database Initialization

The init_db function creates the database and the urls table if it doesn't exist:

# Initialize the SQLite database
def init_db():
    """Create the database and the urls table if it doesn't exist."""
    conn = sqlite3.connect(DATABASE)
    cursor = conn.cursor()
    cursor.execute('''
        CREATE TABLE IF NOT EXISTS urls (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            original_url TEXT NOT NULL,
            short_url TEXT NOT NULL UNIQUE
        )
    ''')
    conn.commit()
    conn.close()

Short URL Generation

The generate_short_url function generates a hash-based short URL identifier for a given original URL:

# Function to generate a short URL identifier
def generate_short_url(original_url):
    """Generate a hash-based short URL identifier for a given original URL."""
    return str(hash(original_url) % 10000)

Main Route

The main route (/) handles URL shortening requests:

@app.route('/', methods=['GET', 'POST'])
def index():
    """Handle URL shortening requests."""
    if request.method == 'POST':
        original_url = request.form['url']
        short_url = generate_short_url(original_url)

        # Ensure unique short URL
        conn = sqlite3.connect(DATABASE)
        cursor = conn.cursor()

        try:
            cursor.execute('INSERT INTO urls (original_url, short_url) VALUES (?, ?)', (original_url, short_url))
            conn.commit()
        except sqlite3.IntegrityError:
            # Handle the case where the short URL already exists
            cursor.execute('SELECT original_url FROM urls WHERE short_url = ?', (short_url,))
            existing_url = cursor.fetchone()
            if existing_url and existing_url[0] == original_url:
                # If the short URL already exists for the same original URL, reuse it
                pass
            else:
                # Collision: generate a new short URL and insert again
                short_url = generate_short_url(original_url + str(os.urandom(16)))  # Slightly modify input to get a different hash
                cursor.execute('INSERT INTO urls (original_url, short_url) VALUES (?, ?)', (original_url, short_url))
                conn.commit()

        conn.close()

        # Get the full URL, including the domain name and protocol
        full_short_url = request.url_root + short_url

        return render_template('index.html', short_url=full_short_url)

    return render_template('index.html')

This route handles both GET and POST requests. When a POST request is made, it generates a short URL, ensures its uniqueness, and inserts it into the database. If the short URL already exists, it either reuses it or generates a new one.

Redirect Route

The redirect route (/<short_url>) redirects to the original URL based on the short URL identifier:

@app.route('/<short_url>')
def redirect_to_original(short_url):
    """Redirect to the original URL based on the short URL identifier."""
    conn = sqlite3.connect(DATABASE)
    cursor = conn.cursor()
    cursor.execute('SELECT original_url FROM urls WHERE short_url = ?', (short_url,))
    row = cursor.fetchone()
    conn.close()

    if row:
        return redirect(row[0])
    else:
        return "URL not found!", 404

This route queries the database for the original URL associated with the short URL and redirects to it if found.

Running the Application

Finally, the application is run using the following code:

if __name__ == '__main__':
    if not os.path.exists(DATABASE):
        init_db()
    app.run(host='0.0.0.0', port=5000)

This code initializes the database if it doesn't exist and runs the application on port 5000.

Full app.py Code:

from flask import Flask, request, redirect, render_template
import sqlite3
import os

app = Flask(__name__)
DATABASE = '/data/urls.db'

# Initialize the SQLite database
def init_db():
    """Create the database and the urls table if it doesn't exist."""
    conn = sqlite3.connect(DATABASE)
    cursor = conn.cursor()
    cursor.execute('''
        CREATE TABLE IF NOT EXISTS urls (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            original_url TEXT NOT NULL,
            short_url TEXT NOT NULL UNIQUE
        )
    ''')
    conn.commit()
    conn.close()

# Function to generate a short URL identifier
def generate_short_url(original_url):
    """Generate a hash-based short URL identifier for a given original URL."""
    return str(hash(original_url) % 10000)

@app.route('/', methods=['GET', 'POST'])
def index():
    """Handle URL shortening requests."""
    if request.method == 'POST':
        original_url = request.form['url']
        short_url = generate_short_url(original_url)

        # Ensure unique short URL
        conn = sqlite3.connect(DATABASE)
        cursor = conn.cursor()

        try:
            cursor.execute('INSERT INTO urls (original_url, short_url) VALUES (?, ?)', (original_url, short_url))
            conn.commit()
        except sqlite3.IntegrityError:
            # Handle the case where the short URL already exists
            cursor.execute('SELECT original_url FROM urls WHERE short_url = ?', (short_url,))
            existing_url = cursor.fetchone()
            if existing_url and existing_url[0] == original_url:
                # If the short URL already exists for the same original URL, reuse it
                pass
            else:
                # Collision: generate a new short URL and insert again
                short_url = generate_short_url(original_url + str(os.urandom(16)))  # Slightly modify input to get a different hash
                cursor.execute('INSERT INTO urls (original_url, short_url) VALUES (?, ?)', (original_url, short_url))
                conn.commit()

        conn.close()

        # Get the full URL, including the domain name and protocol
        full_short_url = request.url_root + short_url

        return render_template('index.html', short_url=full_short_url)

    return render_template('index.html')

@app.route('/<short_url>')
def redirect_to_original(short_url):
    """Redirect to the original URL based on the short URL identifier."""
    conn = sqlite3.connect(DATABASE)
    cursor = conn.cursor()
    cursor.execute('SELECT original_url FROM urls WHERE short_url = ?', (short_url,))
    row = cursor.fetchone()
    conn.close()

    if row:
        return redirect(row[0])
    else:
        return "URL not found!", 404

if __name__ == '__main__':
    if not os.path.exists(DATABASE):
        init_db()
    app.run(host='0.0.0.0', port=5000)

templates/index.html:

This is the HTML template that renders the form for users to input the URL they want to shorten. It also displays the shortened URL.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>URL Shortener</title>
    <link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
</head>
<body>
    <div class="container">
        <h1>LinkShrinker</h1>
        <form method="post">
            <label for="url">Enter URL:</label>
            <input type="text" id="url" name="url" required>
            <button type="submit">Shorten</button>
        </form>
        {% if short_url %}
        <div class="short-url-container">
            <p>Short URL:</p>
            <input type="text" id="short-url" value="{{ short_url }}" readonly>
            <button id="copy-btn">Copy</button>
        </div>
        {% endif %}
    </div>

    <script>
        const copyBtn = document.getElementById('copy-btn');
        const shortUrlInput = document.getElementById('short-url');

        copyBtn.addEventListener('click', () => {
            shortUrlInput.select();
            document.execCommand('copy');
            alert('Short URL copied to clipboard!');
        });
    </script>
</body>
</html>

Here's a breakdown of the HTML code:

HTML Structure The code starts with the basic HTML structure, including the <!DOCTYPE html> declaration, <html>, <head>, and <body> tags.

Head Section In the <head> section, we have:

<meta charset="UTF-8">: specifies the character encoding of the document.
<meta name="viewport" content="width=device-width, initial-scale=1.0">: sets the viewport settings for mobile devices.
<title>URL Shortener</title>: sets the title of the page.
<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">: links to an external stylesheet named style.css located in the static folder. The {{ url_for }} syntax is likely using a templating engine like Jinja2 to generate the URL.

Body Section In the <body> section, we have:

<div class="container">: a container element that wraps the entire content.
<h1>LinkShrinker</h1>: a heading element that displays the title "LinkShrinker".
<form method="post">: a form element that submits data to the server using the POST method.
- <label for="url">Enter URL:</label>: a label element that associates with the input field.
- <input type="text" id="url" name="url" required>: a text input field that accepts a URL. The required attribute makes it mandatory to fill in.
- <button type="submit">Shorten</button>: a submit button that triggers the form submission.
{% if short_url %}: a conditional statement that checks if a short_url variable is defined. If true, the code inside the block is rendered.
- <div class="short-url-container">: a container element that wraps the short URL display.
  - <p>Short URL:</p>: a paragraph element that displays the text "Short URL:".
  - <input type="text" id="short-url" value="{{ short_url }}" readonly>: a read-only text input field that displays the shortened URL. The value attribute is set to the short_url variable.
  - <button id="copy-btn">Copy</button>: a button element that triggers the copy functionality.

JavaScript Code The JavaScript code is included at the end of the HTML file and is used to handle the copy functionality:

const copyBtn = document.getElementById('copy-btn');: gets a reference to the copy button element.
const shortUrlInput = document.getElementById('short-url');: gets a reference to the short URL input field.
copyBtn.addEventListener('click', () => { ... });: adds an event listener to the copy button that listens for click events.
- shortUrlInput.select();: selects the entire short URL text.
- document.execCommand('copy');: executes the copy command to copy the selected text to the clipboard.
- alert('Short URL copied to clipboard!');: displays an alert message to the user indicating that the short URL has been copied.

Overall, this code creates a simple URL shortener web page that accepts a URL input, submits it to the server, and displays a shortened URL with a copy button.

static/style.css:

This file contains the styling for the HTML page.

/* Global Styles */

* {
    box-sizing: border-box;
    margin: 0;
    padding: 0;
  }

  body {
    font-family: Arial, sans-serif;
    background-color: #f0f0f0;
    line-height: 1.6;
  }

  /* Container Styles */

  .container {
    max-width: 400px;
    margin: 40px auto;
    padding: 20px;
    background-color: #fff;
    border: 1px solid #ddd;
    border-radius: 10px;
    box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
  }

  /* Heading Styles */

  h1 {
    text-align: center;
    margin-bottom: 20px;
  }

  /* Form Styles */

  form {
    margin-bottom: 20px;
  }

  label {
    display: block;
    margin-bottom: 10px;
  }

  input[type="text"] {
    width: 100%;
    height: 40px;
    margin-bottom: 20px;
    padding: 10px;
    border: 1px solid #ccc;
    border-radius: 5px;
  }

  button[type="submit"] {
    width: 100%;
    height: 40px;
    background-color: #4CAF50;
    color: #fff;
    padding: 10px;
    border: none;
    border-radius: 5px;
    cursor: pointer;
  }

  button[type="submit"]:hover {
    background-color: #3e8e41;
  }

  /* Short URL Styles */

  .short-url-container {
    margin-top: 20px;
  }

  #short-url {
    width: 100%;
    height: 40px;
    margin-bottom: 10px;
    padding: 10px;
    border: 1px solid #ccc;
    border-radius: 5px;
  }

  #copy-btn {
    width: 100%;
    height: 40px;
    background-color: #4CAF50;
    color: #fff;
    padding: 10px;
    border: none;
    border-radius: 5px;
    cursor: pointer;
  }

  #copy-btn:hover {
    background-color: #3e8e41;
  }

Here's a breakdown of the CSS code:

Global Styles The first section defines global styles that apply to all elements on the page.

* { ... }: This is a wildcard selector that targets all elements on the page. It sets:
- box-sizing: border-box;: This tells the browser to include padding and border widths in the element's width and height calculations.
- margin: 0; and padding: 0;: These reset the default margins and padding on all elements to 0.

Body Styles The next section targets the <body> element.

body { ... }: This sets:
- font-family: Arial, sans-serif;: The font family for the entire page is set to Arial, with a fallback to sans-serif if Arial is not available.
- background-color: #f0f0f0;: The background color of the page is set to a light gray (#f0f0f0).
- line-height: 1.6;: The line height (space between lines of text) is set to 1.6, which is a common value for readability.

Container Styles The .container class is defined to style a container element that will hold the blog's content.

.container { ... }: This sets:
- max-width: 400px;: The maximum width of the container is set to 400 pixels.
- margin: 40px auto;: The container has a margin of 40 pixels on the top and bottom, and is centered horizontally using auto.
- padding: 20px;: The container has a padding of 20 pixels on all sides.
- background-color: #fff;: The background color of the container is set to white (#fff).
- border: 1px solid #ddd;: A 1-pixel solid border is added around the container, with a color of #ddd (a light gray).
- border-radius: 10px;: The container's corners are rounded with a radius of 10 pixels.
- box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);: A subtle box shadow is added to the container, with a horizontal and vertical offset of 0, a blur radius of 10 pixels, and an opacity of 0.1.

Heading Styles The h1 element is targeted to style headings.

h1 { ... }: This sets:
- text-align: center;: Headings are centered horizontally.
- margin-bottom: 20px;: A margin of 20 pixels is added below headings.

Form Styles

form { margin-bottom: 20px; }: This sets a margin of 20 pixels below each form element, creating some space between the form and the next element on the page.
label { display: block; margin-bottom: 10px; }: This styles the label elements within the form:
- display: block;: Labels are displayed as block elements, which means they take up the full width of their parent element and have a line break before and after them.
- margin-bottom: 10px;: A margin of 10 pixels is added below each label, creating some space between the label and the next element.
input[type="text"] { ... }: This targets text input fields and sets several styles:
- width: 100%;: Input fields take up the full width of their parent element.
- height: 40px;: Input fields have a height of 40 pixels.
- margin-bottom: 20px;: A margin of 20 pixels is added below each input field, creating some space between the input field and the next element.
- padding: 10px;: Input fields have a padding of 10 pixels on all sides, creating some space between the input field's border and its content.
- border: 1px solid #ccc;: A 1-pixel solid border is added around input fields, with a color of #ccc (a light gray).
- border-radius: 5px;: Input fields have rounded corners with a radius of 5 pixels.
button[type="submit"] { ... }: This targets submit buttons and sets several styles:
- width: 100%;: Submit buttons take up the full width of their parent element.
- height: 40px;: Submit buttons have a height of 40 pixels.
- background-color: #4CAF50;: Submit buttons have a green background color (#4CAF50).
- color: #fff;: Submit buttons have white text.
- padding: 10px;: Submit buttons have a padding of 10 pixels on all sides, creating some space between the button's border and its content.
- border: none;: Submit buttons have no border.
- border-radius: 5px;: Submit buttons have rounded corners with a radius of 5 pixels.
- cursor: pointer;: The cursor changes to a pointer when hovering over the submit button.
button[type="submit"]:hover { background-color: #3e8e41; }: This sets the hover state for submit buttons, changing the background color to a darker green (#3e8e41) when the user hovers over the button.

Short URL Styles

.short-url-container { margin-top: 20px; }: This sets a margin of 20 pixels above the short URL container, creating some space between the container and the previous element on the page.
#short-url { ... }: This targets the short URL input field and sets several styles:
- width: 100%;: The short URL input field takes up the full width of its parent element.
- height: 40px;: The short URL input field has a height of 40 pixels.
- margin-bottom: 10px;: A margin of 10 pixels is added below the short URL input field, creating some space between the input field and the next element.
- padding: 10px;: The short URL input field has a padding of 10 pixels on all sides, creating some space between the input field's border and its content.
- border: 1px solid #ccc;: A 1-pixel solid border is added around the short URL input field, with a color of #ccc (a light gray).
- border-radius: 5px;: The short URL input field has rounded corners with a radius of 5 pixels.
#copy-btn { ... }: This targets the copy button and sets several styles:
- width: 100%;: The copy button takes up the full width of its parent element.
- height: 40px;: The copy button has a height of 40 pixels.
- background-color: #4CAF50;: The copy button has a green background color (#4CAF50).
- color: #fff;: The copy button has white text.
- padding: 10px;: The copy button has a padding of 10 pixels on all sides, creating some space between the button's border and its content.
- border: none;: The copy button has no border.
- border-radius: 5px;: The copy button has rounded corners with a radius of 5 pixels.
- cursor: pointer;: The cursor changes to a pointer when hovering over the copy button.
#copy-btn:hover { background-color: #3e8e41; }: This sets the hover state for the copy button, changing the background color to a darker green (#3e8e41) when the user hovers over the button.

This is how the final UI will look:

`nginx/` Directory

This directory holds the NGINX configuration file that serves as a reverse proxy to the Flask application.

nginx.conf: Configuration file for NGINX.

server {
    listen 80;

    location / {
        proxy_pass http://web:5000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

This is an NGINX configuration block, which is used to define how the NGINX server should behave when it receives a request on port 80 (the default HTTP port). Let’s break down each part of the configuration Line by Line:

server { ... }
- The server block defines the configuration for a virtual server that listens for HTTP requests. NGINX can handle multiple server blocks, each representing a different domain or service.
listen 80;
- This line tells NGINX to listen for incoming requests on port 80, which is the standard port for HTTP. Any HTTP request directed to this server (such as from a web browser) will be captured here.
location / { ... }
- The location / block specifies how to handle requests that match the root URL (i.e., requests that match the server’s domain with or without any additional path).
- In this case, it catches all requests that are sent to the root of the server (/), meaning it will handle any request made to the base domain (e.g., http://example.com/ or http://example.com/anything).
proxy_pass http://web:5000;
- The proxy_pass directive tells NGINX to pass (or "proxy") the requests to another server—in this case, http://web:5000.
  - web refers to a service that is likely defined in Docker Compose or elsewhere in the environment. It could be the Flask application that is running on the web service, listening on port 5000.
  - NGINX will forward any request it receives on port 80 to the Flask app, which is running on port 5000 inside the web container.
- This is an example of reverse proxying, where NGINX serves as a middle layer between the client (user's browser) and the application (Flask app).
proxy_set_header Host $host;
- This line sets the Host HTTP header in the proxied request to the value of $host, which represents the original host header from the client request. This ensures that the backend server (web) knows the original host that was used to access the NGINX server.
proxy_set_header X-Real-IP $remote_addr;
- This sets the X-Real-IP header to the client's IP address ($remote_addr). This is useful for logging or security purposes, as the backend server (Flask app) will know the actual IP address of the client, not just the NGINX server's IP.
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
- The X-Forwarded-For header is commonly used to pass along the original client’s IP address through proxies.
  - $proxy_add_x_forwarded_for appends the client’s IP address to the existing X-Forwarded-For header, ensuring that if there are multiple proxies, the request chain can be traced.
proxy_set_header X-Forwarded-Proto $scheme;
- This header sets X-Forwarded-Proto to the protocol used by the client ($scheme), which could be http or https. This helps the backend server know whether the original request was made using a secure connection (HTTPS) or not.

Purpose of the Configuration:

Reverse Proxy: NGINX forwards incoming requests on port 80 to the Flask application running on the web service (port 5000).
Header Handling: It ensures that the Flask app receives information about the client, such as the original Host, IP address, and protocol (HTTP/HTTPS).

This setup allows NGINX to handle incoming traffic and forward it to the backend application (Flask) while maintaining important client information.

`docker-compose.yml` file

Defines the Docker services for the Flask application and NGINX. This is the main file where we need to define Docker volumes and networks.

version: '3.8'

services:
  web:
    build: .
    volumes:
      - url_data:/data
    networks:
      - url_network

  nginx:
    image: nginx:latest
    ports:
      - "80:80"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf
    networks:
      - url_network

networks:
  url_network:
    driver: bridge

volumes:
  url_data:

Here's a breakdown of the Docker Compose file:

Version The file starts by specifying the version of Docker Compose being used, which is 3.8.

Services The services section defines two services: web and nginx.

Web Service

build: . indicates that the Docker image for the web service will be built from the current directory (i.e., the directory containing the Docker Compose file).
volumes: mounts a volume at /data inside the container, which is linked to a named volume url_data (defined later in the file).
networks: specifies that the web service will be connected to a network named url_network (defined later in the file).

Nginx Service

image: nginx:latest uses the latest available version of the official Nginx Docker image.
ports: exposes port 80 from the container to the host machine, allowing incoming requests to be routed to the Nginx server.
volumes: mounts a volume at /etc/nginx/conf.d/default.conf inside the container, which is linked to a file nginx.conf in the current directory (i.e., ./nginx/nginx.conf). This allows the Nginx server to use a custom configuration file.
networks: specifies that the nginx service will be connected to the same url_network network as the web service.

Networks

The networks section defines a single network named url_network with a bridge driver. This allows the web and nginx services to communicate with each other.

Volumes

The volumes section defines a single named volume url_data, which is used by the web service to persist data.

`Dockerfile`

The Dockerfile for building the Flask application container.


# Set the working directory in the container
WORKDIR /app

# Copy the application files into the container
COPY app/ .

# Install the dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Expose port 5000
EXPOSE 5000

# Run the application
CMD ["python", "app.py"]

This is a Dockerfile, which is a text file that contains a set of instructions for building a Docker image. Here's a breakdown of what each line does:

WORKDIR /app:
- This sets the working directory in the container to /app. This means that any subsequent commands will be executed from this directory.
COPY app/ .:
- This copies the files from the app/ directory on the host machine (i.e., the machine building the Docker image) into the current working directory (/app) in the container. This is likely copying the application code into the container.
RUN pip install --no-cache-dir -r requirements.txt:
- This runs the pip command to install the dependencies specified in the requirements.txt file. The --no-cache-dir flag tells pip not to cache the package installations, which can help reduce the size of the Docker image.
EXPOSE 5000:
- This exposes port 5000 from the container to the host machine. This means that when the container is running, it will listen on port 5000 and allow incoming connections from the host machine.
CMD ["python", "app.py"]:
- This sets the default command to run when the container is started. In this case, it runs the python command with the argument app.py, which is likely the main application file. This means that when the container is started, it will run the app.py file using the Python interpreter.

Overall, this Dockerfile is building a Docker image that contains a Python application, installs the required dependencies, and sets up the application to run on port 5000.

`requirements.txt`

This file lists the dependencies required by the Flask application.

Generate requirements.txt: Once your environment is active, use the pip freeze command to generate a list of installed packages and their exact versions:

 pip freeze > requirements.txt

Docker Volumes: Persisting the Database

In Docker, a volume is a way to save data created and used by Docker containers. Normally, data inside a Docker container is temporary. This means that once the container stops or is removed, the data is lost. For many applications, including our URL shortener, we need a way to save data even after the containers stop or are removed.

Why Use Volumes?

Persistence: Volumes provide a reliable way to store data, like our SQLite database, that must survive container restarts.
Decoupling: By decoupling data storage from the container's life cycle, you allow for greater flexibility and scalability. Containers can be replaced or updated without data loss.
Performance: Volumes are built to manage high data flow and lower I/O overhead.

Code Implementation

In the docker-compose.yml file, we define a volume for our SQLite database:

# Define the services section, which lists the services that will be 
# created and managed by Docker Compose
services:
  web:
    build: .
    # Define the volumes section for the web service
    volumes:
       # Mount a volume named url_data to the container's /data directory
      - url_data:/data     
    networks:
      - url_network

  nginx:
    image: nginx:latest
    ports:
      - "80:80"
    # Define the volumes section for the nginx service
    volumes:
      # Mount a volume containing the nginx configuration file (nginx.conf) 
      # to the container's /etc/nginx/conf.d/default.conf location
      - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf
    networks:
      - url_network

networks:
  url_network:
    driver: bridge

# Define the volumes section, which lists the volumes that will be created
# and managed by Docker Compose
volumes:
  url_data:

Here’s what’s happening:

In this example, the url_data volume is used to persist data for the web service.
volumes: url_data: defines a named volume called url_data. This volume is not tied to a specific directory on the host machine, so Docker will create a directory for it under /var/lib/docker/volumes/ on Linux systems.
The SQLite database is stored in /data/urls.db, ensuring that our URL mappings persist even if the container stops or is recreated.
./nginx/nginx.conf:/etc/nginx/conf.d/default.conf mounts a volume containing the nginx configuration file (nginx.conf) to the container's /etc/nginx/conf.d/default.conf location.

Docker Networks: Enabling Container Communication

A network in Docker is a way to allow containers to communicate with each other, isolated from other containers not on the same network. Docker networks are essential for enabling secure, scalable communication between different parts of an application.

Why Networks Matter?

Isolation and Security: Containers on the same network can communicate, but they’re isolated from containers on other networks.
Service Discovery: Docker networks automatically handle DNS resolution, allowing containers to refer to each other by name.
Scalability: Networks can easily be extended to include new containers, facilitating dynamic and scalable architectures.

Code Implementation

We define a custom network in the docker-compose.yml file:

# Define the services section, which lists the services that will be 
# created and managed by Docker Compose
services:
  web:
    build: .
    volumes:
      - url_data:/data
     # Define the networks section for the web service
    networks:
      # Connect the web service to the url_network network
      - url_network

  nginx:
    image: nginx:latest
    ports:
      - "80:80"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf
    # Define the networks section for the nginx service
    networks:
      # Connect the nginx service to the url_network network
      - url_network

# Define the networks section, which lists the networks that will be 
# created and managed by Docker Compose
networks:
  # Define the url_network network
  url_network:
    # Specify the driver for the url_network network 
    # (in this case, a bridge driver)
    driver: bridge

volumes:
  url_data:

Here’s what’s happening:

The url_network is a custom bridge network that we’ve created using bridge driver, which is the default Docker network type.
Both the web and nginx services are attached to this network, allowing Nginx to proxy requests to the Flask application securely.
This allows Nginx to forward incoming HTTP requests to the Flask application seamlessly.

Deployment

Step 1: Set Up AWS EC2 Instance

Log in to your AWS Management Console.
Navigate to the EC2 Dashboard and click "Launch Instance".
Choose an Amazon Machine Image (AMI), preferably a Linux-based one like Amazon Linux 2.
Select an instance type (e.g., t2.micro for free tier eligibility).
Configure instance details, including network settings.
Add storage (default settings are usually sufficient).
Add tags (optional).
Configure security group to allow HTTP (port 80) and SSH (port 22) access.
Review and launch the instance, then download the key pair for SSH access.

For more details - Check here

Step 2: Connect to Your EC2 Instance

Open a terminal on your local machine.

Connect to your instance using SSH:

  ssh -i key-pair.pem ec2-user@your-ec2-public-dns

Copy the example SSH command. The following is an example, where key-pair-name.pem is the name of your private key file, ec2-user is the username associated with the image, and the string after the @ symbol is the public DNS name of the instance.

For more details and options for connecting to your EC2 instance: Click here

Step 3: Install Docker and Docker Compose

Install Docker:

  sudo yum update -y
  sudo yum install docker -y
  sudo service docker start
  sudo usermod -aG docker ec2-user

Log out and back in to apply Docker group changes:
```
  exit
```
Reconnect to your instance using SSH.

Install Docker Compose:

  sudo curl -L "https://github.com/docker/compose/releases/download/$(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep -Po '"tag_name": "\K.*\d')/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
  sudo chmod +x /usr/local/bin/docker-compose

Verify Docker and Docker Compose installation:

  docker --version
  docker-compose --version

For a brief description of these installation steps : Check here

Step 4: Prepare the Application on EC2

Clone Your Application Repository (or transfer files):

If your code is in a Git repository, clone it:
```
  git clone https://github.com/yourusername/url-shortener.git
  cd url-shortener
```
Alternatively, use SCP to copy your project files to the EC2 instance.
For more details on Connecting securely to Amazon EC2 instance: Click here

Step 5: Build and Run Your Docker Containers

Navigate to the project directory in your terminal:
```
  cd url-shortener
```
Build the Docker image:
```
  docker-compose build
```
Run the Docker containers:
```
  docker-compose up -d
```
The -d flag runs the containers in detached mode.
OR Build and start the services using Docker Compose in single command:
```
  docker-compose up -d --build
```

Step 6: Access the application

Open a web browser and navigate to your EC2 instance's public DNS or IP address to verify that your URL shortener application is running.
Input a long URL and submit to get a shortened URL.
Visit the shortened URL to be redirected to the original URL.

Verify Data Persistence

To verify data persistence across container restarts:

Stop the containers:
```
 docker-compose down
```
Restart the containers:
```
 docker-compose up -d
```
The previously created shortened URLs should still be accessible, demonstrating that the SQLite database persisted.
To verify that the Docker volume is created and being used:
```
 docker volume ls
```

docker volume ls, lists all of the Docker volumes on the system
The output of the command shows the following information about the volumes:
- DRIVER: The driver used for the volume. In this case, the driver is "local", which means the volume is stored on the host machine.
- VOLUME NAME: The name of the volume. In this case, the volume is named "url-shortener_url_data".

    docker volume inspect url-shortener_url_data

docker volume inspect url-shortener_url_data, inspects a Docker volume named "url-shortener_url_data".
The output of the command shows the following information about the volume:
- CreatedAt: The volume was created on September 1, 2024 at 12:40 PM UTC.
  
  Driver: The volume is using the "local" driver, which means it is stored on the host machine.
  
  Labels: The volume has the following labels:
  - com.docker.compose.project: "url-shortener"
  - com.docker.compose.version: "2.29.2"
  - com.docker.compose.volume: "url data"

Mountpoint: The volume is mounted to the directory /var/lib/docker/volumes/url-shortener_url_data/_data on the host machine.

Name: The name of the volume is "url-shortener_url_data".

Options: The volume has no options.

Scope: The volume is local, which means it is only accessible on the host machine.

The volume url-shortener_url_data should be listed, indicating it is mounted to persist data across container restarts.

Check Docker Network Usage

To see the Docker network that was created and inspect its configuration:

docker network ls

docker network ls, lists all of the Docker networks on the system.
The output of the command shows the following information about the networks:
- NETWORK ID: The unique identifier for the network.
- NAME: The name of the network.
- DRIVER: The driver used for the network.
- SCOPE: The scope of the network. In this case, all networks have a scope of "local", which means they are only accessible on the host machine.

The networks listed in the output are:

bridge: A default network created by Docker.
host: A network that provides direct access to the host network.
none: A network with no networking capabilities.
url-shortener_url-net: A network created by the "url-shortener" project.
url-shortener_url network: A network created by the "url-shortener" project.

docker network inspect url-shortener_url_network

docker network inspect url-shortener_url_network inspects a specific network named "url-shortener_url_network".
Here's a breakdown of the information provided in the output:

Network Details:
- Name: "url-shortener url network"
- ID: A unique identifier for the network
- Created: Timestamp of when the network was created
- Scope: "local", indicating the network is only accessible within the host machine
- Driver: "bridge", specifying the type of network driver used
- EnableIPv6: False, meaning IPv6 is disabled for this network
- IPAM:
  - Driver: "default", using the default IPAM driver
  - Options: null, no specific IPAM options are configured
  - Config:
    - Subnet: "192.168.192.0/20", the network's IP address range
    - Gateway: "192.168.192.1", the default gateway for the network
- Internal: False, meaning the network is not accessible from outside the Docker host
- Attachable: False, indicating containers cannot directly attach to this network
- Ingress: False, preventing incoming traffic to the network
- ConfigFrom: Empty, no configuration is inherited from other networks
- Network:
  - ConfigOnly: False, the network is not configured to be used only for container networking
  - Containers: A list of containers connected to the network:
    - Container 1:
      - Name: "url-shortener-nginx-1"
      - EndpointID: A unique identifier for the container's connection to the network
      - MacAddress: The container's MAC address
      - IPv4Address: The container's IPv4 address within the network
    - Container 2:
      - Name: "url-shortener-web-1"
      - EndpointID: A unique identifier for the container's connection to the network
      - MacAddress: The container's MAC address
      - IPv4Address: The container's IPv4 address within the network
- Options: Empty, no additional network options are specified
- Labels:
  - com.docker.compose.network: "url network", indicating the network was created by Docker Compose
  - com.docker.compose.project: "url-shortener", the name of the Docker Compose project
  - com.docker.compose.version: "2.29.2", the version of Docker Compose used

In summary, this output provides detailed information about the "url-shortener_url_network", including its configuration, connected containers, and IP address assignments.

Conclusion

Demonstrated Docker Networking: We explicitly created a Docker network (url_network) for the services, ensuring proper container communication between Nginx and Flask.
Utilized Docker Volumes: A Docker volume (url_data) was used to persist the SQLite database, ensuring data persistence across container restarts.
Incorporated Nginx: Nginx is used as a reverse proxy, managing incoming traffic to the Flask app and adding an extra layer of security and scalability.
Deployed on AWS EC2: The application is deployed on an AWS EC2 instance, accessible to users over the internet.

This setup effectively demonstrates the use of Docker networking, volumes, and Nginx in a real-world application, providing a robust and scalable solution for a URL shortening service.

Command Palette

Blog Article Overview

Project Overview

Brief Code Explanation: LinkShrinker

Project Structure Overview for URL Shortener Application

Let’s break down each component:

app/ Directory

app.py file :

templates/index.html:

static/style.css:

This is how the final UI will look:

nginx/ Directory

Purpose of the Configuration:

docker-compose.yml file

Web Service

Nginx Service

Networks

Volumes

Dockerfile

requirements.txt

Docker Volumes: Persisting the Database

Code Implementation

Docker Networks: Enabling Container Communication

Code Implementation

Deployment

Step 1: Set Up AWS EC2 Instance

Step 2: Connect to Your EC2 Instance

Step 3: Install Docker and Docker Compose

Step 4: Prepare the Application on EC2

Step 5: Build and Run Your Docker Containers

Step 6: Access the application

Verify Data Persistence

Check Docker Network Usage

Conclusion

DevOps - Planning to Production

Comprehensive Guide to Jenkins: CI/CD, Pipelines, and Multi-Node Cluster Architecture

More from this blog

`app/` Directory

`app.py` file :

`nginx/` Directory

`docker-compose.yml` file

`Dockerfile`

`requirements.txt`