Adding Services With OurCompose Role and Branch

As we have continued to mention in the podcast, especially Episode 35 - Akaunting for a new service, we are continuing to grow the serivces offered by OurCompose. This post highlights some of the major changes and requirements for adding a service to the OurCompose Collection.

Over the past few weeks we have been adding more services to the OurCompose suite and since our core OurCompose Collection and Playbooks are Open Source this post will explain how adding a service can be done. Note for more comprehensive documentation see the README of the Collection. In this example we will be reviewing the changes made for adding Akaunting, an open source Accounting tool for small businesses and individuals.

We will walk through variables, the tasks, and any necessary configuration files.

Variables

Taking a look at the role our first step is to add the necessary role variables in the defaults/main.yml file including the service under compositional_services, and then a section for the new service in the variables file that looks like the following (these variables will be broken down):

# 
# Akaunting
# 
compositional_akaunting_pull: yes
compositional_akaunting_state: present
compositional_akaunting_version: '2.1.25'
compositional_akaunting_storage: 'local'
compositional_akaunting_backend_password: 'testpassword'
compositional_akaunting_admin_email: "admin@{{ environment_domain }}"
compositional_akaunting_admin_password: "testpassword"
compositional_akaunting_bind_mountpoints:
  - {location: '/akaunting/public/css/', directory: '/var/www/html/public/css'}
  - {location: '/akaunting/public/files/', directory: '/var/www/html/public/files'}
  - {location: '/akaunting/public/img/', directory: '/var/www/html/public/img'}
  - {location: '/akaunting/public/vendor/', directory: '/var/www/html/public/vendor'}
  - {location: '/akaunting/public/js/', directory: '/var/www/html/public/js'}
compositional_akaunting_mysql_script: |
  CREATE USER IF NOT EXISTS 'akaunting'@'%' IDENTIFIED BY '{{ compositional_akaunting_backend_password }}';
  CREATE DATABASE IF NOT EXISTS akaunting;
  GRANT ALL PRIVILEGES ON akaunting.* TO 'akaunting'@'%' IDENTIFIED BY '{{ compositional_akaunting_backend_password }}';
compositional_akaunting_healthcheck: |
  wget --quiet --no-verbose --tries=1 --spider localhost:80 \
  && wget --quiet --no-verbose --tries=1 --spider --no-check-certificate proxy$$(wget --quiet --no-verbose --tries=1 localhost:3000/portal -O - | grep -oe "[A-Za-z0-9/_-]\+.css" | head -n 1 | sed 's/^{{ environment_domain.split(".")[-1] }}//') \
  || exit 1

Breaking down the variables we have:

  • compositional_akaunting_pull Responsible for stating the docker image will need to be pulled from dockerhub as it is not local to the machine
  • compositional_akaunting_state Responsible for ensuring the image is present on the local machine
  • compositional_akaunting_version The version from Dockerhub that will be pulled
  • compositional_akaunting_storage storage configuraiton setting for the container during runtime (another docker setting)
  • compositional_akaunting_backend_password This is the default password we use for the connection to the SQL backend
  • compositional_akaunting_admin_email The admin email for the service
  • compositional_akaunting_admin_password The admin password for the service
  • compositional_akaunting_bind_mountpoints Bind mountpoints are added to reduce latency and essentially move assets (javascript/css/images) from the image itself to the nginx container moving assets closer to the user
  • compositional_akaunting_mysql_script This is usually a multiline script used for initial database configuration on the backend (SQL) container
  • compositional_akaunting_healthcheck Healthchecks are used to check the status of the service to confirm the container is running properly

These variables make each service configurable and give us (the developers and maintainers) the ability to easily add new services.

Tasks

Moving to the tasks there are a few things we need to do to ensure our services is stood up correctly. These include:

  • Ensuring the frontend container (nginx in this case) is configured to properly proxy requests to the container
  • Ensuring our image is pulled down and running on the server
  • Configuring the application container to work with the backend database
  • Configuring bind mountpoints

To ensure the frontend container is configured properly we use a jinja template in the templates directory for the nginx configuration. This template includes an almost minimal configuration for bind mountpoints and a proxy pass for our frontend (nginx) to the container. This is how the configuration looks for our Akaunting service:

{% for bind_mountpoint in compositional_akaunting_bind_mountpoints %}
location {{ bind_mountpoint['location'] }} {
    access_log /var/log/nginx/services/akaunting_{{ bind_mountpoint['location'].split('/')[-2] }}_access.log;
}
{% endfor %}

location /akaunting/ {
    access_log /var/log/nginx/services/akaunting_main_access.log;
    proxy_headers_hash_max_size 512;
    proxy_headers_hash_bucket_size 64;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    add_header Front-End-Https on;
    proxy_pass http://akaunting/akaunting/;
}

There is nothing much of note here except the proxy_pass line which varies per service. For Akaunting as an example, we point to the container which is running and serving requests at http://akaunting/akaunting/ – Note because this is a php application we had to pull a trick to symlink root of the web serivce / to serve files from /akaunting. This symlink trick looks like the following in the tasks file:

- name: (akaunting) Symlink akaunting directory
  shell: docker exec -i akaunting bash -c "ln -snfT /var/www/html/ /var/www/html/akaunting"

After the service is linked to the frontend, we need to ensure it is running. For the time being we are using docker compose. This looks fairly simple, but we pass in a few variables to ensure our configurations are correct. Below is what the docker compose configuration looks like:

- name: (akaunting) The accounting service is built and {{ compositional_akaunting_state }}
  docker_compose:
    project_name: akaunting
    definition: 
      version: '3.6'
      services:
          akaunting:
              image: "akaunting/akaunting:{{ compositional_akaunting_version }}"
              container_name: akaunting
              restart: always
              networks:
                - frontend
                - backend
              environment:
                DB_HOST: "database"
                DB_DATABASE: "akaunting"
                DB_USERNAME: "akaunting"
                APP_URL: https://{{ environment_domain }}/akaunting
                DB_PASSWORD: "{{ compositional_akaunting_backend_password }}"
                ADMIN_EMAIL: "{{ compositional_akaunting_admin_email }}"
                PASSWORD: "{{ compositional_akaunting_admin_password }}"
      networks:
          frontend:
              external: true
          backend:
              external: true
    pull: "{{ compositional_akaunting_pull }}"
    state: "{{ compositional_akaunting_state }}"
    restarted: "{{ compositional_akaunting_restarted }}"
  register: compositional_akaunting_output_1

Note we pass in our compositional_akaunting_version variable we created in the variables file, as well as some backend accounts and passwords.

Once our image is running we symlink the php application (as mentioned above) to ensure files are being served from a subdirectory of root.

After our application is up and running and symlinked properly, a database configuration is needed for the database to be initialized from the application side. Because Akaunting can’t run a seed on itself if the application has already been initialized, a check is run to see if tables have been created. If they have, we do not re-initialize the database, if they haven’t been then the database for the application is initialized. Below is what this looks like:

- name: (akaunting) Register the number of tables
  shell: if [ $(docker exec -i database mysql -uroot -p{{ compositional_database_root_password }} <<< "select count(*) as totaltables from INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA=\"akaunting\"" | tail -n 1) -gt 5 ]; then echo "HasTables"; else echo "MissingTables"; fi
  args: 
    executable: '/bin/bash'
  register: number_of_tables_status

- name: (akaunting) Configure Database
  shell: docker exec -i akaunting php artisan install --db-host="database" --db-name="akaunting" --db-username="akaunting" --db-password="{{ compositional_akaunting_backend_password }}" --admin-email="{{ compositional_akaunting_admin_email }}" --admin-password="{{ compositional_akaunting_admin_password }}"
  args: 
    executable: '/bin/bash'
  no_log: "{{ compositional_no_log }}"

After the database is online, our application is up and running. This means our final step is to configure bind mountpoints for faster responses by moving our assets to the nginx container. Bind mountpoints deserve their own post, so for now this is what the bind mountpoints look like:

#
# Bind Mountpoints
#
- name: (akaunting) Find source filesystem directory
  shell: for i in $(docker inspect --format {{.GraphDriver.Data.LowerDir}} akaunting | tr ':' ' '); do if [[ -d ${i}{{ item['directory'] }} ]]; then echo ${i}; fi; done | head -n 1
  args:
    executable: /bin/bash
  when: not item['directory'].startswith('/srv')
  loop: "{{ compositional_akaunting_bind_mountpoints }}"
  register: compositional_akaunting_src_dirs

- name: (akaunting) Register akaunting non-volume bind-mountpoints for proxy
  set_fact:
    compositional_proxy_bind_mountpoints: "{{ compositional_proxy_bind_mountpoints + [{'location': item['item']['location'], 'directory': item['stdout'] + item['item']['directory']}] }}"
  when: not item['item']['directory'].startswith('/srv')
  loop: "{{ compositional_akaunting_src_dirs['results'] }}"

- name: (akaunting) Register akaunting volume bind-mountpoints for proxy
  set_fact:
    compositional_proxy_bind_mountpoints: "{{ compositional_proxy_bind_mountpoints + [item] }}"
  when: item['directory'].startswith('/srv')
  loop: "{{ compositional_akaunting_bind_mountpoints }}"

- name: (akaunting) Reset the bind mountpoints in order to get akaunting healthy
  include_tasks: ./bind_mountpoints.yml

Note that the first task should have jinja2 raw and endraw tags around the {{.GraphDriver.Data.LowerDir}} part.

Essentially for each bind mountpoint we are using the variables we set in defaults/main.yml to have the assets moved to the nginx container. From here they are registered on the nginx container and our application is configured!

If you are interested in contributing check out the repository, or reach out to us directly with any questions via the contact page on the OurCompose Contact page.

Want to learn more?

Fill out our Contact Form, or do some more research at OurCompose.com