Migrate Gitlab PostgreSQL Database to Custom Location Using Ansible

GitLab, a popular web-based Git repository manager, uses PostgreSQL as its default database. In a self-managed GitLab, the storage we allocated may not be able to fulfill our requirement in the long run, and we might need to move our database to a new location.

This blog will show how to migrate our existing database to a custom location using Ansible. We will cover the steps and configurations required to achieve this automation and provide a step-by-step guide to help you get started.

Read more about: AWS vs Google Cloud Vs Microsoft Azure

You can easily migrate GitLab PostgreSQL to a custom location with just three steps using Ansible; create a custom location for migration, migrate the existing database to the new site, changing and apply the GitLab configuration file.

We already have some custom configurations on my self-managed GitLab server. I will show how we can change the configuration file to use the custom location for the GitLab PostgreSQL database.

Prerequisites

• An up-and-running self-managed GitLab server.
• User with appropriate permission to make changes to GitLab configuration file.
• The private key to SSH to the remote server using Ansible.

So, let’s dive in and explore how to automate GitLab PostgreSQL database migration using Ansible.

File structure

|__ roles
|_______migrate
|__________defaults
|_____________main.yml
|__________tasks
|_____________main.yml
|_______reconfigure
|__________tasks
|_____________main.yml
|__________handler
|_____________main.yml
|__________template
|_____________gitlab.rb.j2
|__________vars
|_____________main.yml
|__ Configure.yml
|__ inventory.txt

Inventory file format

[hosts]
 gitlab ansible_host= <host ip>

[all:vars]
ansible_python_interpreter=/usr/bin/python3
ansible_user= <user_name>

The inventory file needs to be in this format. The “inventory_hostname “value is required in the later step.

Step 1: Create a custom location for migrating

We will create a custom location in the initial step if we don’t use the ansible task yet. Run the following code on the Ansible playbook role:

- name: Create a directory if it does not exist
  ansible.builtin.file:
    path: <custom path>
    state: directory

make sure to change the <custom path> with your own path

Step 2: Migrating the existing database data to a new custom location using Ansible

To ensure no data changes occur during the migration, we must stop the GitLab and PostgreSQL services.

Read More about: How To Insert Data Into a DynamoDB Table with Boto3

After halting the GitLab service, we now have to migrate our PostgreSQL server to the new custom location by running this code:

- name: stop Gitlab service
  service:
    name: gitlab-runsvdir
    state: stopped    

- name: Ensure database_path dir exists
  ansible.builtin.file:
    path: "{{ database_path }}"
    state: directory
    recurse: yes

- name: Synchronize postresql to new path.
  ansible.posix.synchronize:
    src: /var/opt/gitlab/  #This is the default path for GitLab postgreSQL Database
    dest: "{{ database_path }}"
  delegate_to: "{{ inventory_hostname }}"

- name: Start Gitlab service
  service:
    name: gitlab-runsvdir
    state: started

Step 3: Changing GitLab configuration

Now is the time we make changes to our GitLab configuration file located at '/etc/gitlab/gitlab.rb' . We will use the ansible template module to change the configuration file and reconfigure our GitLab service.

Explore more about “How to Install and Upgrade the AWS CDK CLI“

We already have some existing configurations in our GitLab, and now we will add path variables for the GitLab PostgreSQL database in our Configuration file.

Add Some Other Configuration

Now we will add other configurations for setting up the database’s custom location. The jinja2 file should look like this and should contain different existing designs.

Now use the following code to perform this task on the Ansible playbook task:

external_url "{{ gitlab_external_url }}"

######################################
#####  other configurations ##########
######################################

postgresql['home'] = "{{ database_path }}/postgresql"
postgresql['data_dir'] = "{{ database_path }}/postgresql/data"
postgresql['unix_socket_directories'] = "{{ database_path }}/postgresql"

You must include another existing configuration line because the template module replaces your configuration file with the jinja2 (.j2) file content, which may misconfigure your current setup.

The ansible task for changing the configuration file and setting. Use the below code to complete these steps:

- name: backup existing configuration file to the /tmp folder
  copy:
      src: "/etc/gitlab/gitlab.rb"
      dest: "/tmp"
      remote_src: yes

- name: Copy GitLab configuration file.
  template:
    src: "{{ gitlab_config_template }}" #location of the .j2 jinja2 file
    dest: /etc/gitlab/gitlab.rb
    owner: root
    group: root
    mode: 0600
    backup: yes
  notify: restart gitlab

The handler task. To handle the job, you need to run the code which is mentioned below:

---
- name: restart gitlab
  command: gitlab-ctl reconfigure
  register: gitlab_restart
  failed_when: gitlab_restart_handler_failed_when | bool

After completing the task, your GitLab database should be migrated to a new destination, as shown in the image below.

Database connection info before migration

Database connection info after migration

Roll Back | Troubleshoot

In case something goes wrong, you can roll back to your previous configuration by copying the last configuration file we copied to /tmp/gitlab.rb . Or the backup file created while Ansible ran the template module. Then reconfiguring the GitLab.
This can also be done using an ansible task that looks like this.

In case of any unexpected error, you may need to SSH into your GitLab server and make the configuration changes manually.

- name: backup existing configuration file to the /tmp folder
  copy:
      src: "/tmp/gitlab.rb"
      dest: "/etc/gitlab/"
      remote_src: yes
  notify: restart gitlab

The handler task:

---
- name: restart gitlab
  command: gitlab-ctl reconfigure
  register: gitlab_restart
  failed_when: gitlab_restart_handler_failed_when | bool

Frequently Asked Questions