
GitLab, a popular web-based Git repository manager, uses PostgreSQL as its default database. In a self-managed GitLab, the storage we allocated may not be able to fulfill our requirement in the long run, and we might need to move our database to a new location.
In this blog we will see how we can migrate our existing database to a custom location using Ansible. We will cover the necessary steps and configurations required to achieve this automation and provide a step-by-step guide to help you get started. You can easily migrate GitLab PostgreSQL to a custom location with just three steps using Ansible; create a custom location for migration, migrate the existing database to the new site, changing and apply the GitLab configuration file.
We already have some custom configurations on my self-managed GitLab server. I will show how we can change the configuration file to use the custom location for the GitLab PostgreSQL database.
Read more about: AWS vs Google Cloud Vs Microsoft Azure
Prerequisites An up-and-running self-managed GitLab server. User with appropriate permission to make changes to GitLab configuration file. The private key to SSH to the remote server using Ansible. So, let’s dive in and explore how to automate GitLab PostgreSQL database migration using Ansible.
Prerequisites
- An up-and-running self-managed GitLab server.
- User with appropriate permission to make changes to GitLab configuration file.
- The private key to SSH to the remote server using Ansible.
So, let’s dive in and explore how to automate GitLab PostgreSQL database migration using Ansible.
- An up-and-running self-managed GitLab server.
- User with appropriate permission to make changes to GitLab configuration file.
- The private key to SSH to the remote server using Ansible.
File structure
|__ roles
|_______migrate
|__________defaults
|_____________main.yml
|__________tasks
|_____________main.yml
|_______reconfigure
|__________tasks
|_____________main.yml
|__________handler
|_____________main.yml
|__________template
|_____________gitlab.rb.j2
|__________vars
|_____________main.yml
|__ Configure.yml
|__ inventory.txt
Inventory file format
[hosts]
gitlab ansible_host=
[all:vars]
ansible_python_interpreter=/usr/bin/python3
ansible_user=
The inventory file needs to be in this format. The “inventory_hostname “value is required in the later step.
Step 1: Create a custom location for migrating
We will create a custom location in the initial step if we don’t use the ansible task yet. Run the following code on the Ansible playbook role:
- name: Create a directory if it does not exist
ansible.builtin.file:path: state: directory
make sure to change the <custom path> with your own path
Step 2: Migrating the existing database data to a new custom location using Ansible
To ensure no data changes occur during the migration, we must stop the GitLab and PostgreSQL services.
Read More about: How To Insert Data Into a DynamoDB Table with Boto3
After halting the GitLab service, we now have to migrate our PostgreSQL server to the new custom location by running this code:
- name: stop Gitlab service
service:name: gitlab-runsvdirstate: stopped- name: Ensure database_path dir exists ansible.builtin.file: path: "{{ database_path }}" state: directory recurse: yes - name: Synchronize postresql to new path. ansible.posix.synchronize: src: /var/opt/gitlab/ #This is the default path for GitLab postgreSQL Database dest: "{{ database_path }}" delegate_to: "{{ inventory_hostname }}" - name: Start Gitlab service service: name: gitlab-runsvdirstate: started
Step 3: Changing GitLab configuration
Now is the time we make changes to our GitLab configuration file located at '/etc/gitlab/gitlab.rb'
. For this we are going to use ansible template module to change the configuration file and reconfigure our GitLab service.
We already have some existing configurations in our GitLab, and now we will add path variables for the GitLab PostgreSQL database in our Configuration file.
Explore more about “How to Install and Upgrade the AWS CDK CLI“
Step 4: add some other Configuration
Now we are going to add some other configurations for setting up the database custom location. The jinja2 file should look like this and have should contain other existing configuration.
Now use the following code to perform this task on the Ansible playbook task:
external_url "{{ gitlab_external_url }}"
########################################### other configurations ################################################postgresql['home'] = "{{ database_path }}/postgresql" postgresql['data_dir'] = "{{ database_path }}/postgresql/data"postgresql['unix_socket_directories'] = "{{ database_path }}/postgresql"
You must include another existing configuration line because the template module replaces your configuration file with the jinja2 (.j2) file content, which may misconfigure your current setup.
The ansible task for changing the configuration file and setting. Use the below code to complete these steps:
- name: backup existing configuration file to the /tmp folder
copy:src: "/etc/gitlab/gitlab.rb"dest: "/tmp"remote_src: yes- name: Copy GitLab configuration file. template: src: "{{ gitlab_config_template }}" #location of the .j2 jinja2 file dest: /etc/gitlab/gitlab.rb owner: root group: root mode: 0600 backup: yesnotify: restart gitlab
The handler task. To handle the job, you need to run the code which is mentioned below:
---
- name: restart gitlabcommand: gitlab-ctl reconfigureregister: gitlab_restartfailed_when: gitlab_restart_handler_failed_when | bool
After all the task is completed, you should have your GitLab database should be migrated to a new destination, as shown in the image below.
Database connection info before migration
Roll Back
/tmp/gitlab.rb
. Or the backup file created while Ansible ran the template module. Then reconfiguring the GitLab.In case of any unexpected error, you may need to SSH into your GitLab server and make the configuration changes manually.
- name: backup existing configuration file to the /tmp folder
copy:src: "/tmp/gitlab.rb"dest: "/etc/gitlab/"remote_src: yesnotify: restart gitlab
The handler task:
---
- name: restart gitlabcommand: gitlab-ctl reconfigureregister: gitlab_restartfailed_when: gitlab_restart_handler_failed_when | bool
Frequently Asked Questions
How can we check the remaining storage? Can it be seen through GitLab’s dashboard? How to check from cli?
To check the remaining space in your GitLab Linux server, you can use the df -h
command to display the disk space usage in a more human-readable format, with sizes in gigabytes, megabytes, etc.Unfortunately, for a self-managed GitLab server, we can’t use the dashboard to view the remaining storage.
df -h
command to display the disk space usage in a more human-readable format, with sizes in gigabytes, megabytes, etc.Will the process be the same if I need to change any other configuration?
Answer:
The simple answer is Yes.
To make any configuration changes to your self-managed GitLab server, you should change the configuration file located in the '/etc/gitlab/gitlab.rb'
and then run the command '
sudo gitlab-ctl reconfigure'
to apply the changes.
For example, you can change the repository storage location, add LDAP authentication, etc.
Can and How I view the other configurations from the GitLab’s Dashboard?
Answer: