Overcome Merge Hell with Trunk based development and Continuous Integration

Overview

Different teams and projects require different workflows, as no one-size-fits-all approach exists. To enhance collaboration and tackle problems like merge hell, software teams have experimented with various techniques over the years.

Version control systems (VCS) revolutionized code management by introducing branches and merging capabilities. This paved the way for various branching and development strategies like Feature Branching and GitFlow. However, despite their benefits, these strategies often face the ongoing challenge known as Merge Hell.

The solution to this problem? Continuous Integration (CI), coupled with trunk-based development, provides a powerful approach to mitigate merge hell. CI ensures frequent integration of changes, while trunk-based development reduces complexity by emphasizing a single, stable main branch as the source of truth, enabling smoother collaboration and reducing the risk of integration issues.

So, in this blog, we will learn about the use and importance of CI and TBD with all its advantages and workflow.

What is Version Control?

Version control is a method or software that keeps track of changes made to files over time, enabling teams or developers to collaborate, track modifications, and easily revert to previous states. It organizes different versions of files and maintains a detailed record of adjustments, including contributors, timestamps, and specific changes made.

Here’s a scenario: Consider that you are editing a document over time. You might occasionally want to review an earlier document version or check to see if anyone else has made any modifications. Using version control is like having a time machine for your files.

Similarly, taking reference to the figure above, let’s consider our application, which has two versions: v0.1 and v0.2. In the event that any issues arise with v0.2, we always have the option to revert to the previous version.

Additionally, we can perform tasks such as comparing different versions, reverting to previous states, and merging changes from different versions. This flexibility allows us to manage our application effectively and address any problems that may arise while incorporating and integrating various updates.

Here, we will focus on git as our default version control.

Also Read: What are the Benefits of Amazon S3 Glacier?

Why branch in version control?

Branching in version control is a powerful feature that allows for parallel development and isolation of changes within a project. It makes it possible for numerous people or teams to work on various features, bug fixes, or experiments simultaneously without interfering with one another’s efforts.

Additionally, branching supports orderly and effective software development operations. Teams can set up branches for certain features, improvements, or bug fixes, enabling developers to work on several projects simultaneously. This concurrent development strategy boosts output and facilitates meeting project deadlines. In addition, branches offer a distinct separation of concerns, making it simpler to track changes, examine code, and, if required, roll back adjustments. Effective branching enables teams to maintain a disciplined and cooperative development process, facilitating easier change integration and reducing codebase conflicts.

Different Development Workflows: A Comparison

Over time, as Git and other version control systems developed to suit the demands of diverse development scenarios, new Git workflows were added.

Git workflows are several methods or techniques that teams can use to coordinate and manage projects while utilizing the Git version control system. Git workflows motivate DevOps teams and developers to use Git efficiently and consistently. Let’s investigate and evaluate a few typical Git workflows:

Centralized Workflow:
A central repository acts as the only reliable source of information in this workflow. The central repository is copied by developers, who then edit it and push the changes back.
Feature Branch Workflow:
This process focuses on creating separate branches for bug fixes or new features. Each new feature or problem receives its branch, which the developers work on before merging into the main branch.
Due to The independence of each feature branch, it enables simultaneous development.
GitFlow Workflow:
Using the branching model known as GitFlow, development is divided into several branches for features, releases, and bug fixes. It makes use of branches such as “develop” for current work, “feature” for new features, “release” for preparing releases, and “hotfix” for urgent fixes.
It works well for projects that need rigorous versioning and frequent releases.
Forking Workflow:
The forking method became more common as open-source development and remote team participation grew. Developers can fork a central repository to distinguish their copy from the original. Developers can freely modify their forks using this approach and then send pull requests to the main repository for review and merging.
Forking encourages a decentralized development approach by enabling contributions from other collaborators while retaining control over the primary repository.

Challenges of Traditional Development Workflows

The traditional process, also known as the centralized workflow, faces a number of difficulties, one of which is referred to as “merge hell”, which we will discuss in the next section.

The centralized workflow allows for numerous developers to work together directly on a single branch, making changes as they go and committing them to the shared repository. Conflicts and challenges arise when integrating modifications from various team members as the number of developers and code changes grows. The resolution of these disputes might become difficult and drawn out.

Also Read: Unlock Cloud Migration with 8 Powerful AWS Migration Tools

Alternative workflows, like the Feature Branch Workflow or GitFlow, were introduced to address these issues. These processes encourage a more organized method of branching and let programmers focus just on one feature or task at a time. As a result, only fewer merge conflicts arise and modifications may be incorporated into the main branch more easily.

But even though alternative workflows aim to reduce merge conflicts and alleviate merge hell, it is still possible for merge conflicts to occur. Here are a few reasons why merge conflicts can still happen:

Long-lived branches:
If feature or development branches have long lifespans, they are more prone to accumulating conflicting changes over time. Conflicts are more likely to arise while attempting to merge a branch the longer it hasn’t been merged into the main branch.
Parallel development:
When several developers work on various features or tasks simultaneously, there is always a chance that some modifications will overlap and conflict when the code is merged. Even with well-organized workflows, this can still occur if engineers don’t consistently update their branches with the most recent modifications from the main branch.
Lack of communication and coordination:
Communication breakdowns or poor teamwork can cause conflicting modifications in several branches in collaborative contexts. Conflicts may occur when attempting to merge these branches if correct synchronization and alignment are not maintained.
Major code refactor changes:
When considerable code refactoring or reorganization occurs, it may result in extensive changes to numerous files and modules. When attempting to integrate the refactored code with existing branches, these changes may conflict with other ongoing development projects, resulting in merge conflicts. While refactoring aims to improve the code’s quality, readability, and maintainability, it can introduce conflicts when merging the refactored code with other branches. These conflicts arise when the changes made during refactoring overlap with modifications made in parallel by other developers or teams.

What is Merge Hell?

Merge hell refers to a situation where frequent and complex merge conflicts occur during the process of merging code changes from multiple branches. It frequently leads to time-consuming and clumsy attempts to resolve these issues.

Let us consider a scenario,

Imagine a software development project where Alice and Bob, two developers, are each working on a feature branch that is descended from the main branch. Bob is working on a feature that will create a notification system, while Alice is developing a feature that will provide user authentication capability.

Alice and Bob starts working on their respective feature branches at once, making frequent changes to the codebase.
After some time, Alice finishes her feature and successfully merges it into the main branch without any issues.
Meanwhile, Bob continues working on his feature branch, making significant changes to various files, including some areas that Alice modified for the user authentication feature.
When Bob finishes his feature and attempts to merge it into the main branch, conflicts arise due to the overlapping changes in the codebase. This is merge hell in action.
Bob discovers that the changes Alice made to user authentication conflict with how he implemented the notification system. He has to alter his code to consider the new authentication feature, which adds more time to the dispute resolution process.
As more conflicts arise during the merge process, Bob’s productivity decreases, and frustration builds up. Determining the root cause of conflicts and finding an optimal resolution becomes challenging.

This illustration shows how developers working on different branches with overlapping modifications might end up in merge hell. One of the ways to reduce such hassle is by implementing Continuous Integration along with Trunk-Based Development.

Intro to Continuous Integration and Trunk-Based Development

Continuous Integration (CI) is a development methodology in which programmers routinely integrate their code changes into a common repository. The primary goal of CI is to automatically integrate and validate code changes in order to identify and fix integration problems as soon as possible.

Here’s how continuous integration works:

Code Repository:
- Developers modify the source code while working on their local codebases.
Automated Build:
- A CI server or build system instantly gets the most recent code when developers commit their modifications to the repository.
Automated Testing:
- A series of automated tests are run after the code has been created. This covers integration tests, unit tests, and other project-specific test types.
Integration and Validation:
- The code is incorporated into a common branch or trunk after it passes the automated tests. To ensure that all modifications coexist without conflict, the CI server combines the changes with the current codebase.
Notification and Feedback:
- The build and test results receive quick feedback from the CI server. The build status, test results, and any potential integration problems are communicated to developers.

Trunk-based development (TBD) is a modern software development approach that emphasizes keeping a single branch, called the “trunk” or “mainline” as the primary development branch.

The idea is you work in master or close to master and you make small simple changes and get them integrated into the main as soon as possible. The goal of trunk-based development is to promote collaboration, streamline development, and enable faster feedback loops.

One of the best things about TBD is that it enables code reviews and makes it much more easier and efficient. It is easier to review smaller chunks of code rather than hopelessly gazing at 1000 lines of code changes.

Trunk-based development also allows for relatively short-lived branches because they are removed as soon as a developer pushes or submits their changes to the trunk.

But why are short-lived branches a good thing?

Think branching as an integration credit card. And just like any credit, you eventually have to pay off the debt at some point. The more the debt remains the more it gets difficult. With short-lived branches, you don’t have to worry about debt accumulating in the form of merge conflicts.

Let's see how Trunk Based Development works,

In trunk-based development, the primary branches are the trunk (also known as the main branch or the master branch) and feature branches. The trunk represents the mainline development, while feature branches are created to work on specific features or changes.

The developer occasionally updates the feature branch with the most recent modifications from the trunk when working with it. The modifications from the trunk into the feature branch are merged or integrated during this process. By doing this, the feature branch maintains sync with the most recent version of the trunk and incorporates any updates or problem fixes.

The developer repeatedly works on the feature branch throughout the development process, adding and changing things as needed. The feature is prepared to be merged back into the trunk once it has been finished and tested.

A final merge operation is carried out by the developer to combine the feature branch with the trunk. The new feature and the current codebase are combined in this process by incorporating the modifications made in the feature branch into the trunk. The completed feature is now a part of the mainstream development because it is now a part of the trunk.

How TBD and CI mitigate the problem of Merge Hell

In the above scenario of Bob and Alice, let's see how they can use TBD and CI to solve their problems.

Create feature branches: Bob and Alice would create their own feature branches from the main/trunk branch to work on their respective features. For example, Alice could create a branch called “Auth Branch” and Bob could create a branch called “Notification Branch.”
Develop features in their branches: Alice would work on implementing the authentication feature in her branch, while Bob would work on the notification system in his branch. They would make regular commits to their feature branches as they progress. Alice completes a part of the feature and merges the changes in the trunk in a short time. Once the feature gets successfully merged, the branch gets deleted.
Bring Trunk up to date: In the meantime, Bob regularly examines any modifications made to the trunk, merges those modifications, and integrates them with his own codebase. After thorough testing, his changes are also merged into the trunk.
Pull Requests: The pull request requests the original repository’s maintainers to review your changes and potentially merge them into the main codebase. Alice and Bob do not directly integrate their code into the main branch in the given situation. Instead, they submit a pull request, which serves as a request to merge their code changes into the primary branch. This pull request undergoes a review process where designated reviewers evaluate the code and decide whether or not to proceed with the merge.
During the review, the reviewers thoroughly examine the code, test its functionality, and assess its adherence to coding standards and best practices. They may also consider performance, scalability, security, and overall system compatibility factors.
Reviewers play a crucial role in ensuring the quality and reliability of the codebase. They have the expertise to identify potential issues, bugs, or design flaws. If any such problems are found, the reviewers can provide valuable feedback and suggest changes that must be addressed before merging the code. These suggestions may include improvements, optimizations, or adjustments to make the code more efficient, maintainable, or compliant with the project’s guidelines.
Continuous Integration: Continuous Integration (CI) is triggered every time Bob or Alice pushes their code changes, or a pull request is created, and it initiates automated build and testing processes. This streamlined approach significantly eases and accelerates the development cycle.
CI springs into action whenever Bob or Alice pushes their code, executing automated tasks. Firstly, it initiates the automated build process, which involves compiling the code, resolving dependencies, and generating the necessary artifacts. Automating this process minimizes potential human errors, ensuring consistency in the build.
Simultaneously, CI kicks off the automation testing phase. This encompasses a suite of tests that verify the functionality and quality of the code. It includes various tests, such as unit tests, integration tests, and other relevant tests specific to the project. The automated tests provide quick and accurate feedback on the correctness of the code changes.
Several well-known CI/CD (Continuous Integration/Continuous Deployment) technologies offer automated build and test capabilities for CI implementation. These are commonly used CI tools: Jenkins, Travis CI, CircleCI, GitLab CI/CD, and GitHub Actions.
Here is an example of CI workflow with automated testing. Every time code is pushed into the trunk, a series of tests runs to ensure the correctness and quality of code.

Conclusion

In conclusion, the combination of continuous integration (CI) and trunk-based development offers significant advantages for the development process. CI detects conflicts and compatibility issues early, preventing them from accumulating and reducing the likelihood of merge conflicts later on. Trunk-based development promotes collaboration, streamlines development efforts, and minimizes the risk of conflicts through iterative and focused changes.

Together, CI and trunk-based development create an efficient and harmonious environment, ensuring an up-to-date and stable codebase, smoother code integration, and faster development cycles. By adopting these practices, teams can deliver high-quality software more effectively and with reduced merge conflicts.