Nearly Automatic Branch Merging With Bitbucket Cloud
This is going to be a story about nearly automatic branch merging under a Gitflow workflow with Atlassian Bitbucket Cloud. If you are familiar with automatic branch merging with Bitbucket Server, you’ll know that it’s a super awesome feature that keeps your newer release branches up-to-date and your team productive when you are doing fixes against older releases. If you are a Bitbucket Cloud user you might know that since BCLOUD-14286, well, the tedious task of managing hotfixes or latefixes at the tail end of a release that’s been cut in concert with current development is all up to you! Hooray for a ball of unmanageable merge conflicts!
The TL;DR is this lack of this feature became somewhat of a minor hellscape for our team so we hacked together a tiny web application to emulate the missing feature in conjunction with Bitbucket’s webhooks. If you just want to go check out the end result and adapt it for your purposes, we’ve put it up on Github. The rest is simply a memorial to this missing feature and my testament to the product prioritization gods.
When I founded Funraise some years back with some of my friends, we didn’t really have a lot of resources up front for a lot of different engineering or technical support roles. This is probably a not unfamiliar notion for any of my hardened software-startup-battle-gnomes out there. At early stage startups, everyone has to wear different hats and take on different responsibilities. As one does, we tried to shift as much supporting infrastructure like issue trackers, source control, build systems, etc to the cloud so no one had to babysit application servers as their side gig. I had really been spoiled by Atlassian Stash in the early 2010s and Atlassian had a new cloud offering for hosted Git. The choice to go cloud on a product we already liked seemed pretty obvious in the moment.
At Funraise we also really love our release trains. We’ve adopted the model where we cut a release branch on a predetermined point in the month every month and whatever new features are on develop at the time, those features will ship in the next release (post rounds of QA and any late fixes that need to happen). It’s been a pretty good experience from a development perspective — we don’t typically try to cram in features to a particular release at the last minute and we don’t pin a release date to the completion of a particular feature. If a major feature isn’t ready by the merge window, it’ll just take the extra time it needs to hop on the next one. We release a major feature release every month and the process has been really smooth.
Things happen though and invariably we have to do a hotfix to the current release (or several) by the time a new major release is ready. We also typically have several major feature branches in flight at any one point in time. And then, there are all those fixes that we are doing against the release in the stabilization phases of prepping a release. Point is — there could be some divergence between develop and the current release depending on what went on between post release cut and pre-deployment.
Now, Bitbucket Server (Formerly Atlassian Stash) is the self-hosted version of Atlassians Bitbucket offering. You know, the one I just told you we didn’t end up using because no one wanted to baby sit app servers. It has a feature that made this workflow a little nicer by automatically creating and merging “child” pull requests for any pull request originally targeted a branch with a lower semantic version. So, if we need to do a hotfix to a particular release say 2021.4.0 that is about to go out the door, when we merge that pull request Bitbucket Server will automatically merge that branch down to develop and any other open semantically higher versioned branch. In the happy path case, there is no action from developers. If a conflict happens, the PR isn’t automatically merged, but assigned to the original author to handle that single set of conflicts in isolation. Nice, no need for developers to open multiple pull requests or some unlucky person to have to figure out ALL the conflicts at once even though they might not be the best person to do so.
Welp, that luxury doesn’t exist in the cloud version of Bitbucket and has been on the product backlog for the last four years.
After our first few releases back when we were a smaller team, and much smaller codebase, we chalked up the lack of the automatic branch merging feature in Bitbucket vs Stash / Bitbucket Server to an unfortunate product gap that we’d just have to work around. No big deal at first, our releases were smaller, and we didn’t really have a sophisticated CI/CD or hotfix strategy. We also mostly just picked our releases and shipped what we had and didn’t have a lot of parallel work going on.
As we grew, and things started to get more complicated, we really started to miss that old automatic merge feature. It was pretty typical post release or after a series of divergences happened to see something like in slack:
any @ui heroes around who can merge master -> develop?
In fact, I can find hundreds of such messages. Sometimes this operation is straightforward, and sometimes its not very because the person doing the merge might not be familiar with all the changes that happened. For a time we tried to mitigate this by having developers submit a couple of pull requests if there were doing a late fix or hotfix. One for the release they were targeting, and one for current development. We found that this tended to hamper productivity and made it so PRs stayed open longer than we liked and sometimes it was difficult to get everyone on the same page of what branches they should be targeting. If we had cascade merging, any latefix or hotfix to a release would target the release branch say release/2020.12.0 which would then get automatically cascaded to develop so any conflicts post release merge back should be small or non-existent.
In addition to the overhead incurred by having to continually perform this branch maintenance ritual there is good chance that on occasion a manual cascade merge might not happen in a timely enough fashion and a follow up hotfix release might not contain fixes from a previous release and you could accidentally revert a fix that you had previously deployed.
You might ask yourself, “What’s going on here? It’s the same product, right? Does Atlassian just hate us or are they trying to encourage people toward on-premise deployments of Bitbucket Server for those sweet Enterprise Bucks??” Nah, they haven’t gone full Emerald City yet. In fact, they’re ending new sales of server licenses effective 2021 with end of life on February 2, 2024. RIP. So I guess no one gets this awesome feature.
This gap in parity between the products is an artifact of a series of acquisitions and product mergers. Atlassian originally released a product called Stash back in 2010 this product was designed like a lot of it’s cousins. It was a Java+Maven type project. Atlassian then purchased Bitbucket which was a hosted source control provider. Bitbucket was written in Python using Django. So, really they’re just entirely different code bases and products. Atlassian just hasn’t gotten around to translating that original cool feature they built into the product that they acquired which is written in python — or who knows, maybe they are trying to rewrite it in their Java native tongue. At any rate, it just hasn’t been on their radar, much to our chagrin and despite how many junior devs we’ve sacrificed on the alter of storms.
Our solution is pretty straight forward. Basically, our web app is a single route that is listening to every type of signal that it can for a particular Bitbucket repository like this:
If we see a POST request from Bitbucket and the X-Event-Key header is pullrequest:fulfilled, we know that a PR just got merged and we should check to see if we need to need to create another pull request that targets an upstream branch. For example, a pull request bugfix/FUN-1234 →release/2021.4.1 got merged. OK — now we need to create a pull request for release/2021.4.1 → release/2021.5.0 if that release has been cut already.
The logic for that is reasonably straight forward. The main gimmick is finding all of the release branches and then picking the next highest semantic version to cascade to. If none can be found, we just target the default development branch which is develop in our case:
func (service *BitbucketService) NextTarget(oldDest string, cascadeTargets *[]string) string {
targets := *cascadeTargets
//Change release/YYYY.M.P --> vYYYY.M.P
destination := strings.ReplaceAll(oldDest, service.ReleaseBranchPrefix, "v")
//Change release/YYYY.M.P --> vYYYY.M.P
for i, _ := range targets {
targets[i] = strings.ReplaceAll(targets[i], service.ReleaseBranchPrefix, "v")
}
sort.SliceStable(targets, func(i, j int) bool {
return semver.Compare(targets[i], targets[j]) < 0
})
for _, target := range targets {
if semver.Compare(destination, target) < 0 {
return strings.ReplaceAll(target, "v", service.ReleaseBranchPrefix)
}
}
return service.DevelopmentBranchName
}And ah-ha! Success. Now when a hotfix pull request comes in, we get nice small cascade pull requests that our team can straight away merge. It’s a little less good than the original Stash feature since automatic cascade was truly automatic in that if the branch could be merged, it was auto merged. Our particular case doesn’t really support that workflow since we have premium features turned on like build validation and some other fun stuff so we’re happy enough to just have a button click merge once the build and our other processes check out.
We’d rather be building cool features for awesome non-profits than mucking around with merge conflicts at the end of the release so hopefully Atlassian adds this feature to Bitbucket Cloud natively soon and makes this humble app completely superfluous.
