Working with manual approvals for multiple builds in AWS CodePipeline

Question

We have a CodePipeline set up to do a build, deploy to a QA ECS environment, then a manual approval step to deploy to Prod.

What gets confusing though, is when there are

Answer 1

I had the same problem. Manual approvals are confusing since several pipeline executions can get queued and it's easy to lose track of things. I think we can blame this on CodePipeline's bad UX.

The workaround I settled with is to have two identical pipelines for the same project. They have the same source stage (same repo/branch) but different deploy stages (one deploys to QA, one deploys to prod). No more manual approval stages. The QA pipeline is set to auto-execute when changes in the source (repo/branch) are detected while the Prod pipeline needs to be manually released.

Basically, we replaced the Manual Approval with Manual Release. Manual release always releases the latest from source unlike manual approvals.

Answer 2

You should place the deploy and approval action in the same stage. This lets you approve exactly what you tested. Why? Because exactly one pipeline execution can be in a pipeline stage at any given time.

...approve the latest build, in case the earlier builds had issues that were fixed by the later builds.

If you want to let later builds catch up, reject the earlier build that is waiting for approval.

Answer 3

Well, we can solve this problem as how you describe it with development, but it might also be a process glitch.

For example: If we have a development branch, a release branch (staging) and a master branch ( production ) we could easily solve this issue.

Development branch Things we develop will be going through the development branch stage where we don't need the manual approval, as we don't want to check every changes. We have setup automated unit tests for that.

Release branch This will deploy to the staging environment where we extensively test the software quality, also based on the regression tests on an acceptance chain with acceptance systems. This should prevent all the big issues before merging towards master branch. Next to that, we could also manually test the release branch on the staging environment. If this works, be happy and easily migrate towards master

Master branch This will deploy to the production environment with a manual approval before the actual deployment is taking place, knowing for sure you will only push 1 change, being the merge from release to master, preventing the issues you've summarized in the ticket.

Another way is to develop a new AWS feature where you can uncheck or check a checkbox saying: always take the latest release, but that will not help adding value to the pipeline integration as things will be pushed without testing well enough.

Answer 4

One option if you don't want to have multiple pipelines is to by default disable stage transitions into your environments that required controlled releases.

When you are ready to deploy into an environment, you enable the stage transition to allow the most recent release from the previous stage to be processed and then disable the transitions again.

It's still a bit clunky, but reasonably effective once you get used to it. Having to reject each change that comes through becomes very slow and cumbersome to manage, so by disabling transitions you choose when to promote a release.

IMO, CodePipeline should have an option to automatically supersede executions if they are paused at the manual approval stage.

Answer 5

In the CodePipeline UI, you can see the history of Manual approvals in your pipelines' History. Click on History to see what's in progress (Manual Approvals that haven't timed out will always be in progress) and the source (git) short-sha that triggered it (if you need to narrow down to the relevant commit).

To know which Manual approval you're approving, in Pipeline view, click on View current revisions next to the Manual step (to get the Execution ID), then find the matching Execution ID in History (should be the oldest one).

Only way I found to get to the latest Approval is to hit reject n-1 times in the pipeline (where n is how many manual approvals are still in progress) until I only have 1 approval left (or until I find matching Execution ID).