I have a submodule in my git repository and my directory structure is like,
app
-- folder1
-- folder2
-- submodule @5855
I have deployed
It has been so long since the issue with submodules was found. But aws cannot fix it. So codepipeline cannot send .git directory to codebuild. So we have to invent new bicecles, my fix is this in buildspec.yml pre-build command
rm -rf $PWD/*
git clone --depth 1 https://<REPO NAME> -b develop .
git submodule update --init --recursive
AWS hurry up, cuz our team is considering move back to github.
Borrowing from the concepts of @MattBucci and @JoshuaEvans. Here is what we did. We could not install git in our pipeline, because of strict access/security issues with ssh. So we ended up doing this thru HTTP instead.
version: 0.2
env:
variables:
token: " token "
secrets-manager:
personalAccessToken: $personalAccessTokenPath
phases:
install:
runtime-versions:
nodejs: 12
commands:
- cd <to the folder where you want the submodules>
- wget --header="Authorization:$token$personalAccessToken" --content-disposition https://github.com/uri/zipball/$branch/
- unzip project*.zip -d project-folder
- rm -rf project*.zip
- cd project-folder
- mv project*/* .
- rm -rf project*
- cd <back to your base folder where the buildspec is>
pre_build:
commands:
- xxx
build:
commands:
- xxx
post_build:
commands:
- xxx
artifacts:
files:
- '**/*'
base-directory: dist
Hope this helps!
I ran into this issue myself and, thanks to the awesome suggestions by @matt-bucci I was able to come up with what seems like a robust solution.
My specific use-case is slightly different - I am using Lambda Layers to reduce lambda redundancy, but still need to include the layers as submodules in the Lambda function repositories so that CodeBuild can build and test PRs. I am also using CodePipeline to assist with continuous delivery - so I need a system that works with both CodePipeline and CodeBuild by itself
I created a new SSH key for use by a "machine user" following these instructions. I am using a machine user in this case so that a new ssh key doesn't need to be generated for every project, as well as for potential support of multiple private submodules
I stored the private key in the AWS Parameter Store as a SecureString. This doesn't actually change anything within CodeBuild, since it's smart enough to just know how to decrypt the key
I gave the "codebuild" role AWS managed property: AmazonSSMReadOnlyAccess - allowing CodeBuild to access the private key
I made my buildspec.yml file, using a bunch of the commands suggested by @matt-bucci, as well as some new ones
# This example buildspec will enable submodules for CodeBuild projects that are both
# triggered directly and via CodePipeline
#
# This buildspec is designed with help from Stack Overflow:
# https://stackoverflow.com/questions/42712542/how-to-auto-deploying-git-repositories-with-submodules-on-aws
version: 0.2 # Always use version 2
env:
variables:
# The remote origin that will be used if building through CodePipeline
remote_origin: "git@github.com:your/gitUri"
parameter-store:
# The SSH RSA Key used by our machine user
ssh_key: "ssh_key_name_goes_here"
phases:
install:
commands:
# Add the "machine user's" ssh key and activate it - this allows us to get private (sub) repositories
- mkdir -p ~/.ssh # Ensure the .ssh directory exists
- echo "$ssh_key" > ~/.ssh/ssh_key # Save the machine user's private key
- chmod 600 ~/.ssh/ssh_key # Adjust the private key permissions (avoids a critical error)
- eval "$(ssh-agent -s)" # Initialize the ssh agent
- ssh-add ~/.ssh/ssh_key # Add the machine user's key to the ssh "keychain"
# SSH Credentials have been set up. Check for a .git directory to determine if we need to set up our git package
- |
if [ ! -d ".git" ]; then
git init # Initialize Git
git remote add origin "$remote_origin" # Add the remote origin so we can fetch
git fetch # Get all the things
git checkout -f "$CODEBUILD_RESOLVED_SOURCE_VERSION" # Checkout the specific commit we are building
fi
# Now that setup is complete, get submodules
- git submodule init
- git submodule update --recursive
# Additional install steps... (npm install, etc)
build:
commands:
# Build commands...
artifacts:
files:
# Artifact Definitions...
This install script performs three discrete steps
It installs and enables the ssh private key used to access private repositories
It determines if there is a .git folder - if there isn't then the script will initialize git and checkout the exact commit that is being built. Note: According to the AWS docs, the $CODEBUILD_RESOLVED_SOURCE_VERSION
envar is not guranteed to be present in CodePipeline builds. However, I have not seen this fail
Finally, it actually gets the submodules
Obviously, this is not a great solution to this problem. However, it's the best I can come up with given the (unnecessary) limitations of CodePipeline. A side effect of this process is that the "Source" CodePipeline stage is completely worthless, since we just overwrite the archived source files - it's only used to listen for changes to the repository
Better functionality has been requested for over 2 years now: https://forums.aws.amazon.com/thread.jspa?threadID=248267
I realized (the hard way) that my previous response didn't support CodePipeline builds, only builds run through CodeBuild directly. When CodeBuild responds to a GitHub Webhook, it will clone the entire GitHub repository, including the .git folder
However, when using CodePipeline, the "Source" action will clone the repository, check out the appropriate branch, then artifact the raw files without the .git folder. This means that we do have to initialize the github repository to get access to submodules
SSH is not needed if you're using CodeCommit as a repository. Use the AWS CLI Credential Helper and clone over https.
git config --global credential.helper '!aws codecommit credential-helper $@'
git config --global credential.UseHttpPath true
git clone https://git-codecommit.[region].amazonaws.com/v1/repos/[repo]
After banging my head against this all day, I've found a simple solution (for Code Pipeline) that doesn't require any SSH key juggling in the buildspec. I am using Bitbucket but I would think this would work for other providers. I'm also cloning my submodule via https, I'm not sure if that's a requirement or not.
Configure your source to do a full clone of the repository. This will pass along the git metadata that you need.
Configure your build role to add a customer-managed UseConnection permission to give your build action access to the credentials you configured for your source. Documentation from AWS here: https://docs.aws.amazon.com/codepipeline/latest/userguide/troubleshooting.html#codebuild-role-connections
Set up your env to include git-credential-helper: yes and clone the submodule in your buildspec.yml:
And that's it! Submodule will be available for build, and without having to do a bunch of key configuration for every submodule you want to use.
Maybe a good addition to the documentation if this ends up being useful for people.
I faced the same issue on AWS CodeBuild. I tick Use Git submodules
like below image to update my submodule
.
When I run the build I got following error,
CLIENT_ERROR: Submodule error error creating SSH agent: "SSH agent requested but SSH_AUTH_SOCK not-specified" for primary source and source version refs/heads/dev
So I googled the above error and got this DOWNLOAD_SOURCE Fails with Git submodules thread from AWS Forum. They've mentioned,
The submodules must be configured as https and not ssh.
I think this is useless, what will happen someone setup submodule
as ssh
. I also did the same, here is my .gitmodules
file.
[submodule "common"]
path = common
url = git@bitbucket.org:organization_id/common.git
Really I don't want to change it to https
. Then I found this Working with Git Submodules in CodePipeline article from medium. I would like to visualize what I did to solve this issue and there was an error that didn't mention in that article. Let's do this in more secure way.
First go to the AWS Key Management Service (KMS) and go to the Customer managed keys
section and click the Create key
to create the key.
Symmetric
and click Next
.bitbucket-credentials
) to create Alias
and click Next
.AWS Role
to configure any of Developer Tools on AWS, so in my case I created a AWS Role
call ecsCodeBuildRole
for AWS CodeBuild and give the Define key administrative permissions
for it and click Next
.Define key usage permissions
for your AWS Role
and click Next
.Finish
to create the CMK.So AWS Key Management Service (KMS) part is done, now go the AWS Systems Manager and find the Parameter Store
section. Click Create parameter
.
id_rsa
and put the same things like below.
value
section, just run cat ~/.ssh/id_rsa
command in your terminal and you'll get the output like below. Add it to the value
section.-----BEGIN RSA PRIVATE KEY-----
qdjbXp+42VTnccC7pxOZcofomfwGXPWuqcv99sQEPtToODvGIxWoooJUpb6qMIWY
1zccEuwAhmqcPvpsJyWhcctZB/wWglNvViZcOYjrQ8HBUBKJT8pF
-----END RSA PRIVATE KEY-----
Create another parameter and name it as id_rsa.pub
. Follow the same steps like above.
For the value
section, just run cat ~/.ssh/id_rsa.pub
command in your terminal and you'll get the output like below. Add it to the value
section.
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGtf8jjkogWxRGNGjJlOZ1G+pWExgDOdA5wVML6TMTT2YtvhPJD60nPx5TfA8zRzGWubwrPp40SPAhSs5wiAAg38HlS4pz9X wasdkiller@wasdkiller
As my research(maybe I'm wrong, please correct me) I don't have any other way to pass ssh
credentials to the AWS CodeBuild without this much of effort. So I changed my buildspec.yml
file manually like this.
version: 0.2
env:
parameter-store:
ssh_key: id_rsa
ssh_pub: id_rsa.pub
phases:
install:
commands:
- mkdir -p ~/.ssh
- echo "$ssh_key" > ~/.ssh/id_rsa
- echo "$ssh_pub" > ~/.ssh/id_rsa.pub
- chmod 600 ~/.ssh/id_rsa
- eval "$(ssh-agent -s)"
- git submodule update --init --recursive
When you continue you'll get below error surely,
Decrypted Variables Error: AccessDeniedException: User: arn:aws:sts::organization_id:assumed-role/ecsCodeBuildRole/AWSCodeBuild-12896abb-bdcf-4cfc-a12b-bcf30d6e96ab is not authorized to perform: ssm:GetParameters on resource: arn:aws:ssm:ap-southeast-2:organization_id:parameter/wasd status code: 400, request id: 23b94bc2-961e-4d86-9b73-d16e3bda357c
It'll ask you for ssm:GetParameters
permission, Just attach AmazonSSMReadOnlyAccess
policy or create policy manually with the ssm:GetParameters
permission and attach it to your AWS Role
, it'll solve this issue.