I have a private repo on Github that houses 3 submodules, all 3 of which are also private.
I have generated 4 SSH keys on my EC2 server and applied them as Github de
github's authentication is a bit odd. They don't use usernames; they just infer based on the public key you presented which user you are. Since you generated four deploy keys, it's anyone's guess which one will be used by your server when it connects to github - github will accept any of them, then reject any access to repositories that don't have that key registered.
As such, the simplest solution is to just use a single deploy key for all of the repositories.
If you can't, however, you can hack around this using ssh host aliases. Add to your server's ~/.ssh/config
stanzas like the following:
Host repo-foo
HostName ssh.github.com
Port 443
User git
IdentityFile /path/to/my-ssh-key-file-for-foo
IdentitiesOnly yes
Host repo-bar
HostName ssh.github.com
Port 443
User git
IdentityFile /path/to/my-ssh-key-file-for-bar
IdentitiesOnly yes
Then point your submodules at repo-bar:username/bar.git
and repo-foo:username/foo.git
rather than using the git@github.com:...
form.
This will effectively cause git and ssh to treat each repository as living on a different server, and pass in an explicit identity file, so there is no confusion over what key to use.