python asyncio gets deadlock if multiple stdin input is needed

Question

I wrote a command-line tool to execute git pull for multiple git repos using python asyncio. It works fine if all repos have ssh password-less login setup. It also

Answer 1

I ended up using a simple solution suggested by @vincent, i.e., disable any existing password mechanism by setting the GIT_ASKPASS environment variable, run async on all repos, and re-run the failed ones synchronously.

The main logic changes to

cache = os.environ.get('GIT_ASKPASS')
os.environ['GIT_ASKPASS'] = 'echo'
errors = utils.exec_async_tasks(
    utils.run_async(path, cmds) for path in repos.values())
# Reset context and re-run
if cache:
    os.environ['GIT_ASKPASS'] = cache
else:
    del os.environ['GIT_ASKPASS']
for path in errors:
    if path:
        subprocess.run(cmds, cwd=path)

In run_async and exec_async_tasks, I simply redirect error and return the repo path if subprocess execution fails.

async def run_async(path: str, cmds: List[str]) -> Union[None, str]:
    """
    Run `cmds` asynchronously in `path` directory. Return the `path` if
    execution fails.
    """
    process = await asyncio.create_subprocess_exec(
        *cmds,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
        cwd=path)
    stdout, stderr = await process.communicate()
    stdout and print(stdout.decode())
    if stderr:
        return path

You can see this pull request for the complete change.

Further Update

The PR above resolves the problem when https type remote requires username/password input, but still has problem when ssh requires password input for multiple repos. Thanks to @gdlmx's comment below.

In version 0.9.1, I basically followed @gdlmx's suggestion: disable user input completely when running in the async mode, and the failed repos will run the delegated command again using subprocess serially.