aws s3 command does not work in a batch file triggered by a Windows task scheduler

Question

I have a batch file C:\\upload_to_s3.bat. In this file, there is a line:

aws s3 sync D:\\S3\\batch1\\ s3://MyBucket/batch1 --exclude *.bat

I have Windows tas

Answer 1

In windows bat file what works is a combination of user2415376 and docesam answers.

Here it is:

set userprofile=C:\Users\%username%
aws configure set AWS_ACCESS_KEY_ID <your_key>
aws configure set AWS_SECRET_ACCESS_KEY <your_secret>
aws configure set default.region <your_region>

other aws commands here

Answer 2

The reason this is happening, is because aws command is not finding the the config or the credentials file! And it happens for Win2012 too.

The problem is that Windows Task Schedule sets the %USERPROFILE% to C:\Users\Default instead of the actual user the task is running under!! ??? And AWS is using that to find the config and credentials files.

So the fix is to put something like this in your script (either BAT or PowerShell)

:: in a BAT script
set userprofile=C:\Users\%username%

# in a PowerShell
$env:USERPROFILE = "c:\users\$env:USERNAME"

Even the %homepath% variable is blank when running from a Scheduled Task.

I just discovered this after procrastinating trying to figure out why this was happening for us.

set AWS_DEFAULT_PROFILE=namedprofile
set AWS_CONFIG_FILE=C:\Users\xxxx\.aws\config

Only until I added the AWS_CONFIG_FILE line above in my script was AWS able to actually see the config file! However it can not find the credentials now, and I don't see anywhere how to set that.

This is the error I get without telling where the config file is:

Traceback (most recent call last):
  File "aws", line 27, in <module>
  File "aws", line 23, in main
  File "awscli\clidriver.pyc", line 50, in main
  File "awscli\clidriver.pyc", line 176, in main
  File "awscli\clidriver.pyc", line 157, in _create_parser
  File "awscli\clidriver.pyc", line 91, in _get_command_table
  File "awscli\clidriver.pyc", line 111, in _build_command_table
  File "botocore\session.pyc", line 680, in emit
  File "botocore\hooks.pyc", line 226, in emit
  File "botocore\hooks.pyc", line 209, in _emit
  File "awscli\customizations\preview.pyc", line 71, in mark_as_preview
  File "awscli\clidriver.pyc", line 349, in service_model
  File "awscli\clidriver.pyc", line 366, in _get_service_model
  File "botocore\session.pyc", line 256, in get_config_variable
  File "botocore\session.pyc", line 277, in _found_in_config_file
  File "botocore\session.pyc", line 349, in get_scoped_config
botocore.exceptions.ProfileNotFound: The config profile (backup) could not be found

And then this is the error I get after setting the AWS_CONFIG_FILE variable:

Unable to locate credentials
Completed 1 part(s) with ... file(s) remaining

Answer 3

Include the following lines in your batch file:

set AWS_ACCESS_KEY_ID=your_access_key_id

set AWS_SECRET_ACCESS_KEY=your_secret_access_key

AWS Documentation

Answer 4

i was able to solve the problem by adding the following 3 lines in the batch file BEFORE your s3 command

aws configure set AWS_ACCESS_KEY_ID <your acess key id>
aws configure set AWS_SECRET_ACCESS_KEY <your secret access key here>
aws configure set default.region eu-west-1

replace the stuff between angled brackets with your data and the region with your bucket's region. and yes ,there is no equal sign between "AWS_ACCESS_KEY_ID" and the key.