问题
I am posting a json to AWS elasticsearch,using a java lambda function.
public Object handleRequest(DynamodbEvent dynamodbEvent, Context context) {
//code to general the json document
AmazonDynamoDBClient amazonDynamoDBClient = new AmazonDynamoDBClient();
List<DynamodbEvent.DynamodbStreamRecord> dynamodbStreamRecordlist = dynamodbEvent.getRecords();
if (!dynamodbStreamRecordlist.isEmpty()) {
DynamodbEvent.DynamodbStreamRecord record = dynamodbStreamRecordlist.get(0);
if(record.getEventSource().equalsIgnoreCase("aws:dynamodb"))
tableName = getTableNameFromARN(record.getEventSourceARN());
}
LaneAnnotation laneAnnotation = new LaneAnnotation();
ScanRequest scanRequest = new ScanRequest().withTableName(tableName);
ScanResult result = amazonDynamoDBClient.scan(scanRequest);
List<Lines> linesFinalList = new ArrayList<Lines>();
if(result != null) {
for (Map<String, AttributeValue> item : result.getItems()) {
//code for looping through the table items and generating a json object for the elastic search model
}
//Code to post the json below -
RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory clientHttpRequestFactory = (SimpleClientHttpRequestFactory)restTemplate.getRequestFactory();
clientHttpRequestFactory.setConnectTimeout(10000);
clientHttpRequestFactory.setReadTimeout(10000);
HttpEntity<String> entity = new HttpEntity<String>(<json goes here>, headers);
try{
restTemplate.exchange(endpoint, HttpMethod.POST, entity, String.class);
}catch(Exception e){
e.printStackTrace();
}
}
However, I see the following error when I test my AWS lambda function -
org.springframework.web.client.HttpClientErrorException: 403 Forbidden
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:700)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:653)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:531)
at com.here.aws.LambdaApplication.handleRequest(LambdaApplication.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at lambdainternal.EventHandlerLoader$PojoMethodRequestHandler.handleRequest(EventHandlerLoader.java:456)
at lambdainternal.EventHandlerLoader$PojoHandlerAsStreamHandler.handleRequest(EventHandlerLoader.java:375)
at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:1139)
at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:285)
at lambdainternal.AWSLambda.<clinit>(AWSLambda.java:57)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:94)
I even modified the access policy and added my IP address. Have others faced this too? How did you resolve it?> Any help will be appreciated.
EDIT1: I am now trying to incorporate signing of the request as is mentioned here - https://aws.amazon.com/blogs/security/how-to-control-access-to-your-amazon-elasticsearch-service-domain/
Will report back if it goes well.
EDIT2:
Here's the second way of sending a request that I tried referring to the link above-
@Override
public Object handleRequest(DynamodbEvent dynamodbEvent, Context context) {
AmazonDynamoDBClient amazonDynamoDBClient = new AmazonDynamoDBClient();
List<DynamodbEvent.DynamodbStreamRecord> dynamodbStreamRecordlist = dynamodbEvent.getRecords();
if (!dynamodbStreamRecordlist.isEmpty()) {
DynamodbEvent.DynamodbStreamRecord record = dynamodbStreamRecordlist.get(0);
if(record.getEventSource().equalsIgnoreCase("aws:dynamodb"))
tableName = getTableNameFromARN(record.getEventSourceARN());
}
LaneAnnotation laneAnnotation = new LaneAnnotation();
ScanRequest scanRequest = new ScanRequest().withTableName(tableName);
ScanResult result = amazonDynamoDBClient.scan(scanRequest);
List<Lines> linesFinalList = new ArrayList<Lines>();
if(result != null) {
for (Map<String, AttributeValue> item : result.getItems()) {
//Generate the json object that needs to be sent in the request
}
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON_UTF8);
Request<?> request = new DefaultRequest<Void>(SERVICE_NAME);
request.setContent(new ByteArrayInputStream(elasticSearchModel.toString().getBytes()));
request.setEndpoint(URI.create(endpoint));
request.setHttpMethod(HttpMethodName.POST);
AWS4Signer signer = new AWS4Signer();
signer.setServiceName(SERVICE_NAME);
signer.setRegionName(Regions.US_EAST_1.getName());
AWSCredentialsProvider credsProvider =
new DefaultAWSCredentialsProviderChain();
AWSCredentials creds = credsProvider.getCredentials();
// Sign request with supplied creds
signer.sign(request, creds);
log.info("Request signed");
ExecutionContext executionContext = new ExecutionContext(true);
ClientConfiguration clientConfiguration = new ClientConfiguration();
AmazonHttpClient client = new AmazonHttpClient(clientConfiguration);
MyHttpResponseHandler<Void> responseHandler = new MyHttpResponseHandler<Void>();
MyErrorHandler errorHandler = new MyErrorHandler();
Response<Void> response =
client.execute(request, responseHandler, errorHandler, executionContext);
return dynamodbEvent;
}
However, I get the following error -
Check the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'GET
/
host:somehostname-XXXXXXXXXXXXXXXX.us-east-1.es.amazonaws.com
x-amz-date:20170130T105736Z
x-amz-security-token:FQoDYXdzEG4aDJJ4ryjXXXXXXXXXXXXXXXX/auMHooYENY6YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
host;x-amz-date;x-amz-security-token
e3b0c4429XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
The String-to-Sign should have been
'AWS4-HMAC-SHA256
20170130T105736Z
20170130/us-east-1/es/aws4_request
9a5b4c92ec121c333f8cdXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
"}"
10:57:36.818 [main] DEBUG org.apache.http.headers - http-outgoing-1 << HTTP/1.1 403 Forbidden
回答1:
AWS Elastic Search has a security gateway that provides authentication. The authentication options are configured in the AWS Elastic Search console.
You are receiving a 403 authentication error because your AWS Elastic Search access policy does not allow the IP for the NAT Gateway that your Lambda uses.
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-createupdatedomains.html#es-createdomain-configure-access-policies
Here is an access policy template that you can use for IP based authentication.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:YOUR-AWS-ACCOUNT-ID:domain/YOUR-ELASTICSEARCH-DOMAIN-NAME/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"YOUR-NAT-GATEWAY-PUBLIC-IP/32"
]
}
}
}
]
}
来源：https://stackoverflow.com/questions/41932026/why-do-i-get-403-forbidden-error-while-posting-a-json-to-elasticsearch-endpoint