How can I access Amazon AWS S3 using GSOAP for C and C++?

Question

I have searched everywhere for this and I could not find a single decent code. How can I access Amazon AWS S3 service using GSOAP?

Answer 1

How can I access Amazon AWS S3 using GSOAP for C and C++?

Step 1

Use gSOAP's wsd2lh tool to convert Amazon's S3 WSDL to an interface header file aws-s3.h:

wsdl2h -t typemap.dat -o aws-s3.h http://doc.s3.amazonaws.com/2006-03-01/AmazonS3.wsdl

Use option -c to generate C source code instead of the default C++ source code. The typemap.dat file is located in the gsoap directory of the gSOAP distribution.

Step 2

Use the soapcpp2 tool on the header file created from the wsdl2h tool.

soapcpp2 -C -j aws-s3.h

This generates client-side code (-C option) with C++ service proxies and objects (-j option) from the aws-s3.h header. Omit -j for C code.

Step 3

Use the auto-generated AmazonS3SoapBindingProxy proxy methods to access AWS S3 and create a base64-encoded, HMAC-SHA1 hashed signature for AWS S3. The signature is a string with the base64-encoded version of the HMAC-SHA1 hashed string "AmazonS3" + OPERATION_NAME + Timestamp:

/*
    createbucket.cpp
    Example AWS S3 CreateBucket service invocation
*/

#include "soapAmazonS3SoapBindingProxy.h"
#include "AmazonS3SoapBinding.nsmap"
#include <fstream>

// Make allocation of primitive values quick and easy:
template<class T>
T * soap_make(struct soap *soap, T val) {
    T *p = (T*)soap_malloc(soap, sizeof(T));
    *p = val;
    return p;
}

// Make base64-encoded, HMAC-SHA1 hashed signature for AWS S3
std::string soap_make_s3__signature(struct soap *soap, char const *operation, char const *key) {
    std::string signature = "AmazonS3";
    signature += operation;
    char UTCstamp[40]; //to hold ISO 8601 time format
    time_t now;
    time(&now);
    strftime(UTCstamp, sizeof UTCstamp, "%Y-%m-%dT%H:%M:%S.000Z", gmtime(&now));
    signature += UTCstamp;
    // Get the HMAC-SHA1 digest of the signature string
    unsigned char * digest;
    digest = HMAC(EVP_sha1(), key, strlen(key), 
    (unsigned char*)(signature.c_str()), 
    signature.length(), NULL, NULL);        
    char signatureBase64[20];
    // Convert the digest to base64
    soap_s2base64(soap, digest, signatureBase64, sizeof signatureBase64);
    return std::string(signatureBase64);
}

// Read access keys from file generated by AWS CLI
bool getAWSKeys(std::string path, std::string user, std::string &accessKey, std::string &secretKey) {
    std::ifstream credentialsFile(path.c_str());
    if (!credentialsFile.is_open())
        return false;
    
    std::string line;
    while (std::getline(credentialsFile, line)) {
        // Keep going until we get to the desired user
        if (line.find(user) == std::string::npos)
            continue;
        
        while (std::getline(credentialsFile, line)) {
            // Keep going until we get to the access key lines
            if (line.find("aws_access_key_id") == std::string::npos)
                continue;

            // Grab keys and trim whitespace
            size_t first, last;
            accessKey = line.substr(line.find_first_of('=')+1);
            first = accessKey.find_first_not_of(' ');
            if (first == std::string::npos)
                return false;
            last = accessKey.find_last_not_of(' ');
            accessKey.substr(first, last-first+1).swap(accessKey); 

            std::getline(credentialsFile, line);
            secretKey = line.substr(line.find_first_of('=')+1);
            first = secretKey.find_first_not_of(' ');
            if (first == std::string::npos)
                return false;
            last = secretKey.find_last_not_of(' ');
            secretKey.substr(first, last-first+1).swap(secretKey);
            
            return true;
        }

    }
    return false;
}

int main(int argc, char **argv) {    
    // Load AWS keys from file
    std::string accessKey, secretKey;
    // Use the path to your AWS credentials file
    std::string credentialsFile = (argc > 2 ? argv[2] : "path_to_aws_credentials_file"); 
    std::string user = "default";
    if (!getAWSKeys(credentialsFile, user, accessKey, secretKey)) {
        std::cout << "Couldn't read AWS keys for user " << user 
                  << " from file " << credentialsFile << '\n';
        return 0;
    }
    
    // Create a proxy to invoke AWS S3 services
    AmazonS3SoapBindingProxy aws(SOAP_XML_INDENT);

    // Create bucket
    
    // Set the arguments of the CreateBucket service operation
    _s3__CreateBucket createBucketReq;
    std::string bucketName = (argc > 1 ? argv[1] : "BucketName");
    createBucketReq.Bucket = bucketName;
    createBucketReq.AWSAccessKeyId  = soap_new_std__string(aws.soap);
    *createBucketReq.AWSAccessKeyId = accessKey;      
    createBucketReq.Timestamp       = soap_make(aws.soap, time(0));
    createBucketReq.Signature       = soap_new_std__string(aws.soap);
    *createBucketReq.Signature      = soap_make_s3__signature(aws.soap, 
                                                              "CreateBucket", 
                                                              secretKey.c_str());
                                                              
    // Store the result of the service
    _s3__CreateBucketResponse createBucketRes;

    // Create a bucket
    if (aws.CreateBucket(&createBucketReq, createBucketRes)) {
        aws.soap_stream_fault(std::cerr);
    }
    /*
        NOTE: you must add the line:
           _s3__CreateBucketResponse = $ s3__CreateBucketResult* CreateBucketResponse;
        to the typemap.dat file because Amazon's response doesn't match
        their promised schema. This adds the variable CreateBucketResponse
        to the _s3__CreateBucketResponse class so we can access the response.
    */
    else if (createBucketRes.CreateBucketResponse) {
        s3__CreateBucketResult &result = *createBucketRes.CreateBucketResponse;
        std::cout << "You are the owner of bucket '" << result.BucketName << "'." << std::endl;
    }

    // Delete all managed data
    aws.destroy();

    return 0;
}

The C code look similar, with the main difference being the use of function calls instead of method invocations, i.e. soap_call___s3__CreateBucket(&createBucketReq, &createBucketRes). All this is explained in the generated aws-s4.h file.

Compile the generated files with your source code:

c++ -DSOAP_MAXDIMESIZE=104857600 -DWITH_OPENSSL -o createbucket createbucket.cpp soapAmazonS3SoapBindingProxy.cpp soapC.cpp stdsoap2.cpp -lssl -lcrypto

The SOAP_MAXDIMESIZE=104857600 ensures that DIME attachment sizes can be large enough while preventing denial of service attacks using DIME. The DIME header has the attachment size, so an attacker could set that arbitrary large to deplete memory resources. Other posts failed to mention this.

Run createbucket, and a new bucket will be created.

In the final .cpp file, note that we check the command line arguments (argv) when setting credentialsFile and bucketName. This allows the program to be called with arguments:

./createbucket BucketName path_to_credentials_file

For more details on all of this, I suggest to read the excellent CodeProject article on How to use AWS S3 in C++ with gSOAP by Chris Moutsos from which parts of this explanation originates.