How to read Snappy Compressed file from S3 in Java

问题

Currently we are running MapReduce job in Hadoop in which the output is compressed into SnappyCompression. Then we are moving the output file to S3. Now I want to read the Compressed file from S3 through Java.

回答1:

I found the answer to read snappy compressed file from S3. First you should get the object content from S3. And then decompress the file.

    S3Object s3object = s3Client.getObject(new GetObjectRequest(bucketName,Path));
    InputStream inContent = s3object.getObjectContent();
    CompressionCodec codec = (CompressionCodec) ReflectionUtils.newInstance(SnappyCodec.class, new Configuration());
    InputStream inStream = codec.createInputStream(new BufferedInputStream(inContent));
    InputStreamReader  inRead = new InputStreamReader(inStream);
    BufferedReader br = new BufferedReader(inRead);
    String line=null;
    while ((line = br.readLine()) != null){
        system.out.println(line);
    }   

来源：https://stackoverflow.com/questions/29816067/how-to-read-snappy-compressed-file-from-s3-in-java