S3 Multiple Files Download Java SDK

问题

We have a feature in which end users can download thousands of photos (e.g. 4-5k) in a collection in a ZIP format. We are using S3 for storing the photos. So an user clicks the "Download" button then a ZIP file starts downloading. The backend code is like below (Just code snippet)

ZipOutputStream zos = new ZipOutputStream(response.getOutputStream());
for (Gallery photo : gallery) {
    S3ObjectInputStream file = null;
    try {
        filePath = title + "_" + id + "_"+ photo.getId() + ".jpg";
        file = zipServerFiles(photo,title, id, filePath, zos);
    } catch (Exception e) {
        LOGGER.error("Error while downloading photo " + photo.getName() + "; project id: " + id, e);
    } finally {
        if (file != null) {
            file.close();
        }
    }
}

public S3ObjectInputStream zipServerFiles(Gallery photo,String title,String id,String fileName,ZipOutputStream zos) throws IOException {
    S3ObjectInputStream file = awsFileUploader.downloadPhoto(photo.photoUrl);
    zipFiles(file, fileName, zos);
    return file;
}

public void zipFiles(S3ObjectInputStream file, String fileName, ZipOutputStream zos) throws IOException {
    zos.putNextEntry(new ZipEntry(fileName));
    IOUtils.copy(file, zos);    
    zos.flush();
    zos.closeEntry();
}

public S3ObjectInputStream downloadPhoto(String key) throws IOException {
    GetObjectRequest request = new GetObjectRequest(CORE_BUNDLE.getString("aws.bucket.name"), key);
    S3ObjectInputStream stream = null;
    try {
        stream =  getS3Client().getObject(request).getObjectContent();
    } catch (Exception e) {
        logger.error("Exception occurred while downloading file", e);
    }
    return stream;
}

Maven Dependency

<dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-s3</artifactId>
        <version>1.11.68</version>
    </dependency>

The problem is that we are getting below exception

Caused by: org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:286)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263)
    at sun.reflect.GeneratedMethodAccessor2679.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)
    at com.amazonaws.http.conn.$Proxy302.get(Unknown Source)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
    at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1115)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:964)

Although we are closing the file object, but don't know why connections are kept opened during the download process.

Help :|

来源：https://stackoverflow.com/questions/54907720/s3-multiple-files-download-java-sdk