Breaking gzipped JSON into chunks of arbitrary size
问题 This is being done in Typescript, but the algorithm is applicable to any language, I think. I am sending log data (parsed AWS ALB logs) to New Relic, and their maximum payload size is 10^6 bytes. What I'm doing right now is encoding the entire ALB log I get from S3 in JSON, gzipping it, and then examining the size via Buffer.byteLength . If it's in excess of 900,000 bytes (I want to leave some headroom, because the gzipped data doesn't exactly scale linearly with the number of log entries) I