ZSTD String compression using luben

问题

I have been struggling to compress a regular string to ZSTD format using the Luben library (v1.4.0-1). I do the following:

byte[] zstdBytes = Zstd.compress(payload.getBytes(StandardCharsets.UTF_8));
ZstdInputStream zstdInputStream= new ZstdInputStream(new ByteArrayInputStream(zstdBytes));

However while reading the zstdInputStream I get the following error:

java.io.IOException: Decompression error: Unknown frame descriptor
   at com.github.luben.zstd.ZstdInputStream.readInternal(ZstdInputStream.java:142)
   at com.github.luben.zstd.ZstdInputStream.read(ZstdInputStream.java:102)
   at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
   at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
   at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
   at java.io.InputStreamReader.read(InputStreamReader.java:184)
   at java.io.BufferedReader.fill(BufferedReader.java:161)
   at java.io.BufferedReader.read(BufferedReader.java:182)

My reading of the zstdInputStream code is as follows:

try (ZstdInputStream zis = new ZstdInputStream(new UncloseableStream(payload))) {
        try (Reader reader = new BufferedReader(new InputStreamReader(zis))) {

            int data = reader.read();
            while (data != -1) {
                System.out.print((char) data);
                data = reader.read();
            }
        }
}

EDIT: My payload is as follows: {"customer":"CA101","source": "live","operation": "add","type": "BC","container": "c_789","token": "BC_mc- l_CA101_20101206_0","priority": 5,"jsonPayload": "{"guid":"89df67","id":"id789","llevel":"INFO","ldate":"20180526","lthread":"Indexing-789","lmethod":"GET","afield":"aaa","bfield":"bbb","cfield":"ccc","dfield":"ddd","efield":"eee","data":"Hello world "fancy","hostname":"uk-sx1-1","instanceid":"swg-89","service":"indexing","grid":"UK","origin":"0","time":1508,"loffset":7845"}"}

回答1:

So I found why it was not working. I shouldn't have used the following line:

ZstdInputStream zstdInputStream= new ZstdInputStream(new   ByteArrayInputStream(zstdBytes));

Basically this works:

byte[] zstdBytes = Zstd.compress(payload.getBytes(StandardCharsets.UTF_8));
InputStream inputStream = new ByteArrayInputStream(zstdBytes);

来源：https://stackoverflow.com/questions/66038814/zstd-string-compression-using-luben

标签

java

file-io

compression