how can I add PAdES-LTV using iText

Question

I am trying to enable LTV in an already signed PDF document without LTV format. I have found the same example in all cases as described in the links How to enable LTV for a

Answer 1

As it turned out in this comment

i want is Adobe LTV-enable

the task is less PAdES related (even though mechanisms introduced in PAdES are used) but focused on an Adobe proprietary signature profile, "LTV enabled" signatures.

Unfortunately, this proprietary signature profile is not properly specified. All Adobe tells us is

LTV enabled means that all information necessary to validate the file (minus root certs) is contained within.

(for details and backgrounds read this answer)

Thus, implementing a way to LTV enable the example signature involved some trial and error, and I cannot guarantee Adobe will accept the outputs of this code as "LTV enabled" in Adobe Acrobat versions to come.

Furthermore, the current iText 5 signature APIs do not suffice out of the box for the task because (as it turned out) Adobe requires certain otherwise optional structures which the iText code does not create (but see the PPS below). The most simple way to fix this was to update the iText class LtvVerification in two aspects, so I'll describe that way here. Alternatively one could have used Java reflection or copied and tweaked quite a bit of code; if you cannot update iText as shown below, you'll have to chose one such alternative approach.

LTV enabling the signatures of a signed PDF

This section shows the code additions and changes with which one can LTV enable documents like the OP's example PDF sign_without_LTV.pdf.

An approach using iText's LtvVerification class

This is the original code which makes use of the LtvVerification class from iText's signature API. Unfortunately for this a functionality has to be added to that class.

Patching LtvVerification

The iText 5 LtvVerification class only offers addVerification methods accepting a signature field name. We need the functionality of these methods also for signatures not bound to a form field, e.g. for OCSP response signatures. For this I added the following overload of that method:

public boolean addVerification(PdfName signatureHash, Collection<byte[]> ocsps, Collection<byte[]> crls, Collection<byte[]> certs) throws IOException, GeneralSecurityException {
    if (used)
        throw new IllegalStateException(MessageLocalization.getComposedMessage("verification.already.output"));
    ValidationData vd = new ValidationData();
    if (ocsps != null) {
        for (byte[] ocsp : ocsps) {
            vd.ocsps.add(buildOCSPResponse(ocsp));
        }
    }
    if (crls != null) {
        for (byte[] crl : crls) {
            vd.crls.add(crl);
        }
    }
    if (certs != null) {
        for (byte[] cert : certs) {
            vd.certs.add(cert);
        }
    }
    validated.put(signatureHash, vd);
    return true;
}

Furthermore, a (per the specification optional) time entry in the final VRI dictionaries is required (but see the PPS below). Thus, I added the a line in the outputDss method as follows:

...
if (ocsp.size() > 0)
    vri.put(PdfName.OCSP, writer.addToBody(ocsp, false).getIndirectReference());
if (crl.size() > 0)
    vri.put(PdfName.CRL, writer.addToBody(crl, false).getIndirectReference());
if (cert.size() > 0)
    vri.put(PdfName.CERT, writer.addToBody(cert, false).getIndirectReference());
// v--- added line
vri.put(PdfName.TU, new PdfDate());
// ^--- added line
vrim.put(vkey, writer.addToBody(vri, false).getIndirectReference());
...

Some low level helper methods

Some helper methods operating on security primitives is required. These methods mostly have been collected from existing iText classes (which could not be used as is because they are private) or derived from code there:

static X509Certificate getOcspSignerCertificate(byte[] basicResponseBytes) throws CertificateException, OCSPException, OperatorCreationException {
    JcaX509CertificateConverter converter = new JcaX509CertificateConverter().setProvider(BouncyCastleProvider.PROVIDER_NAME);
    BasicOCSPResponse borRaw = BasicOCSPResponse.getInstance(basicResponseBytes);
    BasicOCSPResp bor = new BasicOCSPResp(borRaw);

    for (final X509CertificateHolder x509CertificateHolder : bor.getCerts()) {
        X509Certificate x509Certificate = converter.getCertificate(x509CertificateHolder);

        JcaContentVerifierProviderBuilder jcaContentVerifierProviderBuilder = new JcaContentVerifierProviderBuilder();
        jcaContentVerifierProviderBuilder.setProvider(BouncyCastleProvider.PROVIDER_NAME);
        final PublicKey publicKey = x509Certificate.getPublicKey();
        ContentVerifierProvider contentVerifierProvider = jcaContentVerifierProviderBuilder.build(publicKey);

        if (bor.isSignatureValid(contentVerifierProvider))
            return x509Certificate;
    }

    return null;
}

static PdfName getOcspSignatureKey(byte[] basicResponseBytes) throws NoSuchAlgorithmException, IOException {
    BasicOCSPResponse basicResponse = BasicOCSPResponse.getInstance(basicResponseBytes);
    byte[] signatureBytes = basicResponse.getSignature().getBytes();
    DEROctetString octetString = new DEROctetString(signatureBytes);
    byte[] octetBytes = octetString.getEncoded();
    byte[] octetHash = hashBytesSha1(octetBytes);
    PdfName octetName = new PdfName(Utilities.convertToHex(octetHash));
    return octetName;
}

static PdfName getCrlSignatureKey(byte[] crlBytes) throws NoSuchAlgorithmException, IOException, CRLException, CertificateException {
    CertificateFactory cf = CertificateFactory.getInstance("X.509");
    X509CRL crl = (X509CRL)cf.generateCRL(new ByteArrayInputStream(crlBytes));
    byte[] signatureBytes = crl.getSignature();
    DEROctetString octetString = new DEROctetString(signatureBytes);
    byte[] octetBytes = octetString.getEncoded();
    byte[] octetHash = hashBytesSha1(octetBytes);
    PdfName octetName = new PdfName(Utilities.convertToHex(octetHash));
    return octetName;
}

static X509Certificate getIssuerCertificate(X509Certificate certificate) throws IOException, StreamParsingException {
    String url = getCACURL(certificate);
    if (url != null && url.length() > 0) {
        HttpURLConnection con = (HttpURLConnection)new URL(url).openConnection();
        if (con.getResponseCode() / 100 != 2) {
            throw new IOException(MessageLocalization.getComposedMessage("invalid.http.response.1", con.getResponseCode()));
        }
        InputStream inp = (InputStream) con.getContent();
        byte[] buf = new byte[1024];
        ByteArrayOutputStream bout = new ByteArrayOutputStream();
        while (true) {
            int n = inp.read(buf, 0, buf.length);
            if (n <= 0)
                break;
            bout.write(buf, 0, n);
        }
        inp.close();

        X509CertParser parser = new X509CertParser();
        parser.engineInit(new ByteArrayInputStream(bout.toByteArray()));
        return (X509Certificate) parser.engineRead();
    }
    return null;
}

static String getCACURL(X509Certificate certificate) {
    ASN1Primitive obj;
    try {
        obj = getExtensionValue(certificate, Extension.authorityInfoAccess.getId());
        if (obj == null) {
            return null;
        }
        ASN1Sequence AccessDescriptions = (ASN1Sequence) obj;
        for (int i = 0; i < AccessDescriptions.size(); i++) {
            ASN1Sequence AccessDescription = (ASN1Sequence) AccessDescriptions.getObjectAt(i);
            if ( AccessDescription.size() != 2 ) {
                continue;
            }
            else if (AccessDescription.getObjectAt(0) instanceof ASN1ObjectIdentifier) {
                ASN1ObjectIdentifier id = (ASN1ObjectIdentifier)AccessDescription.getObjectAt(0);
                if ("1.3.6.1.5.5.7.48.2".equals(id.getId())) {
                    ASN1Primitive description = (ASN1Primitive)AccessDescription.getObjectAt(1);
                    String AccessLocation =  getStringFromGeneralName(description);
                    if (AccessLocation == null) {
                        return "" ;
                    }
                    else {
                        return AccessLocation ;
                    }
                }
            }
        }
    } catch (IOException e) {
        return null;
    }
    return null;
}

static ASN1Primitive getExtensionValue(X509Certificate certificate, String oid) throws IOException {
    byte[] bytes = certificate.getExtensionValue(oid);
    if (bytes == null) {
        return null;
    }
    ASN1InputStream aIn = new ASN1InputStream(new ByteArrayInputStream(bytes));
    ASN1OctetString octs = (ASN1OctetString) aIn.readObject();
    aIn = new ASN1InputStream(new ByteArrayInputStream(octs.getOctets()));
    return aIn.readObject();
}

static String getStringFromGeneralName(ASN1Primitive names) throws IOException {
    ASN1TaggedObject taggedObject = (ASN1TaggedObject) names ;
    return new String(ASN1OctetString.getInstance(taggedObject, false).getOctets(), "ISO-8859-1");
}

static byte[] hashBytesSha1(byte[] b) throws NoSuchAlgorithmException {
    MessageDigest sh = MessageDigest.getInstance("SHA1");
    return sh.digest(b);
}

(as in MakeLtvEnabled)

They aren't optimized yet, one certainly can make them more performant and more elegant.

Adding LTV information

Based on these additions and helpers one can add the LTV information required for LTV enabled signatures with this method makeLtvEnabled:

public void makeLtvEnabled(PdfStamper stp, OcspClient ocspClient, CrlClient crlClient) throws IOException, GeneralSecurityException, StreamParsingException, OperatorCreationException, OCSPException {
    stp.getWriter().addDeveloperExtension(new PdfDeveloperExtension(PdfName.ADBE, new PdfName("1.7"), 8));
    LtvVerification v = stp.getLtvVerification();
    AcroFields fields = stp.getAcroFields();

    Map<PdfName, X509Certificate> moreToCheck = new HashMap<>();

    ArrayList<String> names = fields.getSignatureNames();
    for (String name : names)
    {
        PdfPKCS7 pdfPKCS7 = fields.verifySignature(name);
        List<X509Certificate> certificatesToCheck = new ArrayList<>();
        certificatesToCheck.add(pdfPKCS7.getSigningCertificate());
        while (!certificatesToCheck.isEmpty()) {
            X509Certificate certificate = certificatesToCheck.remove(0);
            addLtvForChain(certificate, ocspClient, crlClient,
                    (ocsps, crls, certs) -> {
                        try {
                            v.addVerification(name, ocsps, crls, certs);
                        } catch (IOException | GeneralSecurityException e) {
                            e.printStackTrace();
                        }
                    },
                    moreToCheck::put
            );
        }
    }

    while (!moreToCheck.isEmpty()) {
        PdfName key = moreToCheck.keySet().iterator().next();
        X509Certificate certificate = moreToCheck.remove(key);
        addLtvForChain(certificate, ocspClient, crlClient,
                (ocsps, crls, certs) -> {
                    try {
                        v.addVerification(key, ocsps, crls, certs);
                    } catch (IOException | GeneralSecurityException e) {
                        e.printStackTrace();
                    }
                },
                moreToCheck::put
        );
    }
}

void addLtvForChain(X509Certificate certificate, OcspClient ocspClient, CrlClient crlClient, VriAdder vriAdder,
        BiConsumer<PdfName, X509Certificate> moreSignersAndCertificates) throws GeneralSecurityException, IOException, StreamParsingException, OperatorCreationException, OCSPException {
    List<byte[]> ocspResponses = new ArrayList<>();
    List<byte[]> crls = new ArrayList<>();
    List<byte[]> certs = new ArrayList<>();

    while (certificate != null) {
        System.out.println(certificate.getSubjectX500Principal().getName());
        X509Certificate issuer = getIssuerCertificate(certificate);
        certs.add(certificate.getEncoded());
        byte[] ocspResponse = ocspClient.getEncoded(certificate, issuer, null);
        if (ocspResponse != null) {
            System.out.println("  with OCSP response");
            ocspResponses.add(ocspResponse);
            X509Certificate ocspSigner = getOcspSignerCertificate(ocspResponse);
            if (ocspSigner != null) {
                System.out.printf("  signed by %s\n", ocspSigner.getSubjectX500Principal().getName());
            }
            moreSignersAndCertificates.accept(getOcspSignatureKey(ocspResponse), ocspSigner);
        } else {
           Collection<byte[]> crl = crlClient.getEncoded(certificate, null);
           if (crl != null && !crl.isEmpty()) {
               System.out.printf("  with %s CRLs\n", crl.size());
               crls.addAll(crl);
               for (byte[] crlBytes : crl) {
                   moreSignersAndCertificates.accept(getCrlSignatureKey(crlBytes), null);
               }
           }
        }
        certificate = issuer;
    }

    vriAdder.accept(ocspResponses, crls, certs);
}

interface VriAdder {
    void accept(Collection<byte[]> ocsps, Collection<byte[]> crls, Collection<byte[]> certs);
}

(MakeLtvEnabled as makeLtvEnabledV2)

Example usage

For a signed PDF at INPUT_PDF and a result output stream RESULT_STREAM you can use the method above like this:

PdfReader pdfReader = new PdfReader(INPUT_PDF);
PdfStamper pdfStamper = new PdfStamper(pdfReader, RESULT_STREAM, (char)0, true);

OcspClient ocsp = new OcspClientBouncyCastle();
CrlClient crl = new CrlClientOnline();
makeLtvEnabledV2(pdfStamper, ocsp, crl);

pdfStamper.close();

(MakeLtvEnabled test method testV2)

Limitations

The methods above only work under some simplifying restrictions, in particular:

• signature time stamps are ignored,
• retrieved CRLs are assumed to be direct and complete,
• the complete certificate chains are assumed to be buildable using AIA entries.

You can improve the code accordingly if these restrictions are not acceptable for you.

An approach using an own utility class

To avoid having to patch an iText class, this approach takes the required code from the methods above and the LtvVerification class from iText's signature API and merges all into a new utility class. This class can LTV enable a document without requiring a patched iText version.

The AdobeLtvEnabling class

This class combines the code above and some LtvVerification code into a utility class for LTV enabling documents.

Unfortunately copying it here pushes the message size beyond the 30000 character limit of stack overflow. You can retrieve the code from github, though:

AdobeLtvEnabling.java

Example usage

For a signed PDF at INPUT_PDF and a result output stream RESULT_STREAM you can use the class above like this:

PdfReader pdfReader = new PdfReader(INPUT_PDF);
PdfStamper pdfStamper = new PdfStamper(pdfReader, RESULT_STREAM, (char)0, true);

AdobeLtvEnabling adobeLtvEnabling = new AdobeLtvEnabling(pdfStamper);
OcspClient ocsp = new OcspClientBouncyCastle();
CrlClient crl = new CrlClientOnline();
adobeLtvEnabling.enable(ocsp, crl);

pdfStamper.close();

(MakeLtvEnabled test method testV3)

Limitations

As this utility class merely repackages the code from the first approach, the same limitations apply.

Behind the scenes

As mentioned at the start, all Adobe tells us about the "LTV enabled" signature profile is that

LTV enabled means that all information necessary to validate the file (minus root certs) is contained within

but they don't tell us how exactly they expect the information to be embedded within the file.

At first I merely collected all this information and made sure it was added to the applicable Document Security Store dictionaries of the PDF (Certs, OCSPs, and CRLs).

But even though all information necessary to validate the file (minus root certs) was contained within, Adobe Acrobat did not consider the file "LTV enabled".

I then LTV enabled the document using Adobe Acrobat and analyzed the differences. As it turned out, the following extra data also were necessary (but see the PPS below):

1. For the signature of each OCSP response Adobe Acrobat requires the presence of a respective VRI dictionary. In the example PDF of the OP this VRI dictionary does not need to contain any certificates, CRLs, or OCSP responses at all, but the VRI dictionary needs to be there.

   In contrast this is not necessary for the signatures of CRLs. This looks a bit arbitrary.

   According to the specifications, both ISO 32000-2 and ETSI EN 319 142-1, the use of these VRI dictionaries is purely optional. For PAdES BASELINE signatures there even is a recommendation against using VRI dictionaries!

2. Adobe Acrobat expects the VRI dictionaries to each contain a TU entry documenting the creation time of the respective VRI dictionary. (Probably a TS would also do, I have not tested that).

   According to the specifications, both ISO 32000-2 and ETSI EN 319 142-1, the use of these TU entries is purely optional. For PAdES signatures there even is a recommendation against using TU or TS entries!

Thus, it is not surprising that by default LTV information added by applications according to the PDF specifications do not result in "LTV enabled" signatures as reported by Adobe Acrobat.

PS

Obviously I had to add trust for some certificate in Adobe Acrobat to get it to consider the result of the above code for the OP's document "LTV enabled" at all. I chose the root certificate "CA RAIZ NACIONAL - COSTA RICA v2".

PPS (2020-03-02)

Apparently meanwhile Adobe Acrobat does not require the presence of VRI dictionaries (let alone TU time stamps) anymore for considering revocation information in the DSS when testing for the LTV enabled state, see the section "Optional elements in the DSS" of this answer.

Thus, quite likely the solutions above can be somewhat simplified.