You see, I have an applet that decrypts some info based on the RSA
private key that is found in a pkcs11 token.
In Brazil, there are now, some new certificates
It has nothing to do with Java key length. For all that's worth, Java may not even know the key size as the key remains on the hardware token. It's much more likely that the attribute CKA_ENCRYPT is set to the CK_BBOOL value of CK_FALSE.
From PKCS#11 v2.20 (cryptoki)
CKR_KEY_FUNCTION_NOT_PERMITTED: An attempt has been made to use a key for a cryptographic purpose that the key’s attributes are not set to allow it to do. For example, to use a key for performing encryption, that key must have its CKA_ENCRYPT attribute set to CK_TRUE (the fact that the key must have a CKA_ENCRYPT attribute implies that the key cannot be a private key). This return value has lower priority than CKR_KEY_TYPE_INCONSISTENT.
You are mixing PKCS#11 (CKR_XXX) and CryptoAPI (AT_XXX). You can use pkcs11-tool or pkcs11-dump (google) to list the properties of objects you have.