C# RSA decrypt parameter is incorrect

问题

I am trying to decrypt communication from web sockets secured with SSL certificate. The certificate is installed on the machine and the private key is present. I have this decrypt function

public static string DecryptTry(X509Certificate2 x509, string stringTodecrypt)
{
        try
        {
            if (x509 == null || string.IsNullOrEmpty(stringTodecrypt))
                throw new Exception("A x509 certificate and string for decryption must be provided");

            if (!x509.HasPrivateKey)
                throw new Exception("x509 certificate does not contain a private key for decryption");

            if(x509 == null)
            {
                Console.WriteLine("certificate is null");
                return "";
            }

            Console.WriteLine("decoding text with private key " + x509.HasPrivateKey);//true

            using (RSA csp = (RSA)x509.PrivateKey)
            {
                byte[] bytestoDecrypt = Encoding.UTF8.GetBytes(stringTodecrypt).Take(256).ToArray();
                Console.WriteLine("key size: " + x509.PrivateKey.KeySize + " received data length: " + stringTodecrypt.Length + " bytes length: " + bytestoDecrypt.Length);
                Console.WriteLine(Encoding.UTF8.GetString(bytestoDecrypt));
                byte[] bytesDecrypted = csp.Decrypt(bytestoDecrypt, RSAEncryptionPadding.OaepSHA256); //ERROR HERE

                return Encoding.UTF8.GetString(bytesDecrypted);
            }
        }
        catch(Exception e)
        {
            Console.WriteLine("Error while decrypting text: " + e.Message + e.HResult +  e.StackTrace);
            return "";
        }
    }

But the csp.Decrypt is throwing an error

parameter is incorrect

I have tried all padding parameters - none of the seemed to make a difference. Does anybody know where the problem might be? Am I missing something? **

EDIT 25.12.2020

** To add some more background info: The website where the WebSocket client is listening is secured HTTPS, the SSL certificate is signed by CA with my full access to all of its information. The initial problem is handshake for the WebSocket communication which comes encrypted. I was thinking I would be able to decrypt it with the private key and that is where the problem occurs. The length of the incoming request (or handshake) is between 490 and 520 bytes, so that is the reason for .Take(256). I was thinking to split the text into multiple, decode them separately and put together after. That, however, brought me here. One final thought: This is a .NET console application. Could the problem be possibly fixed by converting it to a format that IIS accepts? The IIS on the machine has the certificate installed... could it possibly make a difference? Thanks in advance!

回答1:

I'm not sure what you want to achieve:

• Decrypt something using RSA
• Decrypt TLS layer of HTTP
• Decrypt something in Websocket protocol

I think that you should give some more informations how it works. Can you show example input for function function, means:

DecryptTry(X509Certificate2 x509, string stringTodecrypt) { ... }

How 'strindTodecypt' looks like? What do you put there? How do you connect to this server to get data to decrypt, via TLS?

来源：https://stackoverflow.com/questions/65432353/c-sharp-rsa-decrypt-parameter-is-incorrect