Corrupted data using Always Encrypted in SQL Server

早过忘川 提交于 2019-12-24 09:14:35

问题


Once we started to write data into an encrypted table, we noticed problems once we tried to read encrypted data. Symptoms are identical to the ones described HERE

Here is a little bit of a background. We have a web app, which writes client information to a "client" table in the SQL Server database. As a transition solution, we created additional table client_enc and updated our app to write to both tables: original one and an encrypted one. We have 4 instances of our web app, hosted on the same VM, same IIS.

All 4 instances of our web app are mapped to the same folder on the file system (no difference in binary code or in web.config).

We noticed that one of the instances randomly writes corrupted values. Those writes happen with no restart/recycling of web app (within few second between writes).

Below is an information of a particular customer:

Client last name: "Hoyer"

Good encrypted value (the one we can read later on):

0x015EF5BB1B1EA45EADFA9EFC3611D3F5661616C4B38BEDB06B33D6B6DC084714F235E0818C14DEEC0A95C5547DE8DC3D3A402A4DB8C992AB3716B651037C8ED2E7

Corrupted encrypted value:

0x01848FA1EA78BA1FCFC615728CEE9882937A52AAF649472F0B7829A28463060E34080F924AC5CD987AA0C5275507C0A480EC9D44B63B256552EFFE7C1562FEC1DA

Environment:

  • Host machine: Windows 2012 R2 (Microsoft Windows NT 6.3 (14393))
  • SQL Server 2016 (v13.0.1742.0)
  • .NET Framework 4.6.2
  • Target framework in web.config: 4.6.2
  • Database has only one master key and only one column key

Could anybody try to guess what can be causing such weird behavior?


回答1:


UPDATE: Initially, I made a wrong judgment due to inaccuracy in test results. I will cross out wrong facts but will leave them here for historical purposes.


After a week of bloody debugging and testing, I came to a conclusion that the root of that behavior is somewhere inside of RSACryptoServiceProvider class in .NET Framework.

Facts which make me think that way:

  • My app is configured to recycle every hour and I noticed that data corruption happens 1 out of 10 times after app restart; If app started to write corrupted data then it was doing it permanently until app is restarted again (or recycled)
  • I used reflection in order to see the internals of objects, involved in AlwaysEncrypted feature:
    • Inside of "System.Data.SqlClient.SqlSymmetricKeyCache" I was watching for result of Column Key decryption ( private field _cache of another private static field _singletonInstance)
    • I replaced default implementation of SqlColumnEncryptionCertificateStoreProvider in order to log all requests to decrypt column encryption key
  • I waited for another data corruption to happen and then looked into my patched provider and cache of decrypted keys. I discovered that decrypted column key, return by SqlColumnEncryptionCertificateStoreProvider was 0x0000000000000000... correct, but in cache appeared to be corrupted (0x0000000000000000...)

I also found this ARTICLE, which makes me think that high-loaded ASP.NET apps may have issues with RSACryptoServiceProvider class, once it is used in the multi-threaded environment. And that's exactly my case and SqlColumnEncryptionCertificateStoreProvider doesn't have any thread synchronization mechanism to avoid the problem, which happens inside of RSACryptoServiceProvider.

After looking at the source code of Always Encrypted related classes I found only ONE PLACE, where decrypted column key is used.

// Decrypt the CEK
// We will simply bubble up the exception from the DecryptColumnEncryptionKey function.
byte[] plaintextKey;
try {
    plaintextKey = provider.DecryptColumnEncryptionKey(keyInfo.keyPath, keyInfo.algorithmName, keyInfo.encryptedKey);
}
catch (Exception e) {
    // Generate a new exception and throw.
    string keyHex = SqlSecurityUtility.GetBytesAsString(keyInfo.encryptedKey, fLast: true, countOfBytes: 10);
    throw SQL.KeyDecryptionFailed(keyInfo.keyStoreName, keyHex, e);
}

encryptionKey = new SqlClientSymmetricKey(plaintextKey);

// If the cache TTL is zero, don't even bother inserting to the cache.
if (SqlConnection.ColumnEncryptionKeyCacheTtl != TimeSpan.Zero) {
    // In case multiple threads reach here at the same time, the first one wins.
    // The allocated memory will be reclaimed by Garbage Collector.
    DateTimeOffset expirationTime = DateTimeOffset.UtcNow.Add(SqlConnection.ColumnEncryptionKeyCacheTtl);
    _cache.Add(cacheLookupKey, encryptionKey, expirationTime);
}

plaintextKey value is 100% correct, as I'm logging it before returning it from DecryptColumnEncryptionKey() method.

Key corruption inside of encryptionKey = new SqlClientSymmetricKey(plaintextKey) is very unlikely, as SqlClientSymmetricKey is a simple wrapper around a byte array.

Key corruption inside of _cache.Add(cacheLookupKey, encryptionKey, expirationTime) is also seems to me very unlikely.

That leaves me with only one logical explanation of how this happens. As byte array (our decrypted key) is passed everywhere as a reference, under particular circumstances, consumer of that key screws up the byte values in the array. But unfortunately, I can't find any place in code to prove that theory.

Workaround. Once I added simple read from an encrypted table before my app starts serving requests (inside of Global.asax), then the problem has gone away. Basically, the trick helps me to guarantee that only one non-concurrent read of data from DB is triggering column key decryption and SqlSymmetricKeyCache initialization.

Will be glad to hear some comments from Microsoft team on that very strange behavior.



来源:https://stackoverflow.com/questions/48592556/corrupted-data-using-always-encrypted-in-sql-server

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!