Adaptation of SHA2 512 gives incorrect results

问题

I am trying to adapt the pure Lua implementation of the SecureHashAlgorithm found here for SHA2 512 instead of SHA2 256. When I try to use the adaptation, it does not give the correct answer.

Here is the adaptation:

--
-- UTILITY FUNCTIONS
--
-- transform a string of bytes in a string of hexadecimal digits
local function str2hexa (s)
    local h = string.gsub(s, ".", function(c)
        return string.format("%02x", string.byte(c))
    end)
    return h
end

-- transforms number 'l' into a big-endian sequence of 'n' bytes
--(coded as a string)
local function num2string(l, n)
    local s = ""
    for i = 1, n do
        --most significant byte of l
        local remainder = l % 256
        s = string.char(remainder) .. s
        --remove from l the bits we have already transformed
        l = (l-remainder) / 256
    end
    return s
end

-- transform the big-endian sequence of eight bytes starting at
-- index 'i' in 's' into a number
local function s264num (s, i)
    local n = 0
    for i = i, i + 7 do
        n = n*256 + string.byte(s, i)
    end
    return n
end

--
-- MAIN SECTION
--

-- FIRST STEP: INITIALIZE HASH VALUES
--(second 32 bits of the fractional parts of the square roots of the first 9th through 16th primes 23..53)

local HH = {}

local function initH512(H)
    H = {0x6a09e667f3bcc908, 0xbb67ae8584caa73b, 0x3c6ef372fe94f82b, 0xa54ff53a5f1d36f1, 0x510e527fade682d1, 0x9b05688c2b3e6c1f, 0x1f83d9abfb41bd6b, 0x5be0cd19137e2179}
    return H
end

-- SECOND STEP: INITIALIZE ROUND CONSTANTS
--(first 80 bits of the fractional parts of the cube roots of the first 80 primes 2..409)

local k = {
    0x428a2f98d728ae22, 0x7137449123ef65cd, 0xb5c0fbcfec4d3b2f, 0xe9b5dba58189dbbc, 0x3956c25bf348b538,
    0x59f111f1b605d019, 0x923f82a4af194f9b, 0xab1c5ed5da6d8118, 0xd807aa98a3030242, 0x12835b0145706fbe,
    0x243185be4ee4b28c, 0x550c7dc3d5ffb4e2, 0x72be5d74f27b896f, 0x80deb1fe3b1696b1, 0x9bdc06a725c71235,
    0xc19bf174cf692694, 0xe49b69c19ef14ad2, 0xefbe4786384f25e3, 0x0fc19dc68b8cd5b5, 0x240ca1cc77ac9c65,
    0x2de92c6f592b0275, 0x4a7484aa6ea6e483, 0x5cb0a9dcbd41fbd4, 0x76f988da831153b5, 0x983e5152ee66dfab,
    0xa831c66d2db43210, 0xb00327c898fb213f, 0xbf597fc7beef0ee4, 0xc6e00bf33da88fc2, 0xd5a79147930aa725,
    0x06ca6351e003826f, 0x142929670a0e6e70, 0x27b70a8546d22ffc, 0x2e1b21385c26c926, 0x4d2c6dfc5ac42aed,
    0x53380d139d95b3df, 0x650a73548baf63de, 0x766a0abb3c77b2a8, 0x81c2c92e47edaee6, 0x92722c851482353b,
    0xa2bfe8a14cf10364, 0xa81a664bbc423001, 0xc24b8b70d0f89791, 0xc76c51a30654be30, 0xd192e819d6ef5218,
    0xd69906245565a910, 0xf40e35855771202a, 0x106aa07032bbd1b8, 0x19a4c116b8d2d0c8, 0x1e376c085141ab53,
    0x2748774cdf8eeb99, 0x34b0bcb5e19b48a8, 0x391c0cb3c5c95a63, 0x4ed8aa4ae3418acb, 0x5b9cca4f7763e373,
    0x682e6ff3d6b2b8a3, 0x748f82ee5defb2fc, 0x78a5636f43172f60, 0x84c87814a1f0ab72, 0x8cc702081a6439ec,
    0x90befffa23631e28, 0xa4506cebde82bde9, 0xbef9a3f7b2c67915, 0xc67178f2e372532b, 0xca273eceea26619c,
    0xd186b8c721c0c207, 0xeada7dd6cde0eb1e, 0xf57d4f7fee6ed178, 0x06f067aa72176fba, 0x0a637dc5a2c898a6,
    0x113f9804bef90dae, 0x1b710b35131c471b, 0x28db77f523047d84, 0x32caab7b40c72493, 0x3c9ebe0a15c9bebc,
    0x431d67c49c100d4c, 0x4cc5d4becb3e42b6, 0x597f299cfc657e2a, 0x5fcb6fab3ad6faec, 0x6c44198c4a475817
}

-- THIRD STEP: PRE-PROCESSING (padding)

local function preprocess(toProcess, len)
    --append a single '1' bit
    --append K '0' bits, where K is the minimum number >= 0 such that L + 1 + K = 896mod1024
    local extra = 128 - (len + 9) % 128
    len = num2string(8 * len, 8)
    toProcess = toProcess .. "\128" .. string.rep("\0", extra) .. len
    assert(#toProcess % 128 == 0)
    return toProcess
end

local function rrotate(rot, n)
    return (rot >> n) | ((rot << 64 - n))
end

local function digestblock(msg, i, H)
    local w = {}
    for j = 1, 16 do w[j] = s264num(msg, i + (j - 1)*4) end
    for j = 17, 80 do
        local v = w[j - 15] 
        local s0 = rrotate(v, 1) ~ rrotate(v, 8) ~ (v >> 7)
        v = w[j - 2] 
        w[j] = w[j - 16] + s0 + w[j - 7] + ((rrotate(v, 19) ~ rrotate(v, 61)) ~ (v >> 6))
    end

    local a, b, c, d, e, f, g, h = H[1], H[2], H[3], H[4], H[5], H[6], H[7], H[8]
    for i = 1, 80 do
        a, b, c, d, e, f, g, h = a , b , c , d , e , f , g , h 
        local s0 = rrotate(a, 28) ~ (rrotate(a, 34) ~ rrotate(a, 39))
        local maj = ((a & b) ~ (a & c)) ~ (b & c)
        local t2 = s0 + maj
        local s1 = rrotate(e, 14) ~ (rrotate(e, 18) ~ rrotate(e, 41))
        local ch = (e & f) ~ (~e & g)
        local t1 = h + s1 + ch + k[i] + w[i]
        h, g, f, e, d, c, b, a = g, f, e, d + t1, c, b, a, t1 + t2
    end

    H[1] = (H[1] + a) 
    H[2] = (H[2] + b) 
    H[3] = (H[3] + c) 
    H[4] = (H[4] + d) 
    H[5] = (H[5] + e) 
    H[6] = (H[6] + f) 
    H[7] = (H[7] + g) 
    H[8] = (H[8] + h) 
end

local function finalresult512 (H)
    -- Produce the final hash value:
    return
    str2hexa(num2string(H[1], 8)..num2string(H[2], 8)..num2string(H[3], 8)..num2string(H[4], 8)..
            num2string(H[5], 8)..num2string(H[6], 8)..num2string(H[7], 8)..num2string(H[8], 8))
end


-- Returns the hash512 for the given string.
local function hash512 (msg)
    msg = preprocess(msg, #msg)
    local H = initH512(HH)

    -- Process the message in successive 1024-bit (128 bytes) chunks:
    for i = 1, #msg, 128 do
        digestblock(msg, i, H)
    end

    return finalresult512(H)
end

Given hash512("a"):

Expect: 1f40fc92da241694750979ee6cf582f2d5d7d28e18335de05abc54d0560e0f5302860c652bf08d560252aa5e74210546f369fbbbce8c12cfc7957b2652fe9a75

Actual: e0b9623f2194cb81f2a62616a183edbe390be0d0b20430cadc3371efc237fa6bf7f8b48311f2fa249131c347fee3e8cde6acfdab286d648054541f92102cfc9c

I know that I am creating a message of the correct bit size (1024 bits) and also working in 1024-bit chunks, or at least I believe I am.

I am not sure if it has to do with the handling of the integers (the standard requires unsigned integers) or whether I made a mistake in one of the utility functions, or both. If it is indeed an issue with the handling of the integers, how would I go about taking care of the problem. I was able to resolve this when working on the 256-bit version of the adaptation by using mod 2^32 when working with numbers in the digestblock method. I attempted to do mod 2^64 and 2^63 with the 512-bit version and it does not correct the problem. I am stumped.

I should mention that I cannot use one of the many library implementations as I am using a sandboxed Lua that does not provide this access, which is why I need a pure lua implementation. Thanks in advance.

回答1:

Unfortunately, after introducing integers in Lua 5.3 writing scripts for Lua becomes a more complicated task.
You must always think about transformations between integers and floating point numbers.
ALWAYS. Yes, that's boring.

One of your mistakes is an excellent example of this "dark corner of Lua".

  local remainder = l % 256
  s = string.char(remainder) .. s
  --remove from l the bits we have already transformed
  l = (l-remainder) / 256

Your value l is initially a 64-bit integer.
After cutting off its first byte l contains (64-8) = 56 bits, but now it's a floating point-number (with 53-bit precision, of course).

Possible solution: use l = l >> 8 or l = l // 256 instead of l = (l-remainder) / 256

---

Another mistake is using s264num(msg, i + (j - 1) * 4) instead of s264num(msg, i + (j - 1) * 8)

---

One more mistake is in the following line:

local extra = 128 - (len + 9) % 128

The correct code is

local extra = - (len + 17) % 128 + 8

(Please note that -a%m+b is not the same as b-a%m due to operator precedence)

---

After fixing these 3 mistakes your code works correctly.

来源：https://stackoverflow.com/questions/50442185/adaptation-of-sha2-512-gives-incorrect-results