I am attempting to repurpose the heightmap shader example found here into one that will work with 32-bits of precision instead of 8. The work-in-progress code is on github: htt
If you encode a wide integer into the components of a RGBA vector it's essential that you turn off filtering so that no interpolation happens between the values. Also OpenGL may internally convert to a different format, but that should only reduce your sample depth.
Your colour:
return new Color((enc >> 24 & 255)/255f, (enc >> 16 & 255)/255f, (enc >> 8 & 255)/255f,
(enc & 255)/255f);
... contains the most significant byte of enc
in r
, the second most significant in g
, etc.
This:
vAmount = dot(bumpData, vec4(1.0, 255.0, 65025.0, 160581375.0));
builds vAmount
with r
in the least significant byte, g
in the next-least significant, etc (though the multiplicands should be 256, 65536, etc*). So the bytes are in the incorrect order. The flatter version:
vAmount = dot(bumpData, vec4(1.0, 1.0/255.0, 1.0/65025.0, 1.0/160581375.0));
gets the bytes in the correct order but scales the output values into the range [0.0, 1.0]
, which is probably why it looks essentially flat.
So switch the order of encoding or of decoding the bytes and pick an appropriate scale.
(*) think about it this way: the smallest number that can go on any channel is 1.0 / 255.0
. The least significant channel will be in the range [0, 1.0]
— from 0 / 255.0
to 255.0 / 255.0
. You want to scale the next channel so that its smallest value is the next thing on that scale. So its smallest value should be 256 / 255.0
. So you need to turn 1.0 / 255.0
into 256.0 / 255.0
. You achieve that by multiplying by 256
, not by 255
.