Skip to main content

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

4
  • $\begingroup$ yes exactly +1 :) $\endgroup$
    – Harry McKenzie
    Commented Jul 29, 2023 at 16:27
  • 1
    $\begingroup$ I made some tests and apparently the implementation underneath uses an 8-bit minifloat. Here's a calculator, if you set 4 as both precision and exponent bits, and then test consecutive numbers 0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6… You will get 0, 0.001953125, 0.00390625, 0.005859375, 0.0078125, 0.009765625, 0.01171875, suspiciously similar ;) $\endgroup$ Commented Jul 29, 2023 at 20:16
  • $\begingroup$ Apparently it uses 3 precision bits (1 sign, 2 mantissa; if there's no sign, then it's just 2 precision bits) and 4 exponent bits, totaling 7 bits (or 6 without a sign), then it skips every other value listed above. $\endgroup$ Commented Jul 29, 2023 at 20:35
  • $\begingroup$ As user43967 points out in the comment section to his answer, the values listed 0.003921568859368563, 0.007843137718737125, 0.011764707043766975, when multiplied by 255 give 1.000000059138983565, 2.000000118277966875, 3.000000296160578625, so the precision is enough to get those low values. However, it starts failing at 0x9, which gives one too much: 0.0390625 ×255 = 9.9609375 instead of 9. This is because the exponent increases and from now on some colors will be skipped. Understandable: need 8 bits to describe 0..255 range, and above I just figured it's 6/7 bits… $\endgroup$ Commented Jul 30, 2023 at 8:42