Skip to main content

Return to Question

Notice removed Draw attention by CommunityBot
Bounty Ended with no winning answer by CommunityBot
actionscript-3 tag was removed
Link
edit approved Feb 8, 2017 at 9:29
Sangbok Lee
edit approved Feb 8, 2017 at 9:29
Sangbok Lee
  • 2.2k
  • 3
  • 16
  • 33
Notice added Draw attention by Markus von Broady
Bounty Started worth 100 reputation by Markus von Broady
update
Source Link
Markus von Broady
Markus von Broady
  • 313
  • 2
  • 16

From docs: seq set-if-equal destination = source1 == source2 ? 1 : 0, component-wise

I haven't yet tested it thoroughly, but so far my fragment shader worked on both machines (desktop pcs), where context3D initialization succeeded as DirectX, but doesn't work on machines where flash falls back to software rendering.

seq ft2.x, ft0.x, fc0.x

ft.x is set to 1 on hardware, when current pixel red value, stored in ft0.x is equal to constant fc0.x, which stores 50/255. So what I want to happen, does happen on #32???? (50 == 0x32) colored pixel on hardware, but doesn't on software.

I already tested for a workaround, and I can replace seq opcodes with a more complex algorithm involving slt (set if less than) or sge (set if greater or equal).

So it seems the problem lies in a comparison of a constant I supply to the GPU (50/255) and the actual red value (which is 50 in the texture). If it was anything else (e.g. RGBA values had a different order), slt and sge would fail as well.

Am I doing something wrong here? Should I somehow round compared values (e.g. multiply by 255 then remove the fractional) in order to be sure it will work in all devices and modes?

Update: One of the machines with software rendering fallback was set to 16 bit graphics, however changing it to 32 bit didn't fix the issue. I also did a blind try to divide the color value by 256, 128 and 127 instead of 255, hoping it could maybe fix the issue if the float had a different precision (and higher and lower numbers would work as long as they would equal to one of pixels inside a 256px long gradient), but my hopes didn't pay off.

Then I tried the workaround of storing the constant as an integer, and inside shader multiplying the value by 255 and removing the fractional, and to my surprise, while it worked on GPU, it failed on software rendering:

mul ft0.x, ft0.x, fc0.y convert ft0.x (red channel) to integer by multiplying it by the constant 255

frc ft4.x, ft0.x get a fractional

sub ft0.x, ft0.x, ft4.x remove fractional, to truncate the integer

Now do the comparisons, e.g. seq ft2.x, ft0.x, fc0.x

add ft0.x, ft0.x, ft4.x add fractional back, this step is probably not necessary

div ft0.x, ft0.x, fc0.y divide the integer value by 255 to convert it back to float (by this I mean a number in 0..1 range)

The next thing I'm going to try as a workaround is to simply make a series of less-than comparisons that set a temp register to 1, which is added to another temp register (a counter), so that by checking the counter I can see inside which range is the value.

From docs: seq set-if-equal destination = source1 == source2 ? 1 : 0, component-wise

I haven't yet tested it thoroughly, but so far my fragment shader worked on both machines (desktop pcs), where context3D initialization succeeded as DirectX, but doesn't work on machines where flash falls back to software rendering.

seq ft2.x, ft0.x, fc0.x

ft.x is set to 1 on hardware, when current pixel red value, stored in ft0.x is equal to constant fc0.x, which stores 50/255. So what I want to happen, does happen on #32???? (50 == 0x32) colored pixel on hardware, but doesn't on software.

I already tested for a workaround, and I can replace seq opcodes with a more complex algorithm involving slt (set if less than) or sge (set if greater or equal).

So it seems the problem lies in a comparison of a constant I supply to the GPU (50/255) and the actual red value (which is 50 in the texture). If it was anything else (e.g. RGBA values had a different order), slt and sge would fail as well.

Am I doing something wrong here? Should I somehow round compared values (e.g. multiply by 255 then remove the fractional) in order to be sure it will work in all devices and modes?

From docs: seq set-if-equal destination = source1 == source2 ? 1 : 0, component-wise

I haven't yet tested it thoroughly, but so far my fragment shader worked on both machines (desktop pcs), where context3D initialization succeeded as DirectX, but doesn't work on machines where flash falls back to software rendering.

seq ft2.x, ft0.x, fc0.x

ft.x is set to 1 on hardware, when current pixel red value, stored in ft0.x is equal to constant fc0.x, which stores 50/255. So what I want to happen, does happen on #32???? (50 == 0x32) colored pixel on hardware, but doesn't on software.

I already tested for a workaround, and I can replace seq opcodes with a more complex algorithm involving slt (set if less than) or sge (set if greater or equal).

So it seems the problem lies in a comparison of a constant I supply to the GPU (50/255) and the actual red value (which is 50 in the texture). If it was anything else (e.g. RGBA values had a different order), slt and sge would fail as well.

Am I doing something wrong here? Should I somehow round compared values (e.g. multiply by 255 then remove the fractional) in order to be sure it will work in all devices and modes?

Update: One of the machines with software rendering fallback was set to 16 bit graphics, however changing it to 32 bit didn't fix the issue. I also did a blind try to divide the color value by 256, 128 and 127 instead of 255, hoping it could maybe fix the issue if the float had a different precision (and higher and lower numbers would work as long as they would equal to one of pixels inside a 256px long gradient), but my hopes didn't pay off.

Then I tried the workaround of storing the constant as an integer, and inside shader multiplying the value by 255 and removing the fractional, and to my surprise, while it worked on GPU, it failed on software rendering:

mul ft0.x, ft0.x, fc0.y convert ft0.x (red channel) to integer by multiplying it by the constant 255

frc ft4.x, ft0.x get a fractional

sub ft0.x, ft0.x, ft4.x remove fractional, to truncate the integer

Now do the comparisons, e.g. seq ft2.x, ft0.x, fc0.x

add ft0.x, ft0.x, ft4.x add fractional back, this step is probably not necessary

div ft0.x, ft0.x, fc0.y divide the integer value by 255 to convert it back to float (by this I mean a number in 0..1 range)

The next thing I'm going to try as a workaround is to simply make a series of less-than comparisons that set a temp register to 1, which is added to another temp register (a counter), so that by checking the counter I can see inside which range is the value.

Source Link
Markus von Broady
Markus von Broady
  • 313
  • 2
  • 16

AGAL seq opcode works on hardware, but doesn't on software emulation (float number comparison is different on both?)

From docs: seq set-if-equal destination = source1 == source2 ? 1 : 0, component-wise

I haven't yet tested it thoroughly, but so far my fragment shader worked on both machines (desktop pcs), where context3D initialization succeeded as DirectX, but doesn't work on machines where flash falls back to software rendering.

seq ft2.x, ft0.x, fc0.x

ft.x is set to 1 on hardware, when current pixel red value, stored in ft0.x is equal to constant fc0.x, which stores 50/255. So what I want to happen, does happen on #32???? (50 == 0x32) colored pixel on hardware, but doesn't on software.

I already tested for a workaround, and I can replace seq opcodes with a more complex algorithm involving slt (set if less than) or sge (set if greater or equal).

So it seems the problem lies in a comparison of a constant I supply to the GPU (50/255) and the actual red value (which is 50 in the texture). If it was anything else (e.g. RGBA values had a different order), slt and sge would fail as well.

Am I doing something wrong here? Should I somehow round compared values (e.g. multiply by 255 then remove the fractional) in order to be sure it will work in all devices and modes?