Is it possible to read arbitrary data at arbitrary array offset in glsl?

Question

TL;DR

In a Vulkan / GLSL ray tracing program, I have several shader files, all accessing the same material buffer of arbitrary data. All materials have an arbitrary layout, e.g. one might be a float, one might contain two vec3s and a float. Can I access a buffer (i.e. an array of uint type) with just an offset within the shader (i.e. the same buffer binding is shared, I cannot use dynamic offsets set on the CPU side) and read out a struct without having to read every member of the struct individually? I am aware of not being able to read a vec3 from a buffer as just 3 values, but that is not my problem. Just assume I only read good data types that don't waste space.

See in the image, I want to access at the given green position with just knowing the size of the struct that will follow at that position.

In C++ it would be something like this:

uint32_t* buffer; //assume this has all the data in it
struct mat2
{
    glm::vec3 data1;
    float data2;
}

//the following line is what I try to achieve
mat2 matData = *((mat2)(buffer+offset);

Long explanation

I am trying to setup a buffer that contains material parameters. The problem is that I have to support a large number of materials, that may differ drastically in number of parameters, e.g. there might be an entry with two uints and then an entry with five floats.

Since I assume using one buffer per material is bad (due to the number of layouts bindings I'd have to use) I wanted to tightly pack the material parameters into a single byte buffer and readout from the shader with a known offset to the material for the hit geometry. I have a shader file per material, so no material is aware of other material structs - it just knows the offset into the array for its own data.

While I have basically set up the buffer (though apperantly the smallest chunks I am allowed to use are 4 bytes), I have trouble actually reading out of this buffer in GLSL. Given a simple struct for one material:

struct BlinnPhongParams
{
    vec3 diffuse;
    vec3 specular;
    float specIntensity;
};

And the material buffer:

layout(binding = 123, set = 0) readonly buffer MaterialParamsBuffer
{
    uint[] data;
}uMaterialParamsBuffer;

Currently I am reading every member from the buffer individually:

BlinnPhongParams getMaterialParams(uint offset)
{
    BlinnPhongParams bpParams = BlinnPhongParams(
        //DiffuseCol
        vec3(
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+1]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+2])),
        //SpecularCol
        vec3(
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+3]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+4]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+5])),
        //SpecIntensity
        uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+6])
    );
    return bpParams;
}

I would like to make reading from this buffer as easy as possible, ideally just say "at this offset, I have a struct that looks like this", e.g. I would like to have something like this:

BlinnPhongParams getMaterialParams(uint offset)
{
    // here is the problem
    BlinnPhongParams params = BlinnPhongParams(&uMaterialParamsBuffer.data[offset]);
}

Is this possible?

Answer 1

The shaders themselves have the structure memory layout in them and indexing into the array is essentially (sizeof(material)+alignment) * offset So what the listed pseudocode is already doing is pretty close to what is needed to index directly. The sizeof(material) would also need to be carefully computed since data alignment inside a struct follows the buffer type layout rules. For example the two vec3's actually align to vec4 boundaries. So internally at least 8 bytes are lost for alignment. This complicates writing the data since it is up to the program to place the data at the correct offset for both the structure alignment and the internal alignment, which is basically identical to what the shader is going to do to access it. For a vec3 25% of the memory is being lost to alignment.

As an example the listed struct layout would look something like the following in C++: (note that I am doing this from memory so please don't write these numbers in stone)

struct phong {
   vec3 diffuse; float alignment;
   vec3 specular; float alignment2;
   int intensity;
};

Where the actual size of the struct would add another 12 bytes to the end of the int so that sizeof(phong) == 48 instead of the expected 28.

The unusable memory issue can be overcome with careful design of materials, like folding the specular intensity into the specular color to align it on a vec4 boundary, etc.

So this is all doable but automating it so that the programmer doesn't have to hand tune every shader is more challenging.