6

I need to call a function in dll and return an array of structures. I do not know the size of the array in advance. How can this be done? The error can not marshal 'returns value' invalid managed / unmanaged

Code in C#:

[DllImport("CppDll"]
public static extern ResultOfStrategy[] MyCppFunc(int countO, Data[] dataO, int countF, Data[] dataF);

in C++:

extern "C" _declspec(dllexport) ResultOfStrategy* WINAPI MyCppFunc(int countO, MYDATA * dataO, int countF, MYDATA * dataF)
{
    return Optimization(countO, dataO, countF, dataF);
}

Return array of struct :

struct ResultOfStrategy
{
bool isGood;
double allProfit;
double CAGR;
double DD;
int countDeals;
double allProfitF;
double CAGRF;
double DDF;
int countDealsF;
Param Fast;
Param Slow;
Param Stop;
Param Tp;
newStop stloss;
};
7
  • 3
    This is very hard to do correctly in C++, that doesn't get better when you pinvoke. There is a nasty memory management problem, the storage for that array needs to be released again, using the exact same allocator that was used to create it. The pinvoke marshaller tells you that it doesn't want to solve that problem. Standard technique is to allow the client program to call the function twice. First with a null pointer for the array argument, all the function then does is return the required size. So in the second call it can pass the correctly sized array. Commented May 28, 2018 at 13:30
  • As always the three problems of marshaling strings and arrays are: who allocates the memory, how much memory needs to be allocated, who (and how) that memory can be released.
    – xanatos
    Commented May 28, 2018 at 13:37
  • @HansPassant Thanks a lot for the answers, now it's a little more clear what to do. At the expense of a double call, I understood. But @ Hanatos said that you can give C++ the ability to allocate memory via C#. I'm too new in this topic, if it's easy, can you write the simplest example please.
    – Fresto
    Commented May 28, 2018 at 13:37
  • @Fresto It is useless in this scenario... Best to use a shared allocator, like the Marshal.AllocHGlobal that is equivalent to LocalAlloc or the Marshal.AllocCoTaskMem that is equivalent to the CoTaskMemAlloc
    – xanatos
    Commented May 28, 2018 at 13:39
  • 1
    And in general what I don't see is how the C++ will return the length of the array.
    – xanatos
    Commented May 28, 2018 at 13:45

1 Answer 1

11

I'll give you two responses. The first one is a method quite basic. The second one is quite advanced.

Given:

C-side:

struct ResultOfStrategy
{
    //bool isGood;
    double allProfit;
    double CAGR;
    double DD;
    int countDeals;
    double allProfitF;
    double CAGRF;
    double DDF;
    int countDealsF;
    ResultOfStrategy *ptr;
};

C#-side:

public struct ResultOfStrategy
{
    //[MarshalAs(UnmanagedType.I1)]
    //public bool isGood;
    public double allProfit;
    public double CAGR;
    public double DD;
    public int countDeals;
    public double allProfitF;
    public double CAGRF;
    public double DDF;
    public int countDealsF;
    public IntPtr ptr;
}

Note that I've removed the bool, because it has some problems with the case 2 (but it works with case 1)... Now...

Case 1 is very basic, and it will cause the .NET marshaler to copy the array built in C to a C# array.

Case 2 as I've written is quite advanced and it tries to bypass this marshal-by-copy and make so that the C and the .NET can share the same memory.

To check the difference I've written a method:

static void CheckIfMarshaled(ResultOfStrategy[] ros)
{
    GCHandle h = default(GCHandle);

    try
    {
        try
        {
        }
        finally
        {
            h = GCHandle.Alloc(ros, GCHandleType.Pinned);
        }

        Console.WriteLine("ros was {0}", ros[0].ptr == h.AddrOfPinnedObject() ? "marshaled in place" : "marshaled by copy");
    }
    finally
    {
        if (h.IsAllocated)
        {
            h.Free();
        }
    }
}

And I've added a ptr field to the struct that contains the original address of the struct (C-side), to see if it has been copied or if it is the original struct.

Case 1:

C-side:

__declspec(dllexport) void MyCppFunc(ResultOfStrategy** ros, int* length)
{
    *ros = (ResultOfStrategy*)::CoTaskMemAlloc(sizeof(ResultOfStrategy) * 2);
    ::memset(*ros, 0, sizeof(ResultOfStrategy) * 2);
    (*ros)[0].ptr = *ros;
    (*ros)[0].allProfit = 100;
    (*ros)[1].ptr = *ros + 1;
    (*ros)[1].allProfit = 200;
    *length = 2;
}

and C#-side:

public static extern void MyCppFunc(
    [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.Struct, SizeParamIndex = 1)] out ResultOfStrategy[] ros, 
    out int length
);

and then:

ResultOfStrategy[] ros;
int length;
MyCppFunc(out ros, out length);

Console.Write("Case 1: ");
CheckIfMarshaled(ros);

ResultOfStrategy[] ros2;

The .NET marshaler knows (because we gave it the information) that the second parameter is the length of the out ResultOfStrategy[] ros (see the SizeParamIndex?), so it can create a .NET array and copy from the C-allocated array the data. Note that in the C code I've used the ::CoTaskMemAlloc to allocate the memory. The .NET wants the memory to be allocated with that allocator, because it then frees it. If you use malloc/new/??? to allocate the ResultOfStrategy[] memory, bad things will happen.

Case 2:

C-Side:

__declspec(dllexport) void MyCppFunc2(ResultOfStrategy* (*allocator)(size_t length))
{
    ResultOfStrategy *ros = allocator(2);
    ros[0].ptr = ros;
    ros[1].ptr = ros + 1;
    ros[0].allProfit = 100;
    ros[1].allProfit = 200;
}

C#-side:

// Allocator of T[] that pins the memory (and handles unpinning)
public sealed class PinnedArray<T> : IDisposable where T : struct
{
    private GCHandle handle;

    public T[] Array { get; private set; }

    public IntPtr CreateArray(int length)
    {
        FreeHandle();

        Array = new T[length];

        // try... finally trick to be sure that the code isn't interrupted by asynchronous exceptions
        try
        {
        }
        finally
        {
            handle = GCHandle.Alloc(Array, GCHandleType.Pinned);
        }

        return handle.AddrOfPinnedObject();
    }

    // Some overloads to handle various possible length types
    // Note that normally size_t is IntPtr
    public IntPtr CreateArray(IntPtr length)
    {
        return CreateArray((int)length);
    }

    public IntPtr CreateArray(long length)
    {
        return CreateArray((int)length);
    }

    public void Dispose()
    {
        FreeHandle();
    }

    ~PinnedArray()
    {
        FreeHandle();
    }

    private void FreeHandle()
    {
        if (handle.IsAllocated)
        {
            handle.Free();
        }
    }
}

[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate IntPtr AllocateResultOfStrategyArray(IntPtr length);

[DllImport("CplusPlusSide.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern void MyCppFunc2(
    AllocateResultOfStrategyArray allocator
);

and then

ResultOfStrategy[] ros;

using (var pa = new PinnedArray<ResultOfStrategy>())
{
    MyCppFunc2(pa.CreateArray);
    ros = pa.Array;

    // Don't do anything inside of here! We have a
    // pinned object here, the .NET GC doesn't like
    // to have pinned objects around!

    Console.Write("Case 2: ");
    CheckIfMarshaled(ros);
}

// Do the work with ros here!

Now this one is interesting... The C function receives an allocator from the C#-side (a function pointer). This allocator will allocate length elements and must then remember the address of the allocated memory. The trick here is that C#-side we allocate a ResultOfStrategy[] of the size required by C, that is then used directly C-side. This will break badly if ResultOfStrategy isn't blittable (a term meaning that you can only use some types inside ResultOfStrategy, mainly numerical types, no string, no char, no bool, see here). The code is quite advanced because on top of all of this, it has to use the GCHandle to pin the .NET array, so that it isn't moved around. Handling this GCHandle is quite complex, so I had to create a ResultOfStrategyContainer that is IDisposable. In this class I even save the reference to the created array (the ResultOfStrategy[] ResultOfStrategy). Note the use of the using. That is the correct way to use the class.

bool and case 2

As I've said, while bool work with case 1, they don't work with case 2... But we can cheat:

C-side:

struct ResultOfStrategy
{
    bool isGood;

C#-side:

public struct ResultOfStrategy
{
    private byte isGoodInternal;
    public bool isGood
    {
        get => isGoodInternal != 0;
        set => isGoodInternal = value ? (byte)1 : (byte)0;
    }

this works:

C-side:

extern "C"
{
    struct ResultOfStrategy
    {
        bool isGood;
        double allProfit;
        double CAGR;
        double DD;
        int countDeals;
        double allProfitF;
        double CAGRF;
        double DDF;
        int countDealsF;
        ResultOfStrategy *ptr;
    };

    int num = 0;
    int size = 10;

    __declspec(dllexport) void MyCppFunc2(ResultOfStrategy* (*allocator)(size_t length))
    {
        ResultOfStrategy *ros = allocator(size);

        for (int i = 0; i < size; i++)
        {
            ros[i].isGood = i & 1;
            ros[i].allProfit = num++;
            ros[i].CAGR = num++;
            ros[i].DD = num++;
            ros[i].countDeals = num++;
            ros[i].allProfitF = num++;
            ros[i].CAGRF = num++;
            ros[i].DDF = num++;
            ros[i].countDealsF = num++;
            ros[i].ptr = ros + i;
        }

        size--;
    }
}

C#-side:

[StructLayout(LayoutKind.Sequential)]
public struct ResultOfStrategy
{
    private byte isGoodInternal;
    public bool isGood
    {
        get => isGoodInternal != 0;
        set => isGoodInternal = value ? (byte)1 : (byte)0;
    }
    public double allProfit;
    public double CAGR;
    public double DD;
    public int countDeals;
    public double allProfitF;
    public double CAGRF;
    public double DDF;
    public int countDealsF;
    public IntPtr ptr;
}

and then

ResultOfStrategy[] ros;

for (int i = 0; i < 10; i++)
{
    using (var pa = new PinnedArray<ResultOfStrategy>())
    {
        MyCppFunc2(pa.CreateArray);
        ros = pa.Array;

        // Don't do anything inside of here! We have a
        // pinned object here, the .NET GC doesn't like
        // to have pinned objects around!
    }

    for (int j = 0; j < ros.Length; j++)
    {
        Console.WriteLine($"row {j}: isGood: {ros[j].isGood}, allProfit: {ros[j].allProfit}, CAGR: {ros[j].CAGR}, DD: {ros[j].DD}, countDeals: {ros[j].countDeals}, allProfitF: {ros[j].allProfitF}, CAGRF: {ros[j].CAGRF}, DDF: {ros[j].DDF}, countDealsF: {ros[j].countDealsF}");
    }

    Console.WriteLine();
}
7
  • Oh, man. Thank you very much! I am very surprised that in english stackoverflow give very detailed answers, even on such difficult questions. I will study in detail the second option, and the first one came up and everything works. Once again many thanks!
    – Fresto
    Commented May 29, 2018 at 10:40
  • @Fresto Because during cut-n-paste I forgot a line :-) I've added it. Search for the public static extern void MyCppFunc(. There is a special annotation SizeParamIndex` that does the magic.
    – xanatos
    Commented May 29, 2018 at 12:07
  • Thank you so much)
    – Fresto
    Commented May 29, 2018 at 12:13
  • Sorry for the question. But, why, if I call my function once, then the results are correct when returning from the C ++ function. But if I call this code using(var container ...) {...} several times with different input values, then for the second time the values are incorrect? Is the reason for the intersection of memory?
    – Fresto
    Commented Jun 1, 2018 at 10:03
  • @Fresto Tested here... Works correctly here. Are you sure you have the same struct both C# and C side?
    – xanatos
    Commented Jun 1, 2018 at 13:50

Not the answer you're looking for? Browse other questions tagged or ask your own question.