Support for a method with an InlinedCallFrame and a reverse p/invoke transition is missing in the jit

[davidwrighton]

Description

When fixing issue #45326 I disabled the ability of the JIT compiler to generate methods with both an inlined call frame and a reverse p/invoke transition with PR #55092. This causes a slight performance degradation in methods which would otherwise have both transitions. This was needed as the existing logic would setup the InlinedCallFrame while not in cooperative mode, thus causing periodic crashes. This should be changed to logic such that the function performs the reverse pinvoke logic before setting up the InlinedCallFrame, and removes the InlinedCallFrame before returning back to preemptive mode.

Configuration

All, although the assertion in bug #45326 can only occur on Windows X86. On other platforms, this would manifest as a much more rare crash.

Regression?

A bit. I don't have data.

category:cq
theme:inlining
skill-level:expert
cost:large
impact:small

[EgorBo]

I wrote a simple benchmark with a native function like this:

extern "C" __declspec(dllexport) void DoWork(MyFunc ptr)
{
    for (int i = 0; i < 100; i++)
    {
        ptr(i);
    }
}

and wasn't able to notice difference in performance calling a C# function that previously had inlined call frame + reverse pinvoke.
Moving to 7.0.0 as the fix for it won't make it to 6.0