Skip to content

[RuntimeAsync] Merge from upstream#3079

Merged
jakobbotsch merged 469 commits intodotnet:feature/async2-experimentfrom
jakobbotsch:merge-from-upstream
Apr 10, 2025
Merged

[RuntimeAsync] Merge from upstream#3079
jakobbotsch merged 469 commits intodotnet:feature/async2-experimentfrom
jakobbotsch:merge-from-upstream

Conversation

@jakobbotsch
Copy link
Member

Fixes:

  • BuildResumptionStubCalliSignature was building a signature with HASTHIS | EXPLICITTHIS calling convention always, even for signatures without any 'this'. With Support HasThis and ExplicitThis calling conventions in AssemblyBuilder and DynamicMethod runtime#113666 this broke as we started reordering the first argument with ret buffers.
  • Update JIT interface getEHinfo to handle getting EH clause information when trying to inline methods with transient IL, like async thunks
  • Update for new refactored jitinterface classes
  • Update for removed g_isNewExceptionHandlingEnabled

cc @VSadov

teo-tsirpanis and others added 30 commits March 23, 2025 18:35
---------

Co-authored-by: Larry Ewing <lewing@microsoft.com>
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
Now that the JIT is able to inline methods with exception handling, we need to disable inlining explicitly to maintain the test invariants. Inlining is no longer disabled as side-effect of exception handling.

Fixes #113813
…0.6998.166 (#113802)

* Automated bump of chrome version

* Update V8 version in BrowserVersions.props

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ilona Tomkowicz <32700855+ilonatommy@users.noreply.github.com>
* Fix possible null reference exception in SslStream.Protocol

* Update src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Protocol.cs

Co-authored-by: Marie Píchová <11718369+ManickaP@users.noreply.github.com>

* code review feedback

* Fix build

---------

Co-authored-by: Marie Píchová <11718369+ManickaP@users.noreply.github.com>
* [mono][sgen] Fix DUMP_GRAPH debug option build for tarjan bridge

* [mono][sgen] Don't create ScanData* during debug dumping of SCCs

It serves no purpose and it would later crash the runtime since we didn't patch the lockword back in place.

* [mono][sgen] Fix some null deref crashes in DUMP_GRAPH debug option

* [mono][tests] Add bridge tests

These are ported from some of the bridge tests we had on mono/mono. In order to test them we compare between the output of the new and the tarjan bridge.
…50317.4 (#113819)

Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Common , Microsoft.DotNet.XHarness.TestRunners.Xunit
 From Version 10.0.0-prerelease.25164.1 -> To Version 10.0.0-prerelease.25167.4

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
When a NULL reference exception occurs in a JIT helper or a VSD stub,
runtime pretends the exception occured in the managed caller. There is a
bug on x86 Windows where the COMPlusThrowCallback considers that frame
to be the frame where the exception actually occurred (based on the
m_crawl.isFirst). In case the call to the helper is the last instruction
in a try region, the exception handler lookup would reject that address
and the exception may not get handled at the right place or at all.

This change fixes it by ensuring that the m_crawl.isFirst is not set
when the frame is not the frame of the failure.

Close #113106
Remove explicit GC Poll from GC.SuppressFinalize().
Remove GC Poll from FCall and move it to managed, Buffer.__BulkMoveWithWriteBarrier().
Remove unneccessary explicit FC_GC_POLL() calls.
Remove unneccessary explicit FC_GC_POLL_RET() calls.
Remove unmanaged GCPoll implementation.
Implement JIT intrinsic for GC Poll, Thread.FastGCPoll.
Add ease-of-use overloads for MLKem API where appropriate.
Only the secret stub parameter goes through this path to create an edge
in the graph, and it was missing a check for whether it was going into
the same register as it was in. This would cause register homing to
generate unnecessary moves into a temporary register because it thought
there was a conflict.

Usually the secret stub parameter has to be allocated to a callee saved
variable, so we typically do not see this inefficiency.
* [RISC-V] Detect Zba & Zbb extensions

* [RISC-V] Reflect constants in cpufeatures.h to HardwareIntrinsicHelpers.cs

* [RISC-V] Use hwprobe syscall directly

* [RISC-V] Check syscall return value and use baseline support on failure

* [RISC-V] Regenerate JIT version GUID

* [RISC-V] Add Zba Zbb extension configuration variables
* Don't fail altjit compiles when RunAltJitCode=0

With RunAltJitCode=0, the JIT returns CORJIT_SKIPPED. We don't want
superpmi to fail these compiles.

* Enable SuperPMI replay for APX ISA testing

Create a new `runtime-coreclr superpmi-replay-apx` pipeline that
does SuperPMI replay an AltJit with various APX enabling configuration
options set. This enables some amount of APX testing even without
hardware available to run the generated code.

Notes:
1. Create a new jit-replay-pipeline.yml template to share the
   superpmi-replay pipeline implementation with the existing pipeline.
2. The existing pipeline is designated the "standard" replay type and
   the new pipeline the "apx" replay type.
3. The APX pipeline does replays for Windows x64 and Linux x64.
4. The configurations tested are:
```
a. "RunAltJitCode=0", "EnableAPX=1", "EnableApxNDD=1"
b. "RunAltJitCode=0", "EnableAPX=1", "JitStressRex2Encoding=1"
c. "RunAltJitCode=0", "EnableAPX=1", "JitStressPromotedEvexEncoding=1"
d. "RunAltJitCode=0", "EnableAPX=1", "JitStressRegs=4000"
e. "RunAltJitCode=0", "EnableAPX=1", "EnableApxNDD=1", "JitStressRex2Encoding=1", "JitStressPromotedEvexEncoding=1", "JitStressRegs=4000"
```
…ence-packages build 20250320.2 (#113820)

Microsoft.SourceBuild.Intermediate.source-build-reference-packages
 From Version 10.0.616401 -> To Version 10.0.617002

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…OT (#113850)

CORJIT_FLAG_READYTORUN and CORJIT_FLAG_PREJIT were always both set
together. Introduce a new name to be more explicit of a concept
that includes both ReadyToRun and NativeAOT (AOT == Ahead Of Time).

Introduce new JIT helpers `IsAot` (for either NativeAOT or R2R),
`IsNativeAot`, and `IsReadyToRun`.

Remove the name "ngen" in a bunch of places in the JIT code.
- Convert to the FCALL fast path/QCALL slow path approach
- Make EnterHelperResult/LeaveHelperAction into enum class so that they can safely be silently marshaled between native and managed
- Move the lockTaken flag handling to managed code so we can share more helpers

Benchmarking indicates this may actually be faster than the previous approach. I'm a bit skeptical, but its not out of the question.
…formation (#112825)

* Fix an edge case in the Tarjan SCC that lead to losing xref information

In the Tarjan SCC bridge processing there's a color graph used to find out
connections between SCCs. There was a rare case which only manifested when
a cycle in the object graph points to another cycle that points to a bridge
object. We only recognized direct bridge pointers but not pointers to other
non-bridge SCCs that in turn point to bridges and where we already calculated
the xrefs. These xrefs were then lost.

* Add test case to sgen-bridge-pathologies and add an assert to catch the original bug

* Add review

---------

Co-authored-by: Vlad Brezae <brezaevlad@gmail.com>
…e-file bundle probing" (#113776)

Undo the revert in dotnet/runtime#113738

Fix on top of the original: Make a copy of the host contract information on initialization instead of just operating on a pointer to the struct. This should address intermittent failures on shutdown where a thread is still running after coreclr_shutdown has completed and the actual contract struct provided by a host may be cleaned up. The assumption is that any function pointers provided on the host contract at initialization should remain valid for the process lifetime.
* Fix SslStreamDisposeTest.Dispose_ParallelWithHandshake_ThrowsODE test failures

* Fix build
The dot.net redirector is currently having issues, we should be using the script directly from the CDN anyway.
* fix cast asserts

* fix containment of CreateScalar

* add tests
When we hit a generic recursion over arrays, we detect it, but there was no cutoff logic to cut off the expansion. The result is a compile time OOM. This adds a cutoff, using the same cutoff number we use to look for deep generics.
tomeksowi and others added 20 commits April 8, 2025 09:25
* Intrinsify LeadingZeroCount

* TrailingZeroCount

* PopCount

* Build fix

* Remove assertion from default case

* cosmetics

* Separate bit counting from System.Math intrinsics

* Fix assert

* Revert unnecessary diffs
* Convert to [Fact]

* rename class

* Sve.IsSupported
Alpine is having problems with not enough file handles.
The public API surface looks exactly like RequiresUnreferencedCode, but
the private implementation details vary slightly. I don't think there's a
meaningful difference in the annotations for RDC vs RUC.
Fix #113458 -- failures on Azure Linux 3 due to running
out of file handles.
prepare-signed-artifacts.yml tries to download artifacts from all items in the buildStages array, but since we disabled the workloads job it fails to find an artifact.

Comment out the entry in yml instead.
Describe current stack buffer overrun detection capabilities.
…set to a directory (#114362)

Currently, host tracing can output to a file specified by the `COREHOST_TRACEFILE` environment variable. With multiple concurrent processes, the tracing gets interleaved in the resulting log.

With this change, if `COREHOST_TRACEFILE` is set to a directory that exists, the host will trace to a file named `<exe_name>.<pid>.log` in that directory. If the specified path does not exist or is not a directory, the host uses the existing behavior of tracing to the file specified.
Implements Vector Round intrinsic in terms of PackedSimd.
* slightly better calculation of stack frame for SVE ldr/str

* add comments

* review feedback
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ilona Tomkowicz <32700855+ilonatommy@users.noreply.github.com>
This PR changes the patchpoint key to be a pair <function
version, IL offset> instead of by IP of the return address. This is in
preparation for runtime async changes where suspension inside OSR
methods have to be resumed via resumption stub -> tier 0 -> OSR method.
The latter step needs to retransition into the right OSR method from the
tier 0 resumption code. With the current key this is problematic
because the return address is not going to be the same as the original
patchpoint that induced the OSR transition.
@jakobbotsch jakobbotsch added the area-async Runtime generate async state machines label Apr 9, 2025
@VSadov
Copy link
Member

VSadov commented Apr 9, 2025

x64 builds and passes runtime tests.

For x86 I see build errors.

  E:\A2\runtimelab\src\coreclr\vm\jithelpers.cpp(1693): error C2039: 'InsureInit': is not a member of 'HelperMethodFrame'
  E:\A2\runtimelab\src\coreclr\vm\frames.h(1204): note: see declaration of 'HelperMethodFrame'
  E:\A2\runtimelab\src\coreclr\inc\jithelpers.h(326): error C2440: 'type cast': cannot convert from 'overloaded-function' to 'LPVOID'
  E:\A2\runtimelab\src\coreclr\inc\jithelpers.h(326): note: Context does not allow for disambiguation of overloaded function
  E:\A2\runtimelab\src\coreclr\vm\jithelpers.cpp(3461): error C2737: 'hlpFuncTable': const object must be initialized

I think I have fixes for this in my other PR.
I'll cherry-pick the fixes and push here.

@VSadov
Copy link
Member

VSadov commented Apr 9, 2025

x86 builds, but a few tests fail - for example JIT\Directed\arglist\vararg_TargetWindows

I think I have a fix for that too

@VSadov
Copy link
Member

VSadov commented Apr 10, 2025

now only R2RDump fails on x86 (readytorun\r2rdump\FrameworkTests\R2RDumpTests\R2RDumpTests.cmd)

I think it is ok to merge and take a look at that later.

Copy link
Member

@VSadov VSadov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@jakobbotsch
Copy link
Member Author

Thanks for fixing that!

@jakobbotsch jakobbotsch merged commit bbb9741 into dotnet:feature/async2-experiment Apr 10, 2025
1 of 7 checks passed
@jakobbotsch jakobbotsch deleted the merge-from-upstream branch April 10, 2025 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-async Runtime generate async state machines

Projects

None yet

Development

Successfully merging this pull request may close these issues.