Skip to content

Copy module metadata for source module, if any.#58508

Merged
maleadt merged 3 commits intomasterfrom
tb/module_metadata
Jan 19, 2026
Merged

Copy module metadata for source module, if any.#58508
maleadt merged 3 commits intomasterfrom
tb/module_metadata

Conversation

@maleadt
Copy link
Copy Markdown
Member

@maleadt maleadt commented May 23, 2025

Instead, rely on the linker automatically inheriting the metadata from the parent module. This avoids warnings and aborts with external users such as GPUCompiler.jl.

@maleadt maleadt requested a review from gbaraldi May 23, 2025 12:13
@maleadt maleadt added compiler:codegen Generation of LLVM IR and native code gpu Affects running Julia on a GPU labels May 23, 2025
@maleadt maleadt force-pushed the tb/module_metadata branch from c8760c3 to a7df949 Compare May 23, 2025 12:59
@gbaraldi
Copy link
Copy Markdown
Member

Looks fine to me. I guess our adhoc linker was fine with this but LLVMs default one complained?

@maleadt
Copy link
Copy Markdown
Member Author

maleadt commented May 23, 2025

Yeah, IIUC another thing that landed as part of #57010.

Copy link
Copy Markdown
Member

@vtjnash vtjnash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The concept that there is a "parent module" seems unsound to me. Does your linker not handle these correctly? I'd expect the LTO linker we're using here to combine these properly.

@maleadt
Copy link
Copy Markdown
Member Author

maleadt commented May 23, 2025

I'm not linking these, that happens as part of jl_emit_native calling jl_merge_module in a loop. And yeah LLVM's linker complains with warnings like:

warning: linking module flags 'Dwarf Version': IDs have conflicting values ('i32 4' from kernel with 'i32 2' from start)

... or an abort when there's a mismatch in stack-protector flags.

I considered alternative approaches, either (1) having the fresh modules in the jl_emit_native loop being clones of the initial llvmmod passed into it (which works for GPUCompiler.jl but isn't compatible with how precompilation invokes staticcompilation), and (2) adding additional flags to jl_new_ts_module to copy the flags from the llvmmod passed in (but that would result in those prototypes getting very complicated; simply omitting the "optional" metadata seemed more straightforward).

@vtjnash
Copy link
Copy Markdown
Member

vtjnash commented May 23, 2025

jl_new_ts_module to copy the flags from the llvmmod passed in

I think this option makes the most sense. In theory LLVM already implements it because it uses this same operation in many places. In practice, it isn't exactly optimal, but it is just CloneModule with a lambda that returns false, followed by a pass to remove all globals (alternatively, just copying out the 10 lines from that function which are relevant) as here: https://llvm.org/doxygen/namespacellvm.html#a61553b705fc9be3d8d0a18a8af1bc152

@maleadt
Copy link
Copy Markdown
Member Author

maleadt commented Jan 17, 2026

Alright, implemented the above suggestion. Since GPUCompiler.jl already calls jl_emit_native with a module that's been set-up correctly, thread that through until it reaches jl_create_llvm_module so that we can copy flags instead.

@maleadt maleadt added the backport 1.13 Change should be backported to release-1.13 label Jan 17, 2026
@maleadt maleadt requested a review from vtjnash January 17, 2026 21:29
Copy link
Copy Markdown
Member

@vtjnash vtjnash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

@maleadt maleadt changed the title Don't emit module metadata for non-toplevel modules. Copy module metadata for source module, if any. Jan 18, 2026
maleadt and others added 3 commits January 18, 2026 22:15
Instead, rely on the linker automatically inheriting the metadata
from the parent module. This avoids warnings and aborts with
external users such as GPUCompiler.jl.
Instead of omitting module flags for non-toplevel modules and relying on
LLVM linker inheritance, copy flags from the source/parent module when
creating child modules. This ensures that custom flags (e.g., custom
Dwarf Version set by GPUCompiler.jl) are properly propagated to child
modules.

The jl_create_llvm_module function now takes an optional Module *source
parameter. When provided, it copies module flags and properties from the
source module; when nullptr, it sets the default Julia flags.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@maleadt maleadt force-pushed the tb/module_metadata branch from b6ed6fa to ccb306e Compare January 18, 2026 21:15
@maleadt maleadt merged commit 897908c into master Jan 19, 2026
8 checks passed
@maleadt maleadt deleted the tb/module_metadata branch January 19, 2026 10:50
@KristofferC KristofferC mentioned this pull request Jan 26, 2026
43 tasks
@KristofferC KristofferC mentioned this pull request Feb 4, 2026
56 tasks
@KristofferC
Copy link
Copy Markdown
Member

@maleadt, this doesn't backport cleanly to 1.13.

maleadt added a commit that referenced this pull request Feb 10, 2026
This avoids warnings and aborts with external users such as GPUCompiler.jl.

(cherry picked from commit 897908c)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
maleadt added a commit that referenced this pull request Feb 11, 2026
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@KristofferC KristofferC removed the backport 1.13 Change should be backported to release-1.13 label Mar 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

compiler:codegen Generation of LLVM IR and native code gpu Affects running Julia on a GPU

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants