SIMD-0167: Loader-v4 by Lichtso · Pull Request #167 · solana-foundation/solana-improvement-documents

Lichtso · 2024-08-20T13:20:14Z

No description provided.

nickfrosty

I also noticed there are several typos on the word "throw"

proposals/0167-loader-v4.md

topointon-jump

Very nice to see this well-written, well-specified SIMD!

Would be great to get more detail on the migration path from v3 to v4.

topointon-jump · 2024-11-01T20:55:03Z

proposals/0167-loader-v4.md

+
+- loader-v3 clears the program proxy account (setting its size to zero)
+- loader-v3 transfers all funds from the programdata to the proxy account
+- loader-v3 gifts the program proxy account to loader-v4


How would the gifting work? Do we need to add a new instruction to the v3 loader? Will this upgrade mechanism be specified in a future SIMD?

I see three options here:

A migrate instruction for loader-v3 which the dApp devs trigger themselves

A migrate instruction for loader-v3 which we the protocol engineers trigger

A feature gate which scans the accounts DB and migrates all loader-v3 programs

I would prefer the second option as that allows us to keep the gathering of the loader-v3 program list and then slowly working through it off-chain and still makes sure that all loader-v3 programs get migrated.

Added a more detailed description of how this would work:
f2d4970

buffalojoec

Getting really close on my side. Based on our last discussion we wanted to add the following to SetAuthority:

If the program is Uninitialized will skip the existing authority check, set the new authority, and set the status to NeverBeenDeployed.
Programs that are Retracted can be finalized only if the program length is 0

proposals/0167-loader-v4.md

buffalojoec

Let's do it!

proposals/0167-loader-v4.md

jstarry · 2025-08-26T01:38:26Z

proposals/0167-loader-v4.md

+- the program account was not deployed within the current slot (delay
+visibility)


Can we define "delay visibility" or "deployment cooldown" in terminology and use the term consistently in this SIMD?

jstarry · 2025-08-26T01:40:58Z

proposals/0167-loader-v4.md

+  - Change the status stored in the program account to `Retracted`
+  - Set the `is_executable` flag to `false`
+
+#### SetAuthority


This instruction does three different actions: initialize, set-authority, or finalize. All of them set the authority field but they are all different enough that they warrant having specific instructions, in my opinion.

Splitting this instruction up would help clarify instruction account requirements:

Initialize does not require an instruction account to specify the current authority

SetAuthority would always require the new authority to be a signer, no need to specify it as optional

Finalize does not require an instruction account to specify a new authority

Additionally, the way SetAuthority is specified to be treated as an "initialization" is an anti-pattern because it's typically assumed that when initializing accounts on solana that the caller allocates the account data before invoking the initialization instruction.

jstarry · 2025-08-26T01:50:37Z

proposals/0167-loader-v4.md

+
+None.
+
+## Detailed Design


@Lichtso I don't think this has been addressed yet (https://github.com/solana-foundation/solana-improvement-documents/pull/167/files#r2184891211)

proposals/0167-loader-v4.md

jstarry · 2025-08-26T01:54:49Z

proposals/0167-loader-v4.md

+
+The feature gate must enable loader-v4 program management and execution.
+
+### Owned Program Accounts


nit: I don't really understand what "owned" is supposed to indicate here. Can we call this section "Program Account Layout"?

jstarry · 2025-08-26T02:13:44Z

proposals/0167-loader-v4.md

+    otherwise throw `InvalidArgument`
+  - Check that there are enough funds in the program account for rent
+  exemption of the new length, otherwise throw `InsufficientFunds`
+  - Set the length of the program account to the requested new size plus


I don't think this was addressed (https://github.com/solana-foundation/solana-improvement-documents/pull/167/files#r2226342431)

proposals/0167-loader-v4.md

jstarry · 2025-10-07T02:31:32Z

proposals/0167-loader-v4.md

+    - Enum variant `0u64`: `Uninitalized`, account was zero-filled externally
+    - Enum variant `1u64`: `NeverBeenDeployed`, used as write buffer
+    - Enum variant `2u64`: `Retracted`, program is in maintenance
+    - Enum variant `3u64`: `Deployed`, program is ready to be executed
+    - Enum variant `4u64`: `Finalized`, same as `Deployed`, but can not be


some typo fixes (u64 -> u32 and Uninitalized -> Uninitialized) and a new naming suggestion for NeverBeenDeployed -> Buffered.

Suggested change

- Enum variant `0u64`: `Uninitalized`, account was zero-filled externally

- Enum variant `1u64`: `NeverBeenDeployed`, used as write buffer

- Enum variant `2u64`: `Retracted`, program is in maintenance

- Enum variant `3u64`: `Deployed`, program is ready to be executed

- Enum variant `4u64`: `Finalized`, same as `Deployed`, but can not be

- Enum variant `0u32`: `Uninitialized`, account was zero-filled externally

- Enum variant `1u32`: `Buffered`, used as write buffer

- Enum variant `2u32`: `Retracted`, program is in maintenance

- Enum variant `3u32`: `Deployed`, program is ready to be executed

- Enum variant `4u32`: `Finalized`, same as `Deployed`, but can not be

jstarry · 2025-10-07T02:37:26Z

proposals/0167-loader-v4.md

+
+### Program Account Header Verification
+
+Verification the program account checks in the following order that:


Suggested change

Verification the program account checks in the following order that:

Verification of the program account checks the following in order:

jstarry · 2025-10-07T02:49:48Z

proposals/0167-loader-v4.md

+
+- the owner of the program account is loader-v4,
+otherwise throw `InvalidAccountOwner`
+- the program account is at least as long enough for the header (48 bytes),


Suggested change

- the program account is at least as long enough for the header (48 bytes),

- the program account is at least long enough for the header (48 bytes),

jstarry · 2025-10-07T02:56:13Z

proposals/0167-loader-v4.md

+Invoking programs owned by loader-v4 checks in the following order that:
+
+- the owner of the program account is loader-v4
+- the program account is at least as long enough for the header


In light of the conversation here should we change this to check that the program account is strictly greater than the length of the header? A finalized program with executable length of zero definitely wouldn't be executable. But this is technically a sub-case of the later "executable verification" check anyways.

jstarry · 2025-10-07T03:29:56Z

proposals/0167-loader-v4.md

+
+### Program Management Instructions
+
+The loader-v4 intructions Deploy and Retract are not authorized in CPI.


Suggested change

The loader-v4 intructions Deploy and Retract are not authorized in CPI.

The loader-v4 instructions Deploy and Retract are not authorized in CPI. If

these instructions are invoked from another program, throw

`ProgramFailedToComplete`.

Was thinking about the CPI restriction some more: Isn't it a problem for multisig and governance? E.g. #167 (comment) would not work, right?

Ah yeah, you're right. Didn't think about that. Let's try to remove this restriction then. If I remember correctly it's because we don't yet allow changing the executable flag from CPI.

jstarry · 2025-10-07T04:54:47Z

proposals/0167-loader-v4.md

+  - Enum variant `3u32`
+  - `u32` The new size after the operation.
+- Behavior:
+  - Charge 32 + new_size_in_bytes / cpi_bytes_per_unit CUs


We should only charge CU's for the increased size (if any)

I think we discussed this before. The reason why we charge for the entire length is because the program runtime has to copy over the existing data into the new allocation.

Ah yeah, you're right never mind.

jstarry · 2025-10-07T05:02:56Z

proposals/0167-loader-v4.md

+- Instruction data:
+  - Enum variant `4u32`
+  - `u32` Byte offset at which to write the given bytes
+  - `[u8]` Chunk of the programs executable file


This looks different from loader-v3 which serializes the length of the chunk as a u64 before the chunk data. Are you implying that the length isn't serialized here and that chunk_length_in_bytes is inferred from ix.data.len - 4 bytes (enum variant) - 4 bytes (offset)? I think we should match loader-v3 behavior and serialize the chunk length as well. But consider serializing the length as u16 or u32 instead of u64 to save some bytes

What is the advantage of doing so? It would introduce additional failure cases of the explicit field and implicit calculation disagreeing.

jstarry · 2025-10-07T05:04:26Z

proposals/0167-loader-v4.md

+
+failing any of the above checks must throw `UnsupportedProgramId`.
+
+### Program Management Instructions


Should also add a note saying that instruction data length is limited to 1232 bytes to match loader-v3 behavior

jstarry · 2025-10-07T05:20:26Z

proposals/0167-loader-v4.md

+    otherwise throw `InvalidArgument`
+  - Check that the executable length is not longer than the program account,
+    otherwise throw `InvalidArgument`
+  - Charge program_length_in_bytes / cpi_bytes_per_unit CUs


We should only charge for the executable length rather than the full account data length

Suggested change

- Charge program_length_in_bytes / cpi_bytes_per_unit CUs

- Charge executable length / cpi_bytes_per_unit CUs

jstarry · 2025-10-07T05:22:37Z

proposals/0167-loader-v4.md

+  - Charge program_length_in_bytes / cpi_bytes_per_unit CUs
+  - Check that the executable file stored in the program account passes
+  executable verification
+  - Change the executable length in the program account header


Suggested change

- Change the executable length in the program account header

- Change the executable length in the program account header to the specified

value.

Lichtso · 2025-11-11T14:29:23Z

If we explicitly encode the ELF length anyway, we might as well move the header down to become a footer:

Lichtso force-pushed the loader-v4 branch from 8d0ec51 to ce5daeb Compare August 20, 2024 13:28

Lichtso mentioned this pull request Aug 20, 2024

Adjustments of loader-v4 anza-xyz/agave#2630

Merged

nickfrosty reviewed Aug 22, 2024

View reviewed changes

proposals/0167-loader-v4.md Outdated Show resolved Hide resolved

Lichtso force-pushed the loader-v4 branch from ce5daeb to 9fedd95 Compare August 22, 2024 18:25

github-actions bot mentioned this pull request Aug 26, 2024

Upstream Updates - Mon Aug 26 00:12:40 UTC 2024 smartcontractkit/chainlink-solana#831

Closed

Lichtso force-pushed the loader-v4 branch 12 times, most recently from fc20b4e to bd5ab89 Compare August 27, 2024 15:00

Lichtso force-pushed the loader-v4 branch from bd5ab89 to 0b3d4c5 Compare September 5, 2024 18:40

Lichtso mentioned this pull request Oct 8, 2024

SIMD-0181: Privileged Syscalls #181

Closed

topointon-jump previously approved these changes Nov 1, 2024

View reviewed changes

Lichtso force-pushed the loader-v4 branch from 434071b to a4f4116 Compare November 19, 2024 15:19

Lichtso mentioned this pull request Dec 30, 2024

SIMD-0164: ExtendProgramChecked loader-v3 instruction #164

Closed

Lichtso force-pushed the loader-v4 branch 2 times, most recently from 1f95ac3 to a788388 Compare January 24, 2025 12:18

Lichtso mentioned this pull request Jan 24, 2025

Feature - Adds LoaderV4Instruction::Copy anza-xyz/agave#4613

Merged

joncinque mentioned this pull request Jan 27, 2025

Refactor - Makes loader-v4 the default anza-xyz/agave#2796

Merged

Lichtso mentioned this pull request Jan 28, 2025

Feature - Adds UpgradeableLoaderInstruction::Migrate anza-xyz/agave#4661

Merged

Lichtso force-pushed the loader-v4 branch 3 times, most recently from 05f0b13 to f2d4970 Compare January 29, 2025 15:09

Lichtso force-pushed the loader-v4 branch from a41ad2b to b777b48 Compare July 23, 2025 19:36

buffalojoec reviewed Jul 24, 2025

View reviewed changes

proposals/0167-loader-v4.md Outdated Show resolved Hide resolved

proposals/0167-loader-v4.md Outdated Show resolved Hide resolved

proposals/0167-loader-v4.md Show resolved Hide resolved

Lichtso force-pushed the loader-v4 branch from b777b48 to aa7c34f Compare July 30, 2025 15:58

Lichtso force-pushed the loader-v4 branch from aa7c34f to f27f77f Compare August 6, 2025 16:06

buffalojoec reviewed Aug 7, 2025

View reviewed changes

proposals/0167-loader-v4.md Show resolved Hide resolved

proposals/0167-loader-v4.md Outdated Show resolved Hide resolved

proposals/0167-loader-v4.md Outdated Show resolved Hide resolved

Lichtso force-pushed the loader-v4 branch from 696d941 to 6a80e94 Compare August 22, 2025 16:30

buffalojoec previously approved these changes Aug 22, 2025

View reviewed changes

jstarry reviewed Aug 26, 2025

View reviewed changes

proposals/0167-loader-v4.md Show resolved Hide resolved

proposals/0167-loader-v4.md Show resolved Hide resolved

jstarry reviewed Aug 26, 2025

View reviewed changes

jstarry reviewed Sep 5, 2025

View reviewed changes

proposals/0167-loader-v4.md Outdated Show resolved Hide resolved

proposals/0167-loader-v4.md Outdated Show resolved Hide resolved

jstarry reviewed Sep 9, 2025

View reviewed changes

proposals/0167-loader-v4.md Outdated Show resolved Hide resolved

jstarry reviewed Sep 9, 2025

View reviewed changes

Lichtso force-pushed the loader-v4 branch from 78cc45a to fec253a Compare September 9, 2025 07:31

Lichtso added 3 commits September 11, 2025 17:52

Suggestions from jstarry, buffalojoec and t-nelson.

efa203b

Reorders instructions.

e5cb64d

Suggestions from jstarry.

6ee6720

Lichtso force-pushed the loader-v4 branch from fec253a to 6ee6720 Compare September 11, 2025 15:57

Review feedback.

ebde062

Lichtso requested a review from jstarry September 29, 2025 20:15

Separates program length and account length.

5ac9b65

jstarry reviewed Oct 7, 2025

View reviewed changes

WIP: Review feedback

b6eaa16

Lichtso dismissed stale reviews from buffalojoec and topointon-jump via 5ac9b65 November 19, 2025 20:19

buffalojoec self-requested a review December 3, 2025 13:58

Lichtso mentioned this pull request Feb 19, 2026

Inconsistent CU charging for ELF size across loaders anza-xyz/agave#10659

Closed

		- the program account was not deployed within the current slot (delay
		visibility)


		The feature gate must enable loader-v4 program management and execution.

		### Owned Program Accounts


		### Program Account Header Verification

		Verification the program account checks in the following order that:

	Verification the program account checks in the following order that:
	Verification of the program account checks the following in order:

	- the program account is at least as long enough for the header (48 bytes),
	- the program account is at least long enough for the header (48 bytes),


		### Program Management Instructions

		The loader-v4 intructions Deploy and Retract are not authorized in CPI.

-The loader-v4 intructions Deploy and Retract are not authorized in CPI.
+The loader-v4 instructions Deploy and Retract are not authorized in CPI. If
+these instructions are invoked from another program, throw
+`ProgramFailedToComplete`.


		failing any of the above checks must throw `UnsupportedProgramId`.

		### Program Management Instructions

	- Charge program_length_in_bytes / cpi_bytes_per_unit CUs
	- Charge executable length / cpi_bytes_per_unit CUs

	- Change the executable length in the program account header
	- Change the executable length in the program account header to the specified
	value.

Conversation

Lichtso commented Aug 20, 2024

Uh oh!

nickfrosty left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

topointon-jump left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Lichtso Jan 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

buffalojoec left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

buffalojoec left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jstarry Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jstarry Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

topointon-jump left a comment •

edited

Loading

Lichtso Jan 16, 2025 •

edited

Loading

jstarry Aug 26, 2025 •

edited

Loading

jstarry Aug 26, 2025 •

edited

Loading