feat: add conversion utility to convert between bytes and elements

[ivokub]

Description

Utility for converting between bytes and nonnative/native. Needed for EIP-4844 #1489 as we need to compare that the "versioned hash" matches the commitment.

It also allows for an use case where if a circuit requires as an input some byte array, then we can provide it as a single native element and then directly decompose it into bytes in-circuit. This allows to significantly decrease the number of public inputs for some circuits.

The technique used here could also be used in non-native arithmetic for asserting less-equal. Currently we use binary decomposition and compare bit by bit (and also when doing strict modular reduction), but using range checker we could significantly reduce number of constraints. I'll do it later in a separate PR.

Currently it is not fully complete, it works for emulated only in case emulation parameters have limb width divisible by 8. It is currently the case, but changes for small fields. But at least unblocks some work elsewhere for now and we can implement it later.

Type of change

• New feature (non-breaking change which adds functionality)

How has this been tested?

• TestBytesToEmulatedDivisible
• TestBytesToNative
• TestNativeToBytes
• TestEmulatedToBytes
• TestAssertBytesLeq

Checklist:

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• I did not modify files generated from templates
• golangci-lint does not output errors locally
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[Copilot]

Pull Request Overview

Adds a conversion utility for transforming between byte arrays and both native/emulated field elements, enabling comparisons of “versioned hashes” and reducing public inputs in circuits.

• Registers the new conversion hints in the global hint registry
• Implements nativeToBytesHint and conversion logic in std/conversion
• Adds end-to-end tests covering bytes↔native and bytes↔emulated conversions

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
std/hints.go	register conversion hints
std/conversion/hints.go	implement nativeToBytesHint
std/conversion/conversion.go	add bytes↔native and bytes↔emulated logic
std/conversion/conversion_test.go	add tests for conversion functions

Comments suppressed due to low confidence (4)

std/conversion/hints.go:32

• Iterating over an int (nbBytes) will not compile. Replace with a loop over the outputs slice, e.g., for i := 0; i < len(outputs); i++ or for i := range outputs.

	for i := range nbBytes {

std/conversion/conversion.go:151

• Iterating over an int (int(effNbBits)/8) is invalid. Use an index-based loop, e.g., for j := 0; j < int(effNbBits)/8; j++.

		for j := range int(effNbBits) / 8 {

std/conversion/conversion.go:198

• Cannot range over nbBytes (an int). Change to iterate over the slice or use a classic for-loop: for i := 0; i < nbBytes; i++.

	for i := range nbBytes {

std/conversion/conversion.go:263

• Ranging over nbLimbBytes (an int) is invalid. Replace with for j := 0; j < nbLimbBytes; j++.

		for j := range nbLimbBytes {

[gbotrel]

lgtm

[ivokub]

Bug: Byte Array Sizing and Indexing Errors

The EmulatedToBytes function has two potential issues related to byte array sizing and indexing. Firstly, a slice bounds panic can occur at line 277: if nbLimbBytes (emulated limb byte length) exceeds nbBytes (native field byte length), uint(nbBytes) - nbLimbBytes underflows, leading to an out-of-bounds slice access on res. Secondly, the resU8 array's allocated size (based on the emulated field modulus bit length) may not match the total bytes written by the loop (len(vr.Limbs) * nbLimbBytes). This mismatch could result in an incorrectly sized output array, potentially causing out-of-bounds writes or leaving parts uninitialized. Both issues are unlikely with typical field parameters but possible with unusual emulated field configurations.

std/conversion/conversion.go#L261-L280

gnark/std/conversion/conversion.go

Lines 261 to 280 in 47cd263

	nbBytes := (api.Compiler().Field().BitLen() + 7) / 8
	nbLimbBytes := fr.BitsPerLimb() / 8
	resU8 := make([]uints.U8, (fr.Modulus().BitLen()+7)/8)
	uapi, err := uints.NewBytes(api)
	if err != nil {
	return nil, fmt.Errorf("new uints: %w", err)
	}
	for i := range vr.Limbs {
	res, err := api.NewHint(nativeToBytesHint, nbBytes, vr.Limbs[len(vr.Limbs)-i-1])
	if err != nil {
	return nil, fmt.Errorf("new hint: %w", err)
	}
	if len(res) != nbBytes {
	return nil, fmt.Errorf("expected %d bytes, got %d", nbBytes, len(res))
	}
	res = res[uint(nbBytes)-nbLimbBytes:] // take only the last nbLimbBytes bytes
	for j := range nbLimbBytes {
	resU8[uint(i)*nbLimbBytes+j] = uapi.ValueOf(res[j])
	}

Fix in Cursor • Fix in Web

Was this report helpful? Give feedback by reacting with 👍 or 👎

Good catch. However we use GetEffectiveFieldParameters which takes into account the native field size and should return nbLimbWidth smaller than native field.

The base branch was changed.