Switch default storage to MemoryRef-based array by jakobnissen · Pull Request #189 · JuliaArrays/FixedSizeArrays.jl

jakobnissen · 2026-04-02T17:58:44Z

Instead of backing default FixedSizeArray with Memory, create a new array type implemented as an immutable struct implementing MemoryRef, and no size. This allow the pointer to inline into the FixedSizeArray structure, instead of being behind a pointer dereference to the Memory object. By avoiding size information in the backing type, the size is not wastefully stored doubly in FSA.

Closes #187

For reviewers:
The upside is faster access to array elements by saving a pointer indirection.
The downsides are:

Struct is now 1 pointer larger (8 bytes on 64-bit)
More likely that some optimization made for Memory is not made for this new type. E.g. copyto! may be more optimized for Memory

The missing codecov appears to be entirely false positives, so I'm not going to address it.

Instead of backing default FixedSizeArray with Memory, create a new array type implemented as an immutable struct implementing MemoryRef, and no size. This allow the pointer to inline into the FixedSizeArray structure, instead of being behind a pointer dereference to the Memory object. By avoiding size information in the backing type, the size is not wastefully stored doubly in FSA.

codecov · 2026-04-02T18:01:12Z

Codecov Report

❌ Patch coverage is 95.23810% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.25%. Comparing base (147851b) to head (2a62db8).

Files with missing lines	Patch %	Lines
src/refarray.jl	94.44%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #189      +/-   ##
==========================================
- Coverage   92.78%   92.25%   -0.54%     
==========================================
  Files           5        6       +1     
  Lines         208      271      +63     
==========================================
+ Hits          193      250      +57     
- Misses         15       21       +6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

oscardssmith · 2026-04-02T18:49:11Z

+Base.firstindex(::UnsafeRefArray) = 1
+
+function Base.getindex(x::UnsafeRefArray, i::Int)
+    @inbounds(memoryref(x.ref, i)[])


which part are you trying to @inbounds?

Both making the ref, and dereferencing it, both of which checks bounds.

giordano · 2026-04-02T22:56:42Z

I haven't looked closely at the code, but doing some quick tests I believe this would fix #142:

julia> code_llvm(Base.mapreduce_impl, (typeof(identity), typeof(Base.add_sum), FixedSizeVectorDefault{Float64}, Int, Int, Int); )

LLVM IR for FixedSizeVectorDefault with this PR

; Function Signature: mapreduce_impl(typeof(Base.identity), typeof(Base.add_sum), FixedSizeArrays.FixedSizeArray{Float64, 1, FixedSizeArrays.UnsafeRefArray{Float64}}, Int64, Int64, Int64)
;  @ reduce.jl:251 within `mapreduce_impl`
; Function Attrs: noinline
define double @julia_mapreduce_impl_0(ptr nocapture noundef nonnull readonly align 8 dereferenceable(24) %"A::FixedSizeArray", ptr nocapture noundef nonnull readonly align 8 dereferenceable(8) %.roots.A, i64 signext %"ifirst::Int64", i64 signext %"ilast::Int64", i64 signext %"blksize::Int64") local_unnamed_addr #0 {
top:
;  @ reduce.jl:253 within `mapreduce_impl`
; ┌ @ promotion.jl:637 within `==`
   %.not = icmp eq i64 %"ifirst::Int64", %"ilast::Int64"
; └
  br i1 %.not, label %L26, label %L34

L26:                                              ; preds = %top
;  @ reduce.jl:254 within `mapreduce_impl`
; ┌ @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/FixedSizeArray.jl:167 within `getindex` @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/refarray.jl:40
; │┌ @ boot.jl:662 within `memoryref`
    %memoryref_data = load ptr, ptr %"A::FixedSizeArray", align 8
    %memoryref_offset = shl i64 %"ifirst::Int64", 3
; │└
; │ @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/FixedSizeArray.jl:167 within `getindex` @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/refarray.jl:40 @ essentials.jl:402
   %0 = getelementptr i8, ptr %memoryref_data, i64 %memoryref_offset
   %memoryref_data2 = getelementptr i8, ptr %0, i64 -8
   %1 = load double, ptr %memoryref_data2, align 8
; └
;  @ reduce.jl:255 within `mapreduce_impl`
  br label %common.ret

common.ret:                                       ; preds = %L174, %L119, %middle.block, %L60, %L26
  %common.ret.op = phi double [ %1, %L26 ], [ %34, %L174 ], [ %6, %L60 ], [ %24, %middle.block ], [ %28, %L119 ]
;  @ reduce.jl within `mapreduce_impl`
  ret double %common.ret.op

L34:                                              ; preds = %top
;  @ reduce.jl:256 within `mapreduce_impl`
; ┌ @ essentials.jl:1216 within `-`
   %2 = sub i64 %"ilast::Int64", %"ifirst::Int64"
; └
; ┌ @ int.jl:83 within `<`
   %.not55 = icmp slt i64 %2, %"blksize::Int64"
; └
  br i1 %.not55, label %L60, label %L174

L60:                                              ; preds = %L34
;  @ reduce.jl:258 within `mapreduce_impl`
; ┌ @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/FixedSizeArray.jl:167 within `getindex` @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/refarray.jl:40
; │┌ @ boot.jl:662 within `memoryref`
    %memoryref_data9 = load ptr, ptr %"A::FixedSizeArray", align 8
    %memoryref_offset11 = shl i64 %"ifirst::Int64", 3
; │└
; │ @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/FixedSizeArray.jl:167 within `getindex` @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/refarray.jl:40 @ essentials.jl:402
   %3 = getelementptr i8, ptr %memoryref_data9, i64 %memoryref_offset11
   %memoryref_data17 = getelementptr i8, ptr %3, i64 -8
   %4 = load double, ptr %memoryref_data17, align 8
; └
;  @ reduce.jl:259 within `mapreduce_impl`
; ┌ @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/FixedSizeArray.jl:167 within `getindex` @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/refarray.jl:40 @ essentials.jl:402
   %5 = load double, ptr %3, align 8
; └
;  @ reduce.jl:260 within `mapreduce_impl`
; ┌ @ reduce.jl:19 within `add_sum`
; │┌ @ float.jl:492 within `+`
    %6 = fadd double %4, %5
; └└
;  @ reduce.jl:261 within `mapreduce_impl`
; ┌ @ simdloop.jl:69 within `macro expansion`
; │┌ @ essentials.jl:1215 within `+`
    %7 = add i64 %"ifirst::Int64", 2
; │└
; │┌ @ range.jl:5 within `Colon`
; ││┌ @ range.jl:417 within `UnitRange`
; │││┌ @ range.jl:428 within `unitrange_last`
; ││││┌ @ operators.jl:479 within `>=`
; │││││┌ @ int.jl:560 within `<=`
        %.not56 = icmp sgt i64 %7, %"ilast::Int64"
; ││││└└
      %8 = add i64 %"ifirst::Int64", 1
      %value_phi = select i1 %.not56, i64 %8, i64 %"ilast::Int64"
; │└└└
; │ @ simdloop.jl:71 within `macro expansion`
; │┌ @ simdloop.jl:51 within `simd_inner_length`
; ││┌ @ range.jl:785 within `length`
; │││┌ @ essentials.jl:1216 within `-`
      %9 = sub i64 %value_phi, %7
; │└└└
; │ @ simdloop.jl:72 within `macro expansion`
; │┌ @ int.jl:83 within `<`
    %10 = icmp ugt i64 %9, 9223372036854775806
; │└
   br i1 %10, label %common.ret, label %L119.preheader

L119.preheader:                                   ; preds = %L60
; │ @ simdloop.jl:75 within `macro expansion`
   %invariant.gep = getelementptr i8, ptr %memoryref_data9, i64 -8
   %11 = xor i64 %"ifirst::Int64", -1
   %12 = add i64 %value_phi, %11
   %min.iters.check = icmp ult i64 %12, 8
   br i1 %min.iters.check, label %L119, label %vector.ph

vector.ph:                                        ; preds = %L119.preheader
   %n.vec = and i64 %12, -8
   %13 = insertelement <2 x double> <double poison, double -0.000000e+00>, double %6, i64 0
   br label %vector.body

vector.body:                                      ; preds = %vector.body, %vector.ph
; │ @ simdloop.jl:76 within `macro expansion`
; │┌ @ simdloop.jl:54 within `simd_index`
; ││┌ @ essentials.jl:1215 within `+`
     %index = phi i64 [ 0, %vector.ph ], [ %index.next, %vector.body ]
     %vec.phi = phi <2 x double> [ %13, %vector.ph ], [ %19, %vector.body ]
     %vec.phi61 = phi <2 x double> [ splat (double -0.000000e+00), %vector.ph ], [ %20, %vector.body ]
     %vec.phi62 = phi <2 x double> [ splat (double -0.000000e+00), %vector.ph ], [ %21, %vector.body ]
     %vec.phi63 = phi <2 x double> [ splat (double -0.000000e+00), %vector.ph ], [ %22, %vector.body ]
     %.reass = add i64 %index, %7
; │└└
; │ @ simdloop.jl:77 within `macro expansion` @ reduce.jl:262
; │┌ @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/FixedSizeArray.jl:167 within `getindex` @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/refarray.jl:40
; ││┌ @ boot.jl:662 within `memoryref`
     %14 = shl i64 %.reass, 3
; ││└
; ││ @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/FixedSizeArray.jl:167 within `getindex` @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/refarray.jl:40 @ essentials.jl:402
    %15 = getelementptr i8, ptr %invariant.gep, i64 %14
    %16 = getelementptr i8, ptr %15, i64 16
    %17 = getelementptr i8, ptr %15, i64 32
    %18 = getelementptr i8, ptr %15, i64 48
    %wide.load = load <2 x double>, ptr %15, align 8
    %wide.load64 = load <2 x double>, ptr %16, align 8
    %wide.load65 = load <2 x double>, ptr %17, align 8
    %wide.load66 = load <2 x double>, ptr %18, align 8
; │└
; │ @ simdloop.jl:77 within `macro expansion` @ reduce.jl:263
; │┌ @ reduce.jl:19 within `add_sum`
; ││┌ @ float.jl:492 within `+`
     %19 = fadd reassoc contract <2 x double> %vec.phi, %wide.load
     %20 = fadd reassoc contract <2 x double> %vec.phi61, %wide.load64
     %21 = fadd reassoc contract <2 x double> %vec.phi62, %wide.load65
     %22 = fadd reassoc contract <2 x double> %vec.phi63, %wide.load66
; │└└
; │ @ simdloop.jl:76 within `macro expansion`
; │┌ @ simdloop.jl:54 within `simd_index`
; ││┌ @ essentials.jl:1215 within `+`
     %index.next = add nuw i64 %index, 8
     %23 = icmp eq i64 %index.next, %n.vec
     br i1 %23, label %middle.block, label %vector.body

middle.block:                                     ; preds = %vector.body
; │└└
; │ @ simdloop.jl:75 within `macro expansion`
   %bin.rdx = fadd reassoc contract <2 x double> %20, %19
   %bin.rdx67 = fadd reassoc contract <2 x double> %21, %bin.rdx
   %bin.rdx68 = fadd reassoc contract <2 x double> %22, %bin.rdx67
   %24 = call reassoc contract double @llvm.vector.reduce.fadd.v2f64(double -0.000000e+00, <2 x double> %bin.rdx68)
   %cmp.n = icmp eq i64 %12, %n.vec
   br i1 %cmp.n, label %common.ret, label %L119

L119:                                             ; preds = %L119, %middle.block, %L119.preheader
   %value_phi3660 = phi i64 [ %25, %L119 ], [ 0, %L119.preheader ], [ %n.vec, %middle.block ]
   %value_phi3559 = phi double [ %28, %L119 ], [ %6, %L119.preheader ], [ %24, %middle.block ]
; │ @ simdloop.jl:76 within `macro expansion`
; │┌ @ simdloop.jl:54 within `simd_index`
; ││┌ @ essentials.jl:1215 within `+`
     %25 = add nuw nsw i64 %value_phi3660, 1
; ││└
; ││┌ @ array.jl:3220 within `getindex`
; │││┌ @ range.jl:955 within `_getindex`
; ││││┌ @ essentials.jl:1215 within `+`
       %26 = add i64 %value_phi3660, %7
; │└└└└
; │ @ simdloop.jl:77 within `macro expansion` @ reduce.jl:262
; │┌ @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/FixedSizeArray.jl:167 within `getindex` @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/refarray.jl:40
; ││┌ @ boot.jl:662 within `memoryref`
     %memoryref_offset46 = shl i64 %26, 3
; ││└
; ││ @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/FixedSizeArray.jl:167 within `getindex` @ /Users/mose/.julia/packages/FixedSizeArrays/gtodJ/src/refarray.jl:40 @ essentials.jl:402
    %gep = getelementptr i8, ptr %invariant.gep, i64 %memoryref_offset46
    %27 = load double, ptr %gep, align 8
; │└
; │ @ simdloop.jl:77 within `macro expansion` @ reduce.jl:263
; │┌ @ reduce.jl:19 within `add_sum`
; ││┌ @ float.jl:492 within `+`
     %28 = fadd reassoc contract double %value_phi3559, %27
; │└└
; │ @ simdloop.jl:75 within `macro expansion`
; │┌ @ int.jl:83 within `<`
    %exitcond.not = icmp eq i64 %value_phi3660, %9
; │└
   br i1 %exitcond.not, label %common.ret, label %L119

L174:                                             ; preds = %L34
; └
;  @ reduce.jl:268 within `mapreduce_impl`
; ┌ @ int.jl:580 within `>>` @ int.jl:573
   %29 = ashr i64 %2, 1
; └
; ┌ @ essentials.jl:1215 within `+`
   %30 = add i64 %29, %"ifirst::Int64"
; └
;  @ reduce.jl:269 within `mapreduce_impl`
  %31 = call double @julia_mapreduce_impl_0(ptr nocapture readonly %"A::FixedSizeArray", ptr nocapture readonly %.roots.A, i64 signext %"ifirst::Int64", i64 signext %30, i64 signext %"blksize::Int64")
;  @ reduce.jl:270 within `mapreduce_impl`
; ┌ @ essentials.jl:1215 within `+`
   %32 = add i64 %30, 1
; └
  %33 = call double @julia_mapreduce_impl_0(ptr nocapture readonly %"A::FixedSizeArray", ptr nocapture readonly %.roots.A, i64 signext %32, i64 signext %"ilast::Int64", i64 signext %"blksize::Int64")
;  @ reduce.jl:271 within `mapreduce_impl`
; ┌ @ reduce.jl:19 within `add_sum`
; │┌ @ float.jl:492 within `+`
    %34 = fadd double %31, %33
; └└
  br label %common.ret
}

This IR looks a lot similar to the one for Vector than FixedSizeVectorDefault in #142 (comment) (the top block is a lot simpler).

Also, most the examples in #62 seem to maintain their good features.

jakobnissen · 2026-04-03T06:22:38Z

FWIW, it origially failed the allocation tests, until I discovered that the call to rand! was what defeated escape analysis - adding the fast path in RandomExt made it work. This suggests that the allocation elision may be overly sensitive to which dispatch paths the array takes, and does not generalize to all functions having the same Effects

giordano · 2026-04-03T13:56:40Z

+struct UnsafeRefArray{T} <: DenseVector{T}
+    ref::MemoryRef{T}
+end


Why we can't use MemoryRef directly?

Because MemoryRef <: Ref, whereas FixedSizeArray must be backed by a DenseVector:

struct FixedSizeArray{T,N,Mem<:DenseVector{T}} <: DenseArray{T,N} mem::Mem size::NTuple{N,Int} end

nsajko · 2026-04-03T18:31:09Z

I have not yet taken a look at the content of this PR. However, I suspect a nicer approach would be to allow using the types from NonResizableVectors.jl as FSA backing storage, instead of adding new internal types to FixedSizeArrays.jl.

This would be:

More comprehensive, as NonResizableVectors.jl implements all the configurations that one could possibly want, I think.
Better for code reuse across the ecosystem, as hypothetically some package might depend on NonResizableVectors.jl without depending on FixedSizeArrays.jl.

We might use a package extension and weak dependency on NonResizableVectors, or we could even make it a strong dep, and change the default backing storage.

Slight tweaks are still necessary at NonResizableVectors to make this possible. In particular, NonResizableVectors do not currently subtype DenseVector.

I invited all of you to the NonResizableVectors repo.

jakobnissen · 2026-04-03T20:16:14Z

Yes, I agree that would be better than this PR. I would make it a strong dependency.

There is some overlap between NonResizableVectors and MemoryViews. A very large overlap, in fact. I would be interested in discussing - separate from this PR - how we could align those two packages. NonResizableVectors seem more fundamental, so perhaps MemoryViews could be built on top of that.

giordano · 2026-04-04T20:20:03Z

One regression I noticed is from #62 (comment): g3 (reshaping a 3-element vector into a 2x2 matrix) isn't consistent with this PR, but it is on main.

jakobnissen · 2026-04-07T13:28:47Z

OK - talked to @nsajko . Here is the direction I want to go, let me know if there are any objections:

Build out NonResizableVectors. In the beginning I just care about MemoryRefVectorImm, so I'm going to implement that and export it. Other types can wait. Implementation will be based on MemoryViews - where possible, this type will construct a MemoryView, then forward its methods to the view. This is to avoid a combinatorial type explosion and to repeat tonnes of identical implementations.
Then rebase this PR on top of NonResizableVectors.

I don't have any particular time line, so it may take a while.

Fixup CI

d2d103e

oscardssmith reviewed Apr 2, 2026

View reviewed changes

Comment thread src/refarray.jl Outdated

oscardssmith reviewed Apr 2, 2026

View reviewed changes

Reviewer comments

2a62db8

giordano reviewed Apr 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch default storage to MemoryRef-based array#189

Switch default storage to MemoryRef-based array#189
jakobnissen wants to merge 3 commits intoJuliaArrays:mainfrom
jakobnissen:memref

jakobnissen commented Apr 2, 2026 •

edited

Loading

Uh oh!

codecov bot commented Apr 2, 2026 •

edited

Loading

Uh oh!

Uh oh!

oscardssmith Apr 2, 2026

Uh oh!

jakobnissen Apr 2, 2026

Uh oh!

giordano commented Apr 2, 2026

Uh oh!

jakobnissen commented Apr 3, 2026

Uh oh!

giordano Apr 3, 2026

Uh oh!

jakobnissen Apr 3, 2026

Uh oh!

nsajko commented Apr 3, 2026

Uh oh!

jakobnissen commented Apr 3, 2026

Uh oh!

giordano commented Apr 4, 2026

Uh oh!

jakobnissen commented Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jakobnissen commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

oscardssmith Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

jakobnissen Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

giordano commented Apr 2, 2026

Uh oh!

jakobnissen commented Apr 3, 2026

Uh oh!

giordano Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

jakobnissen Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

nsajko commented Apr 3, 2026

Uh oh!

jakobnissen commented Apr 3, 2026

Uh oh!

giordano commented Apr 4, 2026

Uh oh!

jakobnissen commented Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jakobnissen commented Apr 2, 2026 •

edited

Loading

codecov bot commented Apr 2, 2026 •

edited

Loading