Test failures with CategoricalArrays 1.0

[palday]

I havne't been able to replicate it in CI, but I have replicated it on both a M1 Mac and and an x86-64 Ubuntu machine locally:

BoundsError: attempt to access 3-element CategoricalArrays.CategoricalRefPool{Union{Missing, CategoricalValue{String, UInt32}}, CategoricalPool{String, UInt32}} with indices 0:2 at index [3]

cc @nalimilan

Details

```julia # some dict encoding coverage: Error During Test at /home/ubuntu/arrow-julia/test/runtests.jl:339 Got exception outside of a @test BoundsError: attempt to access 3-element CategoricalArrays.CategoricalRefPool{Union{Missing, CategoricalValue{String, UInt32}}, CategoricalPool{String, UInt32}} with indices 0:2 at index [3] Stacktrace: [1] throw_boundserror(A::CategoricalArrays.CategoricalRefPool{Union{Missing, CategoricalValue{String, UInt32}}, CategoricalPool{String, UInt32}}, I::Tuple{Int64}) @ Base ./abstractarray.jl:737 [2] checkbounds @ ./abstractarray.jl:702 [inlined] [3] getindex @ ~/.julia/packages/CategoricalArrays/ptzwC/src/array.jl:1161 [inlined] [4] getindex @ ~/.julia/packages/ArrowTypes/IVpuP/src/ArrowTypes.jl:445 [inlined] [5] iterate @ ./abstractarray.jl:1216 [inlined] [6] Arrow.ValidityBitmap(x::ToArrow{Union{Missing, String}, CategoricalArrays.CategoricalRefPool{Union{Missing, CategoricalValue{String, UInt32}}, CategoricalPool{String, UInt32}}}) @ Arrow ~/arrow-julia/src/arraytypes/arraytypes.jl:228 [7] arrowvector(::ListKind{true}, x::ToArrow{Union{Missing, String}, CategoricalArrays.CategoricalRefPool{Union{Missing, CategoricalValue{String, UInt32}}, CategoricalPool{String, UInt32}}}, i::Int64, nl::Int64, fi::Int64, de::Dict{Int64, Any}, ded::Vector{Arrow.DictEncoding}, meta::Base.ImmutableDict{String, String}; largelists::Bool, kw::@kwargs{dictencode::Bool, maxdepth::Int64, dictencodenested::Bool, compression::Nothing}) @ Arrow ~/arrow-julia/src/arraytypes/list.jl:226 [8] arrowvector @ ~/arrow-julia/src/arraytypes/list.jl:224 [inlined] [9] #arrowvector#10 @ ~/arrow-julia/src/arraytypes/arraytypes.jl:150 [inlined] [10] arrowvector @ ~/arrow-julia/src/arraytypes/arraytypes.jl:148 [inlined] [11] arrowvector(x::CategoricalArrays.CategoricalRefPool{Union{Missing, CategoricalValue{String, UInt32}}, CategoricalPool{String, UInt32}}, i::Int64, nl::Int64, fi::Int64, de::Dict{Int64, Any}, ded::Vector{Arrow.DictEncoding}, meta::Nothing; dictencoding::Bool, dictencode::Bool, maxdepth::Int64, kw::@kwargs{dictencodenested::Bool, compression::Nothing}) @ Arrow ~/arrow-julia/src/arraytypes/arraytypes.jl:113 [12] arrowvector(::DictEncodedKind, x::Arrow.DictEncode{Union{Missing, CategoricalValue{String, UInt32}}, CategoricalVector{Union{Missing, String}, UInt32, String, CategoricalValue{String, UInt32}, Missing}}, i::Int64, nl::Int64, fi::Int64, de::Dict{Int64, Any}, ded::Vector{Arrow.DictEncoding}, meta::Base.ImmutableDict{String, String}; dictencode::Bool, dictencodenested::Bool, kw::@kwargs{maxdepth::Int64, compression::Nothing}) @ Arrow ~/arrow-julia/src/arraytypes/dictencoding.jl:236 [13] arrowvector @ ~/arrow-julia/src/arraytypes/dictencoding.jl:202 [inlined] [14] #arrowvector#10 @ ~/arrow-julia/src/arraytypes/arraytypes.jl:150 [inlined] [15] arrowvector @ ~/arrow-julia/src/arraytypes/arraytypes.jl:148 [inlined] [16] arrowvector(x::CategoricalVector{Union{Missing, String}, UInt32, String, CategoricalValue{String, UInt32}, Missing}, i::Int64, nl::Int64, fi::Int64, de::Dict{Int64, Any}, ded::Vector{Arrow.DictEncoding}, meta::Nothing; dictencoding::Bool, dictencode::Bool, maxdepth::Int64, kw::@kwargs{compression::Nothing}) @ Arrow ~/arrow-julia/src/arraytypes/arraytypes.jl:113 [17] arrowvector @ ~/arrow-julia/src/arraytypes/arraytypes.jl:68 [inlined] [18] toarrowvector(x::CategoricalVector{Union{Missing, String}, UInt32, String, CategoricalValue{String, UInt32}, Missing}, i::Int64, de::Dict{Int64, Any}, ded::Vector{Arrow.DictEncoding}, meta::Nothing; compression::Nothing, kw::@kwargs{}) @ Arrow ~/arrow-julia/src/arraytypes/arraytypes.jl:47 [19] toarrowvector @ ~/arrow-julia/src/arraytypes/arraytypes.jl:36 [inlined] [20] toarrowvector(x::CategoricalVector{Union{Missing, String}, UInt32, String, CategoricalValue{String, UInt32}, Missing}) @ Arrow ~/arrow-julia/src/arraytypes/arraytypes.jl:36 [21] macro expansion @ ~/arrow-julia/test/runtests.jl:347 [inlined] [22] macro expansion @ ~/.julia/juliaup/julia-1.10.10+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined] [23] macro expansion @ ~/arrow-julia/test/runtests.jl:341 [inlined] [24] macro expansion @ ~/.julia/juliaup/julia-1.10.10+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined] [25] macro expansion @ ~/arrow-julia/test/runtests.jl:115 [inlined] [26] macro expansion @ ~/.julia/juliaup/julia-1.10.10+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined] [27] top-level scope @ ~/arrow-julia/test/runtests.jl:54 [28] include(fname::String) @ Base.MainInclude ./client.jl:494 [29] top-level scope @ none:6 [30] eval @ ./boot.jl:385 [inlined] [31] exec_options(opts::Base.JLOptions) @ Base ./client.jl:296 [32] _start() @ Base ./client.jl:557 ```

[nalimilan]

That's really weird. I was able to reproduce it locally, but now I can't. It looks like firstindex returned 1 when it should have returned 0.

[nalimilan]

Are you able to reproduce it systematically? From the stacktrace, it appears you're using released ArrowTypes but Arrow from ~/arrow-julia. Could there be a discrepancy between the two? My setup was also a mess when I reproduced the bug. Now I can't find a way to see it again.