SCS fails to install on Julia 1.6

[Edenharder]

Error information:

julia> using SCS
[ Info: Precompiling SCS [c946c3f1-0d1f-5ce8-9dea-7daa1f7e2d13]
ERROR: LoadError: InitError: could not load library "/home/x/.julia/artifacts/2c8f4935f4def75a6baccd200074a08352795a51/lib/libscsdir.so"
libopenblas64_.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] dlopen(s::String, flags::UInt32)
   @ Base.Libc.Libdl ./libdl.jl:114
 [2] macro expansion
   @ ~/.julia/packages/JLLWrappers/WnWcZ/src/products/library_generators.jl:63 [inlined]
 [3] __init__()
   @ SCS_jll ~/.julia/packages/SCS_jll/aatM2/src/wrappers/x86_64-linux-gnu.jl:10
 [4] top-level scope
   @ ~/.julia/packages/SCS/xTUCf/src/SCS.jl:21
 [5] top-level scope (repeats 2 times)
   @ none:1
during initialization of module SCS_jll
in expression starting at /home/x/.julia/packages/SCS/xTUCf/src/SCS.jl:1
ERROR: Failed to precompile SCS [c946c3f1-0d1f-5ce8-9dea-7daa1f7e2d13] to /home/x/.julia/compiled/v1.6/SCS/jl_GTs14c.
Stacktrace:
 [1] compilecache(pkg::Base.PkgId, path::String, internal_stderr::Base.TTY, internal_stdout::Base.TTY)
   @ Base ./loading.jl:1360

[odow]

What is your versioninfo()? (And what build of linux are you running?)

What happens if you update and re-build? import Pkg; Pkg.update(); Pkg.build("SCS")

[Edenharder]

@odow Thank you for the prompt reply. I'm using Manjaro.

versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb* (2021-03-24 12:55 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.1.0 (ORCJIT, broadwell)

Nothing happens when I run

import Pkg; Pkg.update(); Pkg.build("SCS")

But the error is still there when I run using SCS.

[odow]

What is:

] add OpenBLAS_jll
@show OpenBLAS_jll.artifact_dir
@show filter(x->contains(x,"openblas"), readdir(joinpath(OpenBLAS_jll.artifact_dir, "lib", "julia"))

[Edenharder]

@odow
The outputs are

julia> @show OpenBLAS_jll.artifact_dir
OpenBLAS_jll.artifact_dir = "/usr"
"/usr"

julia> @show filter(x->contains(x,"openblas"), readdir(joinpath(OpenBLAS_jll.artifact_dir, "lib", "julia")))
filter((x->begin
            #= REPL[7]:1 =#
            contains(x, "openblas")
        end), readdir(joinpath(OpenBLAS_jll.artifact_dir, "lib", "julia"))) = ["libopenblas64_.so"]
1-element Vector{String}:
 "libopenblas64_.so"

Then I tried to rebuild SCS again, still the same error.

[kalmarek]

@Edenharder is this a julia installed by pacman, or downloaded from official page?

[Edenharder]

@kalmarek It's installed by pacman. I have used Julia 1.5 which is also installed by pacman. It gets well with SCS.

[kalmarek]

@Edenharder I remembered times when those _ were exactly caused by pacman distribution patching up julia to use system blas. You may try linking the non-underscore version to underscore in your /usr, but julia actually may use recipe different from pacman to compile it. Expect some segfaults ;)

I'm actually very surprised it did work on 1.5. Can you check what was the blas julia-1.5 used?
EDIT: I checked and pacmans libopenblas doesn't come with underscored files.

oh, there is a chance libblastrampoline is causing this? JuliaLang/julia#39455
EDIT: No, it is not, that's only included in master, not 1.6

[Edenharder]

You may try linking the non-underscore version to underscore in your /usr

Could you explain more about this step?

Is it possible to solve this problem within the package SCS? Or is it really caused by Pacman and nothing can be done about that here?

[kalmarek]

@Edenharder you may try linking /usr/lib/libopenblas64_.so → /usr/lib/libopenblas.so and hope for the best.

In general I'd advise against using julia shipped by pacman (this takes you to the "unsupported setup" category:), and use the one from julialang directly. If you find managing versions/links/etc tedious (as I did) there're a few scripts to automate this. I found jill useful.

[Edenharder]

@kalmarek Thank you for the detailed solution. Finally, I installed julia with jill. Everything works well now.

[odow]

@kalmarek I think the trampoline will be part of 1.7, precisely to overcome this issue.

@Edenharder there is a large backstory to this. Julia uses Yggdrasil/BinaryBuilder to provide binary JLL packages. That comes at a cost of us occasionally shipping an extra binary that you may already have on your system, but it comes at the benefit of always working. This has relieved a significant paint point for us. The exception is when distro's patch Julia to use the native libraries... and we get back to the broken state of things. See also: https://github.com/jump-dev/SCS.jl#custom-installation.

[kalmarek]

Indeed, trampoline is in master, not in 1.6;

I'm not really sure how accurate custom-installation is nowadays, I haven't done it in a while...
EDIT: in particular JULIA_BLAS_PATH should be set to something based on OpenBLAS_jll.artifact_dir for modern julia installations (I actually didn't know about this path)

EDIT2: just checkout scs master and the instructions work with JULIA_BLAS_PATH=joinpath(OpenBLAS_jll.artifact_dir, "lib", "julia")
that's always a pleasant surprise to see something working after such long time ;)