IpoptMKL: new recipe

[dpo]

Is this of interest? There have been a few requests for Ipopt/MKL recently over at Ipopt.jl.

[dpo]

Not understanding why irrelevant platforms are being built here...

[giordano]

What do you mean by "irrelevant"?

[dpo]

Oh, I apologize. I forgot that Windows was in there...

[dpo]

I'm quite confused that when I try to build for Windows locally, I get

$ BINARYBUILDER_RUNNER=docker julia --project=../../yggdrasil --color=yes build_tarballs.jl --verbose --debug x86_64-w64-mingw32-libgfortran5-cxx11

ERROR: LoadError: IOError: open: no such file or directory (ENOENT)
Stacktrace:
 [1] open at ./filesystem.jl:79 [inlined]
 [2] sendfile(::String, ::String) at ./file.jl:909
 [3] cp(::String, ::String; force::Bool, follow_symlinks::Bool) at ./file.jl:351
 [4] cp at ./file.jl:345 [inlined]
 [5] (::BinaryBuilder.var"#57#61"{Prefix,Windows,Array{String,1}})() at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/Prefix.jl:447
 [6] activate(::BinaryBuilder.var"#57#61"{Prefix,Windows,Array{String,1}}, ::String) at /builddir/build/BUILD/julia/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:915
 [7] setup_dependencies(::Prefix, ::Array{Pkg.Types.PackageSpec,1}, ::Windows) at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/Prefix.jl:405
 [8] autobuild(::AbstractString, ::AbstractString, ::VersionNumber, ::Array{#s1079,1} where #s1079<:BinaryBuilder.AbstractSource, ::AbstractString, ::Array{T,1} where T, ::Array{#s1078,1} where #s1078<:Product, ::Array{#s1077,1} where #s1077<:BinaryBuilder.AbstractDependency; verbose::Bool, debug::Bool, skip_audit::Bool, ignore_audit_errors::Bool, autofix::Bool, code_dir::Union{Nothing, String}, require_license::Bool, lazy_artifacts::Bool, meta_json_stream::Any, kwargs::Any) at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/AutoBuild.jl:597
 [9] build_tarballs(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any; kwargs::Any) at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/AutoBuild.jl:183
 [10] build_tarballs(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any) at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/AutoBuild.jl:84
 [11] top-level scope at /home/mgi.polymtl.ca/doorba/dev/julia/Yggdrasil/I/IpoptMKL/build_tarballs.jl:62
in expression starting at /home/mgi.polymtl.ca/doorba/dev/julia/Yggdrasil/I/IpoptMKL/build_tarballs.jl:62

Any idea what this error means?

[giordano]

Any idea what this error means?

Not on top of my head, I'd go up in the stacktrace, to understand what's going on.

[dpo]

Modifying src/Prefix.jl on my Mac as follows:

diff --git a/src/Prefix.jl b/src/Prefix.jl
index c282ced..0264235 100644
--- a/src/Prefix.jl
+++ b/src/Prefix.jl
@@ -454,6 +454,8 @@ function setup_dependencies(prefix::Prefix, dependencies::Vector{PkgSpec}, platf
             # Copy the artifact from the global installation location into this build-specific artifacts collection
             src_path = Pkg.Artifacts.artifact_path(artifact_hash)
             dest_path = joinpath(prefix, "artifacts", basename(src_path))
+            @show name src_path dest_path
+            @show readdir(src_path) readdir(dest_path)
             cp(src_path, dest_path)
 
             # Keep track of our dep paths for later symlinking

gives me:

name = "MKL_jll"
src_path = "/Users/dpo/.julia/artifacts/b096ea2066a0eb73885a511bef6608de3845feae"
dest_path = "/Users/dpo/dev/julia/Yggdrasil/I/IpoptMKL/build/x86_64-w64-mingw32/bkDYE2SI/artifacts/b096ea2066a0eb73885a511bef6608de3845feae"
ERROR: LoadError: SystemError: unable to read directory /Users/dpo/.julia/artifacts/b096ea2066a0eb73885a511bef6608de3845feae: No such file or directory

On another machine, I get a similar error with CompilerSupportLibraries_jll.

[giordano]

I think this PR needs JuliaPackaging/BinaryBuilder.jl#786

[staticfloat]

Heh, yeah, I just spent like an hour trying to figure out why the new arctic3 installation wasn't installing artifacts properly 😅

[dpo]

Thanks, the fix in BinaryBuilder appears to be working.

Unfortunately, I've no idea how to link with MKL on MinGW. Does anyone know? configure keeps complaining that it's not finding the symbol dsyev, which is a LAPACK symbol and should be found in the MKL.

[svigerske]

Thanks, the fix in BinaryBuilder appears to be working.

Unfortunately, I've no idea how to link with MKL on MinGW. Does anyone know? configure keeps complaining that it's not finding the symbol dsyev, which is a LAPACK symbol and should be found in the MKL.

Check the config.log for details.

[dpo]

Disabling Windows entirely as apparently, the MKL is incompatible with gcc compilers on Windows.

[giordano]

Good to go?

[dpo]

I think so.

[vchuravy]

Should this be loadable? JuliaRegistries/General#15232

[dpo]

I just tried on my Mac and it works. Not sure why CI is failing.

(@v1.4) pkg> activate ipopt-mkl
 Activating new environment at `/private/tmp/ipopt-mkl/Project.toml`

(ipopt-mkl) pkg> add https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl
    Cloning git-repo `https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl`
   Updating git-repo `https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl`
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
Downloading artifact: IpoptMKL
######################################################################## 100.0%#=#=-#  #             Updating `/private/tmp/ipopt-mkl/Project.toml`
  [6af6c270] + IpoptMKL_jll v3.13.2+0 #master (https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl)
   Updating `/private/tmp/ipopt-mkl/Manifest.toml`
  [ae81ac8f] + ASL_jll v0.1.1+3
  [e66e0078] + CompilerSupportLibraries_jll v0.3.3+0
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+0
  [6af6c270] + IpoptMKL_jll v3.13.2+0 #master (https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl)
  [856f044c] + MKL_jll v2020.1.216+0
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

julia> using IpoptMKL_jll
[ Info: Precompiling IpoptMKL_jll [6af6c270-9bc2-5f6c-a1c9-61e255f839ff]
Downloading artifact: MKL
######################################################################## 100.0%#=#=-#  #

julia> IpoptMKL_jll.libipopt
"@rpath/libipopt.3.dylib"

julia> IpoptMKL_jll.libipopt_handle
Ptr{Nothing} @0x00007fe4c2ec9610

[giordano]

I just tried on my Mac and it works. Not sure why CI is failing.

The registry uses a Linux system

Should this be loadable? JuliaRegistries/General#15232

I fear that's because libmkl_intel_lp64 is not a product, or even worse because we don't run autofix on Linux because of #922

[dpo]

Correction: it was picking up another libmkl on my system. I confirm that it doesn't load on macOS either.

[giordano]

What error do you get? We might just need to add the lib to the product (and autofix is run on Linux, so that's not the problem)

[dpo]

I get

julia> using IpoptMKL_jll
ERROR: InitError: could not load library "/Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib"
dlopen(/Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib, 1): Library not loaded: @rpath/libmkl_intel_lp64.dylib
  Referenced from: /Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib
  Reason: image not found

Shouldn't all MKL libs be products?

shell> otool -L /Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib
/Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib:
	@rpath/libipopt.3.dylib (compatibility version 17.0.0, current version 17.2.0)
	@rpath/libmkl_intel_lp64.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_core.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_intel_thread.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libiomp5.dylib (compatibility version 5.0.0, current version 5.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)

[giordano]

Shouldn't all MKL libs be products?

Yes, they should... However I think that MKL might have different different libraries on different platforms, which makes thing slightly more complicated. Mind opening a PR?

[dpo]

#1075