Building not working on Windows with existing Mosek installation

[dourouc05]

I've just installed a very recent version of MOSEK (9.1.13), but building the package (1.1.1) does not work properly:

(v1.3) pkg> add MosekTools
  Updating registry at `C:\Users\Thibaut\.julia\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed JSExpr ─────────────────────── v0.5.2
 Installed OpenSpecFun_jll ────────────── v0.5.3+3
 Installed PDMats ─────────────────────── v0.9.12
 Installed MosekTools ─────────────────── v0.9.3
 Installed MacroTools ─────────────────── v0.5.5
 Installed MutableArithmetics ─────────── v0.2.7
 Installed Mosek ──────────────────────── v1.1.1
 Installed MbedTLS_jll ────────────────── v2.16.0+1
 Installed OpenBLAS_jll ───────────────── v0.3.7+8
 Installed Inflate ────────────────────── v0.1.2
 Installed Tables ─────────────────────── v1.0.3
 Installed CSV ────────────────────────── v0.5.26
 Installed FileIO ─────────────────────── v1.2.4
 Installed Contour ────────────────────── v0.5.2
 Installed HTTP ───────────────────────── v0.8.13
 Installed Rmath_jll ──────────────────── v0.2.2+0
 Installed MbedTLS ────────────────────── v1.0.1
 Installed CompilerSupportLibraries_jll ─ v0.3.1+0
 Installed Parsers ────────────────────── v0.3.12
 Installed Blosc ──────────────────────── v0.6.0
 Installed JuliaInterpreter ───────────── v0.7.13
 Installed PlotUtils ──────────────────── v0.6.4
 Installed GeometryTypes ──────────────── v0.7.10
 Installed Rmath ──────────────────────── v0.6.1
 Installed Graphics ───────────────────── v1.0.2
 Installed StatsBase ──────────────────── v0.32.2
  Updating `C:\Users\Thibaut\.julia\environments\v1.3\Project.toml`
  [1ec41992] + MosekTools v0.9.3
  Updating `C:\Users\Thibaut\.julia\environments\v1.3\Manifest.toml`
  [a74b3585] ↑ Blosc v0.5.1 ⇒ v0.6.0
  [336ed68f] ↑ CSV v0.5.25 ⇒ v0.5.26
  [e66e0078] ↑ CompilerSupportLibraries_jll v0.2.0+1 ⇒ v0.3.1+0
  [d38c429a] ↑ Contour v0.5.1 ⇒ v0.5.2
  [5789e2e9] ↑ FileIO v1.2.2 ⇒ v1.2.4
  [4d00f742] ↑ GeometryTypes v0.7.9 ⇒ v0.7.10
  [a2bd30eb] ↑ Graphics v1.0.1 ⇒ v1.0.2
  [cd3eb016] ↑ HTTP v0.8.11 ⇒ v0.8.13
  [d25df0c9] ↑ Inflate v0.1.1 ⇒ v0.1.2
  [97c1335a] ↑ JSExpr v0.5.0 ⇒ v0.5.2
  [aa1ae85d] ↑ JuliaInterpreter v0.7.12 ⇒ v0.7.13
  [1914dd2f] ↑ MacroTools v0.5.4 ⇒ v0.5.5
  [739be429] ↑ MbedTLS v0.7.0 ⇒ v1.0.1
  [c8ffd9c3] + MbedTLS_jll v2.16.0+1
  [6405355b] + Mosek v1.1.1
  [1ec41992] + MosekTools v0.9.3
  [d8a4904e] ↑ MutableArithmetics v0.2.6 ⇒ v0.2.7
  [4536629a] ↑ OpenBLAS_jll v0.3.7+5 ⇒ v0.3.7+8
  [efe28fd5] ↑ OpenSpecFun_jll v0.5.3+2 ⇒ v0.5.3+3
  [90014a1f] ↑ PDMats v0.9.11 ⇒ v0.9.12
  [69de0a69] ↑ Parsers v0.3.11 ⇒ v0.3.12
  [995b91a9] ↑ PlotUtils v0.6.3 ⇒ v0.6.4
  [79098fc4] ↑ Rmath v0.6.0 ⇒ v0.6.1
  [f50d1b31] + Rmath_jll v0.2.2+0
  [2913bbd2] ↑ StatsBase v0.32.1 ⇒ v0.32.2
  [bd369af6] ↑ Tables v1.0.2 ⇒ v1.0.3
  Building Mosek → `C:\Users\Thibaut\.julia\packages\Mosek\r4MUf\deps\build.log`
┌ Error: Error building `Mosek`:
│ ERROR: LoadError: Failed to get MOSEK download host
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] top-level scope at C:\Users\Thibaut\.julia\packages\Mosek\r4MUf\deps\build.jl:150
│  [3] include at .\boot.jl:328 [inlined]
│  [4] include_relative(::Module, ::String) at .\loading.jl:1105
│  [5] include(::Module, ::String) at .\Base.jl:31
│  [6] include(::String) at .\client.jl:424
│  [7] top-level scope at none:5
│ in expression starting at C:\Users\Thibaut\.julia\packages\Mosek\r4MUf\deps\build.jl:105
└ @ Pkg.Operations D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Pkg\src\backwards_compatible_isolation.jl:649
  Building Blosc → `C:\Users\Thibaut\.julia\packages\Blosc\ZgcAM\deps\build.log`

(v1.3) pkg> build Mosek
  Building Mosek → `C:\Users\Thibaut\.julia\packages\Mosek\r4MUf\deps\build.log`
┌ Error: Error building `Mosek`:
│ ERROR: LoadError: Failed to get MOSEK download host
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] top-level scope at C:\Users\Thibaut\.julia\packages\Mosek\r4MUf\deps\build.jl:150
│  [3] include at .\boot.jl:328 [inlined]
│  [4] include_relative(::Module, ::String) at .\loading.jl:1105
│  [5] include(::Module, ::String) at .\Base.jl:31
│  [6] include(::String) at .\client.jl:424
│  [7] top-level scope at none:5
│ in expression starting at C:\Users\Thibaut\.julia\packages\Mosek\r4MUf\deps\build.jl:105
└ @ Pkg.Operations D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Pkg\src\backwards_compatible_isolation.jl:649

MOSEK is installed properly: if I open a command prompt, I have direct access to MOSEK.

Microsoft Windows [Version 10.0.18363.720]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\Users\Thibaut>mosek

MOSEK Version 9.1.13 (Build date: 2020-2-20 14:26:57)
Copyright (c) MOSEK ApS, Denmark. WWW: mosek.com
Platform: Windows/64-X86


*** No input file specfied. No optimization is performed.

Return code - 0  [MSK_RES_OK]

MOSEK's environment variables are available from within Julia:

julia> ENV
Base.EnvDict with 52 entries:
  "=::"                       => "::\\"
  "=C:"                       => "C:\\Users\\Thibaut\\AppData\\Local\\Julia-1.3.0"
  "ALLUSERSPROFILE"           => "C:\\ProgramData"
  "APPDATA"                   => "C:\\Users\\Thibaut\\AppData\\Roaming"
  "CLASSPATH"                 => "C:\\Program Files\\Mosek\\9.1\\tools\\platform\\win64x86\\bin\\mosek.jar"
  "COMMONPROGRAMFILES"        => "C:\\Program Files\\Common Files"
  "COMMONPROGRAMFILES(X86)"   => "C:\\Program Files (x86)\\Common Files"
  "COMMONPROGRAMW6432"        => "C:\\Program Files\\Common Files"
  "COMPUTERNAME"              => "MSI"
  "COMSPEC"                   => "C:\\WINDOWS\\system32\\cmd.exe"
  "CONFIGSETROOT"             => "C:\\WINDOWS\\ConfigSetRoot"
  "CPLEX_STUDIO_BINARIES1210" => "C:\\Program Files\\IBM\\ILOG\\CPLEX_Studio1210\\opl\\bin\\x64_win64;C:\\Program Files\\IBM\\ILOG\\CPLEX_Studio1210\\opl\\oplide\\;C:\\Program Files\\IBM\\ILOG\\CPLEX_Studio1210\\cplex\\bin\\x64_win64;C:…
  "CPLEX_STUDIO_BINARIES129"  => "C:\\Program Files\\IBM\\ILOG\\CPLEX_Studio129\\opl\\bin\\x64_win64;C:\\Program Files\\IBM\\ILOG\\CPLEX_Studio129\\opl\\oplide\\;C:\\Program Files\\IBM\\ILOG\\CPLEX_Studio129\\cplex\\bin\\x64_win64;C:\\P…
  "CPLEX_STUDIO_DIR1210"      => "C:\\Program Files\\IBM\\ILOG\\CPLEX_Studio1210"
  "CPLEX_STUDIO_DIR129"       => "C:\\Program Files\\IBM\\ILOG\\CPLEX_Studio129"
  "DRIVERDATA"                => "C:\\Windows\\System32\\Drivers\\DriverData"
  "GUROBI_HOME"               => "C:\\gurobi900\\win64"
  "HOMEDRIVE"                 => "C:"
  "HOMEPATH"                  => "\\Users\\Thibaut"
  "LOCALAPPDATA"              => "C:\\Users\\Thibaut\\AppData\\Local"
  "MOSEK_9_1_BINDIR"          => "C:\\Program Files\\Mosek\\9.1\\tools\\platform\\%(BLDARCH)s\\bin"
  "MOSEK_9_1_EXE"             => "C:\\Program Files\\Mosek\\9.1\\tools\\platform\\%(BLDARCH)s\\bin\\mosek.exe"
  "MOSEK_9_1_INSTALLDIR"      => "C:\\Program Files\\Mosek\\9.1"
  "NUMBER_OF_PROCESSORS"      => "12"
  …

The variables MOSEK_9_1_BINDIR and MOSEK_9_1_EXE do not seem to have a correct value (what is this BLDARCH thing? it's not even an environment variable), even though they were set by the installer. Looking at the code (build.jl), however, this does not seem to be a problem, as the variable MOSEK_9_1_BINDIR is not used, but only MOSEKBINDIR (https://github.com/JuliaOpt/Mosek.jl/blob/master/deps/build.jl#L106-L112)… I guess it's a problem on MOSEK.jl's side?

Furthermore, couldn't this package look for MOSEK in the PATH (like CPLEX: https://github.com/JuliaOpt/CPLEX.jl/blob/master/deps/build.jl#L21-L26)?

Platform: Julia 1.3.0, x64, on Windows 10 x64 version 1909 (build 18363.720).

[ahmadreza-marandi]

I got exactly the same error...

[ulfworsoe]

I think it is because 9.1 is put up as a kind of 'beta' release, so perhaps there is an inconsistency in the naming that breaks something. I'll see what I can do.

[ulfworsoe]

I cannot reproduce the issue and it doesn't seem to have anything to do with the beta status. Can you post the build.log for the failed build?

[ahmadreza-marandi]

Here it is:

ERROR: LoadError: Failed to get MOSEK download host
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] top-level scope at C:\Users\20176914\.julia\dev\Mosek\deps\build.jl:150
 [3] include at .\boot.jl:317 [inlined]
 [4] include_relative(::Module, ::String) at .\loading.jl:1044
 [5] include(::Module, ::String) at .\sysimg.jl:29
 [6] include(::String) at .\client.jl:392
 [7] top-level scope at none:0
in expression starting at C:\Users\20176914\.julia\dev\Mosek\deps\build.jl:105

It is the same as the first post.

[dourouc05]

I can't reproduce anymore by ]build Moseking. Maybe it's because it could build successfully before? I checked my environment variables, and the new variables I defined previously to make the build work do not seem to be there anymore.

Here is my build.log:

[ Info: mskbindir = C:/Program Files/Mosek/9.1/tools/platform/win64x86/bin
[ Info: Found MOSEK 9.1.1 at C:/Program Files/Mosek/9.1/tools/platform/win64x86/bin

[ulfworsoe]

Hm. That was less informative than I hoped.

If you update to latest revision of branch b1.1-msk9.1 it should print the download command and some other info when you build. That might give a hint.

[ahmadreza-marandi]

Here is the build.log:

[ Info: Download command: powershell -file 'C:\Users\20176914\.julia\dev\Mosek\deps\winget.ps1' https://www.mosek.com/downloads/default_dns.txt 'C:\Users\20176914\.julia\dev\Mosek\deps\downloads\downloadhostname'
ERROR: LoadError: Failed to get MOSEK download host
Stacktrace:
[1] error(::String) at .\error.jl:33
[2] top-level scope at C:\Users\20176914.julia\dev\Mosek\deps\build.jl:153
[3] include at .\boot.jl:317 [inlined]
[4] include_relative(::Module, ::String) at .\loading.jl:1044
[5] include(::Module, ::String) at .\sysimg.jl:29
[6] include(::String) at .\client.jl:392
[7] top-level scope at none:0
in expression starting at C:\Users\20176914.julia\dev\Mosek\deps\build.jl:106

[ulfworsoe]

Thanks for the quick answer. Can you run the download command from the command line?

powershell -file 'C:\Users\20176914\.julia\dev\Mosek\deps\winget.ps1' https://www.mosek.com/downloads/default_dns.txt dlhostname

[ahmadreza-marandi]

This is the response from PowerShell:

Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
At C:\Users\20176914.julia\dev\Mosek\deps\winget.ps1:5 char:1

• (new-object net.webclient).DownloadFile($url, $fname)

•   + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : WebException
  

[ahmadreza-marandi]

It seems the problem is resolved! I have a "Restricted" policy in PowerShell. I changed it to "unrestricted", then build Mosek in Julis and it worked.

[ulfworsoe]

That is great! I'll see if I can make the error message more useful.

[ulfworsoe]

If the download fails it will now print any error messages from the shell command. This will be included in the next release.

[dourouc05]

I just updated to Julia 1.4.1 (Windows 10, x64, still), with the latest versions of Mosek (9.2.4) and Mosek.jl (master), installation still does not work:

(@v1.4) pkg> add Mosek#master
    Cloning git-repo `https://github.com/JuliaOpt/Mosek.jl.git`
   Updating git-repo `https://github.com/JuliaOpt/Mosek.jl.git`
   Updating registry at `C:\Users\Thibaut\.julia\registries\General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
   Updating `C:\Users\Thibaut\.julia\environments\v1.4\Project.toml`
  [6405355b] ~ Mosek v1.1.2 ⇒ v1.1.2 #master (https://github.com/JuliaOpt/Mosek.jl.git)
   Updating `C:\Users\Thibaut\.julia\environments\v1.4\Manifest.toml`
  [6405355b] ~ Mosek v1.1.2 ⇒ v1.1.2 #master (https://github.com/JuliaOpt/Mosek.jl.git)
   Building Mosek → `C:\Users\Thibaut\.julia\packages\Mosek\XLZS9\deps\build.log`
┌ Error: Error building `Mosek`:
│ [ Info: Download command: `powershell -file 'C:\Users\Thibaut\.julia\packages\Mosek\XLZS9\deps\winget.ps1' https://www.mosek.com/downloads/default_dns.txt 'C:\Users\Thibaut\.julia\packages\Mosek\XLZS9\deps\downloads\downloadhostname'`
│ ┌ Error: File C:\Users\Thibaut\.julia\packages\Mosek\XLZS9\deps\winget.ps1 cannot be loaded because running scripts is disabled
│ │ on this system. For more information, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170.│ │     + CategoryInfo          : SecurityError: (:) [], ParentContainsErrorRecordException
│ │     + FullyQualifiedErrorId : UnauthorizedAccess
│ └ @ Main C:\Users\Thibaut\.julia\packages\Mosek\XLZS9\deps\build.jl:176
│ ERROR: LoadError: Failed to get MOSEK download host
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] top-level scope at C:\Users\Thibaut\.julia\packages\Mosek\XLZS9\deps\build.jl:177
│  [3] include(::String) at .\client.jl:439
│  [4] top-level scope at none:5
│ in expression starting at C:\Users\Thibaut\.julia\packages\Mosek\XLZS9\deps\build.jl:124
└ @ Pkg.Operations D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Operations.jl:892

I still have to manually set an environment variable to have the installation work properly:

julia> ENV["MOSEKBINDIR"] = "C:\\Program Files\\Mosek\\9.2\\tools\\platform\\win64x86\\bin"
"C:\\Program Files\\Mosek\\9.2\\tools\\platform\\win64x86\\bin"

(@v1.4) pkg> build Mosek
   Building Mosek → `C:\Users\Thibaut\.julia\packages\Mosek\XLZS9\deps\build.log`

[ulfworsoe]

I think the issue is that some powershell system setting restricts download, most likely the same issue as mentioned above: #195 (comment)
We depend on a powershell scripts to do the actual download, so if Powershell is restricted, it will fail.