Add support for the Visual Studio 2017 linker

[poke]

With Visual Studio 2017, Microsoft has changed the folder structure it uses for everything. link.exe is now located at C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.10.24728\bin\HostX64\x64\link.exe. As you can see, the folder is dependent on the actual edition of Visual Studio as well.

Currently, Rust does not seem to recognize this path correctly. I don’t know exactly what Rust needs to figure out the paths, but if you need some information (e.g. registry values, contents of vcvarsall.bat or something), just tell me what you need and I’ll try my best to provide you with it.

[retep998]

I already spoke with the Visual C++ team a few months ago and someone was going to investigate adding support for this. I'll poke them and see if any progress has been made, and if not I'll end up doing it myself. I have VS 2017 myself so there should be nothing I'd need from you.

[retep998]

For reference, a code sample for finding VS 2017 instances https://code.msdn.microsoft.com/windowsdesktop/Visual-Studio-Setup-0cedd331

[chris-morgan]

Concerning a workaround: if this is the same as Visual Studio “15” Preview 4 (which had the new installer and directory structures): you can run vcvarsall.bat to set up the current terminal and then it all works.

For me, this works:

"C:\Program Files (x86)\Microsoft Visual Studio\VS15Preview\Common7\IDE\VC\vcvarsall.bat" x64

[poke]

So, I spent some time with the Rust compiler’s source code and with vcvarsall.bat and my Visual Studio environment to figure out a few things:

• get_vc_dir – At least on my machine, in order to find the Visual Studio install directory for the version code 15.0, we also have to check the Wow6432Node node at HKLM:\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VS7. Fortunately, this also does appear to work for older versions, so it might be enough to simply check there first, and then fall back to HKLM:\SOFTWARE\Microsoft\VisualStudio\SxS\VS7.
• vcvars.bat then loads the tools version from %VSINSTALLDIR%VC\Auxiliary\Build\Microsoft.VCToolsVersion.default.txt which is a text file that contains the version number, currently __VCVARS_TOOLS_VERSION=14.10.24728.
• VCToolsInstallDir then becomes %VSINSTALLDIR%VC\Tools\MSVC\%__VCVARS_TOOLS_VERSION%\".
• link.exe is then inside %VCToolsInstallDir%bin%__VCVARS_HOST_DIR%%__VCVARS_HOST_NATIVEDIR%

The values for __VCVARS_HOST_DIR and _VCVARS_HOST_NATIVEDIR are dependant on the platform:

if /I "%VSCMD_ARG_HOST_ARCH%" == "x86" (
    set __VCVARS_HOST_DIR=\HostX86
    set __VCVARS_HOST_NATIVEDIR=\x86
)
if /I "%VSCMD_ARG_HOST_ARCH%" == "x64" (
    set __VCVARS_HOST_DIR=\HostX64
    set __VCVARS_HOST_NATIVEDIR=\x64
)
if /I "%VSCMD_ARG_HOST_ARCH%" == "arm" (
    set __VCVARS_HOST_DIR=\HostARM
    set __VCVARS_HOST_NATIVEDIR=\arm
)

Maybe that helps.

@chris-morgan Yeah, using the vcvarsall.bat to set up the environment variables always works. But it’s also enough to just augment the PATH environment variable to include the path to the linker and to set the LIB environment variable.

[whoisj]

Visual Studio 2017 supports multi-instance installation, meaning you can install it more than once and each instance can have its own setup of options enabled/disabled. Therefore there is no longer a canonical location for Visual Studio to exists. That said, the "%ProgramFiles(x86)%\Microsoft Visual Studio\2017" is still the best shot.

Ideally, users can compile Rustlang projects from a "Developer Command Prompt for Visual Studio 2017", this will resolve most (if not all) of the issues.

For users developing Rust in an IDE (like VS Code) can benefit from launching the IDE from the developer command prompt or by leveraging the logic provided by @poke, which should work for the common case.

@artl93 for a more authoritative information.

[artl93]

Hi. Let me go and start by saying I am not familiar with rust particulars, so I am going to draw some conclusions based on the thread above. Please bear with me, as must of what I am about to go into just relates to how we want to think of tools, the application and the system. 

We wanted to make Visual Studio not impact the operating system. We wanted the fact that Visual Studio is installed to no longer impact the system-wide environment. By impacting the system environment, we made it impossible to have our compilers or tools sit side-by-side without having to ship a new side-by-side version of Visual Studio. We really wanted customers to be more free to install newer versions without the fear of corrupting their build environments and working tools, making it an easier call to try out new stuff without the fear of being able to get work done.

To accomplish this, we really separated VS from the system-wide SDKs and runtimes that VS works with. We really wanted VS to be an "application" that consumes these things, rather than a part of each these. This meant taking the time to de-couple VS and the SDKs from the environment, making a much more flexible system for everyone. That also means we're going to have to understand how tools really relate to the IDE so that they are coupled in the best way possible.

The VC++ toolset is interesting when we think of this model. In doing the work to de-couple SDKs from the IDE, the VC tools in particular turned out to be tightly coupled to the IDE. Therefore, the VC toolset ships as a part of the IDE application folder, and not something that is deployed system-wide. Therefore, each Visual Studio application on the system has its own, compatible copy of the VC tools.

In the case of Rust, it sounds like it's heavily dependent on the VC tools, based on the context in the thread. As such, if you think of rust being heavily dependent on the VC tools, then they are by transitive closure dependent on the Visual Studio environment. Since the Visual Studio environment is a part of the application environment (rather than something that is shared and creating system-impact), I think we could think of the rust toolset as being a part of that environment. Therefore, I think we no longer want to think about the tools as being system-wide, but rather an extension to the VS IDE (application) environment. Therefore, you'd want to just extend the environment for the VS application, and not some system wide setting.

To make this work, you can extend the environment for the particular application install using VSIXes (the new version). To add environment variables to the developer command prompt, you can add your own bat files (yes, *.bat) to: Common7\tools\vsdevcmd\ext. In the file, define a call_script_helper.

You can see how this is set up by looking at vsdevcmd.bat:
for /F %%a in ( 'dir "%VS150COMNTOOLS%vsdevcmd\ext\*.bat" /b /a-d-h /on' ) do (call :call_script_helper ext\%%a)

You can also look at examples in the ext folder as to how others have set this up. I'm not sure if this is documented, but it should be. I can confirm with the team that did the work.

The one last thing: if you depend on the  VC tools, then you'll want to declare that dependency in your VSIX. 

Does that make sense, or am I way off base here?

[whoisj]

@artl93 I do not believe that Rust has a Vsix, I believe that they just want to use the MS provided linker (and masm?) tools. Basically, how can a Rust make file find the location of the linker (and assembler?)?

[poke]

@whoisj

Visual Studio 2017 supports multi-instance installation, meaning you can install it more than once

I’m not sure how that works though. When I rerun the installer, I can only modify my existing 2017 Enterprise installation but not install a second parallel one. Since the installer is the primary entry point for this case, I would say we can assume that there would only be a single installation at the default location.

That said, the "%ProgramFiles(x86)%\Microsoft Visual Studio\2017" is still the best shot.

That won’t work though since the actual edition is also part of the path now, e.g. …\2017\Enterprise in my case.

Ideally, users can compile Rustlang projects from a "Developer Command Prompt for Visual Studio 2017", this will resolve most (if not all) of the issues.

The dev command prompt is a very limited tool though, being way inferior to PowerShell or other command line interfaces. I assume Git Bash is also a common alternative shell for use with Rust projects. Requiring the developer command prompt there by default does not seem to be a good idea there.

@artl93

Therefore, the VC toolset ships as a part of the IDE application folder, and not something that is deployed system-wide.

But is there technically any difference between a VC toolset 14.10.24728 installed by VS2017 Enterprise, and a VC toolset 14.10.24728 installed by another edition? Would it not have been possible to ship this with the IDE but still place it in a more generic folder instead?

In the case of Rust, it sounds like it's heavily dependent on the VC tools, based on the context in the thread.

It’s actually not as complicated as it may sound. Rust actually only depends on the linker link.exe and the libraries that ship with VC. That’s why it’s enough to add the VC bin folder to PATH and set up the LIB environment variable to make Rust work.

That’s why I don’t really agree that Rust should convert into some extension to Visual Studio or the VC tools. And as said above, the fact that the Visual Studio tools depend that much on a cmd command prompt can be pretty limiting.

---

Anyway, to get back to what I am trying to get at: Rust already figures out which Visual Studio version is available. The MSVC detection written by @retep998 is already very sophisticated and works fine in many cases. It’s just that the detection is not yet compatible with the changed path structure in VS2017, so it simply won’t work yet. What this issue is about is fixing that detection to work with a standard VS2017 installation. We can’t—and probably shouldn’t even attempt to—handle edge cases or highly customized setups. For situations like that, it’s always possible for users to set up the environment variables themselves, and Rust will simply use that.

So if you have multiple versions of Visual Studio installed (for whatever reason), and Rust ends up using a different version than the one you want it to use (not perfectly sure why that would make an actual difference though), then you will have to tell Rust to use the correct version anyway. That is not limited to just VS2017 but applies to any version.

The steps I’ve written above are actually not that complicated. The current code that figures out where the MSVC linker is is already very well written to support this additional logic. And I’m absolutely willing to provide the patches necessary to make it work myself. It’s just that I’m just beginning with Rust, so I don’t feel comfortable contributing to the compiler just yet. I would eventually give it a try though.

[artl93]

@poke

But is there technically any difference between a VC toolset 14.10.24728 installed by VS2017 Enterprise, and a VC toolset 14.10.24728 installed by another edition? Would it not have been possible to ship this with the IDE but still place it in a more generic folder instead?

The VC tools are tightly coupled to the IDE, therefore version X of the IDE must use a specific version X of the toolset. They are a part of the application folder. 
In the future, we anticipate that 2017's updates can be installed side-by-side with the RTW or previous updates, so picking the compiler associated with your install is critical to getting the environment set-up correctly.

[whoisj]

@artl93 I believe the @poke specifically doesn't care about the IDE, but only the toolset. Given that information, and a system with VS 2017 installed (at least once): what is the best solution for the Rust make system to find the necessary tools provided by VS 2017>

[poke]

Yeah, what Rust basically needs to do is to be able to answer the question: “Where is the VC linker and library folder?”

Currently it does that by explicitly checking against every Visual Studio version starting with the latest known version. Obviously, this requires to keep adding new logic as newer versions of Visual Studio are released. In an ideal world, there would be a single constant and future-proof location in the registry we could look at to find the path to the most-current VC toolset.

Until we are there, we keep adding logic, with the stategy outlined above being my suggestion to find the current tools for (hopefully) any VS2017 installation.

[retep998]

I'd really rather not hardcode all those relative paths; it would be incredibly brittle. The proper solution involves using special COM interfaces to enumerate the VC++ instances. Unless someone from Microsoft decides to do it themselves, I'm likely going to do the COM stuff myself. It's just a bit icky doing COM in rustc itself without winapi to help out.