Enable MemberImportVisibility check on all targets #497
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rnro
force-pushed
the
enable_MemberImportVisibility_check
branch
from
e795944 to
e5e447f
Compare
rnro
changed the title
rnro
force-pushed
the
enable_MemberImportVisibility_check
branch
2 times, most recently
from
a8eedcb to
720591b
Compare
rnro
added
🔨 semver/patch
rnro
force-pushed
the
enable_MemberImportVisibility_check
branch
from
8d62050 to
1185bd6
Compare
glbrntt
reviewed
Dec 17, 2024
Comment on lines
17
to
19
| #if os(Linux) || os(FreeBSD) || os(Android) | ||
| import CNIOLinux | ||
| #endif |
Contributor
There was a problem hiding this comment.
You should be able to unconditionally import this -- it if-defs itself so that it's empty on other platforms.
Contributor
Author
There was a problem hiding this comment.
Thanks, I forgot I did that here
unconditionally import CNIOLinux
glbrntt
approved these changes
Dec 17, 2024
simonjbeaumont
added a commit
to simonjbeaumont/vapor
that referenced
this pull request
Nov 11, 2025
## Motivation Since adding `MemberImportVisibility`, when Vapor is compiled in highly parallel environments it fails with high probability: ```console % git rev-parse HEAD ac3aeb7 % rm -rf .build ~/.cache/org.swift.swiftpm/manifests/ && swift build -j 64 ... Building for debugging... /pwd/Sources/Vapor/Utilities/String+IsIPAddress.swift:10:24: error: initializer 'init()' is not available due to missing import of defining module 'CNIOLinux' [#Membe rImportVisibility] 1 | import Foundation 2 | import NIOCore 3 | #if canImport(Android) | `- note: add import of module 'CNIOLinux' 4 | import Android 5 | #endif : 8 | func isIPAddress() -> Bool { 9 | // We need some scratch space to let inet_pton write into. 10 | var ipv4Addr = in_addr() | `- error: initializer 'init()' is not available due to missing import of defining module 'CNIOLinux' [#MemberImportVisibility] 11 | var ipv6Addr = in6_addr() 12 | ---[ similar error for in6_addr too ]--- ``` Building with `-j 1` always succeeds. I've spent quite some time looking into this and it appears that we're hitting a combination of: 1. The Glibc module map is broken. It only declares a top-level `SwiftGlibc.h`, which then `#include`s many of the system headers—relevant to this this issue, it includes `in.h`. As a result of this investigation, @al45tair has filed swiftlang/swift#85427, but we'll need a solution in the interim. 2. The CNIOLinux non-product C target of Swift NIO also has an umbrella `CNIOLinux.h` header, which also `#include`s many headers, including a transitive include of `in.h`. 3. Vapor has started building with `MemberImportVisibility`. We can see in the above error that the Swift compiler is suggesting to add `import CNIOLinux`, which is a dubious suggestion, since you'd expect the fix to be to add an `import Glibc` or similar. However, if we poke around at the failed build output we can see that `in.h` is _only_ part of the `CNIOLinux` and _absent_ from `Glibc`: ```console % find .build/aarch64-unknown-linux-gnu/debug/ModuleCache/ -name '*.pcm' | grep -e NIOLinux -e Glibc | xargs -n 1 -t clang -cc1 -module-file-info 2>&1 | grep -e "clang -cc1" -e "Input file:.*/in\\.h" clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/29YJZD87ZLI67/SwiftGlibc-IGSGJKVMPVR1.pcm clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/29YJZD87ZLI67/CNIOLinux-1MNQXZXDCO5H5.pcm Input file: /usr/include/linux/in.h [System] clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/1NHH1B2IBMMHE/SwiftGlibc-IGSGJKVMPVR1.pcm clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/1NHH1B2IBMMHE/CNIOLinux-1MNQXZXDCO5H5.pcm Input file: /usr/include/linux/in.h [System] clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/1O3SENRIBS9OC/SwiftGlibc-IGSGJKVMPVR1.pcm ``` Compare this to the build output of a successful build (`-j 1`), where we can see that `in.h` is part of `Glibc`: ```console % find .build-j1/aarch64-unknown-linux-gnu/debug/ModuleCache/ -name '*.pcm' | grep Glibc | xargs -n 1 -t clang -cc1 -module-file-info 2>&1 | grep -e "clang -cc1" -e "Input file:.*/in\\.h" clang -cc1 -module-file-info .build-j1/aarch64-unknown-linux-gnu/debug/ModuleCache/29YJZD87ZLI67/SwiftGlibc-IGSGJKVMPVR1.pcm Input file: /usr/include/netinet/in.h [System] Input file: /usr/include/bits/in.h [System] clang -cc1 -module-file-info .build-j1/aarch64-unknown-linux-gnu/debug/ModuleCache/1NHH1B2IBMMHE/SwiftGlibc-IGSGJKVMPVR1.pcm Input file: /usr/include/netinet/in.h [System] Input file: /usr/include/bits/in.h [System] clang -cc1 -module-file-info .build-j1/aarch64-unknown-linux-gnu/debug/ModuleCache/1O3SENRIBS9OC/SwiftGlibc-IGSGJKVMPVR1.pcm Input file: /usr/include/netinet/in.h [System] Input file: /usr/include/bits/in.h [System] ``` What's happening is that a header file can only belong to exactly one Clang module and how header files are attributed to modules is non-deterministic in some cases. `CNIOLinux` is an implicit module——it does _not_ have a `.modulemap`——and is inferred from the presence and contents of `Sources/CNIOLinux/include/CNIOLinux.h`. `Glibc` is an explicit module——it _does_ have a `.modulemap`——but its module map does not explicitly list all the headers that belong to the module. It only lists `SwiftGlibc.h`. Header files that are explicitly listed in a module map are deterministically attributed to that module, but header files that are transitively included using `#include` are attributed to modules on a first-come-first-attributed basis. To illustrate this, I can make the `-j64` build succeed with the following hot-patch of the `Glibc.modulemap` in my container: ```diff module SwiftGlibc [system] { // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 23) link "m" // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 26) link "pthread" // FIXME: util contains rarely used functions and not usually needed. Unfortunately // link directive doesn't work in the submodule yet. // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 30) link "util" // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 33) // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 35) link "dl" // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 37) // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 43) header "SwiftGlibc.h" + header "netinet/in.h" // <assert.h>'s use of NDEBUG requires textual inclusion. textual header "assert.h" export * } ``` We'll have to track swiftlang/swift#85427 for a real fix for this issue, but in the meantime, we'll need a workaround in Vapor if we want it to work with the `MemberImportVisibility` compiler setting. As it happens, it looks like the compiler hint——`import CNIOLinux`——might be the best we can do for today, and it turns out that there are other projects that are downstream of NIO that added this when they added `MemberImportVisibilty` too: - apple/swift-nio-ssl#497 - apple/swift-nio-extras#240 - swift-server/async-http-client#794 ## Modifications Add missing imports, with a link to the Swift issue to track. ## Result Vapor will now reliably build when built in parallel.
0xTim
pushed a commit
to vapor/vapor
that referenced
this pull request
Nov 11, 2025
## Motivation Since adding `MemberImportVisibility`, when Vapor is compiled in highly parallel environments it fails with high probability: ```console % git rev-parse HEAD ac3aeb7 % rm -rf .build ~/.cache/org.swift.swiftpm/manifests/ && swift build -j 64 ... Building for debugging... /pwd/Sources/Vapor/Utilities/String+IsIPAddress.swift:10:24: error: initializer 'init()' is not available due to missing import of defining module 'CNIOLinux' [#Membe rImportVisibility] 1 | import Foundation 2 | import NIOCore 3 | #if canImport(Android) | `- note: add import of module 'CNIOLinux' 4 | import Android 5 | #endif : 8 | func isIPAddress() -> Bool { 9 | // We need some scratch space to let inet_pton write into. 10 | var ipv4Addr = in_addr() | `- error: initializer 'init()' is not available due to missing import of defining module 'CNIOLinux' [#MemberImportVisibility] 11 | var ipv6Addr = in6_addr() 12 | ---[ similar error for in6_addr too ]--- ``` Building with `-j 1` always succeeds. I've spent quite some time looking into this and it appears that we're hitting a combination of: 1. The Glibc module map is broken. It only declares a top-level `SwiftGlibc.h`, which then `#include`s many of the system headers—relevant to this this issue, it includes `in.h`. As a result of this investigation, @al45tair has filed swiftlang/swift#85427, but we'll need a solution in the interim. 2. The CNIOLinux non-product C target of Swift NIO also has an umbrella `CNIOLinux.h` header, which also `#include`s many headers, including a transitive include of `in.h`. 3. Vapor has started building with `MemberImportVisibility`. We can see in the above error that the Swift compiler is suggesting to add `import CNIOLinux`, which is a dubious suggestion, since you'd expect the fix to be to add an `import Glibc` or similar. However, if we poke around at the failed build output we can see that `in.h` is _only_ part of the `CNIOLinux` and _absent_ from `Glibc`: ```console % find .build/aarch64-unknown-linux-gnu/debug/ModuleCache/ -name '*.pcm' | grep -e NIOLinux -e Glibc | xargs -n 1 -t clang -cc1 -module-file-info 2>&1 | grep -e "clang -cc1" -e "Input file:.*/in\\.h" clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/29YJZD87ZLI67/SwiftGlibc-IGSGJKVMPVR1.pcm clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/29YJZD87ZLI67/CNIOLinux-1MNQXZXDCO5H5.pcm Input file: /usr/include/linux/in.h [System] clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/1NHH1B2IBMMHE/SwiftGlibc-IGSGJKVMPVR1.pcm clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/1NHH1B2IBMMHE/CNIOLinux-1MNQXZXDCO5H5.pcm Input file: /usr/include/linux/in.h [System] clang -cc1 -module-file-info .build/aarch64-unknown-linux-gnu/debug/ModuleCache/1O3SENRIBS9OC/SwiftGlibc-IGSGJKVMPVR1.pcm ``` Compare this to the build output of a successful build (`-j 1`), where we can see that `in.h` is part of `Glibc`: ```console % find .build-j1/aarch64-unknown-linux-gnu/debug/ModuleCache/ -name '*.pcm' | grep Glibc | xargs -n 1 -t clang -cc1 -module-file-info 2>&1 | grep -e "clang -cc1" -e "Input file:.*/in\\.h" clang -cc1 -module-file-info .build-j1/aarch64-unknown-linux-gnu/debug/ModuleCache/29YJZD87ZLI67/SwiftGlibc-IGSGJKVMPVR1.pcm Input file: /usr/include/netinet/in.h [System] Input file: /usr/include/bits/in.h [System] clang -cc1 -module-file-info .build-j1/aarch64-unknown-linux-gnu/debug/ModuleCache/1NHH1B2IBMMHE/SwiftGlibc-IGSGJKVMPVR1.pcm Input file: /usr/include/netinet/in.h [System] Input file: /usr/include/bits/in.h [System] clang -cc1 -module-file-info .build-j1/aarch64-unknown-linux-gnu/debug/ModuleCache/1O3SENRIBS9OC/SwiftGlibc-IGSGJKVMPVR1.pcm Input file: /usr/include/netinet/in.h [System] Input file: /usr/include/bits/in.h [System] ``` What's happening is that a header file can only belong to exactly one Clang module and how header files are attributed to modules is non-deterministic in some cases. `CNIOLinux` is an implicit module——it does _not_ have a `.modulemap`——and is inferred from the presence and contents of `Sources/CNIOLinux/include/CNIOLinux.h`. `Glibc` is an explicit module——it _does_ have a `.modulemap`——but its module map does not explicitly list all the headers that belong to the module. It only lists `SwiftGlibc.h`. Header files that are explicitly listed in a module map are deterministically attributed to that module, but header files that are transitively included using `#include` are attributed to modules on a first-come-first-attributed basis. To illustrate this, I can make the `-j64` build succeed with the following hot-patch of the `Glibc.modulemap` in my container: ```diff module SwiftGlibc [system] { // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 23) link "m" // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 26) link "pthread" // FIXME: util contains rarely used functions and not usually needed. Unfortunately // link directive doesn't work in the submodule yet. // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 30) link "util" // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 33) // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 35) link "dl" // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 37) // ###sourceLocation(file: "/home/build-user/swift/stdlib/public/Platform/glibc.modulemap.gyb", line: 43) header "SwiftGlibc.h" + header "netinet/in.h" // <assert.h>'s use of NDEBUG requires textual inclusion. textual header "assert.h" export * } ``` We'll have to track swiftlang/swift#85427 for a real fix for this issue, but in the meantime, we'll need a workaround in Vapor if we want it to work with the `MemberImportVisibility` compiler setting. As it happens, it looks like the compiler hint——`import CNIOLinux`——might be the best we can do for today, and it turns out that there are other projects that are downstream of NIO that added this when they added `MemberImportVisibilty` too: - apple/swift-nio-ssl#497 - apple/swift-nio-extras#240 - swift-server/async-http-client#794 ## Modifications Add missing imports, with a link to the Swift issue to track. ## Result Vapor will now reliably build when built in parallel.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enable MemberImportVisibility check on all targets. Use a standard string header and footer to bracket the new block for ease of updating in the future with scripts.