WCOW/tests: revisit skipped integration tests to run on Windows where applicable

[profnandaa]

I'm opening this issue as a tracker for the explicitly skipped integration tests on Windows. Some should be skipped by design but others need to be enabled on Windows too.

Here is the list (might keep growing as new tests come in):

Script

  $ grep -r -B 1 ".SkipOnPlatform(" | grep -v "^--" | grep -v "integration.SkipOn" |  sed 's/-/ /' | sed 's/^/- [ ] /'
  

/client:

• client/build_test.go testClientGatewayEmptyImageExec
• client/client_test.go testCacheExportCacheKeyLoop
• client/client_test.go testBridgeNetworkingDNSNoRootless
• client/client_test.go testExportedImageLabels
• client/client_test.go testExportBusyboxLocal
• client/client_test.go testHostnameLookup
• client/client_test.go testHostnameSpecifying
• client/client_test.go testStdinClosed
• client/client_test.go testSSHMount
• client/client_test.go testExtraHosts
• client/client_test.go testShmSize
• client/client_test.go testUlimit
• client/client_test.go testNetworkMode
• client/client_test.go testSecurityMode
• client/client_test.go testSecurityModeSysfs
• client/client_test.go testSecurityModeErrors
• client/client_test.go testSecretMounts
• client/client_test.go testSecretEnv
• client/client_test.go testBuildHTTPSource
• client/client_test.go testSourceDateEpochImageExporter
• client/client_test.go testBuildExportWithForeignLayer
• client/client_test.go testBuildExportZstd
• client/client_test.go testPullZstdImage
• client/client_test.go testZstdLocalCacheExport
• client/client_test.go testCacheExportIgnoreError
• client/client_test.go testBasicS3CacheImportExport
• client/client_test.go testBasicAzblobCacheImportExport
• client/client_test.go testSnapshotWithMultipleBlobs
• client/client_test.go testExportLocalNoPlatformSplit
• client/client_test.go testMoveParentDir
• client/client_test.go testSchema1Image
• client/client_test.go testMountWithNoSource
• client/client_test.go testReadonlyRootFS
• client/client_test.go testRmSymlink
• client/client_test.go testProxyEnv
• client/client_test.go testParallelLocalBuilds
• client/client_test.go testMultipleCacheExports
• client/client_test.go testMountStubsDirectory
• client/client_test.go testMountStubsTimestamp
• client/client_test.go testLLBMountPerformance

/cmd/buildctl:

• cmd/buildctl/buildctl_test.go testUsage
• cmd/buildctl/build_test.go testBuildWithLocalFiles
• cmd/buildctl/build_test.go testBuildLocalExporter
• cmd/buildctl/build_test.go testBuildContainerdExporter
• cmd/buildctl/build_test.go testBuildMetadataFile
• cmd/buildctl/diskusage_test.go testDiskUsage
• cmd/buildctl/prune_test.go testPrune

/frontend:

• frontend/dockerfile/dockerfile_addchecksum_test.go testAddChecksum
• frontend/dockerfile/dockerfile_addgit_test.go testAddGit
• frontend/dockerfile/dockerfile_heredoc_test.go testCopyHeredoc
• frontend/dockerfile/dockerfile_heredoc_test.go testCopyHeredocSpecialSymbols
• frontend/dockerfile/dockerfile_heredoc_test.go testRunBasicHeredoc
• frontend/dockerfile/dockerfile_heredoc_test.go testRunFakeHeredoc
• frontend/dockerfile/dockerfile_heredoc_test.go testRunShebangHeredoc
• frontend/dockerfile/dockerfile_heredoc_test.go testRunComplexHeredoc
• frontend/dockerfile/dockerfile_heredoc_test.go testHeredocIndent
• frontend/dockerfile/dockerfile_heredoc_test.go testHeredocVarSubstitution
• frontend/dockerfile/dockerfile_heredoc_test.go testOnBuildHeredoc
• frontend/dockerfile/dockerfile_mount_test.go testMountContext
• frontend/dockerfile/dockerfile_mount_test.go testMountTmpfs
• frontend/dockerfile/dockerfile_mount_test.go testMountInvalid
• frontend/dockerfile/dockerfile_mount_test.go testMountRWCache
• frontend/dockerfile/dockerfile_mount_test.go testCacheMountUser
• frontend/dockerfile/dockerfile_mount_test.go testCacheMountDefaultID
• frontend/dockerfile/dockerfile_mount_test.go testMountEnvVar
• frontend/dockerfile/dockerfile_mount_test.go testMountArg
• frontend/dockerfile/dockerfile_mount_test.go testMountEnvAcrossStages
• frontend/dockerfile/dockerfile_mount_test.go testMountMetaArg
• frontend/dockerfile/dockerfile_mount_test.go testMountFromError
• frontend/dockerfile/dockerfile_mount_test.go testMountTmpfsSize
• frontend/dockerfile/dockerfile_mount_test.go testMountDuplicate
• frontend/dockerfile/dockerfile_outline_test.go testOutlineArgs
• frontend/dockerfile/dockerfile_outline_test.go testOutlineSecrets
• frontend/dockerfile/dockerfile_outline_test.go testOutlineDescribeDefinition
• frontend/dockerfile/dockerfile_provenance_test.go testProvenanceAttestation
• frontend/dockerfile/dockerfile_provenance_test.go testGitProvenanceAttestation
• frontend/dockerfile/dockerfile_provenance_test.go testMultiPlatformProvenance
• frontend/dockerfile/dockerfile_provenance_test.go testClientFrontendProvenance
• frontend/dockerfile/dockerfile_provenance_test.go testClientLLBProvenance
• frontend/dockerfile/dockerfile_provenance_test.go testSecretSSHProvenance
• frontend/dockerfile/dockerfile_provenance_test.go testOCILayoutProvenance
• frontend/dockerfile/dockerfile_provenance_test.go testNilProvenance
• frontend/dockerfile/dockerfile_provenance_test.go testDuplicatePlatformProvenance
• frontend/dockerfile/dockerfile_provenance_test.go testDockerIgnoreMissingProvenance
• frontend/dockerfile/dockerfile_provenance_test.go testFrontendDeduplicateSources
• frontend/dockerfile/dockerfile_provenance_test.go testDuplicateLayersProvenance
• frontend/dockerfile/dockerfile_runnetwork_test.go testRunGlobalNetwork
• frontend/dockerfile/dockerfile_secrets_test.go testSecretFileParams
• frontend/dockerfile/dockerfile_secrets_test.go testSecretRequiredWithoutValue
• frontend/dockerfile/dockerfile_ssh_test.go testSSHSocketParams
• frontend/dockerfile/dockerfile_ssh_test.go testSSHFileDescriptorsClosed
• frontend/dockerfile/dockerfile_targets_test.go testTargetsList
• frontend/dockerfile/dockerfile_targets_test.go testTargetsDescribeDefinition
• frontend/dockerfile/dockerfile_test.go testDefaultEnvWithArgs
• frontend/dockerfile/dockerfile_test.go testEnvEmptyFormatting
• frontend/dockerfile/dockerfile_test.go testDockerignoreOverride
• frontend/dockerfile/dockerfile_test.go testEmptyDestDir
• frontend/dockerfile/dockerfile_test.go testExportCacheLoop
• frontend/dockerfile/dockerfile_test.go testTarExporter
• frontend/dockerfile/dockerfile_test.go testWorkdirCreatesDir
• frontend/dockerfile/dockerfile_test.go testCacheReleased
• frontend/dockerfile/dockerfile_test.go testSymlinkedDockerfile
• frontend/dockerfile/dockerfile_test.go testCopyChownExistingDir
• frontend/dockerfile/dockerfile_test.go testCopyWildcardCache
• frontend/dockerfile/dockerfile_test.go testEmptyWildcard
• frontend/dockerfile/dockerfile_test.go testWorkdirUser
• frontend/dockerfile/dockerfile_test.go testWorkdirCopyIgnoreRelative
• frontend/dockerfile/dockerfile_test.go testWorkdirExists
• frontend/dockerfile/dockerfile_test.go testCopyChownCreateDest
• frontend/dockerfile/dockerfile_test.go testCopyThroughSymlinkContext
• frontend/dockerfile/dockerfile_test.go testCopyThroughSymlinkMultiStage
• frontend/dockerfile/dockerfile_test.go testCopySocket
• frontend/dockerfile/dockerfile_test.go testIgnoreEntrypoint
• frontend/dockerfile/dockerfile_test.go testQuotedMetaArgs
• frontend/dockerfile/dockerfile_test.go testMultiArgs
• frontend/dockerfile/dockerfile_test.go testDefaultShellAndPath
• frontend/dockerfile/dockerfile_test.go testExportMultiPlatform
• frontend/dockerfile/dockerfile_test.go testContextChangeDirToFile
• frontend/dockerfile/dockerfile_test.go testNoSnapshotLeak
• frontend/dockerfile/dockerfile_test.go testCopyFollowAllSymlinks
• frontend/dockerfile/dockerfile_test.go testCopySymlinks
• frontend/dockerfile/dockerfile_test.go testHTTPDockerfile
• frontend/dockerfile/dockerfile_test.go testCmdShell
• frontend/dockerfile/dockerfile_test.go testPullScratch
• frontend/dockerfile/dockerfile_test.go testGlobalArg
• frontend/dockerfile/dockerfile_test.go testDockerfileDirs
• frontend/dockerfile/dockerfile_test.go testDockerfileInvalidCommand
• frontend/dockerfile/dockerfile_test.go testDockerfileInvalidInstruction
• frontend/dockerfile/dockerfile_test.go testDockerfileADDFromURL
• frontend/dockerfile/dockerfile_test.go testDockerfileAddArchive
• frontend/dockerfile/dockerfile_test.go testDockerfileAddArchiveWildcard
• frontend/dockerfile/dockerfile_test.go testDockerfileAddChownExpand
• frontend/dockerfile/dockerfile_test.go testSymlinkDestination
• frontend/dockerfile/dockerfile_test.go testDockerfileScratchConfig
• frontend/dockerfile/dockerfile_test.go testExposeExpansion
• frontend/dockerfile/dockerfile_test.go testDockerignore
• frontend/dockerfile/dockerfile_test.go testDockerignoreInvalid
• frontend/dockerfile/dockerfile_test.go testDockerfileLowercase
• frontend/dockerfile/dockerfile_test.go testExportedHistory
• frontend/dockerfile/dockerfile_test.go testUser
• frontend/dockerfile/dockerfile_test.go testUserAdditionalGids
• frontend/dockerfile/dockerfile_test.go testCopyChmod
• frontend/dockerfile/dockerfile_test.go testCopyOverrideFiles
• frontend/dockerfile/dockerfile_test.go testCopyVarSubstitution
• frontend/dockerfile/dockerfile_test.go testCopyWildcards
• frontend/dockerfile/dockerfile_test.go testCopyRelative
• frontend/dockerfile/dockerfile_test.go testAddURLChmod
• frontend/dockerfile/dockerfile_test.go testDockerfileFromGit
• frontend/dockerfile/dockerfile_test.go testDockerfileFromHTTP
• frontend/dockerfile/dockerfile_test.go testMultiStageImplicitFrom
• frontend/dockerfile/dockerfile_test.go testMultiStageCaseInsensitive
• frontend/dockerfile/dockerfile_test.go testLabels
• frontend/dockerfile/dockerfile_test.go testWildcardRenameCache
• frontend/dockerfile/dockerfile_test.go testOnBuildCleared
• frontend/dockerfile/dockerfile_test.go testCacheMultiPlatformImportExport
• frontend/dockerfile/dockerfile_test.go testImageManifestCacheImportExport
• frontend/dockerfile/dockerfile_test.go testCacheImportExport
• frontend/dockerfile/dockerfile_test.go testReproducibleIDs
• frontend/dockerfile/dockerfile_test.go testImportExportReproducibleIDs
• frontend/dockerfile/dockerfile_test.go testNoCache
• frontend/dockerfile/dockerfile_test.go testPlatformArgsImplicit
• frontend/dockerfile/dockerfile_test.go testPlatformArgsExplicit
• frontend/dockerfile/dockerfile_test.go testBuiltinArgs
• frontend/dockerfile/dockerfile_test.go testTarContext
• frontend/dockerfile/dockerfile_test.go testTarContextExternalDockerfile
• frontend/dockerfile/dockerfile_test.go testFrontendUseForwardedSolveResults
• frontend/dockerfile/dockerfile_test.go testFrontendInputs
• frontend/dockerfile/dockerfile_test.go testFrontendSubrequests
• frontend/dockerfile/dockerfile_test.go testDockefileCheckHostname
• frontend/dockerfile/dockerfile_test.go testShmSize
• frontend/dockerfile/dockerfile_test.go testUlimit
• frontend/dockerfile/dockerfile_test.go testCgroupParent
• frontend/dockerfile/dockerfile_test.go testNamedImageContext
• frontend/dockerfile/dockerfile_test.go testNamedImageContextPlatform
• frontend/dockerfile/dockerfile_test.go testNamedImageContextTimestamps
• frontend/dockerfile/dockerfile_test.go testNamedImageContextScratch
• frontend/dockerfile/dockerfile_test.go testNamedLocalContext
• frontend/dockerfile/dockerfile_test.go testNamedOCILayoutContext
• frontend/dockerfile/dockerfile_test.go testNamedOCILayoutContextExport
• frontend/dockerfile/dockerfile_test.go testNamedInputContext
• frontend/dockerfile/dockerfile_test.go testNamedMultiplatformInputContext
• frontend/dockerfile/dockerfile_test.go testNamedFilteredContext
• frontend/dockerfile/dockerfile_test.go testSourceDateEpochWithoutExporter
• frontend/dockerfile/dockerfile_test.go testSBOMScannerImage
• frontend/dockerfile/dockerfile_test.go testSBOMScannerArgs
• frontend/dockerfile/dockerfile_test.go testMultiPlatformWarnings
• frontend/dockerfile/dockerfile_test.go testReproSourceDateEpoch
• frontend/dockerfile/dockerfile_test.go testCopyUnicodePath
• frontend/dockerfile/errors_test.go testErrorsSourceMap
• frontend/frontend_test.go testRefReadFile **
• frontend/frontend_test.go testRefReadDir **
• frontend/frontend_test.go testRefStatFile **
• frontend/frontend_test.go testRefEvaluate **

/solver:

• solver/jobs_test.go testParallelism

Previously Skipped Tests:

• cache/contenthash/checksum_test.go createRef
• cache/manager_test.go TestSnapshotExtract
• cache/manager_test.go TestExtractOnMutable
• cache/manager_test.go TestMergeOp
• cache/manager_test.go TestDiffOp
• source/git/source_test.go testRepeatedFetch
• source/git/source_test.go testFetchBySHA
• source/git/source_test.go testFetchByTag
• source/git/source_test.go testMultipleRepos
• source/git/source_test.go TestCredentialRedaction
• source/git/source_test.go testSubdir
• source/http/source_test.go TestHTTPSource
• source/http/source_test.go TestHTTPChecksum
• source/http/source_test.go TestHTTPDefaultName

Misc. Tasks

• cross comp container/distribution binaries to avoid drifting like freebsd one and most of this logic should be moved to https://github.com/moby/buildkit/blob/master/.github/workflows/.test.yml where we are self-contained and run integration tests through a WCOW container. ref: tests: enable integration test run on windows as baseline #4479 (review)
• adding the --parallel option for gotestsum, re: tests: enabling integration tests on windows #5027 (comment)
• support multiple architecture for code image mirror logic in util/testutil/integration/run_windows.go

[profnandaa]

/cc. @gabriel-samfira

[tonistiigi]

@profnandaa Look at some of these cases as well https://github.com/moby/buildkit/blame/master/cache/manager_test.go#L577 in cache and source packages. Can some of these skips be removed now?

[profnandaa]

@tonistiigi -- sure, let me rake through the previously skipped cases too, I'd missed that.

[profnandaa]

Found 13 tests that were previously skipped, I've added them at the end of the list.