Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ your contribution to Apple and the community, and agree by submitting the patch
that your contributions are licensed under the Apache 2.0 license (see
`LICENSE.txt`).


## How to submit a bug report

Please ensure to specify the following:
Expand All @@ -20,7 +19,6 @@ Please ensure to specify the following:
* OS version and the output of `uname -a`
* Network configuration


### Example

```
Expand Down Expand Up @@ -89,7 +87,7 @@ SwiftNIO has been created to be high performance. The integration tests cover s

### Formatting

Try to keep your lines less than 120 characters long so github can correctly display your changes.
Try to keep your lines less than 120 characters long so GitHub can correctly display your changes.

SwiftNIO uses the [swift-format](https://github.com/swiftlang/swift-format) tool to bring consistency to code formatting. There is a specific [.swift-format](./.swift-format) configuration file. This will be checked and enforced on PRs. Note that the check will run on the current most recent stable version target which may not match that in your own local development environment.

Expand All @@ -106,7 +104,6 @@ act --container-architecture linux/amd64 --action-offline-mode --bind workflow_c

This will run the format checks, binding to your local checkout so the edits made are to your own source.


### Extensibility

Try to make sure your code is robust to future extensions. The public interface is very hard to change after release - please refer to the [API guidelines](./docs/public-api.md)
Expand All @@ -115,4 +112,4 @@ Try to make sure your code is robust to future extensions. The public interface

Please open a pull request at https://github.com/apple/swift-nio. Make sure the CI passes, and then wait for code review.

After review you may be asked to make changes. When you are ready, use the request re-review feature of github or mention the reviewers by name in a comment.
After review you may be asked to make changes. When you are ready, use the request re-review feature of GitHub or mention the reviewers by name in a comment.
2 changes: 1 addition & 1 deletion Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ let swiftAtomics: PackageDescription.Target.Dependency = .product(name: "Atomics
let swiftCollections: PackageDescription.Target.Dependency = .product(name: "DequeModule", package: "swift-collections")
let swiftSystem: PackageDescription.Target.Dependency = .product(name: "SystemPackage", package: "swift-system")

// These platforms require a depdency on `NIOPosix` from `NIOHTTP1` to maintain backward
// These platforms require a dependency on `NIOPosix` from `NIOHTTP1` to maintain backward
// compatibility with previous NIO versions.
let historicalNIOPosixDependencyRequired: [Platform] = [.macOS, .iOS, .tvOS, .watchOS, .linux, .android]

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ SwiftNIO | Minimum Swift Version
`2.43.0 ..< 2.51.0` | 5.5.2
`2.51.0 ..< 2.60.0` | 5.6
`2.60.0 ..< 2.65.0` | 5.7
`2.65.0 ..< 2.76.0 | 5.8
`2.65.0 ..< 2.76.0` | 5.8
`2.76.0 ...` | 5.9

### SwiftNIO 1
Expand Down Expand Up @@ -116,7 +116,7 @@ SemVer and SwiftNIO's Public API guarantees should result in a working program w

SwiftNIO is fundamentally a low-level tool for building high-performance networking applications in Swift. It particularly targets those use-cases where using a "thread-per-connection" model of concurrency is inefficient or untenable. This is a common limitation when building servers that use a large number of relatively low-utilization connections, such as HTTP servers.

To achieve its goals SwiftNIO extensively uses "non-blocking I/O": hence the name! Non-blocking I/O differs from the more common blocking I/O model because the application does not wait for data to be sent to or received from the network: instead, SwiftNIO asks for the kernel to notify it when I/O operations can be performed without waiting.
To achieve its goals, SwiftNIO extensively uses "non-blocking I/O": hence the name! Non-blocking I/O differs from the more common blocking I/O model because the application does not wait for data to be sent to or received from the network: instead, SwiftNIO asks for the kernel to notify it when I/O operations can be performed without waiting.

SwiftNIO does not aim to provide high-level solutions like, for example, web frameworks do. Instead, SwiftNIO is focused on providing the low-level building blocks for these higher-level applications. When it comes to building a web application, most users will not want to use SwiftNIO directly: instead, they'll want to use one of the many great web frameworks available in the Swift ecosystem. Those web frameworks, however, may choose to use SwiftNIO under the covers to provide their networking support.

Expand All @@ -139,7 +139,7 @@ All SwiftNIO applications are ultimately constructed of these various components

#### EventLoops and EventLoopGroups

The basic I/O primitive of SwiftNIO is the event loop. The event loop is an object that waits for events (usually I/O related events, such as "data received") to happen and then fires some kind of callback when they do. In almost all SwiftNIO applications there will be relatively few event loops: usually only one or two per CPU core the application wants to use. Generally speaking event loops run for the entire lifetime of your application, spinning in an endless loop dispatching events.
The basic I/O primitive of SwiftNIO is the event loop. The event loop is an object that waits for events (usually I/O related events, such as "data received") to happen and then fires some kind of callback when they do. In almost all SwiftNIO applications there will be relatively few event loops: usually only one or two per CPU core the application wants to use. Generally speaking, event loops run for the entire lifetime of your application, spinning in an endless loop dispatching events.

Event loops are gathered together into event loop *groups*. These groups provide a mechanism to distribute work around the event loops. For example, when listening for inbound connections the listening socket will be registered on one event loop. However, we don't want all connections that are accepted on that listening socket to be registered with the same event loop, as that would potentially overload one event loop while leaving the others empty. For that reason, the event loop group provides the ability to spread load across multiple event loops.

Expand Down
51 changes: 26 additions & 25 deletions Sources/NIOFileSystem/CopyStrategy.swift
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@
//===----------------------------------------------------------------------===//

/// How to perform copies. Currently only relevant to directory level copies when using
/// ``FileSystemProtocol/copyItem(at:to:strategy:shouldProceedAfterError:shouldCopyItem:)``
/// or other overloads that use the default behaviour
/// ``FileSystemProtocol/copyItem(at:to:strategy:shouldProceedAfterError:shouldCopyItem:)`` or other
/// overloads that use the default behaviour.
public struct CopyStrategy: Hashable, Sendable {
// Avoid exposing to prevent alterations being breaking changes
// Avoid exposing to prevent breaking changes
internal enum Wrapped: Hashable, Sendable {
// platformDefault is reified into one of the concrete options below:

case sequential
// Constraints on this value are enforced only on making `CopyStrategy`,
// the early error check there is desirable over validating on downstream use.
// Constraints on this value are enforced only on creation of `CopyStrategy`. The early
// error check is desirable over validating on downstream use.
case parallel(_ maxDescriptors: Int)
}

Expand All @@ -33,55 +33,56 @@ public struct CopyStrategy: Hashable, Sendable {

// These selections are relatively arbitrary but the rationale is as follows:
//
// - Never exceed the default OS limits even if 4 such operations
// were happening at once
// - Never exceed the default OS limits even if 4 such operations were happening at once.
// - Sufficient to enable significant speed up from parallelism
// - Not wasting effort by pushing contention to the underlying storage device
// Further we assume an SSD or similar underlying storage tech.
// Users on spinning rust need to account for that themselves anyway
// - Not wasting effort by pushing contention to the underlying storage device. Further we
// assume an SSD or similar underlying storage tech. Users on spinning rust need to account
// for that themselves anyway.
//
// That said, empirical testing for this has not been performed, suggestions welcome
// That said, empirical testing for this has not been performed, suggestions welcome.
//
// Note: for now we model the directory scan as needing two handles because, during the creation
// of the destination directory we hold the handle for a while copying attributes
// a much more complex internal state machine could allow doing two of these if desired
// This may not result in a faster copy though so things are left simple
internal static func determinePlatformDefault() -> Wrapped {
#if os(macOS) || os(Linux) || os(Windows)
// 4 concurrent file copies/directory scans.
// Avoiding storage system contention is the dominant aspect here.
// Empirical testing on an SSD copying to the same volume with a dense directory of small
// files and sub directories of similar shape totalling 12GB showed improvements in elapsed
// time for (expected) increases in CPU time up to parallel(8), beyond this the increases
// in CPU came with only moderate gains.
// 4 concurrent file copies/directory scans. Avoiding storage system contention is of utmost
// importance.
//
// Testing was performed on an SSD, while copying objects (a dense directory of small files
// and subdirectories of similar shape) to the same volume, totalling 12GB. Results showed
// improvements in elapsed time for (expected) increases in CPU time up to parallel(8).
// Beyond this, the increases in CPU led to only moderate gains.
//
// Anyone tuning this is encouraged to cover worst case scenarios.
return .parallel(8)
#elseif os(iOS) || os(tvOS) || os(watchOS) || os(Android)
// Reduced maximum descriptors in embedded world
// This is chosen based on biasing to safety, not empirical testing.
// Reduced maximum descriptors in embedded world. This is chosen based on biasing towards
// safety, not empirical testing.
return .parallel(4)
#else
// Safety first, if we have no view on it keep it simple.
// Safety first. If we do not know what system we run on, we keep it simple.
return .sequential
#endif
}
}

extension CopyStrategy {
// A copy fundamentally can't work without two descriptors unless you copy
// everything into memory which is infeasible/inefficeint for large copies.
// everything into memory which is infeasible/inefficient for large copies.
private static let minDescriptorsAllowed = 2

/// Operate in whatever manner is deemed a reasonable default for the platform.
/// This will limit the maximum file descriptors usage based on 'reasonable' defaults.
/// Operate in whatever manner is deemed a reasonable default for the platform. This will limit
/// the maximum file descriptors usage based on reasonable defaults.
///
/// Current assumptions (which are subject to change):
/// - Only one copy operation would be performed at once
/// - The copy operation is not intended to be the primary activity on the device
public static let platformDefault: Self = Self(Self.determinePlatformDefault())

/// The copy is done asynchronously, but only one operation will occur at a time.
/// This is the only way to guarantee only one callback to the `shouldCopyItem` will happen at a time
/// The copy is done asynchronously, but only one operation will occur at a time. This is the
/// only way to guarantee only one callback to the `shouldCopyItem` will happen at a time.
public static let sequential: Self = Self(.sequential)

/// Allow multiple IO operations to run concurrently, including file copies/directory creation and scanning
Expand Down
Loading