Skip to content

Enforce RFC 6455 (§5.1) frame-masking rules in NIOWebSocket#3653

Open
rafaelcepeda wants to merge 1 commit into
apple:mainfrom
rafaelcepeda:rc-websocket-masking-enforcement
Open

Enforce RFC 6455 (§5.1) frame-masking rules in NIOWebSocket#3653
rafaelcepeda wants to merge 1 commit into
apple:mainfrom
rafaelcepeda:rc-websocket-masking-enforcement

Conversation

@rafaelcepeda

Copy link
Copy Markdown
Contributor

Motivation:

WebSocketFrameDecoder is direction-agnostic: it only tracks maxFrameSize and never inspects the mask bit. validateState() checks frame length, fragmented control frames, and control-frame length, but nothing about masking.

RFC 6455 (§5.1) is directional: a client must mask every frame it sends to a server, and a server must not mask frames it sends to a client. Because the decoder never enforced this, a server would silently accept unmasked client frames (and a client would accept masked server frames), rather than failing the connection as the spec requires. The masking requirement exists to prevent crafted frames from being misinterpreted by caching intermediaries, so this is an RFC-compliance gap.

Modifications:

  • Added WebSocketMaskingVerification (.disabled / .serverExpectsMaskedFrames / .clientExpectsUnmaskedFrames) and a maskingVerification parameter on WebSocketFrameDecoder. It defaults to .disabled, so direct decoder users see no behavioural change.
  • Enforced the selected rule in WSParser.validateState, rejecting the offending frame before its body is buffered. Two new NIOWebSocketError cases — .maskedFrame and .unmaskedFrame — surface the violation and map to the existing 1002 (protocol error) close code.
  • Enabled enforcement by default in the server and client upgraders (NIOWebSocketServerUpgrader, NIOTypedWebSocketServerUpgrader, NIOWebSocketClientUpgrader, NIOTypedWebSocketClientUpgrader) via a new enforceMaskingRules: Bool = true parameter, which can be set to false for peers known to be non-compliant.
  • Added decoder tests covering both directions (reject/accept) plus a default no-enforcement test, and opted the raw-client end-to-end test out of enforcement since it drives the server with an intentionally unmasked client.

Result:

Servers and clients configured through the WebSocket upgraders now reject frames that violate RFC 6455's masking rules by default, closing the connection with a 1002 protocol-error close frame.

This is a behavioural change for the upgrader paths: a peer that previously accepted non-compliant (un)masked frames will now close those connections. Users who need the old behaviour can pass enforceMaskingRules: false.
The raw WebSocketFrameDecoder default (.disabled) is unchanged, so direct decoder users are unaffected. The change is API-additive (no source break).

@rafaelcepeda rafaelcepeda added the 🆕 semver/minor Adds new public API. label Jul 9, 2026
WebSocketFrameDecoder was direction-agnostic and never inspected the mask
bit, so a server would silently accept unmasked client frames (and a client
would accept masked server frames), violating RFC 6455 (§5.1).

- Add WebSocketMaskingVerification (disabled / serverExpectsMaskedFrames /
  clientExpectsUnmaskedFrames) and enforce the selected rule in
  WSParser.validateState, rejecting the frame before its body is buffered.
- Surface violations via a dedicated NIOWebSocketMaskingError (maskedFrame /
  unmaskedFrame) rather than new NIOWebSocketError cases, keeping the change
  purely additive. WebSocketProtocolErrorHandler maps it to the 1002
  protocol-error close code.
- Enable enforcement by default in the server and client upgraders via an
  enforceMaskingRules opt-out, added as new initializer overloads so the
  existing initializers keep their signatures (no API breakage).
- WebSocketFrameDecoder gains a maskingVerification initializer overload,
  defaulting behaviour to no enforcement for direct decoder users.
- Add decoder tests for both directions plus a default no-enforcement test;
  opt the raw-client end-to-end test out of enforcement.

Verified: 'swift package diagnose-api-breaking-changes main' reports no
breaking changes in NIOWebSocket.
@rafaelcepeda rafaelcepeda force-pushed the rc-websocket-masking-enforcement branch from 1c13339 to c25cb91 Compare July 9, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🆕 semver/minor Adds new public API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant