Repository navigation
fix(serialization): cap codec allocations - #10340
Conversation
There was a problem hiding this comment.
Pull request overview
This PR hardens Orleans serialization deserialization against corrupted length prefixes by introducing a shared “remaining bytes” concept on Reader and using it to validate length-prefixed allocations (strings, buffers, arrays/collections, and protobuf payloads), while also ensuring non-seekable streams don’t break these checks.
Changes:
- Add
Reader.RemainingplusReader.EnsureAvailable(uint)and apply it to length-prefixed reads/allocation sites. - Update many collection/array codecs and protobuf codecs to validate declared lengths against remaining input before allocating.
- Add/extend unit tests to cover remaining-bytes behavior, forked readers, malformed lengths, and non-seekable streams.
Show a summary per file
| File | Description |
|---|---|
| test/Orleans.Serialization.UnitTests/ReaderWriterTests.cs | Adds unit tests for Remaining, forked-reader semantics, malformed length rejection, and non-seekable stream behavior. |
| src/Serializers/Orleans.Serialization.Protobuf/RepeatedFieldCodec.cs | Switches repeated-field allocation guard to reader.EnsureAvailable(...). |
| src/Serializers/Orleans.Serialization.Protobuf/ProtobufCodec.cs | Validates protobuf payload length against remaining input before reading/allocating. |
| src/Serializers/Orleans.Serialization.Protobuf/MapFieldCodec.cs | Switches map-field allocation guard to reader.EnsureAvailable(...). |
| src/Orleans.Serialization/Codecs/StringCodec.cs | Ensures string byte-length is available and within supported bounds before reading/allocating. |
| src/Orleans.Serialization/Codecs/StackCodec.cs | Switches stack length guard to reader.EnsureAvailable(...) before allocating backing array. |
| src/Orleans.Serialization/Codecs/QueueCodec.cs | Adds EnsureAvailable guard before allocating queue capacity. |
| src/Orleans.Serialization/Codecs/MultiDimensionalArrayCodec.cs | Adds a remaining-input-based sanity check for declared multi-dimensional array sizes. |
| src/Orleans.Serialization/Codecs/ListCodec.cs | Switches list length guards to reader.EnsureAvailable(...) before allocating/ensuring capacity. |
| src/Orleans.Serialization/Codecs/HashSetCodec.cs | Switches set length guards to reader.EnsureAvailable(...) before allocating/rehydrating. |
| src/Orleans.Serialization/Codecs/DictionaryCodec.cs | Switches dictionary length guards to reader.EnsureAvailable(...) before allocating/rehydrating. |
| src/Orleans.Serialization/Codecs/CollectionCodec.cs | Switches collection length guards to reader.EnsureAvailable(...) before allocating/rehydrating. |
| src/Orleans.Serialization/Codecs/ByteArrayCodec.cs | Adds remaining-input validation for BitArray and PooledBuffer payload lengths. |
| src/Orleans.Serialization/Codecs/ArrayCodec.cs | Switches array/memory/segment length guards to reader.EnsureAvailable(...) before allocating. |
| src/Orleans.Serialization/Buffers/Reader.cs | Introduces Remaining and EnsureAvailable, applies it to byte reads, and updates stream handling for non-seekable inputs. |
| src/api/Orleans.Serialization/Orleans.Serialization.cs | Updates public API surface to include Reader.Remaining and Reader.EnsureAvailable(uint). |
Copilot's findings
- Files reviewed: 16/16 changed files
- Comments generated: 2
There was a problem hiding this comment.
Copilot's findings
Suppressed comments (17)
src/Orleans.Serialization/Buffers/Reader.cs:944
ReadBytes(uint count)can be passed a declared length >int.MaxValue(e.g., from corrupted length prefixes). In that case it will attemptnew byte[count]and later castcounttoint, which can overflow/throw or try to allocate an enormous array—especially for non-seekable streams whereEnsureAvailablecannot bound the remaining input. Add an explicit maximum-length check and allocate using anintlength.
EnsureAvailable(count);
var bytes = new byte[count];
if (IsReadOnlySequenceInput || IsSpanInput || IsBufferSliceInput || IsArcBufferInput)
{
src/Orleans.Serialization/Codecs/ListCodec.cs:140
- Same issue as above:
(int)UInt32Codec.ReadValue(...)can overflow for encoded lengths >int.MaxValue, and the subsequent(uint)lengthwrap undermines the guard. ValidateencodedLengthbefore casting and before using it for capacity/allocations.
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
src/Orleans.Serialization/Codecs/ArrayCodec.cs:236
- The declared length is decoded as
uintbut cast directly toint. If the value is >int.MaxValue, the cast overflows and can lead to invalid allocations or exceptions. Add an explicit max-length check before casting and allocating.
length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
result = new T[length];
src/Orleans.Serialization/Codecs/ArrayCodec.cs:395
- The length prefix is cast from
uinttointwithout checking> int.MaxValue. Overflow here can produce a negative length and/or huge allocation attempts. Validate the encoded length before casting and allocating.
length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
result = new T[length];
ReferenceCodec.RecordObject(reader.Session, result, placeholderReferenceId);
src/Orleans.Serialization/Codecs/ArrayCodec.cs:556
- The length prefix is cast from
uinttointwithout checking> int.MaxValue. Malformed values can overflow and cause invalid/huge allocation attempts. Validate the encoded value before casting and allocating.
length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
result = new T[length];
ReferenceCodec.RecordObject(reader.Session, result, placeholderReferenceId);
src/Orleans.Serialization/Codecs/DictionaryCodec.cs:295
- Same overflow risk here: the encoded length is cast to
intwithout bounds checking, then used for capacity/constructor arguments. Validate theuintlength before casting to avoid negative/overflowed lengths and huge allocation attempts.
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
src/Orleans.Serialization/Codecs/HashSetCodec.cs:160
- Same overflow risk in the reinitialization path: casting the encoded
uintlength tointwithout validating> int.MaxValuecan overflow and lead to invalid capacity/constructor arguments. Validate before casting.
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
src/Orleans.Serialization/Codecs/CollectionCodec.cs:143
- Same overflow risk in the deserialize path: the encoded
uintlength is cast tointwithout validating> int.MaxValue. Validate before casting to avoid negative/overflowed lengths affecting control flow.
case 0:
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
break;
src/Orleans.Serialization/Codecs/StackCodec.cs:89
- The stack length is decoded as
uintand cast directly toint. If the encoded value is >int.MaxValue, the cast overflows and can lead to negative lengths and invalid allocations. Validate the encoded length before casting and allocating the backing array.
case 0:
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
array = new T[length];
i = length - 1;
break;
src/Orleans.Serialization/Codecs/ListCodec.cs:76
- The length prefix is read as
uintand immediately cast toint. Values >int.MaxValuewill overflow to a negativelength, and the subsequent(uint)lengthcan wrap, potentially bypassing the intended allocation guard or leading to confusing exceptions. Validate the encoded value before casting.
This issue also appears on line 139 of the same file.
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
src/Orleans.Serialization/Codecs/ArrayCodec.cs:85
- The array length is read as
uintthen cast tointwithout bounds checking. Encoded values >int.MaxValuewill overflow and can result in attempted huge allocations or inconsistent exceptions. Validate the encoded length before casting/allocating.
This issue also appears in the following locations of the same file:
- line 233
- line 391
- line 552
length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
result = new T[length];
src/Orleans.Serialization/Codecs/DictionaryCodec.cs:109
- The declared dictionary length is read as
uintthen cast tointwithout checking> int.MaxValue. Overflow can produce a negative length and lead to inconsistent exceptions or attempted huge allocations. Validate the encoded value before casting.
This issue also appears on line 293 of the same file.
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
src/Orleans.Serialization/Codecs/HashSetCodec.cs:89
- The declared set length is cast from
uinttointwithout checking> int.MaxValue. Encoded values aboveint.MaxValueoverflow and can cause negative lengths or large allocation attempts. Validate the encoded length before casting and using it for capacity/allocations.
This issue also appears on line 158 of the same file.
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
result = CreateInstance(length, comparer, reader.Session, placeholderReferenceId);
src/Orleans.Serialization/Codecs/CollectionCodec.cs:80
- The declared collection length is cast from
uinttointwithout checking> int.MaxValue. Overflow can produce a negativelengthand inconsistent exceptions or attempted huge allocations (e.g.,new List<T>(length)). Validate the encoded value before casting.
This issue also appears on line 139 of the same file.
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
result = new Collection<T>(new List<T>(length));
ReferenceCodec.RecordObject(reader.Session, result, placeholderReferenceId);
src/Orleans.Serialization/Codecs/QueueCodec.cs:82
- The queue length is cast from
uinttointwithout checking> int.MaxValue. Encoded values aboveint.MaxValueoverflow and can lead to negative lengths or large allocation attempts. Validate the encoded value before casting and using it for capacity.
case 0:
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
result = new Queue<T>(length);
ReferenceCodec.RecordObject(reader.Session, result, placeholderReferenceId);
src/Serializers/Orleans.Serialization.Protobuf/RepeatedFieldCodec.cs:88
- The RepeatedField length prefix is decoded as
uintbut cast directly toint. Values >int.MaxValueoverflow to a negativelength, which can result in invalid capacity settings or inconsistent exceptions. Validate the encoded length before casting.
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
result = new RepeatedField<T>{ Capacity = length };
ReferenceCodec.RecordObject(reader.Session, result, placeholderReferenceId);
src/Serializers/Orleans.Serialization.Protobuf/MapFieldCodec.cs:97
- The MapField length prefix is decoded as
uintbut cast directly toint. Values >int.MaxValueoverflow and can lead to invalid allocations/capacity behavior. Validate the encoded value before casting.
var length = (int)UInt32Codec.ReadValue(ref reader, header);
reader.EnsureAvailable((uint)length);
result = CreateInstance(reader.Session, placeholderReferenceId);
break;
- Files reviewed: 16/16 changed files
- Comments generated: 0 new
Validate declared lengths against the remaining serialized payload before allocating strings, buffers, arrays, and collections. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc2895a9-c8ad-41f4-bd7a-cffe1e35e5b0
Replace checked length casts with explicit validation and actionable exceptions before buffer allocation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc2895a9-c8ad-41f4-bd7a-cffe1e35e5b0
031c907 to
5def074
Compare
There was a problem hiding this comment.
Copilot's findings
Suppressed comments (3)
src/Orleans.Serialization/Buffers/Reader.cs:483
EnsureAvailablerelies onRemaining, which returnslong.MaxValuefor non-seekable streams, so this method will not necessarily throw even when fewer thancountbytes are actually available. The doc comment should reflect that it only enforces the check when remaining length can be determined.
/// <summary>
/// Throws if fewer than <paramref name="count"/> bytes remain in the input.
/// </summary>
/// <param name="count">The number of bytes which are required.</param>
src/Orleans.Serialization/Buffers/Reader.cs:944
ReadBytes(uint count)allocates usingnew byte[count]without guarding againstcount > int.MaxValue. For corrupted length prefixes on inputs whereRemainingis very large (or unknown), this can result in runtimeOverflowException/OutOfMemoryExceptioninstead of a controlledIndexOutOfRangeException(and may attempt a huge allocation). Consider explicitly rejecting lengths aboveint.MaxValueand allocating using anintlength.
EnsureAvailable(count);
var bytes = new byte[count];
if (IsReadOnlySequenceInput || IsSpanInput || IsBufferSliceInput || IsArcBufferInput)
{
src/Orleans.Serialization/Buffers/Reader.cs:445
- The XML doc for
Reader<TInput>.Remainingsays it returns the number of unread bytes, but for non-seekable streamsStreamReaderInput.Remainingreturnslong.MaxValue. The doc should clarify the "unknown" case to avoid misleading API consumers.
This issue also appears on line 480 of the same file.
/// <summary>
/// Gets the number of unread bytes in the input.
/// </summary>
public long Remaining
- Files reviewed: 16/16 changed files
- Comments generated: 0 new
Corrupted length prefixes could cause codecs to allocate objects larger than the remaining serialized payload.
Add a shared remaining-byte check to the reader and apply it before allocating strings, buffers, arrays, collections, and protobuf payloads. Keep exception construction off hot paths and cover malformed lengths, forked readers, and non-seekable streams.
Microsoft Reviewers: Open in CodeFlow