-
Notifications
You must be signed in to change notification settings - Fork 524
CFP AVAD: Adds Container IsFeedRangePartOfAsync support for FeedRange #4566
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CFP AVAD: Adds Container IsFeedRangePartOfAsync support for FeedRange #4566
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All good!
…titionKey and FeedRange.
…MSFT/add_feedranges_to_avad_contract
NaluTripician
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Microsoft.Azure.Cosmos.Encryption.Custom/src/EncryptionContainer.cs
Outdated
Show resolved
Hide resolved
FabianMeiswinkel
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM except for the two NITs
4ebbad9
Pull Request Template
Description
Details #4483
The customer wants to validate if one FeedRange is a subset of another FeedRange. This means checking if all the data in the y FeedRange is entirely within the boundaries of the x FeedRange, ensuring that no data is processed twice or skipped. It is understood that FeedRange could also be represented by a PartitionKey or FeedRangeEpk.
Feature: PartitionKey and FeedRange Validation Scenario: Validate if the y (FeedRange) is part of the x (PartitionKey) Given x represents the PartitionKey And y represents the FeedRange And both x and y have inclusive minimum and maximum boundaries (x.IsMinInclusive=true, x.IsMaxInclusive=true, y.IsMinInclusive=true, y.IsMaxInclusive=true) Then the scenario is valid, and both x and y include their start and end values Scenario: Validate NotSupportedException for incompatible boundaries between x (PartitionKey) and y (FeedRange) Given x represents the PartitionKey And y represents the FeedRange And x has an inclusive maximum boundary (x.IsMaxInclusive=true) And y has an exclusive maximum boundary (y.IsMaxInclusive=false) When the boundaries are compared Then a NotSupportedException is thrown due to incompatible boundary configurations Scenario: Validate minimum boundary inclusiveness for y (FeedRange) when compared to x (PartitionKey) Given x represents the PartitionKey And y represents the FeedRange And y has an exclusive minimum boundary (y.IsMinInclusive=false) And x has an inclusive minimum boundary (x.IsMinInclusive=true) When the boundaries are compared Then an error is thrown because y's minimum value must be inclusive (IsMinInclusive must be true) Scenario: Validate that both y's (FeedRange) minimum and maximum are exclusive when compared to x (PartitionKey) Given x represents the PartitionKey And y represents the FeedRange And both y's minimum and maximum values are exclusive (y.IsMinInclusive=false, y.IsMaxInclusive=false) When the boundaries are compared Then an error is thrown because y's minimum value must be inclusive (IsMinInclusive must be true)Feature: FeedRange and PartitionKey Validation Scenario: Validate that both x (FeedRange) and y (PartitionKey) are fully inclusive Given x represents the FeedRange And y represents the PartitionKey And x has inclusive minimum and maximum boundaries (x.IsMinInclusive=true, x.IsMaxInclusive=true) And y has inclusive minimum and maximum boundaries (y.IsMinInclusive=true, y.IsMaxInclusive=true) Then the scenario is valid because both x and y include their start and end values Scenario: Validate that x (FeedRange) has inclusive minimum and exclusive maximum while y (PartitionKey) is fully inclusive Given x represents the FeedRange And y represents the PartitionKey And x has inclusive minimum and exclusive maximum boundaries (x.IsMinInclusive=true, x.IsMaxInclusive=false) And y has inclusive minimum and maximum boundaries (y.IsMinInclusive=true, y.IsMaxInclusive=true) Then the scenario is valid because x and y are compatible in terms of their boundaries Scenario: Validate that x's (FeedRange) minimum is exclusive while y (PartitionKey) is fully inclusive Given x represents the FeedRange And y represents the PartitionKey And x has an exclusive minimum and inclusive maximum boundary (x.IsMinInclusive=false, x.IsMaxInclusive=true) And y has inclusive minimum and maximum boundaries (y.IsMinInclusive=true, y.IsMaxInclusive=true) Then the scenario is invalid because x's minimum value must be inclusive (IsMinInclusive must be true) Scenario: Validate that both x's (FeedRange) minimum and maximum are exclusive while y (PartitionKey) is fully inclusive Given x represents the FeedRange And y represents the PartitionKey And x has exclusive minimum and maximum boundaries (x.IsMinInclusive=false, x.IsMaxInclusive=false) And y has inclusive minimum and maximum boundaries (y.IsMinInclusive=true, y.IsMaxInclusive=true) Then the scenario is invalid because x's minimum value must be inclusive (IsMinInclusive must be true)Feature: FeedRange and FeedRange Validation Scenario: Validate when both x (FeedRange) and y (FeedRange) have inclusive minimum and maximum boundaries Given x represents the FeedRange And y represents the FeedRange And both x and y have inclusive minimum and maximum boundaries (x.IsMinInclusive=true, x.IsMaxInclusive=true, y.IsMinInclusive=true, y.IsMaxInclusive=true) Then the scenario is valid because both x and y include their start and end values Scenario: Validate when x (FeedRange) has inclusive boundaries and y (FeedRange) has an exclusive maximum boundary Given x represents the FeedRange And y represents the FeedRange And x has inclusive minimum and maximum boundaries (x.IsMinInclusive=true, x.IsMaxInclusive=true) And y has an exclusive maximum boundary (y.IsMinInclusive=true, y.IsMaxInclusive=false) Then a NotSupportedException is thrown because y's maximum boundary is exclusive while x's is inclusive Scenario: Validate when x (FeedRange) has inclusive boundaries and y (FeedRange) has an exclusive minimum boundary Given x represents the FeedRange And y represents the FeedRange And x has inclusive minimum and maximum boundaries (x.IsMinInclusive=true, x.IsMaxInclusive=true) And y has an exclusive minimum boundary (y.IsMinInclusive=false, y.IsMaxInclusive=true) Then the scenario is invalid because y's minimum boundary must be inclusive Scenario: Validate when x (FeedRange) has inclusive boundaries and y (FeedRange) has both exclusive minimum and maximum boundaries Given x represents the FeedRange And y represents the FeedRange And x has inclusive minimum and maximum boundaries (x.IsMinInclusive=true, x.IsMaxInclusive=true) And y has both exclusive minimum and maximum boundaries (y.IsMinInclusive=false, y.IsMaxInclusive=false) Then the scenario is invalid because y's minimum boundary must be inclusive Scenario: Validate when x (FeedRange) has an inclusive minimum boundary and an exclusive maximum boundary, and y (FeedRange) has fully inclusive boundaries Given x represents the FeedRange And y represents the FeedRange And x has an inclusive minimum and exclusive maximum boundary (x.IsMinInclusive=true, x.IsMaxInclusive=false) And y has fully inclusive boundaries (y.IsMinInclusive=true, y.IsMaxInclusive=true) Then the scenario is valid because x's and y's boundaries are compatible Scenario: Validate when both x (FeedRange) and y (FeedRange) have inclusive minimum boundaries and exclusive maximum boundaries Given x represents the FeedRange And y represents the FeedRange And both x and y have inclusive minimum boundaries and exclusive maximum boundaries (x.IsMinInclusive=true, x.IsMaxInclusive=false, y.IsMinInclusive=true, y.IsMaxInclusive=false) Then the scenario is valid because both ranges are compatible Scenario: Validate when x (FeedRange) has an inclusive minimum boundary and an exclusive maximum boundary, and y (FeedRange) has an exclusive minimum boundary Given x represents the FeedRange And y represents the FeedRange And x has an inclusive minimum boundary and an exclusive maximum boundary (x.IsMinInclusive=true, x.IsMaxInclusive=false) And y has an exclusive minimum boundary but an inclusive maximum boundary (y.IsMinInclusive=false, y.IsMaxInclusive=true) Then the scenario is invalid because y's minimum boundary must be inclusive Scenario: Validate when x (FeedRange) has an inclusive minimum boundary and an exclusive maximum boundary, and y (FeedRange) has both exclusive minimum and maximum boundaries Given x represents the FeedRange And y represents the FeedRange And x has an inclusive minimum boundary and an exclusive maximum boundary (x.IsMinInclusive=true, x.IsMaxInclusive=false) And y has both exclusive minimum and maximum boundaries (y.IsMinInclusive=false, y.IsMaxInclusive=false) Then the scenario is invalid because y's minimum boundary must be inclusive Scenario: Validate when x (FeedRange) has an exclusive minimum boundary and an inclusive maximum boundary, and y (FeedRange) has fully inclusive boundaries Given x represents the FeedRange And y represents the FeedRange And x has an exclusive minimum boundary and an inclusive maximum boundary (x.IsMinInclusive=false, x.IsMaxInclusive=true) And y has fully inclusive boundaries (y.IsMinInclusive=true, y.IsMaxInclusive=true) Then the scenario is invalid because x's minimum boundary must be inclusive Scenario: Validate when x (FeedRange) has an exclusive minimum boundary and an inclusive maximum boundary, and y (FeedRange) has an exclusive maximum boundary Given x represents the FeedRange And y represents the FeedRange And x has an exclusive minimum boundary and an inclusive maximum boundary (x.IsMinInclusive=false, x.IsMaxInclusive=true) And y has an inclusive minimum but an exclusive maximum boundary (y.IsMinInclusive=true, y.IsMaxInclusive=false) Then the scenario is invalid because x's minimum boundary must be inclusive Scenario: Validate when both x (FeedRange) and y (FeedRange) have exclusive minimum boundaries and inclusive maximum boundaries Given x represents the FeedRange And y represents the FeedRange And both x and y have exclusive minimum boundaries and inclusive maximum boundaries (x.IsMinInclusive=false, x.IsMaxInclusive=true, y.IsMinInclusive=false, y.IsMaxInclusive=true) Then the scenario is invalid because both x's and y's minimum boundaries must be inclusive Scenario: Validate when both x (FeedRange) and y (FeedRange) have exclusive minimum and maximum boundaries Given x represents the FeedRange And y represents the FeedRange And both x and y have exclusive minimum and maximum boundaries (x.IsMinInclusive=false, x.IsMaxInclusive=false, y.IsMinInclusive=false, y.IsMaxInclusive=false) Then the scenario is invalid because both x's and y's minimum boundaries must be inclusive Scenario: Validate when x (FeedRange) has exclusive minimum and maximum boundaries, and y (FeedRange) has fully inclusive boundaries Given x represents the FeedRange And y represents the FeedRange And x has exclusive minimum and maximum boundaries (x.IsMinInclusive=false, x.IsMaxInclusive=false) And y has fully inclusive boundaries (y.IsMinInclusive=true, y.IsMaxInclusive=true) Then the scenario is invalid because x's minimum boundary must be inclusiveAPI
Levelsetting
Container
Diagram 1: Final state of PR
classDiagram class Container { <<abstract>> +IsFeedRangePartOfAsync() : Task<bool> } class ContainerInternal { +IsFeedRangePartOfAsync() : Task<bool> } class ContainerCore { -container : Container +IsFeedRangePartOfAsync() : Task<bool> } class ContainerInlineCore { -container : Container +IsFeedRangePartOfAsync() : Task<bool> } class EncryptionContainer { -container : Container +IsFeedRangePartOfAsync() : Task<bool> } class CustomEncryptionContainer { -container : Container +IsFeedRangePartOfAsync() : Task<bool> } ContainerInternal --|> Container : "is a" ContainerCore --|> ContainerInternal : "is a" ContainerInlineCore --|> ContainerCore : "is a" EncryptionContainer --|> Container : "is a" CustomEncryptionContainer --|> Container : "is a" ContainerCore --> Container : "has a" ContainerInlineCore --> Container : "has a" EncryptionContainer --> Container : "has a" CustomEncryptionContainer --> Container : "has a"FeedRange
classDiagram class FeedRange { <<abstract>> } class FeedRangeInternal { -range : Range } class FeedRangeEpk { -range : Range } class FeedRangePartitionKey { -range : Range } class FeedRangePartitionKeyRange { -range : Range } class Range { +Min : string +Max : string +IsMinInclusive : bool +IsMaxInclusive : bool } FeedRangeInternal --|> FeedRange : "is a" FeedRangeEpk --|> FeedRangeInternal : "is a" FeedRangePartitionKey --|> FeedRangeInternal : "is a" FeedRangePartitionKeyRange --|> FeedRangeInternal : "is a" FeedRangeInternal --> Range : "has a" FeedRangePartitionKeyRange --> Range : "has a" FeedRangePartitionKey --> Range : "has a" FeedRangeEpk --> Range : "has a"https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.cosmos.feedrange?view=azure-dotnet
A FeedRange in Azure Cosmos DB represents a segment of data, often based on partition keys, that can be processed or monitored. FeedRanges are typically used in the context of change feed processing to allow multiple workers to consume data in parallel by splitting the container’s data into smaller, manageable pieces. Each FeedRange corresponds to a specific partition key range.
Min: Lower boundary of a range
Max Upper boundary of a range
IsMinInclusive: Is the lower boundary inclusive or exclusive?
IsMaxInclusive: Is the upper boundary inclusive or exclusive?
"MaxExclusive": The upper boundary of a range is exclusive; the range does not include the maximum value itself.
Examples based on unit tests
FeedRange
A FeedRange represents a segment of data that can be processed or read in parallel. FeedRanges can be used with or without partition keys, depending on the context. They are especially useful in change feed processing, where multiple workers can process different ranges in parallel.
Hierarchical PartitionKey
In hierarchical partitioning, the partition key is composed of multiple levels (or paths), which can be combined in a hierarchical manner. This means that data is partitioned based on a combination of keys, such as {"region": "US", "userId": "123"}, which makes the partitioning more granular and allows for more specific partition key ranges.
Partition Key
In non-hierarchical partitioning, the partition key is a single value (or path). For instance, a partition key might be {"userId": "123"}. Data is distributed based solely on this one-dimensional key.
Invalid FeedRangeEpk with IsMinInclusive: false throws ArgumentOutOfRangeException
FeedRangeEpk is a more specific form of FeedRange that focuses on effective partition keys (EPK), which represent the internal structure of how data is distributed across partitions in Azure Cosmos DB.
FeedRangeEpk targets specific physical partitions, and these boundaries are managed internally by Cosmos DB. These boundaries are fixed based on the hash values of the partition keys and are not modifiable by the user. As a result, there is no need for the notion of inclusivity, because the partition boundary is rigid and defined by the physical partition structure, not by a logical partition key.
EnsureConsistentInclusitivy
Validates if all ranges in the list have consistent inclusivity for both IsMinInclusive and IsMaxInclusive boundaries.
Throws an InvalidOperationException if there are inconsistencies.
IsSubset
Determines whether the y range is a subset of the x range.
Type of change
Please delete options that are not relevant.
Closing issues
To automatically close an issue: closes #4483