Skip to content

Ensure ActiveIssue attribute is only used with issue links #32535

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stephentoub merged 2 commits into from
Feb 19, 2020
Merged

Ensure ActiveIssue attribute is only used with issue links #32535

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ac38d8d
Ensure ActiveIssue attribute is only used with issue links
akoeplinger Feb 19, 2020
ff4a41b
PR feedback
akoeplinger Feb 19, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/pr-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ In case CI indicates failures which are **highly unlikely** to be caused by chan
* An existing issue in the repository should be searched for. Usually the test method's or the test assembly's name (in case of a crash) are good parameters.
* If there's an existing issue, a comment should be placed that includes a) the link to the build, b) the affected configuration (ie `netcoreapp-Windows_NT-Release-x64-Windows.81.Amd64.Open`) and c) the Error message and Stack trace. This is necessary as retention policies are in place that recycle _old_ builds. In case the issue is already closed, it should be reopened and labels should be updated to reflect the current failure state.
* If there's no existing issue, an issue should be created with the same information outlined above.
* In a follow-up Pull Request, the failing test(s) should be disabled with the corresponding issue number, e.g. `[ActiveIssue(x)]`, and the tracking issue should be labeled as `disabled-test`.
* In a follow-up Pull Request, the failing test(s) should be disabled with the corresponding issue link, e.g. `[ActiveIssue(x)]`, and the tracking issue should be labeled as `disabled-test`.
* A comment should be placed in the original Pull Request that links to the created or updated issues.

There are plenty of possible bugs, e.g. race conditions, where a failure might highlight a real problem and it won't manifest again on a retry. Therefore these steps should be followed for every iteration of the PR build, e.g. before retrying/rebuilding.
Expand Down
10 changes: 5 additions & 5 deletions docs/workflow/testing/libraries/filtering-tests.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,22 +44,22 @@ This attribute returns the 'failing' category, which is disabled by default.

**Disable for all platforms and all target frameworks:**
```cs
[ActiveIssue(int issue)]
[ActiveIssue(string issue)]
```
**Disable for specific platform:**
```cs
[ActiveIssue(int issue, TestPlatforms platforms)]
[ActiveIssue(string issue, TestPlatforms platforms)]
```
**Disable for specific runtime:**
```cs
[ActiveIssue(string issue, TestRuntimes runtimes)]
```
**Disable for specific target frameworks:**
```cs
[ActiveIssue(int issue, TargetFrameworkMonikers frameworks)]
[ActiveIssue(string issue, TargetFrameworkMonikers frameworks)]
```
**Disable for specific test platforms and target frameworks:**
```cs
[ActiveIssue(int issue, TestPlatforms platforms, TargetFrameworkMonikers frameworks)]
[ActiveIssue(string issue, TestPlatforms platforms, TargetFrameworkMonikers frameworks)]
```
Use this attribute over test methods to skip failing tests only on the specific platforms and the specific target frameworks.
Expand Down Expand Up @@ -142,7 +142,7 @@ public class TestClass
}
```

**Note that all of the attributes above must include an issue number/link and/or have a comment next to them briefly justifying the reason. ActiveIssueAttribute and SkipOnTargetFrameworkAttribute should use their constructor parameters to do this**
**Note that all of the attributes above must include an issue link and/or have a comment next to them briefly justifying the reason. ActiveIssueAttribute and SkipOnTargetFrameworkAttribute should use their constructor parameters to do this**

_**A few common examples with the above attributes:**_

Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Collections.Immutable/tests/ImmutableSortedSetTest.nonnetstandard.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ namespace System.Collections.Immutable.Tests
public partial class ImmutableSortedSetTest : ImmutableSetTest
{
[Fact]
[ActiveIssue("Sporadic failure, needs a port of https://github.com/dotnet/coreclr/pull/4340", TargetFrameworkMonikers.NetFramework)]
[SkipOnTargetFramework(TargetFrameworkMonikers.NetFramework, "Sporadic failure, needs a port of https://github.com/dotnet/coreclr/pull/4340")]
public void EmptyTest()
{
this.EmptyTestHelper(Empty<int>(), 5, null);
Expand Down
25 changes: 0 additions & 25 deletions src/libraries/System.Runtime.Serialization.Xml/tests/DataContractSerializer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3490,31 +3490,6 @@ public static void DCS_BasicPerSerializerRoundTripAndCompare_SampleTypes_SampleI
TestObjectWithDifferentPayload(valueSampleICollectionTExplicitWithCDCContainsPrivateDC, netcorePayloadSampleICollectionTExplicitWithCDCContainsPrivateDC, desktopPayloadSampleICollectionTExplicitWithCDCContainsPrivateDC, setting);
}

/// <summary>
/// This case is a part of DCS_BasicPerSerializerRoundTripAndCompare_SampleTypes, but in these scenarios it was not support in current version.
/// </summary>
[Fact]
[ActiveIssue("Not support")]
public static void DCS_BasicPerSerializerRoundTripAndCompare_SampleTypes_Notsupport()
{
TestObjectInObjectContainerWithSimpleResolver(new SerializationTestTypes.ArrayListWithCDCFilledWithMixedTypes(true), @"<ObjectContainer xmlns=""http://schemas.datacontract.org/2004/07/SerializationTestTypes"" xmlns:i=""http://www.w3.org/2001/XMLSchema-instance""><_data z:Id=""i1"" i:type=""a:SerializationTestTypes.ArrayListWithCDCFilledWithMixedTypes***"" xmlns:z=""http://schemas.microsoft.com/2003/10/Serialization/"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.ArrayListWithCDCFilledWithMixedTypes***""><List xmlns:b=""http://schemas.microsoft.com/2003/10/Serialization/Arrays""><b:anyType z:Id=""i2"" i:type=""c:SerializationTestTypes.PublicDC***"" xmlns:c=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicDC***""><Data>55cb1688-dec7-4106-a6d8-7e57590cb20a</Data></b:anyType><b:anyType z:Id=""i3"" i:type=""c:SerializationTestTypes.PublicDCClassPublicDM***"" xmlns:c=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicDCClassPublicDM***""><Data>No change</Data></b:anyType><b:anyType z:Id=""i4"" i:type=""c:SerializationTestTypes.PublicDCClassPrivateDM_DerivedDCClassPublic***"" xmlns:c=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicDCClassPrivateDM_DerivedDCClassPublic***""><_data/></b:anyType><b:anyType z:Id=""i5"" i:type=""PrivateDCClassPublicDM_DerivedDCClassPrivate""><Data>Data</Data></b:anyType><b:anyType z:Id=""i6"" i:type=""PrivateDCClassPrivateDM""><_data>No change</_data></b:anyType><b:anyType z:Id=""i7"" i:type=""PrivateCallBackSample_IDeserializationCallback""><Data>Data</Data></b:anyType><b:anyType z:Id=""i8"" i:type=""PrivateCallBackSample_OnDeserialized""><Data>Data</Data></b:anyType><b:anyType z:Id=""i9"" i:type=""PrivateCallBackSample_OnSerialized""><Data>Data</Data></b:anyType><b:anyType i:type=""PrivateDCStruct""><Data>2147483647</Data></b:anyType><b:anyType i:type=""c:SerializationTestTypes.PrivateDefaultCtorIXmlSerializables***"" xmlns:c=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PrivateDefaultCtorIXmlSerializables***"">68656C6C6F20776F726C64</b:anyType><b:anyType i:type=""PrivateIXmlSerializables"">68656C6C6F20776F726C64</b:anyType><b:anyType z:Id=""i10"" i:type=""c:SerializationTestTypes.Derived_Override_Prop_GetPrivate_Private***"" xmlns:c=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.Derived_Override_Prop_GetPrivate_Private***""><Data>No change</Data><Data>No change</Data></b:anyType><b:anyType i:type=""c:SerializationTestTypes.DerivedFromPriC***"" xmlns:c=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.DerivedFromPriC***""><a>0</a><b i:nil=""true""/><c>100</c><d>100</d></b:anyType></List></_data><_data2 z:Ref=""i1"" xmlns:z=""http://schemas.microsoft.com/2003/10/Serialization/""/></ObjectContainer>");

TestObjectInObjectContainerWithSimpleResolver(new SerializationTestTypes.CollectionBaseWithCDCFilledWithMixedTypes(true), @"<ObjectContainer xmlns=""http://schemas.datacontract.org/2004/07/SerializationTestTypes"" xmlns:i=""http://www.w3.org/2001/XMLSchema-instance""><_data z:Id=""i1"" i:type=""a:SerializationTestTypes.CollectionBaseWithCDCFilledWithMixedTypes***"" xmlns:z=""http://schemas.microsoft.com/2003/10/Serialization/"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.CollectionBaseWithCDCFilledWithMixedTypes***""><anyType z:Id=""i2"" i:type=""b:SerializationTestTypes.PublicDC***"" xmlns:b=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicDC***""><Data>55cb1688-dec7-4106-a6d8-7e57590cb20a</Data></anyType><anyType z:Id=""i3"" i:type=""b:SerializationTestTypes.PublicDCClassPublicDM***"" xmlns:b=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicDCClassPublicDM***""><Data>No change</Data></anyType><anyType z:Id=""i4"" i:type=""b:SerializationTestTypes.PublicDCClassPrivateDM_DerivedDCClassPublic***"" xmlns:b=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicDCClassPrivateDM_DerivedDCClassPublic***""><_data/></anyType><anyType z:Id=""i5"" i:type=""PrivateDCClassPublicDM_DerivedDCClassPrivate""><Data>Data</Data></anyType><anyType z:Id=""i6"" i:type=""PrivateDCClassPrivateDM""><_data>No change</_data></anyType><anyType z:Id=""i7"" i:type=""PrivateCallBackSample_IDeserializationCallback""><Data>Data</Data></anyType><anyType z:Id=""i8"" i:type=""PrivateCallBackSample_OnDeserialized""><Data>Data</Data></anyType><anyType z:Id=""i9"" i:type=""PrivateCallBackSample_OnSerialized""><Data>Data</Data></anyType><anyType i:type=""PrivateDCStruct""><Data>2147483647</Data></anyType><anyType i:type=""b:SerializationTestTypes.PrivateDefaultCtorIXmlSerializables***"" xmlns:b=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PrivateDefaultCtorIXmlSerializables***"">68656C6C6F20776F726C64</anyType><anyType i:type=""PrivateIXmlSerializables"">68656C6C6F20776F726C64</anyType><anyType z:Id=""i10"" i:type=""b:SerializationTestTypes.Derived_Override_Prop_GetPrivate_Private***"" xmlns:b=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.Derived_Override_Prop_GetPrivate_Private***""><Data>No change</Data><Data>No change</Data></anyType><anyType i:type=""b:SerializationTestTypes.DerivedFromPriC***"" xmlns:b=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.DerivedFromPriC***""><a>0</a><b i:nil=""true""/><c>100</c><d>100</d></anyType></_data><_data2 z:Ref=""i1"" xmlns:z=""http://schemas.microsoft.com/2003/10/Serialization/""/></ObjectContainer>");

TestObjectInObjectContainerWithSimpleResolver(new SerializationTestTypes.DCHashtableContainerMixedTypes(true), "", skipStringCompare: true);

TestObjectInObjectContainerWithSimpleResolver(new SerializationTestTypes.IReadWriteXmlWriteBinHex_EqualityDefined(), @"<ObjectContainer xmlns=""http://schemas.datacontract.org/2004/07/SerializationTestTypes"" xmlns:i=""http://www.w3.org/2001/XMLSchema-instance""><_data i:type=""a:SerializationTestTypes.IReadWriteXmlWriteBinHex_EqualityDefined***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.IReadWriteXmlWriteBinHex_EqualityDefined***"">68656C6C6F20776F726C64</_data><_data2 i:type=""a:SerializationTestTypes.IReadWriteXmlWriteBinHex_EqualityDefined***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.IReadWriteXmlWriteBinHex_EqualityDefined***"">68656C6C6F20776F726C64</_data2></ObjectContainer>");

TestObjectInObjectContainerWithSimpleResolver(new SerializationTestTypes.PrivateDefaultCtorIXmlSerializables(true), @"<ObjectContainer xmlns=""http://schemas.datacontract.org/2004/07/SerializationTestTypes"" xmlns:i=""http://www.w3.org/2001/XMLSchema-instance""><_data i:type=""a:SerializationTestTypes.PrivateDefaultCtorIXmlSerializables***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PrivateDefaultCtorIXmlSerializables***"">68656C6C6F20776F726C64</_data><_data2 i:type=""a:SerializationTestTypes.PrivateDefaultCtorIXmlSerializables***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PrivateDefaultCtorIXmlSerializables***"">68656C6C6F20776F726C64</_data2></ObjectContainer>");

TestObjectInObjectContainerWithSimpleResolver(new SerializationTestTypes.PublicIXmlSerializablesWithPublicSchemaProvider(), @"<ObjectContainer xmlns=""http://schemas.datacontract.org/2004/07/SerializationTestTypes"" xmlns:i=""http://www.w3.org/2001/XMLSchema-instance""><_data i:type=""a:SerializationTestTypes.PublicIXmlSerializablesWithPublicSchemaProvider***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicIXmlSerializablesWithPublicSchemaProvider***"">68656C6C6F20776F726C64</_data><_data2 i:type=""a:SerializationTestTypes.PublicIXmlSerializablesWithPublicSchemaProvider***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicIXmlSerializablesWithPublicSchemaProvider***"">68656C6C6F20776F726C64</_data2></ObjectContainer>");

TestObjectInObjectContainerWithSimpleResolver(new SerializationTestTypes.PublicExplicitIXmlSerializablesWithPublicSchemaProvider(), @"<ObjectContainer xmlns=""http://schemas.datacontract.org/2004/07/SerializationTestTypes"" xmlns:i=""http://www.w3.org/2001/XMLSchema-instance""><_data i:type=""a:SerializationTestTypes.PublicExplicitIXmlSerializablesWithPublicSchemaProvider***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicExplicitIXmlSerializablesWithPublicSchemaProvider***"">68656C6C6F20776F726C64</_data><_data2 i:type=""a:SerializationTestTypes.PublicExplicitIXmlSerializablesWithPublicSchemaProvider***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicExplicitIXmlSerializablesWithPublicSchemaProvider***"">68656C6C6F20776F726C64</_data2></ObjectContainer>");

TestObjectInObjectContainerWithSimpleResolver(new SerializationTestTypes.PublicIXmlSerializablesWithPrivateSchemaProvider(), @"<ObjectContainer xmlns=""http://schemas.datacontract.org/2004/07/SerializationTestTypes"" xmlns:i=""http://www.w3.org/2001/XMLSchema-instance""><_data i:type=""a:SerializationTestTypes.PublicIXmlSerializablesWithPrivateSchemaProvider***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicIXmlSerializablesWithPrivateSchemaProvider***"">68656C6C6F20776F726C64</_data><_data2 i:type=""a:SerializationTestTypes.PublicIXmlSerializablesWithPrivateSchemaProvider***"" xmlns:a=""http://schemas.datacontract.org/2004/07/SerializationTestTypes.PublicIXmlSerializablesWithPrivateSchemaProvider***"">68656C6C6F20776F726C64</_data2></ObjectContainer>");

}

[Fact]
public static void DCS_BasicPerSerializerRoundTripAndCompare_Collections()
{
Expand Down