Skip to content
This repository was archived by the owner on Sep 25, 2026. It is now read-only.

Test restructure - #2113

Closed
tlakollo wants to merge 15 commits into
dotnet:mainfrom
tlakollo:TestRestructure
Closed

tlakollo wants to merge 15 commits into
dotnet:mainfrom
tlakollo:TestRestructure

Conversation

@tlakollo

@tlakollo tlakollo commented Jun 25, 2021 •

Copy link
Copy Markdown
Contributor

Add ProducedBy enum to define where the diagnostic can be generated
ExpectedWarningAttribute and LogContainsAttribute now have a property to define where the diagnostic can be generated using the ProducedBy enum
Modified the different tests environments to support the new property, also test will only warn if the id is supported by the current running analyzer
RunTest function now accepts additional references to be inserted before compilation
RequiresUnreferencedCodeCapability.cs file is now only RequiresCapability.cs since is now agnostic of which kind of RequiresAttribute is testing
Fixes #2077
Fixes #2107

tlakollo added 7 commits June 24, 2021 18:34
Add ProducedBy enum to define where the diagnostic can be generated
ExpectedWarningAttribute and LogContainsAttribute noew have a property
to define where the diagnostic can be generated using the ProducedBy
enum
Modified the differet tests environments to support the new property,
also test will only warn if the id is supported by the current running
analyzer
RunTest function now accepts additional references to be inserted before
compilation
RequiresUnreferencedCodeCapability.cs file is now only
RequiresCapability.cs since is now agnostic of which kind of
RequiresAttribute is testing
test file and the linkertestcases file
Add function to search for dependency files and transform them into
metadatareferences
…cture

Add missing attribute in compiler generated type and method
Uncomment the expected warnings
tlakollo added 2 commits July 1, 2021 13:05
Change names to be more generic
Analyzer test infra now understand attributes in accessors and in properties
@tlakollo tlakollo self-assigned this Jul 2, 2021
@tlakollo
tlakollo marked this pull request as ready for review July 2, 2021 07:22
@tlakollo
tlakollo requested a review from marek-safar as a code owner July 2, 2021 07:22
@tlakollo
tlakollo requested review from agocke and mateoatr July 2, 2021 07:22
Update static constructor Compiler Generated conditions to only check
for fields and properties
@tlakollo
tlakollo requested a review from vitek-karas July 7, 2021 00:20
Comment on lines +148 to +151
case "Dependencies":
foreach (var file in Directory.EnumerateFiles (subDir, "*.cs"))
yield return file;
break;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess for now this is OK, but ideally we would react to the same attributes linker tests do. I guess that would require us to parse the test file twice - once to just go over the attributes to detect additional dependencies. And then second time to do the actual test compilation. But it would more precisely match what the linker tests do.

Comment on lines 18 to +19
[RequiresUnreferencedCode ("Message for --Method--")]
[RequiresAssemblyFiles (Message = "Message for --Method--")]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that the two attributes can't look the same is really not ideal. I filed dotnet/runtime#55274 to track changing it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #2156 to update linker sdk along with analyzer Microsoft.NetCore.App.Ref package to get the changes made in RAF

Comment on lines 24 to +25
[RequiresUnreferencedCode ("Message for --UncalledMethod--")]
[RequiresAssemblyFiles (Message = "Message for --UncalledMethod--")]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fragile - in that if somebody adds a new test case and forgets to add both attributes we're loosing coverage (and it's really hard to find). Maybe we can "fix" this by:

  • Adding a unit test which goes over the test classes in the right namespace and validates that if one attribute is there, there's also the second one
  • Would need some way to tell this test that there are cases where it's intentional that there's a difference.

bool IsProducedByLinker (CustomAttribute attr)
{
var propertyObject = attr.GetPropertyValue ("ProducedBy");
ProducedBy diagnosticProducedBy = propertyObject is null ? ProducedBy.LinkerAndAnalyzer : (ProducedBy) propertyObject;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:

Suggested change
ProducedBy diagnosticProducedBy = propertyObject is null ? ProducedBy.LinkerAndAnalyzer : (ProducedBy) propertyObject;
return propertyObject is null ? true : ((ProducedBy) propertyObject).HasFlag (ProducedBy.Linker);

public void RequiresCapability (MethodDeclarationSyntax m, List<AttributeSyntax> attrs)
private static Lazy<IEnumerable<MetadataReference>> _additionalReferences = new Lazy<IEnumerable<MetadataReference>> (LoadDependencies);

private static IEnumerable<MetadataReference> LoadDependencies ()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not put this in LinkerTestCases.cs? It feels like GetReferenceFilesByDirName should only be used there and have the dependencies resolved before compilation (ideally through looking at the linker attributes that we use to add assemblies to the compilation).

List<MetadataReference> additionalReferences = new List<MetadataReference> ();
var s_refFiles = GetReferenceFilesByDirName ();
foreach (var refFile in s_refFiles["Dependencies"]) {
if (refFile.Contains ("RequiresCapability"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels somewhat fragile looking into the string of a filename here.

var ancestor = attr.Ancestors ().OfType<MemberDeclarationSyntax> ().First ();
if (ancestor.IsKind (SyntaxKind.MethodDeclaration) || ancestor.IsKind (SyntaxKind.PropertyDeclaration) || ancestor.IsKind (SyntaxKind.GetAccessorDeclaration) || ancestor.IsKind (SyntaxKind.SetAccessorDeclaration))
return true;
return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think it helps readability to have one line between the if () <exp>; <exp>;. I've noticed this pattern in the codebase, although I didn't find any coding style doc (searched in runtime repo) that clearly states this.

public static async Task<Compilation> GetCompilation (string src)
{
var srctree = CSharpSyntaxTree.ParseText (src);
var mdRef = MetadataReference.CreateFromFile (typeof (Mono.Linker.Tests.Cases.Expectations.Metadata.BaseMetadataAttribute).Assembly.Location);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Why not have a using and put typeof (BaseMetadataAttribute) here?

{
var srctree = CSharpSyntaxTree.ParseText (src);
var mdRef = MetadataReference.CreateFromFile (typeof (Mono.Linker.Tests.Cases.Expectations.Metadata.BaseMetadataAttribute).Assembly.Location);
var comp = CSharpCompilation.Create (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could probably be a good place to call a method that pulls in dependencies. Probably by understanding the SetupCompileBefore/SetupCompileAfter attributes.

return builder.ToImmutable ();
}

public static IEnumerable<string> GetReferenceFiles ()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only called by GetReferenceFilesByDirName, should we collapse these two methods into just one?

{
GetDirectoryPaths (out var rootSourceDir, out _);

foreach (var subDir in Directory.EnumerateDirectories (rootSourceDir, "*", SearchOption.AllDirectories)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use Directory.GetDirectories(rootSourceDir) instead?


private bool IsProducedByAnalyzer (Dictionary<string, ExpressionSyntax> args)
{
return !args.TryGetValue ("ProducedBy", out var diagnosticProducedBy) || (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we do something similar to what we have in IsProducedByLinker instead of having this very long expression?

@tlakollo

tlakollo commented Nov 2, 2021

Copy link
Copy Markdown
Contributor Author

Closing in favor of #2336

@tlakollo tlakollo closed this Nov 2, 2021
@tlakollo
tlakollo deleted the TestRestructure branch February 7, 2023 21:27
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support analyzer tests with additional assemblies Analyzer testing restructure

4 participants