Skip to content

Fix UITest screenshot taking on MacCatalyst#27531

Merged
rmarinho merged 16 commits intodotnet:mainfrom
albyrock87:fix-macos-uitest-screenshot-verification
Feb 6, 2025
Merged

Fix UITest screenshot taking on MacCatalyst#27531
rmarinho merged 16 commits intodotnet:mainfrom
albyrock87:fix-macos-uitest-screenshot-verification

Conversation

@albyrock87
Copy link
Contributor

@albyrock87 albyrock87 commented Feb 3, 2025

Description of Change

MacCatalyst VerifyScreenshot now extracts the window's content instead of the entire screen.

I've added an option to include titlebar (for testing purposes).

In the future, we may want to replace constants with the actual title bar height.
In the past the title bar height changed on MacOS (11) so we should be aware of that.

The trick on MacOS may be to look at the Y coordinate of the first maroon-like pixel.
ImageMagick provides access to pixels colors, but pay attention that night "Night Shift" or "True Tone" may affect the output color, so we may need to verify a given color range.

On Windows this is not working and we need to find a better strategy.
So for now we rolled back to constants.

image

This is the code we used as tentative.

protected override void FixtureSetup()
{
    base.FixtureSetup();
#if MACUITEST || WINTEST
    _titleBarHeight = GetTitleBarHeight();
#endif
}

int GetTitleBarHeight()
{
#if MACUITEST
    var bytes = TakeScreenshot();
#else
    var bytes = App.Screenshot();
#endif
    using var image = new MagickImage(bytes);
    var pixels = image.GetPixels();
    var y = 0;

    const int minR = 0x73;
    const int maxR = 0x83;
    const int maxG = 0x20;
    const int maxB = 0x20;

    while (true)
    {
        if (y >= image.Height)
        {
            Assert.Fail("Failed to find the title bar color");
        }

        var pixelColor = pixels[3, y]!.ToColor()!;
        var r = pixelColor.R;
        var g = pixelColor.G;
        var b = pixelColor.B;

        ++y;

        if (r is >= minR and <= maxR && g <= maxG && b <= maxB)
        {
            break;
        }
    }

    return y;
}

Issues Fixed

Taking screenshot on MacCatalyst in CI and on developer machine is a nightmare:

  • Unreliable due to maximize/minimize actions
  • Developers may have different screen sizes compared to CI

@jsuarezruiz
Copy link
Contributor

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@jsuarezruiz
Copy link
Contributor

New macOS snapshots available. I am going to commit the changes.

Copy link
Contributor

@jsuarezruiz jsuarezruiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Include a condition for macOS to crop off the TitleBar:

@PureWeen PureWeen added this to the .NET 9 SR4 milestone Feb 3, 2025
@jsuarezruiz jsuarezruiz self-assigned this Feb 3, 2025
@jsuarezruiz jsuarezruiz added the area-testing Unit tests, device tests label Feb 3, 2025
@jsuarezruiz jsuarezruiz marked this pull request as draft February 3, 2025 18:00
@PureWeen
Copy link
Member

PureWeen commented Feb 3, 2025

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@PureWeen
Copy link
Member

PureWeen commented Feb 3, 2025

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@PureWeen
Copy link
Member

PureWeen commented Feb 3, 2025

/rebase

@github-actions github-actions bot force-pushed the fix-macos-uitest-screenshot-verification branch from 665944d to 3b1ede7 Compare February 3, 2025 22:31
@dotnet dotnet deleted a comment from azure-pipelines bot Feb 4, 2025
@dotnet dotnet deleted a comment from azure-pipelines bot Feb 5, 2025
@jsuarezruiz
Copy link
Contributor

jsuarezruiz commented Feb 5, 2025

/azp run

@jsuarezruiz jsuarezruiz marked this pull request as ready for review February 5, 2025 13:07
@PureWeen
Copy link
Member

PureWeen commented Feb 5, 2025

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@PureWeen PureWeen modified the milestones: .NET 9 SR5, .NET 9 SR4 Feb 5, 2025
@dotnet dotnet deleted a comment from azure-pipelines bot Feb 6, 2025
@azure-pipelines
Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@jsuarezruiz
Copy link
Contributor

image

@rmarinho rmarinho merged commit ad5a690 into dotnet:main Feb 6, 2025
119 of 124 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Mar 9, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-testing Unit tests, device tests community ✨ Community Contribution

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants