[UI Tests] Replace pixel-by-pixel image comparison with perceptual hash (pHash) for improved visual similarity detection (#40653)
Some checks failed
Spell checking / Check Spelling (push) Has been cancelled
Spell checking / Report (Push) (push) Has been cancelled
Spell checking / Report (PR) (push) Has been cancelled
Spell checking / Update PR (push) Has been cancelled

<!-- Enter a brief description/summary of your PR here. What does it
fix/what does it change/how was it tested (even manually, if necessary)?
-->
## Summary of the Pull Request
This PR replaces the previous pixel-by-pixel image comparison logic with
a perceptual hash (pHash)-based comparison using the
CoenM.ImageSharp.ImageHash library.

**Removes the need for golden images from CI pipelines**
Since the comparison is perceptual rather than binary, we no longer need
to fetch pixel-perfect golden images from pipelines for validation.
Developers can now capture screenshots locally and still get meaningful,
robust comparisons.

### Why pHash?
Unlike direct pixel comparison (which fails on minor rendering
differences), pHash focuses on the overall structure and visual
perception of the image. This provides several benefits:

- Robust to minor differences: tolerates compression artifacts,
anti-aliasing, subtle rendering changes, and border padding.
- Resilient to resolution or format changes: works even if images are
scaled or compressed differently.
- Closer to human perception: more accurately reflects whether two
images "look" the same to a person.

<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist

- [ ] **Closes:** #xxx
- [ ] **Communication:** I've discussed this with core contributors
already. If the work hasn't been agreed, this work might be rejected
- [ ] **Tests:** Added/updated and all pass
- [ ] **Localization:** All end-user-facing strings can be localized
- [ ] **Dev docs:** Added/updated
- [ ] **New binaries:** Added on the required places
- [ ] [JSON for
signing](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ESRPSigning_core.json)
for new binaries
- [ ] [WXS for
installer](https://github.com/microsoft/PowerToys/blob/main/installer/PowerToysSetup/Product.wxs)
for new binaries and localization folder
- [ ] [YML for CI
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ci/templates/build-powertoys-steps.yml)
for new test projects
- [ ] [YML for signed
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/release.yml)
- [ ] **Documentation updated:** If checked, please file a pull request
on [our docs
repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys)
and link it here: #xxx

<!-- Provide a more detailed description of the PR, other things fixed,
or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments

<!-- Describe how you validated the behavior. Add automated tests
wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
This commit is contained in:
leileizhang 2025-07-18 10:32:09 +08:00 committed by GitHub
parent 6642c805b7
commit d37105bf84
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 63 additions and 14 deletions

View File

@ -218,6 +218,7 @@ coclass
CODENAME
codereview
Codespaces
Coen
COINIT
colid
colorconv

View File

@ -9,6 +9,7 @@
<PackageVersion Include="Microsoft.Bot.AdaptiveExpressions.Core" Version="4.23.0" />
<PackageVersion Include="Appium.WebDriver" Version="4.4.5" />
<PackageVersion Include="Azure.AI.OpenAI" Version="1.0.0-beta.17" />
<PackageVersion Include="CoenM.ImageSharp.ImageHash" Version="1.3.6" />
<PackageVersion Include="CommunityToolkit.Common" Version="8.4.0" />
<PackageVersion Include="CommunityToolkit.Mvvm" Version="8.4.0" />
<PackageVersion Include="CommunityToolkit.WinUI.Animations" Version="8.2.250402" />

View File

@ -1496,6 +1496,7 @@ SOFTWARE.
- AdaptiveCards.Templating 2.0.5
- Appium.WebDriver 4.4.5
- Azure.AI.OpenAI 1.0.0-beta.17
- CoenM.ImageSharp.ImageHash 1.3.6
- CommunityToolkit.Common 8.4.0
- CommunityToolkit.Labs.WinUI.Controls.MarkdownTextBlock 0.1.250703-build.2173
- CommunityToolkit.Mvvm 8.4.0

View File

@ -20,6 +20,7 @@
<PackageReference Include="System.Net.Http" />
<PackageReference Include="System.Private.Uri" />
<PackageReference Include="System.Text.RegularExpressions" />
<PackageReference Include="CoenM.ImageSharp.ImageHash" />
</ItemGroup>
</Project>

View File

@ -6,7 +6,11 @@ using System.Diagnostics;
using System.Diagnostics.CodeAnalysis;
using System.Drawing;
using System.IO;
using CoenM.ImageHash;
using CoenM.ImageHash.HashAlgorithms;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;
namespace Microsoft.PowerToys.UITest
{
@ -127,13 +131,40 @@ namespace Microsoft.PowerToys.UITest
}
/// <summary>
/// Test if two images are equal bit-by-bit
/// Test if two images are equal using ImageHash comparison
/// </summary>
/// <param name="baselineImage">baseline image</param>
/// <param name="testImage">test image</param>
/// <returns>true if are equal,otherwise false</returns>
private static bool AreEqual(Bitmap baselineImage, Bitmap testImage)
{
try
{
// Define a threshold for similarity percentage
const int SimilarityThreshold = 95;
// Use CoenM.ImageHash for perceptual hash comparison
var hashAlgorithm = new AverageHash();
// Convert System.Drawing.Bitmap to SixLabors.ImageSharp.Image
using var baselineImageSharp = ConvertBitmapToImageSharp(baselineImage);
using var testImageSharp = ConvertBitmapToImageSharp(testImage);
// Calculate hashes for both images
var baselineHash = hashAlgorithm.Hash(baselineImageSharp);
var testHash = hashAlgorithm.Hash(testImageSharp);
// Compare hashes using CompareHash method
// Returns similarity percentage (0-100, where 100 is identical)
var similarity = CompareHash.Similarity(baselineHash, testHash);
// Consider images equal if similarity is very high
// Allow for minor rendering differences (threshold can be adjusted)
return similarity >= SimilarityThreshold; // 95% similarity threshold
}
catch
{
// Fallback to pixel-by-pixel comparison if hash comparison fails
if (baselineImage.Width != testImage.Width || baselineImage.Height != testImage.Height)
{
return false;
@ -157,4 +188,18 @@ namespace Microsoft.PowerToys.UITest
return true;
}
}
/// <summary>
/// Convert System.Drawing.Bitmap to SixLabors.ImageSharp.Image
/// </summary>
/// <param name="bitmap">The bitmap to convert</param>
/// <returns>ImageSharp Image</returns>
private static Image<Rgba32> ConvertBitmapToImageSharp(Bitmap bitmap)
{
using var memoryStream = new MemoryStream();
bitmap.Save(memoryStream, System.Drawing.Imaging.ImageFormat.Png);
memoryStream.Position = 0;
return SixLabors.ImageSharp.Image.Load<Rgba32>(memoryStream);
}
}
}