Added a new test to check whether some transformations is applied during conversion or completion

[alien-cyber]

This checks whether a specific set of transformations has been applied during conversion or compilation for particular architectures

• The set of transformations that needed to be checked for a particular architecture can be defined in a separate file called expected_transformations.txt
• The test case runs for each architecture
• The model is loaded and compiled in a separate subprocess so that the logs can be captured effectively
• If a transformation is applied during conversion, then the log definitely contains its name along with time and '+' symbol
• We run a scan of the logs using an algorithm to check if the transformations are applied based on that logic
• Sometimes we may misspell the name of the transformation, so I used get_close_matches library which does fuzzy search and returns the possible matches

Closes #1645

Please review and let me know if any further improvements are needed.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[MissLostCodes]

@popovaan @rkazants @alien-cyber

Hi !

I had a few suggestions and potential issues:

• The test currently captures stdout, but OpenVINO logs are emitted to stderr, so this may miss the relevant transformation logs.
• Setting OV_ENABLE_PROFILE_PASS at the top level leaks into the global test environment it seems sufficient to keep it scoped inside the subprocess.
• The subprocess call may be expensive since it can trigger model export for each test case. Adding a cache_dir (or similar reuse strategy) could help reduce overhead.
• The substring matching logic (key in a) could lead to false positives (e.g., MatMul matching MatMulFusionExtra). Exact matching would be safer.

Overall, the structure and idea of the test look great — addressing these points should make it more robust and CI-friendly.

[MissLostCodes]

i think setting OV_ENABLE_PROFILE_PASS at the top level might unintentionally leak into the global test environment . This variable is already being set inside the subprocess - line 49 . we can safely remove this line .

[MissLostCodes]

I think this should use stderr instead of stdout since OpenVINO logs are emitted to stderr
even function name says capture stderr

- return result.stdout
+ return result.stderr

[MissLostCodes]

I think the subprocess code here may be quite expensive since it invokes OVModelForCausalLM.from_pretrained(...) inside the string, which can trigger model export on every test run.

[MissLostCodes]

can add this param

cache_dir="./ov_cache"

we could pass a cache_dir so that models are reused across runs instead of being re-exported each time.

This should help make the test more scalable.

[MissLostCodes]

I think substring matching here could cause false positives like MatMul matching MatMulFusionExtra
Using exact matching like if key in applied_norm: would be safer.

[alien-cyber]

Hi @MissLostCodes
Thanks for the analysis.

• Now the string matching function is Fixed.
• stderr and stdout are captured in the same place.
• unnecessary OV_ENABLE_PROFILE_PASS is removed

Regarding cache_dir="./ov_cache", I believe it is not necessary in this context, as it mainly changes the default directory used for caching model files. The default Hugging Face cache should already handle reuse of downloaded model configuration.