在高分辨率下，屏幕元素识别不准确

[freedomxie]

Version Information

• Commit Hash:

Error Message

 "QHD": Resolution(width=2560, height=1440),   # 16:9     
 "3K": Resolution(width=2880, height=1620),   # 16:9     
 "4K": Resolution(width=3840, height=2160),    # 16:9

Description

能否适以上分辨率

Current Behavior

Expected Behavior

[yuruotong1]

我感觉可能需要对图片进行缩放！

[yuruotong1]

我手头确实没有这个分辨率的电脑，这个问题可能稍晚才能解决。

[greenflute]

1. image scaling, in higher resolution, mac just need the origin image not the 2x, so i don’t think it will cause scaling issue
2. tested under 2014x1280, prompt is „open terminal then cd to root and ls -lha“, with previous executed ls in terminal, automate can correctly recognise the results thus considering the task is already finished. discard the logic issue here, automate can actually correctly recognise icons and text in terminal window. so i don’t think higher resolution will cause problem. except when some old app was forced under higher resolution to maintain proper size using „zoom“, which will definitely cause „blurred“ images, and that may interfere automate’s recognising.

[krkawzq]

屏幕元素识别的部分是omniparser负责的，所以这个可能依赖于更好的detection模型或者新的框架