We are happy to release MMBench-GUI, a hierarchical, multi-platform benchmark framework and toolbox, to evaluate GUI agents. MMBench-GUI is comprising four evaluation levels: GUI Content Understanding ...

OMG-Agent is an open-source Mobile GUI Agent desktop client that drives AI to automatically operate Android phones via natural language instructions. This project is for learning, research, and ...

Abstract: GUI testing automatically using the capture-replay technique is beneficial in shortening testing time. Some of the automated testing tools that implement this technique are Selenium Ide, ...

One of the principal challenges in building VLM-powered GUI agents is visual grounding, i.e., localizing the appropriate screen region for action execution based on both the visual content and the ...