随着多模态大语言模型（MLLM）的飞速发展，能够像人类一样通过视觉输入操作图形用户界面（GUI）的智能体（Agent）正逐渐成为现实。然而，在通往通用计算机控制的道路上，如何让模型精准地将自然语言指令对应到屏幕上的具体元素 —— 即 GUI ...

在更接近真实场景的MobileWorld测试集上，MAI-UI-235B-A22B整体成功率41.7%，比其他端到端模型高出20.8个百分点。在需要主动询问用户的任务上成功率37.5%，在需要调用MCP工具的任务上成功率51.1%，分别比之前最好的成绩高出32.1和18.7个百分点。

人机交互方式正在发生变革。 本论文的主要作者 Chaoyun Zhang、Shilin He、Liqun Li，Si Qin 等均来自 Data, Knowledge, and Intelligence (DKI) 团队，为微软 Windows GUI Agent UFO 的核心开发团队的成员。 图形用户界面（Graphical User Interface, ...

A graphical user interface (or GUI, often pronounced "gooey"), is a particular case of user interface for interacting with a computer which employs graphical images and widgets in addition to text to ...

A graphical user interface (GUI, pronounced “gooey”) is a computer environment that simplifies the user’s interaction with the computer by representing programs, commands, files, and other options as ...

A graphical user interface (GUI) allows users to interact with graphics appearing on electronic devices (eg, smartphones, tablets and netbooks). Typically, a user interacts with a GUI by pressing ...

Those old enough to remember the command line interfaces of yesteryear are only too aware of what a godsend the Graphical User Interfaces (GUI) of today are. However, the human computer interface (HCI ...

Software that lets a programmer or user develop a graphical user interface by dragging and dropping icons from a toolbar onto the interface window and editing them with graphics tools. Behind the ...

This is an Insight article, written by a selected contributor as part of WTR's co-published content. Read more on Insight A graphical user interface (GUI) allows users to interact with graphics ...

It wasn't just cost and Moore's law. The graphical user interface -- now known as the GUI ("gooey") -- is what really made computing widespread, personal and ubiquitous. Its friendly icons and ...