Overview
OpenGCUA is a general computer-use agent (GCUA) technology stack open-sourced by Alibaba Tongyi's MAI-UI team in October 2026, licensed under Apache-2.0, with code hosted on GitHub. Its positioning is not limited to GUI operations; rather, it integrates accessibility tree reading, application API calls, shell command execution, and other methods, and writes GUI operations as code, combining them into programs with loops and conditions to enable background window operations on the same computer shared with the user. This open-source release includes a macOS real-machine harness, a cross-device SDK, OSWorld integration examples, and evaluation trajectories of 8 frontier models. The GCUA model is planned for release later.
Key Features
- Multimodal Operation Fusion: It does not only operate the GUI but also reads the accessibility tree, calls application APIs, executes shell commands, and writes GUI operations as code, combining them into programs with loops and conditions to improve automation flexibility and efficiency.
- macOS Real-Machine Harness: Provides a macOS real-machine harness in the form of a menu bar application, requires macOS 14+, and supports running and debugging agents in real environments.
- Cross-Device SDK: The same set of interfaces covers computers, browsers, and phones, making it easy for developers to build and deploy computer-use agents across platforms.
- OSWorld Integration Examples: Includes two types of reference agents, screenshot-action and code-action, helping developers get started quickly and integrate into the OSWorld evaluation environment.
- Frontier Model Evaluation Trajectories: Releases evaluation trajectories for 8 frontier models (GPT-6 Astra, Claude Opus 5, Gemini 3.8 Flash, GLM-5.3-Flash, Qwen3.8-Max, Qwen3.8-Flash, Kimi K3, DeepSeek-V4.1-Flash) for easy reproduction and analysis.
- Significant Performance Improvement: On the OSWorld-v2 offline set, compared with the baseline of GUI plus batch actions only, the average partial score of 8 models increased from 45.3 to 64.9 (+19.6 points), and the average number of steps per task decreased by 53%; writing operations as code adds another 6.1 points over the GUI+CLI agent and reduces steps by 14.8%.
Use Cases
- Automating cross-device task flows, such as collaborative operations among computers, browsers, and phones.
- Building and debugging general computer-use agents on macOS, using the real-machine harness for real-environment testing.
- Quickly developing reference agents for screenshot actions or code actions based on the OSWorld integration examples.
- Using the released evaluation trajectories to reproduce frontier model performance for comparative analysis and research.
- Writing GUI operations as code and combining them into programs with loops and conditions to achieve complex background automation.
Pros
- Open source and licensed under Apache-2.0, facilitating commercial use and secondary development.
- Integrates multiple operation methods including GUI, API, shell, and code, improving task-handling capability.
- Provides a cross-device SDK and macOS real-machine harness, covering multi-platform development needs.
- Releases evaluation trajectories of 8 frontier models, enhancing reproducibility and research value.
- Demonstrates quantified effects of partial score improvement and step reduction on the OSWorld-v2 offline set.
- The project plans to release the GCUA model later, with potential for continuous evolution.
Pricing
OpenGCUA is an open-source project licensed under Apache-2.0, with code hosted on GitHub, and can be obtained and used for free. No commercial pricing information is currently provided; please refer to the official website for details.
Summary
OpenGCUA is a general computer-use agent technology stack open-sourced by Alibaba Tongyi's MAI-UI team, integrating GUI, API, shell, and code operations, and providing a macOS harness, cross-device SDK, and OSWorld integration examples. On the OSWorld-v2 offline set, the average partial score of 8 models improved by 19.6 points and the average number of steps decreased by 53%, and evaluation trajectories of frontier models have been released. The project is licensed under Apache-2.0, with the GCUA model planned for release later.
Version History
- Alibaba Tongyi MAI-UI open-sources OpenGCUA, a full stack for generalist computer-use agents (2026-10-19): Alibaba Tongyi MAI-UI open-sourced OpenGCUA under Apache-2.0, with a technical report at arXiv 2610.22200. This release opens a macOS real-device harness (macOS 14+ required), a cross-device SDK, OSWorld integration examples, and evaluation trajectories for eight frontier models including GPT-6 Astra, Claude Opus 5 and Gemini 3.8 Flash. On the OSWorld-v2 offline set, mean partial score across the eight models rises from 45.3 to 64.9 and steps per task fall by 53% on average, while the gap between the strongest and weakest model narrows from 39.1 to 18.0 points.