Skill UI

Empowers building AI agents that perceive screens, plan via vision-language reasoning, and control mouse/keyboard for desktop automation, with attention to sandboxing and safety across Anthropic, OpenAI, and open-source paradigms.

Use for live Windows or macOS GUI tasks that structured tools cannot complete. Discover approved apps and windows, observe current state, act via semantic elements or shortcuts, and verify every requested outcome. Includes an operating loop for reliable desktop automation and visual verification.

Build AI agents that mimic human computer interaction, including viewing screens, moving cursors, clicking buttons, and typing text. This skill covers frameworks like Anthropic's Computer Use and OpenAI's Operator, focusing on vision-based control, sandboxing, and security. Ideal for desktop automation, GUI automation, and browser agents.

Choose the next legal action in browser, desktop, game, or simulation. Provides structured decision-making for agents, with options for real AI judgment or simulation. Includes safety checks, mode selection, and workflow examples.