#cmux Computer Use
Give AI coding agents access to native Mac apps. Read app interfaces, take screenshots, click controls, and type text with cmux Computer Use.
#Supported agents
Native Computer Use tools attach automatically to Claude Code and Codex sessions launched inside cmux. Other agents can run in the terminal, but do not yet receive this automatic Mac-control integration.
#Get started
- Install and open cmux for Mac. In Settings → Computer Use, enable Computer Use.
- Use the permission controls to grant Accessibility and Screen Recording to cmux Computer Use in System Settings. If the helper is missing from the list, add or drag its app tile into the permission pane.
- Start a new Claude Code or Codex session inside cmux. Ask it to use Computer Use and name the app and task. The tools connect automatically; no manual MCP configuration is required.
#Copy a Computer Use prompt
Use the $cmux-cua skill for this task. If it is unavailable, install it with your normal skill installer. If permissions are missing, ask me to open cmux Settings → Computer Use. Inspect before acting and verify after each change. Task: [describe the task here]
#Permissions and setup
Accessibility allows the helper to read and interact with app interfaces. Screen Recording allows it to capture screenshots. Grant both permissions to cmux Computer Use. Opening cmux or starting an agent does not grant access or open setup automatically.
Computer Use permissions are managed by a separate helper, so your terminal sessions stay open during onboarding.
On macOS Tahoe, approve the additional screen-capture confirmation during setup. Wait until cmux shows both permissions as granted before retrying the task.
#Try it with Calculator
Use cmux Computer Use to open Calculator, click 100 + 105, and show me the result.Inspect the target first, act on controls from that snapshot, then inspect again to verify the result. Element indexes belong to the current snapshot; refresh them after navigation or layout changes. A successful click acknowledgement alone does not prove the task finished.
#Agent tools
cmux-cua is the bundled MCP server, which gives agents structured tools for native app control. These are tool names, not shell commands. The exact arguments depend on the agent profile; use the schemas attached to the session.
#Codex
Codex uses get_app_state with an app name or bundle identifier to receive a screenshot and accessibility tree. Act on a current element_index or screenshot coordinates, then call get_app_state again. Its tool set is:
list_apps, get_app_state, click, perform_secondary_action,
set_value, select_text, scroll, drag, press_key, type_text#Claude Code
Claude Code uses list_apps and list_windows to find a target, then get_window_state with its pid and window_id. Prefer element tokens or fresh indexes. The native profile also supports perform_actions for a group of actions that does not invalidate later targets.
list_apps → list_windows → get_window_state
click / type_text / press_key / scroll / drag
get_window_state#Follow and stop Computer Use
The Computer Use menu-bar item shows the active agent. Focus Computer Use follows the app it is driving; Focus Calling Terminal returns to the originating terminal. Returning to the terminal does not stop automation. Use the menu’s stop action to stop the driving session.
Turn off Computer Use in Settings to disable it. To exclude the tools from a new agent launch, set CMUX_COMPUTER_USE_MCP_DISABLED=1 before launching that agent.
#Skills
The cmux-cua skill explains the inspection and action workflow. It is optional and does not grant macOS permissions. Use your agent’s skill installer for a persistent installation; cmux does not install global skills by default.
#Native apps and browser automation
Computer Use operates native Mac windows through screenshots and accessibility. For cmux browser panes, use cmux browser commands for DOM snapshots, selectors, forms, and JavaScript. A browser surface reference is different from a native app element index.
#Troubleshooting
#The agent has no Computer Use tools
Enable Computer Use in cmux Settings and start a fresh Claude Code or Codex session inside cmux. Existing sessions may retain their original tool configuration. Check that Computer Use is not disabled by an environment variable or a managed policy.
#Setup is incomplete, clicks fail, or screenshots are blank
Reopen Settings → Computer Use and finish both permission steps, including any extra capture confirmation. Retry after setup completes. If the helper needs to restart, your terminal sessions can remain open.
#The wrong window is targeted or a click is blocked
Inspect the app again and select a fresh element from its current state. If another window covers the target, reveal the intended window and inspect again before retrying. Do not reuse stale coordinates or element indexes.