Overview
The agent:- Interprets natural language instructions
- Interacts with device UI elements
- Executes multi-step workflows autonomously
- Returns results and structured output
Configuration
Task Prompt
The prompt describes what the agent should do. Write clear, specific instructions for best results.Model Selection
Choose the LLM model that powers the agent. Different models offer different tradeoffs:
Pass one of the following identifiers as the
llmModel parameter in the REST API or the SDK, or
select it in the Playground. This table is the canonical catalog of models available for cloud
agent tasks.
These provider prefixed identifiers are specific to the cloud API. The open source
Framework selects models in a different way, by provider class and model
ID such as
GoogleGenAI with gemini-3.1-flash-lite-preview. See the
SDK configuration for details.Execution Limits
Features
Reasoning Mode
Enables logical thinking and step-by-step analysis. The agent will reason through complex tasks before acting.
Vision
Allows the agent to process and understand visual content on screen. Required for most UI automation tasks.
Stealth
Enable human-like device interactions for automations where detection matters.
Memory
Cross-task memory personalization. Use
memoryNamespace to isolate memory between different workflows.Platform Settings
Structured Output
Define a JSON schema to receive structured responses from the agent. This is useful for:- Data extraction tasks
- Validation workflows
- Integration with downstream systems
Running the Agent
Via Playground
Use the Playground for interactive testing:- Configure your task and settings
- Select a device
- Click Run
- Monitor execution in the task stream
Via API
Run tasks programmatically using the API:Via MCP
Use the MCP integration to run tasks from AI assistants and IDEs.Execution Flow
- Initialization - Agent connects to the selected device
- Observation - Agent captures the current screen state
- Planning - Agent determines the next action based on the prompt
- Action - Agent executes the action (tap, type, scroll, etc.)
- Repeat - Steps 2-4 repeat until the task is complete or limits are reached
- Output - Agent returns results and any structured output