App Instruction Cards

​

What Are App Cards?

• How to navigate the app’s UI
• Where to find buttons and features
• App-specific shortcuts and gestures
• Search syntax and filters (for apps like Gmail)
• Common workflows and best practices

• The compose button is at the bottom-right
• Search supports filters like from:sender@email.com or has:attachment
• Swiping right archives emails, swiping left deletes them

​

Why Use App Cards?

• Agents guess how to navigate unfamiliar apps
• Trial-and-error wastes time and tokens
• Success rates drop for complex workflows

• ✅ Agents know exactly where to find features
• ✅ First-attempt success for common tasks
• ✅ Reduced token usage (less exploration needed)
• ✅ Better handling of app-specific quirks

​

Quick Start

# App cards are enabled by default
mobilerun run "Send an email to john@example.com" --reasoning

• Gmail (com.google.android.gm) - Email navigation, search, composition

​

How App Cards Work

​

Automatic Loading

1. Detection: Agent detects the current foreground app (e.g., Gmail)
2. Loading: Mobilerun loads the app card for that package name
3. Injection: App card content is added to the agent’s prompt
4. Guidance: Agent uses the instructions to make better decisions

​

When Are They Used?

# App cards enabled (Manager uses them for planning)
mobilerun run "Archive all unread emails" --reasoning

# App cards not used (direct execution mode)
mobilerun run "Tap the button"

​

Creating Custom App Cards

​

Step 1: Find the Package Name

# Get all apps
adb shell pm list packages

# Or search for a specific app
adb shell pm list packages | grep keyword

• Chrome: com.android.chrome
• WhatsApp: com.whatsapp
• Instagram: com.instagram.android
• YouTube: com.google.android.youtube

​

Step 2: Create the Files

mkdir -p config/app_cards
touch config/app_cards/app_cards.json
touch config/app_cards/chrome.md

# Chrome App Guide

## Navigation
- Address bar at the top for entering URLs
- Three-dot menu (top-right) for settings and history
- Tabs button (top-right) to switch between tabs

## Search
- Type queries directly in the address bar
- Use voice search via the microphone icon

## Common Actions
- **New Tab**: Tap the tabs button → Plus icon
- **Close Tab**: Swipe tab away in tab switcher
- **Refresh**: Pull down from the top of the page
- **Bookmarks**: Three-dot menu → Bookmarks

## Tips
- Incognito mode available via three-dot menu
- Downloads accessible via three-dot menu → Downloads

​

Step 3: Register the App Card

{
  "com.google.android.gm": "gmail.md",
  "com.android.chrome": "chrome.md"
}

{
  "com.whatsapp": "social/whatsapp.md",
  "com.instagram.android": "social/instagram.md"
}

​

Step 4: Test

mobilerun run "Open Chrome and search for mobilerun" --reasoning --debug

Loaded app card for com.android.chrome from config/app_cards/chrome.md

​

Configuration

agent:
  app_cards:
    enabled: true
    app_cards_dir: config/app_cards

agent:
  app_cards:
    enabled: false

​

App Card Best Practices

​

Content Guidelines

• Be concise and actionable
• Focus on UI patterns and workflows
• Include search syntax and special features
• Mention common pitfalls or quirks
• Use bullet points and clear headings

• Write essays or lengthy explanations
• Describe every single feature
• Include information that changes frequently (version-specific details)
• Duplicate general Android knowledge (agents already know how to tap, swipe, etc.)

​

Example: Good vs Bad

Gmail is an email application developed by Google. It has many features
including the ability to send and receive emails. To compose an email,
you need to first understand that Gmail uses a material design interface
with a floating action button, which is a circular button...

## Composing Emails
- Tap the floating compose button (bottom-right)
- Fill recipient, subject, and body
- Send via paper plane icon (top-right)

​

Troubleshooting

​

App Card Not Loading

1. Is the package name correct?
   adb shell dumpsys window windows | grep -E 'mCurrentFocus'
   
2. Is the mapping correct in app_cards.json?
   {
     "com.your.app": "yourapp.md"
   }
   
3. Does the markdown file exist?
   ls config/app_cards/yourapp.md
   
4. Are app cards enabled?
   agent:
     app_cards:
       enabled: true
   
5. Are you using reasoning mode?
   mobilerun run "command" --reasoning
   

​

Debug Mode

mobilerun run "Open Gmail" --reasoning --debug

Loaded app_cards.json with 2 entries
Loaded app card for com.google.android.gm from config/app_cards/gmail.md

​

Related Documentation

• CLI Usage - Mobilerun CLI command reference
• Configuration - Configuration system details
• Agent Architecture - How agents work
• Manager Agent - Agent that uses app cards