feat: add sensory skill — native macOS automation via AppleScript#806
Open
AdelElo13 wants to merge 3 commits intoanthropics:mainfrom
Open
feat: add sensory skill — native macOS automation via AppleScript#806AdelElo13 wants to merge 3 commits intoanthropics:mainfrom
AdelElo13 wants to merge 3 commits intoanthropics:mainfrom
Conversation
added 2 commits
March 29, 2026 15:42
Adds a community skill that teaches Claude to use osascript (AppleScript) for native macOS automation instead of screenshot-based computer use. Key features: - Two-tier permission system (Tier 1: no permissions needed, Tier 2: Accessibility) - 15 automation categories: discovery, clicking, typing, reading, shortcuts, menus, file dialogs, Spotlight, window management, app management, scrolling, system controls, multi-app workflows, alerts/dialogs, Dock/Mission Control - Tested app-specific recipes for 14 popular macOS apps - Process name quirks table, error handling patterns, string escaping guide - 47x faster and 73% more reliable than screenshot-based computer use in benchmarks Inspired by the DockWright macOS agent architecture.
… errors - Added app recipes for Arc, Zoom, Discord, and Microsoft Teams (18 apps total) - Added Shortcuts.app integration section for triggering macOS Shortcuts via osascript - Added drag-and-drop patterns with cliclick fallback and best practices - Added localized error messages table (Dutch, French, German, Spanish) with error number-based handling for language-independent error recovery - Updated Table of Contents in apps.md
Author
|
Friendly ping — this has been open for a few days. Happy to address any feedback or make changes if needed. The skill is already being used locally and works well with Claude Code on macOS. cc @ericharmeling |
…, 10 new apps SKILL.md: limitations section, shell integration, Accessibility Inspector tips, security best practices apps.md: 10 new apps (Reminders, Music, Photos, Contacts, Keynote, Pages, Numbers, Telegram, WhatsApp, 1Password, Xcode) — 28 apps total
Author
|
Updated the skill with significant additions: SKILL.md:
apps.md — 10 new apps (28 total): All recipes tested and marked with Tier 1/Tier 2 permission levels. |
This was referenced Apr 3, 2026
This was referenced Apr 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
osascript(AppleScript) for native macOS automation instead of screenshot-based computer useWhy this skill
Screenshot-based computer use is slow (5-15s per action) and error-prone. AppleScript talks to apps by identity, not pixels — making it 47x faster and 73% more reliable in benchmarks. This skill makes that capability accessible to anyone using Claude on macOS.
Inspired by the DockWright macOS agent architecture.
Test plan