{"id":"manushi4-screenhand","name":"Screenhand","homepage":"https://screenhand.com","repo_url":"https://github.com/manushi4/Screenhand","category":"devtools","subcategories":[],"tags":["mcp","mcp-server","desktop-automation","accessibility","ui-automation","browser-automation","cdp","ocr","typescript","agent-tools","local-first"],"what_it_does":"ScreenHand is an open-source MCP server (stdio transport) that gives AI agents local control over macOS and Windows desktop UIs (via Accessibility APIs) and optionally browser automation (Chrome DevTools Protocol). It exposes a large set of tools for UI inspection, interaction (click/type/keys/scroll/menus/drag), perception/OCR fallbacks, job orchestration, and per-app “app mastery map” learning.","use_cases":["Desktop automation for repetitive UI tasks (forms, menus, navigation)","Browser automation with CDP for scripted workflows","Assistive UI operations driven by an AI agent","Cross-app workflows that move data between browser/app and desktop apps","QA/smoke testing style automation","Workflow recording/playbooks based on observed successful actions"],"not_for":["Running in fully headless environments without a desktop session","Highly sensitive operations without user review/controls (it can trigger real UI actions)","Environments where installing local tooling or granting accessibility permissions is not allowed","Controlling devices outside the local machine/session"],"best_when":"You want a local-first AI agent that can directly operate native apps and (optionally) Chrome with low latency, and you can grant the required local OS permissions.","avoid_when":"You cannot safely grant accessibility/automation permissions or you need a networked SaaS API with built-in policy enforcement/audit trails.","alternatives":["Anthropic Computer Use (cloud/screenshot-based)","MCP servers and automation stacks using OS automation APIs (custom tools)","Playwright/Selenium for web-only automation","UI testing tools (e.g., Playwright desktop, Appium, Robot Framework) for deterministic UI automation"],"af_score":62.2,"security_score":24.8,"reliability_score":28.8,"package_type":"mcp_server","discovery_source":["github"],"priority":"high","status":"evaluated","version_evaluated":null,"last_evaluated":"2026-03-30T15:33:28.540280+00:00","interface":{"has_rest_api":false,"has_graphql":false,"has_grpc":false,"has_mcp_server":true,"mcp_server_url":null,"has_sdk":false,"sdk_languages":["TypeScript"],"openapi_spec_url":null,"webhooks":false},"auth":{"methods":["None explicitly described for MCP/stdio server usage"],"oauth":false,"scopes":false,"notes":"No user authentication described for the local MCP server; access is effectively whoever can run the process and connect via stdio. OS-level permissions (Accessibility on macOS) are required for desktop control."},"pricing":{"model":null,"free_tier_exists":false,"free_tier_limits":null,"paid_tiers":[],"requires_credit_card":false,"estimated_workload_costs":null,"notes":"Project is presented as open-source and local-first; ongoing costs would primarily be any downstream LLM/API usage by your AI client (the README claims zero LLM calls for click/typing once tools are invoked)."},"requirements":{"requires_signup":false,"requires_credit_card":false,"domain_verification":false,"data_residency":[],"compliance":[],"min_contract":null},"agent_readiness":{"af_score":62.2,"security_score":24.8,"reliability_score":28.8,"mcp_server_quality":88.0,"documentation_accuracy":70.0,"error_message_quality":0.0,"error_message_notes":null,"auth_complexity":95.0,"rate_limit_clarity":10.0,"tls_enforcement":0.0,"auth_strength":25.0,"scope_granularity":15.0,"dependency_hygiene":50.0,"secret_handling":40.0,"security_notes":"Security posture appears to rely on local execution and OS permissions rather than network auth. The README claims: local-first (no screen data externally), blocking dangerous browser protocols (e.g., javascript:/data:), and audit-logging of AppleScript and browser JS execution; however, details of enforcement, logging destinations, and secret redaction are not verifiable from the provided text. TLS is not applicable because transport is stdio to a local process. Lack of explicit authentication/authorization means any party that can run/connect to the MCP server could drive the desktop.","uptime_documented":0.0,"version_stability":35.0,"breaking_changes_history":20.0,"error_recovery":60.0,"idempotency_support":"false","idempotency_notes":"No idempotency/side-effect semantics documented in provided README; UI actions generally have effects and would need client-side safeguards.","pagination_style":"none","retry_guidance_documented":false,"known_agent_gotchas":["Requires macOS Accessibility permission for the terminal app to allow UI control.","Browser automation requires launching Chrome with --remote-debugging-port=9222 and a running instance with remote debugging enabled.","Tool calls can have side effects (click/type/JS execution); agents should use confirmation/guardrails for destructive actions.","Cross-app control assumes UI state stability; dynamic layouts may still require fallback strategies (Accessibility→CDP→OCR→coordinates) and careful recovery."]}}