autonomous-ai/autonomous-os

Audio Control adjusts speaker volume, plays test tones, and records raw microphone audio. Use it for hardware-level audio tasks like muting, diagnostics, or capturing WAV input, not for speech output.

Camera controls the device camera to take snapshots, view a live stream, and toggle privacy. Use it when asked to look at something or to turn the camera on or off.

Claude Buddy coordinates with Claude Desktop and Claude Code on a Mac by relaying approval prompts and activity state. Use it when you need to answer tool permission requests by voice instead of clicking.

Computer Use controls a paired Mac through Autonomous Buddy to open apps, navigate URLs, type, use shortcuts, copy text, and click labeled UI elements. Use it for explicit requests to do something on the user's computer; use vision for screen-reading or unlabeled UI tasks.

Connectors detects which linked services are available on this device and uses them to answer questions or take actions in Gmail, Calendar, Drive, Notion, Figma, Asana, Linear, and GitHub. Use it when you need to check connections or work with a connected account.

Display controls the device’s round LCD for manual eye expressions or brief info text like time and weather. Use it only when you need direct display changes; normal conversation reactions are auto-synced by Emotion.

Emotion Expression adds inline emotion markers to conversational replies so a device can drive servos, LEDs, and display eyes with each response. Use it for reactive, lifelike behavior; it is not for ambient scenes or custom LED colors.

Face Enroll manages the device’s face recognition roster by enrolling new faces from a photo, a live snapshot, or a familiar-stranger prompt. It also lets you check status, remove entries, or reset the enrolled set.

Guard Mode turns a device into an alert watchdog for friends who say they are leaving or ask for security mode. While active, guard-tagged sensing events trigger dramatic reactions and Telegram alerts instead of normal sensing responses.

Habit tracks repeating behavioral patterns from wellbeing, presence, mood, music, and posture logs. Use it to answer questions about routines or enrich wellbeing and posture nudges with habit-aware phrasing.

Input Branching routes realtime voice-agent messages using tags like [voice-instruction], [transcript], and [HANDLED]. It is used to choose the correct input path, ignore noisy transcripts, and return NO_REPLY for handled history entries.

LED Control sets the device's RGB light to a specific color, effect, or off state when asked. Use it for direct lighting commands, not for ambient scenes or emotion responses.

Lighting Scenes activates predefined lighting presets for activities like reading, focusing, relaxing, movies, and sleep. Use it when you want ambient lighting that sets both brightness and color temperature.

Local Web UI FAQ explains how to use the repository's local web interface and its Settings controls. Use it when answering questions about Monitor, Setup, or other visible UI behavior.

Mood tracks the user's mood from camera, voice, and Telegram signals, then synthesizes a decision for downstream music and wellbeing skills. It is used when you need the latest inferred mood, not device-directed emotion commands.

Music plays or stops songs through the device speaker by searching YouTube when a user asks to listen, sing, or play music. It can handle specific requests or choose a song from mood and habit patterns.

Music Suggestion proposes a song or genre when a mood event suggests the user may want music and playback is idle. It is used for proactive suggestions, not for user-initiated music requests.

Sensing reacts to passive device events like presence, sound, light, and fire hazard with a matching emotion marker and a short spoken line or NO_REPLY. Use it when routing sensor triggers that should be handled differently from activity or emotion events.

Sensing Track queries flow event logs to answer questions about past sensing events and activity. Use it to check presence, motion, voice, emotion, and snapshot history over a time range.

Servo Control directs the device’s servos to aim in fixed directions, hold or release position, and play named animations. Use it when you need the device to look, point, or react physically.

Skill Creator helps you draft new skills, improve existing ones, and run evaluations to see how they perform. Use it when you want to create, tune, benchmark, or rewrite a skill description for better triggering.

Speaker Recognizer enrolls a speaker’s voice from mic or Telegram introductions and can list, identify, or remove stored voices. Use it when a transcript includes a self-introduction or when you need to manage known speakers.

User Emotion Detection maps a detected facial or vocal emotion into a mood log entry, then routes to music, check-in, or an action. Use it when the current turn includes an [emotion] or [speech_emotion] event.

Vision Tracking tracks and follows a named movable object using camera vision and servo movement. Use it when you want the system to lock onto items like a cup, phone, person, or pet, and stop when tracking is no longer needed.

Voice speaks additional text through TTS and exposes hardware mute markers for privacy and silent mode. Use it for meetings, calls, stop-listening requests, or when speech needs to happen separately from the normal reply.