SamOS

Smart Autonomous Mind Operating System

Not just an AI.
A personal friend who lives on your Mac.

Sam listens, remembers, sees, learns, and grows smarter with every conversation. Voice-first. Memory-persistent. Self-learning. Private by design.

Download on GitHub See How Sam Thinks ↓
Voice ConversationPersistent MemoryAgentic ReasoningSemantic EmbeddingsRAG PipelineComputer VisionSelf-Learning SkillsContradiction DetectionWeb ResearchHybrid SearchKnowledge AttributionFace RecognitionSkillForgeAutonomous Research30 Built-in Tools Voice ConversationPersistent MemoryAgentic ReasoningSemantic EmbeddingsRAG PipelineComputer VisionSelf-Learning SkillsContradiction DetectionWeb ResearchHybrid SearchKnowledge AttributionFace RecognitionSkillForgeAutonomous Research30 Built-in Tools
How Sam Thinks

AI that actually reasons.

Sam doesn't just respond — she reasons, remembers, learns, and improves. Here's what happens every time you speak.

1

Listen & Understand

Sam detects your wake word locally, captures your voice, and transcribes it — all without sending audio to the cloud.

Local Wake Word Whisper STT Follow-up Capture
2

Remember & Retrieve

Before answering, Sam searches her memory. Facts, preferences, websites, behavioural lessons — all retrieved and ranked using a multi-signal hybrid search.

Semantic Embeddings BM25 + Vector Search Recency Boost IDF Weighting
3

Reason & Plan

Sam builds a multi-step plan, injects relevant knowledge via RAG, and decides what tools to use — or if a spoken answer is enough. Agentic reasoning, not just autocomplete.

Agentic Reasoning RAG Pipeline Dual LLM Routing Adaptive Tokens
4

Act & Respond

Sam executes her plan — calling tools, searching the web, checking the camera, or simply speaking naturally. 30 tools across 8 categories, plus any skill she's taught herself.

30 Built-in Tools Self-Learned Skills Output Canvas Natural Voice
5

Learn & Improve

After every conversation, Sam gets smarter. She saves new memories, detects contradictions, records behavioural patterns, and refines how she responds to you over time.

Auto-Save Memories Contradiction Detection Self-Learning SkillForge
Under the Hood

Built smarter from the ground up.

The intelligence that makes Sam more than just another chatbot wrapper.

🧠

Agentic Reasoning

Sam builds multi-step plans — tool calls, follow-up questions, conditional logic — then executes them autonomously. Real reasoning, not pattern matching.

🔎

Real Semantic Embeddings

Memory retrieval uses vector similarity, not just keywords. "What's my pet's name?" finds "Your dog Bailey is a golden retriever" — no exact match needed.

📚

RAG Pipeline

Every response is augmented with relevant memories, learned websites, and behavioural lessons — injected directly into the LLM context window.

⚡

Hybrid Search

Combines BM25 lexical matching, cosine similarity, phrase containment, IDF weighting, and recency decay in a single multi-signal retrieval stack.

🔄

Contradiction Detection

Sam catches conflicting memories — "you love coffee" vs "you avoid caffeine" — and resolves them instead of silently storing both.

🎓

Self-Learning

Sam detects behavioural patterns across conversations and adjusts — learning to be more concise, more clear, or more proactive based on how you interact.

🔧

SkillForge

"Learn how to set reminders" — Sam builds the skill autonomously through a 5-stage AI pipeline: draft, refine, review, validate, install.

🌐

Autonomous Research

"Research solar panels for 10 minutes" — Sam browses the web independently, extracts key insights, and saves them to memory for future conversations.

📈

Knowledge Attribution

Every response shows what percentage came from local memory versus external AI. Full transparency into exactly where Sam's answer came from.

What Sam Can Do

Your personal AI, fully loaded.

30 tools, 8 categories, and the ability to learn more. Here's what you get out of the box.

Voice Conversation

Talk to your Mac like a friend.

Say "Hey Sam" and just talk. Sam listens with local speech recognition, responds with a natural voice, and keeps listening for follow-ups — no wake word needed between exchanges.

  • Local wake word detection — always private
  • Whisper STT runs on-device, or stream via OpenAI Realtime
  • Barge-in — interrupt Sam mid-sentence
  • Natural ElevenLabs voice with audio caching
You
Hey Sam, what time is it in Tokyo?
Sam
It's 2:34 AM in Tokyo right now — probably not the best time to call.
You
What about London?
Sam
London is 5:34 PM — still business hours. Good time to reach someone.
Persistent Memory

Sam remembers what matters.

Sam automatically detects and saves facts, preferences, and notes from your conversations. Smart deduplication ensures she never stores the same thing twice, and hybrid search finds relevant memories even with different wording.

  • Auto-saves without being asked — detects memorable facts
  • 4 memory types with intelligent TTLs (7 days to 1 year)
  • Contradiction detection resolves conflicting memories
  • Semantic search — "my pet" finds "your dog Bailey"
Fact Your dog Bailey is a golden retriever
Pref You prefer dark chocolate over milk
Fact Your anniversary is March 15th
Note Working on the SamOS website this week
Fact You're allergic to peanuts
Computer Vision

Sam sees what you see.

8 vision tools powered by Apple's Vision framework — all running locally on your Mac. Describe scenes, find objects, read text, recognise faces, and track changes over time.

  • Object detection, OCR text reading, face detection
  • Enrol faces for local recognition — no cloud processing
  • Visual Q&A — "Is the door open?" with evidence-based answers
  • Inventory snapshots track what's changed on your desk
Laptop 94% Coffee mug 87% Notebook 79% Live camera feed
Self-Learning Skills

Teach Sam anything.

Tell Sam "learn how to set reminders" and she'll build the capability herself — a 5-stage AI pipeline drafts, refines, reviews, validates, and installs new skills that work like built-in features.

  • Natural language goal → fully functional skill
  • 5-stage pipeline: draft, refine, review, validate, install
  • Skills trigger by voice naturally, just like built-in tools
  • Parameter slots with smart date/number extraction
✓Draft skill from goal
✓Refine with OpenAI
✓Review implementation
4Validating triggers & slots...
5Install to skill library
Web Learning & Research

Sam does the homework.

Share a URL and Sam summarises, indexes, and remembers it. Or say "research AI for 10 minutes" and Sam browses the web independently — extracting insights and saving them for future conversations.

  • Learn any URL — full-text extraction with key highlights
  • Autonomous research mode with configurable duration
  • Searches DuckDuckGo, Wikipedia, and HackerNews
  • Find recipes, videos, images, and local files
Autonomous Research Session
✅ Learned: Solar panel efficiency trends 2024
✅ Learned: Perovskite cell breakthroughs
✅ Learned: Cost per watt comparison by technology
✅ Learned: Home installation requirements (AU)
⏳ Researching: Government rebate programs...
12 sources · 36 insights · 4 min remaining
Daily Assistant

The everyday stuff, handled.

Alarms with snooze, timers, weather forecasts, time zones, file search. The basics done right — with a voice interface that actually works.

  • Alarms & timers with interactive dismiss/snooze cards
  • Weather + 7-day forecast via Open-Meteo (free, no key)
  • Time zones for 400+ cities worldwide
  • Search Downloads & Documents by name or type
⏰ Alarm — 7:00 AM
Good morning! Time to start the day.
Dismiss Snooze 5m
Melbourne, Australia
⛅ Partly cloudy 23°C
30 Built-in Tools

Everything you need, one voice command away.

Plus the ability to learn more through SkillForge.

show_text
Render markdown on the Output Canvas
show_image
Display remote images with fallbacks
find_image
Search Google Images by query
find_video
Search YouTube for videos
find_recipe
Find recipes with ingredients & steps
find_files
Search Downloads & Documents
describe_camera_view
Describe the live camera scene
find_camera_objects
Find specific objects in frame
get_camera_face_presence
Detect faces in camera view
enroll_camera_face
Enrol a face for recognition
recognize_camera_faces
Identify previously enrolled faces
camera_visual_qa
Answer questions about what Sam sees
camera_inventory_snapshot
Track visible object changes
save_camera_memory_note
Save a camera observation to memory
save_memory
Save a fact, preference, note, or check-in
list_memories
View all saved memories
delete_memory
Remove a specific memory
clear_memories
Clear all stored memories
schedule_task
Set alarms and timers
cancel_task
Cancel a scheduled task
list_tasks
View pending scheduled tasks
learn_website
Learn and index a webpage
autonomous_learn
Self-directed web research
stop_autonomous_learn
Stop a research session
get_time
Time and timezone conversions
get_weather
Weather and 7-day forecast
start_skillforge
Build a new skill from a goal
forge_queue_status
View the forge build queue
forge_queue_clear
Stop and clear forge jobs
capability_gap_prompt
Generate build prompt for missing capability
Display Search Camera Memory Scheduler Learning Info SkillForge
Privacy

Private by design. Not by afterthought.

Sam lives on your Mac — not in a data centre. Here's exactly what stays local and what uses the network.

Runs Locally on Your Mac

Wake word detection (Porcupine) Local
Speech-to-text (Whisper) Local
Computer vision (Apple Vision) Local
Face recognition (AES-encrypted) Local
All memories (SQLite) Local
API keys (macOS Keychain) Local
Learned skills (JSON on disk) Local

Network (Only When Needed)

LLM routing (OpenAI / Ollama) Network
Voice synthesis (ElevenLabs) Network
Web learning (URL fetching) Network
Weather data (Open-Meteo) Network
Image & video search Network

"Your voice is transcribed locally. Your face data is encrypted. Your memories never leave your Mac. Sam is private because she lives with you — not in a data centre."

Try Saying

Just talk naturally.

No commands to memorise. No syntax to learn. Just say what you need.

"Hey Sam, find me a recipe for pasta carbonara"
Here's a classic carbonara — I've put the full recipe on the canvas. You'll need eggs, pecorino, guanciale, and black pepper.
"Remember that my anniversary is March 15th"
Got it — I'll remember your anniversary is March 15th.
"What do you see through my camera?"
I can see a desk with a laptop, a coffee mug, and what looks like a notebook. There's text on the screen but it's too small to read.
"Learn how to control my smart lights"
I'll build that skill now. Give me a moment — I'm drafting the capability and will validate it before installing.
"Research renewable energy for 10 minutes"
Starting an autonomous research session on renewable energy. I'll browse, extract key insights, and report back when I'm done.
"What's the weather in Melbourne this week?"
Melbourne's looking at 23 degrees and partly cloudy today. I've put the 7-day forecast on the canvas — rain expected Thursday.

Say hello to Sam.

Free. Open source. Runs on macOS 14+.

1 Clone from GitHub
2 Add your API keys
3 Say "Hey Sam"
Download on GitHub Read the Docs

Requires macOS Sonoma · OpenAI API key · ElevenLabs API key · Porcupine access key