Inside The Machine

Free · private · runs on your computer

Run AI on your own computer.

No subscription. Nothing leaves your machine. Tell us your memory and what you want to do. We show you exactly what to install.

Scan my machinePick a goal Watch on YouTube

New to local AI? Start here: the basic words in 5 minutes.

1How much memory does your computer have?

Not sure? Scan my machine. One read-only command finds it for you.
How do I find this?

Mac: Apple menu → About This Mac → Memory.
Windows: Settings → System → About → Installed RAM.
PC with a graphics card: Task Manager → Performance → GPU → Dedicated GPU memory. Pick that number.

You can do5 of 6 goals below
Best model for youK2 Horizon 7B
About 36% as smart as Claude Opus 5.5, the best paid AI today.
See all 47 models that fit
2Pick a goal

What do you want to do?

Each one is a short recipe: what to install, which model, and the exact steps.

A private ChatGPTChat, write and brainstorm. Nothing leaves your computer.Ollama + Open WebUIRuns well~10 minA coding helperAn assistant that reads and edits your code.Ollama + OpenCode + AiderRuns, but tight~15 minChat with your filesAsk questions about your PDFs, notes and documents.Ollama + Open WebUI + DoclingRuns, but tight~20 minSpeech to textTurn meetings and voice notes into text, offline.whisper.cpp + HandyRuns well~10 minMake imagesCreate pictures from a text prompt on your own machine.Draw Things + ComfyUIRuns, but tight~20 minAn agent that does tasksIt remembers you, uses tools and learns new skills.Hermes Agent + MCP Servers (official)Needs 32 GB+~30 min
How it works

Three steps, about ten minutes.

Words like model, Ollama or 27B new to you? Start here explains them in plain words.

  1. 1Check your memoryIt decides which AI models fit on your computer.
  2. 2Pick a goalChat, code, documents, voice, images or an agent.
  3. 3Follow the recipeCopy a few commands. Or paste one prompt and let your AI agent do the setup.
The honest part

How good is AI on your own computer?

Good, not the best. Here is the gap, on the same test (Artificial Analysis Intelligence Index).

  • Best paid AIClaude Opus 5.5100%
  • Best free model (needs a server)MiMo-V2.6-Pro80%
  • Best on a 32 GB computerQwen3.8 27B59%
  • Best on a 16 GB computerK2 Horizon 7B36%
  • Best on an 8 GB computerQwen3.5 4B23%
Good forEveryday chat, writing, summaries, simple code, transcription.
Still behind onHard reasoning, big coding jobs, long multi-step tasks.
Why run it anywayPrivate, works offline, no monthly bill, no usage limits.

Next: we test each recipe on a 128 GB Mac Studio and publish a Gap Score for it.

Go deeper

Want to compare for yourself?

All 124 text modelsPlus 70 speech and image models. Filter by maker, task and what fits your memory.Which computer to buyMemory, speed and price for 54 machines.242 toolsThe full directory, sorted into 19 groups.Advanced agent guideAfter the recipes: an agent that sets itself up and improves.
The Gap Report

Get the first scores.

From Kit, the AI agent behind the channel.

Or follow on YouTube
Free. New scores and setups by email. Unsubscribe any time.
AGENT VIEW · text/markdown · or send Accept: text/markdown
# Inside The Machine

> Agent-first directory of 242 local-AI tools, models and repos (runtimes, agents, memory, RAG, routing, MCP, UIs, voice, image/video, fine-tuning, quantization, evals, scaffolds, observability, hardware utilities). 13 "runbook" listings have install/verify/rollback checked against official docs; the rest are "catalog" entries with verified metadata and unverified, README-extracted install commands. Gap Scores are pending test on Mac Studio.

Append .md or .json to any /setups/<slug> URL, or send Accept: text/markdown.

## Start here

- [Zero to working local AI](https://insidethemachine.ai/start.md): 5-step runbook + AGENTS.md/SOUL.md + lean-context guide. JSON: https://insidethemachine.ai/start.json
- [Models vs memory](https://insidethemachine.ai/models.md): which open models fit which RAM tier. JSON: https://insidethemachine.ai/models.json (124 text models; plus speech.json and images.json)
- [Machines](https://insidethemachine.ai/machines.md): hardware guide. JSON: https://insidethemachine.ai/machines.json

## Your machine, one-command installs, what changed

- [Scan my machine](https://insidethemachine.ai/scan.sh): read-only; prints a https://insidethemachine.ai/my?... link. Windows: https://insidethemachine.ai/scan.ps1. Report for agents: https://insidethemachine.ai/my.json?ram=<GB>&os=<mac|linux|win>&chip=<slug>&gpu=<slug>&vram=<GB> (also /my.md)
- [One-command installs](https://insidethemachine.ai/install.md): 19 scripts with install, verify, rollback and an honest run status. JSON: https://insidethemachine.ai/install.json
- [Install: A private ChatGPT](https://insidethemachine.ai/install/chat.md) [ran]: `curl -fsSL https://insidethemachine.ai/install/chat.sh | bash`
- [Install: A coding helper](https://insidethemachine.ai/install/code.md) [partly-ran]: `curl -fsSL https://insidethemachine.ai/install/code.sh | bash`
- [Install: Chat with your files](https://insidethemachine.ai/install/docs.md) [partly-ran]: `curl -fsSL https://insidethemachine.ai/install/docs.sh | bash`
- [Install: Speech to text](https://insidethemachine.ai/install/voice.md) [ran]: `curl -fsSL https://insidethemachine.ai/install/voice.sh | bash`
- [Install: Make images](https://insidethemachine.ai/install/images.md) [not-run]: `curl -fsSL https://insidethemachine.ai/install/images.sh | bash`
- [Install: An agent that does tasks](https://insidethemachine.ai/install/agent.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/agent.sh | bash`
- [Install: MLX LM](https://insidethemachine.ai/install/mlx-lm.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/mlx-lm.sh | bash`
- [Install: Ollama](https://insidethemachine.ai/install/ollama.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/ollama.sh | bash`
- [Install: llama.cpp](https://insidethemachine.ai/install/llama-cpp.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/llama-cpp.sh | bash`
- [Install: LM Studio](https://insidethemachine.ai/install/lm-studio.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/lm-studio.sh | bash`
- [Install: Open WebUI](https://insidethemachine.ai/install/open-webui.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/open-webui.sh | bash`
- [Install: LiteLLM](https://insidethemachine.ai/install/litellm.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/litellm.sh | bash`
- [Install: Aider](https://insidethemachine.ai/install/aider.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/aider.sh | bash`
- [Install: Hermes Agent](https://insidethemachine.ai/install/hermes-agent.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/hermes-agent.sh | bash`
- [Install: Mem0](https://insidethemachine.ai/install/mem0.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/mem0.sh | bash`
- [Install: MCP Servers (official)](https://insidethemachine.ai/install/mcp-servers.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/mcp-servers.sh | bash`
- [Install: whisper.cpp](https://insidethemachine.ai/install/whisper-cpp.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/whisper-cpp.sh | bash`
- [Install: Docling](https://insidethemachine.ai/install/docling.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/docling.sh | bash`
- [Install: AGENTS.md](https://insidethemachine.ai/install/agents-md.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/agents-md.sh | bash`
- [What changed](https://insidethemachine.ai/changes.md): dated new models (with the memory they need), tools, machines, and what broke. JSON: https://insidethemachine.ai/changes.json · RSS: https://insidethemachine.ai/changes.xml

## Pages (every page also answers Accept: text/markdown)

- [Start here: local AI in plain words](https://insidethemachine.ai/start-here.md): primer + recipe list + the full runbook. JSON: https://insidethemachine.ai/start-here.json
- [Recipes](https://insidethemachine.ai/do.md): 6 goals with memory needs. JSON: https://insidethemachine.ai/do.json
- [Recipe: A private ChatGPT](https://insidethemachine.ai/do/chat.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/chat.json
- [Recipe: A coding helper](https://insidethemachine.ai/do/code.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/code.json
- [Recipe: Chat with your files](https://insidethemachine.ai/do/docs.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/docs.json
- [Recipe: Speech to text](https://insidethemachine.ai/do/voice.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/voice.json
- [Recipe: Make images](https://insidethemachine.ai/do/images.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/images.json
- [Recipe: An agent that does tasks](https://insidethemachine.ai/do/agent.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/agent.json
- [Harnesses](https://insidethemachine.ai/harnesses.md): which app for which job, Claude Code vs Codex with local models. JSON: https://insidethemachine.ai/harnesses.json
- [Advanced: an agent that improves itself](https://insidethemachine.ai/guides/agent.md): same as start.md. JSON: https://insidethemachine.ai/guides/agent.json
- [Gap Score](https://insidethemachine.ai/gap-score.md): methodology, no scores yet. JSON: https://insidethemachine.ai/gap-score.json

## Core

- [Catalog JSON](https://insidethemachine.ai/api/index.json): every listing with links
- [Listing JSON Schema](https://insidethemachine.ai/schema/listing.schema.json)
- [Full text](https://insidethemachine.ai/llms-full.txt)
- [Dropped / archived items](https://insidethemachine.ai/api/dropped.json)
- [About / Gap Score methodology](https://insidethemachine.ai/about.md)

## Runtimes & inference

- [Ollama](https://insidethemachine.ai/setups/ollama.md): One command to download and run a model. [runbook]
- [SGLang](https://insidethemachine.ai/setups/sglang.md): High-throughput serving for LLMs and VLMs.
- [MLX](https://insidethemachine.ai/setups/mlx.md): Apple's array framework for Apple silicon.
- [llamafile](https://insidethemachine.ai/setups/llamafile.md): Package a model and runtime into one executable.
- [MLC LLM](https://insidethemachine.ai/setups/mlc-llm.md): Compile and run LLMs on any device.
- [KoboldCpp](https://insidethemachine.ai/setups/koboldcpp.md): One-file GGUF runner with a built-in UI.
- [MLX Examples](https://insidethemachine.ai/setups/mlx-examples.md): Official MLX example models and scripts.
- [MLX LM](https://insidethemachine.ai/setups/mlx-lm.md): Runs LLMs natively on Apple silicon. [runbook]
- [llama-swap](https://insidethemachine.ai/setups/llama-swap.md): Hot-swaps models behind one OpenAI-style endpoint.
- [MLX-VLM](https://insidethemachine.ai/setups/mlx-vlm.md): Vision-language models on MLX.
- [LM Studio](https://insidethemachine.ai/setups/lm-studio.md): A desktop app for local models. [runbook]
- [RamaLama](https://insidethemachine.ai/setups/ramalama.md): Serve models from any source in containers.
- [vLLM Metal](https://insidethemachine.ai/setups/vllm-metal.md): vLLM plugin for Apple Silicon.
- [MLX OpenAI Server](https://insidethemachine.ai/setups/mlx-openai-server.md): OpenAI-compatible API server for MLX models.
- [ML Drift](https://insidethemachine.ai/setups/ml-drift.md): Google's GPU inference engine for edge devices.
- [llama.cpp](https://insidethemachine.ai/setups/llama-cpp.md): The engine behind most local AI. [runbook]
- [vLLM](https://insidethemachine.ai/setups/vllm.md): High-throughput LLM serving engine.

## Model hubs

- [Transformers](https://insidethemachine.ai/setups/transformers.md): The reference library for open model weights.
- [Unsloth notebooks](https://insidethemachine.ai/setups/unsloth-notebooks.md): 250+ ready fine-tuning notebooks.
- [Hugging Face Hub](https://insidethemachine.ai/setups/hugging-face-hub.md): The main hub for open model weights.
- [Ollama Library](https://insidethemachine.ai/setups/ollama-library.md): Ready-to-pull models for Ollama.
- [LM Studio model catalog](https://insidethemachine.ai/setups/lm-studio-models.md): Curated models you can load in LM Studio.
- [mlx-community (Hugging Face)](https://insidethemachine.ai/setups/mlx-community.md): Community MLX conversions of popular models.
- [Unsloth (Hugging Face)](https://insidethemachine.ai/setups/unsloth-models.md): Unsloth GGUF and fine-tune-ready model uploads.
- [lmstudio-community (Hugging Face)](https://insidethemachine.ai/setups/lmstudio-community.md): LM Studio's community GGUF/MLX uploads.
- [Kaggle Models](https://insidethemachine.ai/setups/kaggle-models.md): Google-run model hub with downloadable weights.

## Models from the channel

- [Qwen-Image-2.1-Turbo](https://insidethemachine.ai/setups/qwen-image-2-1-turbo.md): 7B image model that paints and edits in 8 steps.
- [Mellum2.1 12B-A2.5B](https://insidethemachine.ai/setups/mellum-2-1.md): JetBrains coding model, 12B with 2.5B active.
- [EmbeddingGemma 2](https://insidethemachine.ai/setups/embeddinggemma-2.md): 740M on-device embedding model for search.
- [Kolibri-1](https://insidethemachine.ai/setups/kolibri-1.md): Open German/English MoE model (about 78 GB of memory, per the video).
- [AstaBrief 8B](https://insidethemachine.ai/setups/astabrief-8b.md): Small Ai2 model that writes cited reports.
- [Clef](https://insidethemachine.ai/setups/cloudflare-clef.md): Cloudflare's small open decision models (Clef, Clef-flash).

## Coding agents

- [OpenCode](https://insidethemachine.ai/setups/opencode.md): Open-source coding agent for your terminal.
- [Pi (pi-mono)](https://insidethemachine.ai/setups/pi-pi-mono.md): Small agent toolkit: LLM API, agent loop and coding CLI.
- [Goose](https://insidethemachine.ai/setups/goose.md): Extensible open-source agent that runs code and tools.
- [Aider](https://insidethemachine.ai/setups/aider.md): A coding agent in your terminal. [runbook]
- [mini-SWE-agent](https://insidethemachine.ai/setups/mini-swe-agent.md): A 100-line agent that fixes GitHub issues.
- [Claude Code](https://insidethemachine.ai/setups/claude-code.md): Anthropic's terminal coding agent (cloud model).
- [Gemini CLI](https://insidethemachine.ai/setups/gemini-cli.md): Google's open-source terminal agent for Gemini.
- [OpenHands](https://insidethemachine.ai/setups/openhands.md): Open platform for autonomous software agents.
- [Cline](https://insidethemachine.ai/setups/cline.md): Coding agent as a VS Code extension, CLI or SDK.
- [Continue](https://insidethemachine.ai/setups/continue.md): Open-source coding agent for IDEs and CI.
- [Tabby](https://insidethemachine.ai/setups/tabby.md): Self-hosted code completion and chat server.
- [Crush](https://insidethemachine.ai/setups/crush.md): Polished terminal coding agent from Charm.
- [Qwen Code](https://insidethemachine.ai/setups/qwen-code.md): Qwen's open-source terminal coding agent.
- [Kilo Code](https://insidethemachine.ai/setups/kilo-code.md): All-in-one agentic coding extension and CLI.
- [Codex CLI](https://insidethemachine.ai/setups/codex-cli.md): OpenAI's lightweight terminal coding agent.
- [GitHub Copilot CLI](https://insidethemachine.ai/setups/copilot-cli.md): GitHub's terminal coding agent; can use Ollama models.

## General & personal agents

- [OpenClaw](https://insidethemachine.ai/setups/openclaw.md): Personal agent that runs on your machine and chats anywhere.
- [Hermes Agent](https://insidethemachine.ai/setups/hermes-agent.md): A personal agent that learns skills and remembers you. [runbook]
- [AutoGPT](https://insidethemachine.ai/setups/autogpt.md): Platform for building autonomous agents.
- [Browser Use](https://insidethemachine.ai/setups/browser-use.md): Let an AI agent drive a web browser.
- [DeerFlow](https://insidethemachine.ai/setups/deerflow.md): Long-running research and coding super-agent.
- [Open Interpreter](https://insidethemachine.ai/setups/open-interpreter.md): Agent that runs code on your computer, for open models.
- [OpenManus](https://insidethemachine.ai/setups/openmanus.md): Open-source general agent (Manus-style).
- [nanobot](https://insidethemachine.ai/setups/nanobot.md): Ultra-light self-hosted personal agent.
- [UI-TARS Desktop](https://insidethemachine.ai/setups/ui-tars-desktop.md): Multimodal agent that operates your desktop.
- [Khoj](https://insidethemachine.ai/setups/khoj.md): Self-hostable AI second brain over your docs and the web.
- [Perplexica](https://insidethemachine.ai/setups/perplexica.md): Open answering engine (formerly Perplexica).
- [NanoClaw](https://insidethemachine.ai/setups/nanoclaw.md): Lightweight, container-sandboxed OpenClaw alternative.
- [GPT Researcher](https://insidethemachine.ai/setups/gpt-researcher.md): Agent that writes deep research reports.
- [Cua](https://insidethemachine.ai/setups/cua.md): Open-source computer-use drivers and sandboxes.
- [LM Studio Bionic](https://insidethemachine.ai/setups/lm-studio-bionic.md): LM Studio's agent that can run beside the app.

## Agent frameworks

- [LiteLLM](https://insidethemachine.ai/setups/litellm.md): One API in front of every model, local or cloud. [runbook]
- [AutoGen](https://insidethemachine.ai/setups/autogen.md): Microsoft's multi-agent programming framework.
- [CrewAI](https://insidethemachine.ai/setups/crewai.md): Role-based multi-agent framework.
- [LangGraph](https://insidethemachine.ai/setups/langgraph.md): Build stateful, resilient agent graphs.
- [Agno](https://insidethemachine.ai/setups/agno.md): Build and run agent systems in Python.
- [DSPy](https://insidethemachine.ai/setups/dspy.md): Program LLMs instead of hand-writing prompts.
- [OpenAI Agents SDK](https://insidethemachine.ai/setups/openai-agents-sdk.md): OpenAI's lightweight multi-agent SDK.
- [smolagents](https://insidethemachine.ai/setups/smolagents.md): Tiny library for agents that think in code.
- [Mastra](https://insidethemachine.ai/setups/mastra.md): TypeScript framework for agents and workflows.
- [Vercel AI SDK](https://insidethemachine.ai/setups/vercel-ai-sdk.md): TypeScript toolkit for AI apps and agents.
- [Haystack](https://insidethemachine.ai/setups/haystack.md): Orchestration framework for RAG and agents.
- [Letta](https://insidethemachine.ai/setups/letta.md): Stateful agents with long-term memory (ex-MemGPT).
- [Google ADK](https://insidethemachine.ai/setups/google-adk.md): Google's code-first agent development kit.
- [Pydantic AI](https://insidethemachine.ai/setups/pydantic-ai.md): Typed Python agent framework.

## Memory

- [Mem0](https://insidethemachine.ai/setups/mem0.md): Long-term memory for your agent. [runbook]
- [claude-mem](https://insidethemachine.ai/setups/claude-mem.md): Captures agent sessions and replays context later.
- [Cognee](https://insidethemachine.ai/setups/cognee.md): Knowledge-graph memory for agents.
- [Graphiti](https://insidethemachine.ai/setups/graphiti.md): Real-time temporal knowledge graphs for agents.
- [Supermemory](https://insidethemachine.ai/setups/supermemory.md): Fast memory engine and app, can run locally.
- [MemOS](https://insidethemachine.ai/setups/memos.md): Memory operating system for LLM agents.
- [Honcho](https://insidethemachine.ai/setups/honcho.md): Memory library that models users across sessions.
- [Basic Memory](https://insidethemachine.ai/setups/basic-memory.md): Markdown-file memory your agent reads and writes.
- [Memobase](https://insidethemachine.ai/setups/memobase.md): User-profile memory for chat apps.
- [memsearch](https://insidethemachine.ai/setups/memsearch.md): Semantic search over agent memory files.
- [MCP Memory Service](https://insidethemachine.ai/setups/mcp-memory-service.md): Persistent memory exposed as an MCP server.

## RAG & document parsing

- [Docling](https://insidethemachine.ai/setups/docling.md): Turns documents into clean text for AI. [runbook]
- [Marker](https://insidethemachine.ai/setups/marker.md): Fast, accurate PDF-to-markdown converter.
- [LightRAG](https://insidethemachine.ai/setups/lightrag.md): Simple, fast graph-based RAG.
- [Qdrant](https://insidethemachine.ai/setups/qdrant.md): High-performance vector database.
- [Chroma](https://insidethemachine.ai/setups/chroma.md): Open-source embedding database.
- [pgvector](https://insidethemachine.ai/setups/pgvector.md): Vector search inside Postgres.
- [LanceDB](https://insidethemachine.ai/setups/lancedb.md): Embedded multimodal vector database.
- [sqlite-vec](https://insidethemachine.ai/setups/sqlite-vec.md): Vector search as a tiny SQLite extension.
- [MarkItDown](https://insidethemachine.ai/setups/markitdown.md): Convert files and Office docs to markdown.
- [RAGFlow](https://insidethemachine.ai/setups/ragflow.md): Deep-document RAG engine with a UI.
- [PaddleOCR](https://insidethemachine.ai/setups/paddleocr.md): OCR and document parsing toolkit.
- [MinerU](https://insidethemachine.ai/setups/mineru.md): Converts complex PDFs to markdown or JSON.
- [AnythingLLM](https://insidethemachine.ai/setups/anythingllm.md): All-in-one desktop app for chat with your docs.
- [Meilisearch](https://insidethemachine.ai/setups/meilisearch.md): Fast search engine with hybrid vector search.

## Routing & gateways

- [Strands Decider](https://insidethemachine.ai/setups/strands-decider.md): Small decision model that routes agent choices.
- [New API](https://insidethemachine.ai/setups/new-api.md): Unified gateway that converts many LLM APIs.
- [One API](https://insidethemachine.ai/setups/one-api.md): LLM API management and distribution gateway.
- [Portkey Gateway](https://insidethemachine.ai/setups/portkey-gateway.md): Fast AI gateway with guardrails and fallbacks.
- [Bifrost](https://insidethemachine.ai/setups/bifrost.md): High-performance LLM and MCP gateway.
- [Plano](https://insidethemachine.ai/setups/plano.md): AI-native proxy with smart LLM routing.
- [LLMRouter](https://insidethemachine.ai/setups/llmrouter.md): Research library of LLM routing methods.
- [Envoy AI Gateway](https://insidethemachine.ai/setups/envoy-ai-gateway.md): Envoy-based gateway for generative AI services.
- [NVIDIA LLM Router](https://insidethemachine.ai/setups/nvidia-llm-router.md): Routes each prompt to the best-fit model.
- [NVIDIA PAIR](https://insidethemachine.ai/setups/nvidia-pair.md): Shares AI requests across the computers on your home network.

## MCP servers & tools

- [MCP Servers (official)](https://insidethemachine.ai/setups/mcp-servers.md): Standard plug-in tools for agents. [runbook]
- [MCP Python SDK](https://insidethemachine.ai/setups/mcp-python-sdk.md): Official Python SDK for MCP servers and clients.
- [LLM CLI](https://insidethemachine.ai/setups/llm-cli.md): CLI and Python library for any LLM, local or cloud.
- [MCP Inspector](https://insidethemachine.ai/setups/mcp-inspector.md): Visual tester for MCP servers.
- [n8n](https://insidethemachine.ai/setups/n8n.md): Workflow automation with AI nodes, self-hostable.
- [Firecrawl](https://insidethemachine.ai/setups/firecrawl.md): Turn websites into LLM-ready data.
- [Awesome MCP Servers](https://insidethemachine.ai/setups/awesome-mcp-servers.md): Big curated list of MCP servers.
- [Crawl4AI](https://insidethemachine.ai/setups/crawl4ai.md): Open-source crawler that outputs clean markdown.
- [Context7](https://insidethemachine.ai/setups/context7.md): Up-to-date library docs for coding agents.
- [Chrome DevTools MCP](https://insidethemachine.ai/setups/chrome-devtools-mcp.md): Chrome DevTools as an MCP server.
- [SearXNG](https://insidethemachine.ai/setups/searxng.md): Self-hosted private metasearch engine.
- [Playwright MCP](https://insidethemachine.ai/setups/playwright-mcp.md): Browser automation MCP server.
- [GitHub MCP Server](https://insidethemachine.ai/setups/github-mcp-server.md): GitHub's official MCP server.
- [Serena](https://insidethemachine.ai/setups/serena.md): Semantic code retrieval and editing over MCP.

## Chat UIs & front-ends

- [Open WebUI](https://insidethemachine.ai/setups/open-webui.md): A ChatGPT-style app for your local models. [runbook]
- [NextChat](https://insidethemachine.ai/setups/nextchat.md): Lightweight cross-platform chat client.
- [LobeChat](https://insidethemachine.ai/setups/lobechat.md): Agent workspace and chat UI.
- [Cherry Studio](https://insidethemachine.ai/setups/cherry-studio.md): Desktop AI studio for many providers.
- [text-generation-webui](https://insidethemachine.ai/setups/text-generation-webui.md): Desktop app for local LLMs (text-generation-webui).
- [LibreChat](https://insidethemachine.ai/setups/librechat.md): Self-hosted multi-provider ChatGPT clone.
- [Jan](https://insidethemachine.ai/setups/jan.md): Offline ChatGPT alternative for your desktop.
- [Chatbox](https://insidethemachine.ai/setups/chatbox.md): Desktop and mobile AI client.
- [SillyTavern](https://insidethemachine.ai/setups/sillytavern.md): Power-user chat and roleplay front-end.
- [Open-LLM-VTuber](https://insidethemachine.ai/setups/open-llm-vtuber.md): Voice chat with an animated avatar, locally.
- [HF Chat UI](https://insidethemachine.ai/setups/hf-chat-ui.md): The open code behind HuggingChat.
- [big-AGI](https://insidethemachine.ai/setups/big-agi.md): Advanced multi-model chat suite.
- [Enchanted](https://insidethemachine.ai/setups/enchanted.md): Native macOS/iOS app for self-hosted models.
- [llama.vscode](https://insidethemachine.ai/setups/llama-vscode.md): Local code completion in VS Code via llama.cpp.

## Voice: speech-to-text & TTS

- [whisper.cpp](https://insidethemachine.ai/setups/whisper-cpp.md): Speech-to-text that runs on your machine. [runbook]
- [MLX-Audio](https://insidethemachine.ai/setups/mlx-audio.md): TTS, STT and speech-to-speech on MLX.
- [WhisperKit](https://insidethemachine.ai/setups/whisperkit.md): On-device speech AI for Apple Silicon (WhisperKit).
- [Parakeet MLX](https://insidethemachine.ai/setups/parakeet-mlx.md): NVIDIA Parakeet speech recognition on MLX.
- [Whisper](https://insidethemachine.ai/setups/whisper.md): OpenAI's original speech recognition model.
- [ChatTTS](https://insidethemachine.ai/setups/chattts.md): Conversational text-to-speech model.
- [Handy](https://insidethemachine.ai/setups/handy.md): Offline push-to-talk dictation app.
- [Fish Speech](https://insidethemachine.ai/setups/fish-speech.md): Open-source text-to-speech with voice cloning.
- [Chatterbox](https://insidethemachine.ai/setups/chatterbox.md): Open-source expressive TTS.
- [faster-whisper](https://insidethemachine.ai/setups/faster-whisper.md): Whisper transcription, faster, via CTranslate2.
- [WhisperX](https://insidethemachine.ai/setups/whisperx.md): Whisper with word timestamps and speaker labels.
- [CosyVoice](https://insidethemachine.ai/setups/cosyvoice.md): Multilingual voice generation model.
- [Dia](https://insidethemachine.ai/setups/dia.md): TTS that generates realistic dialogue.
- [NeMo (Parakeet)](https://insidethemachine.ai/setups/nemo-parakeet.md): NVIDIA NeMo speech models (incl. Parakeet).

## Image & video generation

- [stable-diffusion.cpp](https://insidethemachine.ai/setups/stable-diffusion-cpp.md): Image and video diffusion in pure C/C++.
- [mflux](https://insidethemachine.ai/setups/mflux.md): MLX-native image and video models for Mac.
- [Qwen-Image 2.1](https://insidethemachine.ai/setups/qwen-image-2-1.md): Qwen's open image generation and editing model.
- [Draw Things](https://insidethemachine.ai/setups/draw-things.md): Community repo for the Draw Things Mac/iOS app.
- [Stable Diffusion WebUI](https://insidethemachine.ai/setups/stable-diffusion-webui.md): The classic Stable Diffusion web UI.
- [ComfyUI](https://insidethemachine.ai/setups/comfyui.md): Node-based GUI and backend for diffusion models.
- [Fooocus](https://insidethemachine.ai/setups/fooocus.md): Prompt-first, low-setup image generator.
- [Diffusers](https://insidethemachine.ai/setups/diffusers.md): PyTorch library for image, video and audio diffusion.
- [Open-Sora](https://insidethemachine.ai/setups/open-sora.md): Open video generation models.
- [InvokeAI](https://insidethemachine.ai/setups/invokeai.md): Professional creative studio for diffusion.
- [Core ML Stable Diffusion](https://insidethemachine.ai/setups/core-ml-stable-diffusion.md): Stable Diffusion on Core ML for Apple Silicon.
- [Wan 2.2](https://insidethemachine.ai/setups/wan-2-2.md): Open video generation models (Wan 2.2).
- [Wan 2.1](https://insidethemachine.ai/setups/wan-2-1.md): Open video generation models (Wan 2.1).
- [ComfyUI Manager](https://insidethemachine.ai/setups/comfyui-manager.md): Install and manage ComfyUI custom nodes.

## Fine-tuning

- [MLX-LM-LoRA](https://insidethemachine.ai/setups/mlx-lm-lora.md): LoRA and full fine-tuning on MLX.
- [Unsloth](https://insidethemachine.ai/setups/unsloth.md): Fast, low-memory fine-tuning with a local UI.
- [LLaMA-Factory](https://insidethemachine.ai/setups/llama-factory.md): Fine-tune 100+ LLMs and VLMs from one tool.
- [nanoGPT](https://insidethemachine.ai/setups/nanogpt.md): The simplest repo to train small GPTs.
- [nanochat](https://insidethemachine.ai/setups/nanochat.md): Train a small ChatGPT end to end on a budget.
- [Open R1](https://insidethemachine.ai/setups/open-r1.md): Open reproduction of DeepSeek-R1 training.
- [verl](https://insidethemachine.ai/setups/verl.md): Reinforcement-learning post-training framework.
- [PEFT](https://insidethemachine.ai/setups/peft.md): LoRA and other parameter-efficient fine-tuning.
- [TRL](https://insidethemachine.ai/setups/trl.md): Train models with SFT, DPO and RL.
- [ms-swift](https://insidethemachine.ai/setups/ms-swift.md): Fine-tune 600+ LLMs with PEFT or full training.
- [Axolotl](https://insidethemachine.ai/setups/axolotl.md): Config-driven fine-tuning toolkit.
- [OpenRLHF](https://insidethemachine.ai/setups/openrlhf.md): Scalable RLHF and agentic RL on Ray.
- [Oumi](https://insidethemachine.ai/setups/oumi.md): Fine-tune, evaluate and deploy open models.
- [MergeKit](https://insidethemachine.ai/setups/mergekit.md): Merge model weights into new models.

## Quantization

- [bitsandbytes](https://insidethemachine.ai/setups/bitsandbytes.md): 4-bit and 8-bit quantization for PyTorch.
- [Core ML Tools](https://insidethemachine.ai/setups/core-ml-tools.md): Convert and compress models for Core ML.
- [TensorRT Model Optimizer](https://insidethemachine.ai/setups/tensorrt-model-optimizer.md): NVIDIA quantization, pruning and distillation.
- [LLM Compressor](https://insidethemachine.ai/setups/llm-compressor.md): Compress models for vLLM serving.
- [Optimum](https://insidethemachine.ai/setups/optimum.md): Hardware-optimized inference and quantization for HF models.
- [torchao](https://insidethemachine.ai/setups/torchao.md): PyTorch-native quantization.
- [ExLlamaV3](https://insidethemachine.ai/setups/exllamav3.md): Fast EXL3 quantization and inference on consumer GPUs.
- [AutoRound](https://insidethemachine.ai/setups/autoround.md): Accurate low-bit post-training quantization.
- [AQLM](https://insidethemachine.ai/setups/aqlm.md): Extreme compression with additive quantization.
- [GPTQModel](https://insidethemachine.ai/setups/gptqmodel.md): GPTQ quantization toolkit across GPU vendors.
- [Optimum Quanto](https://insidethemachine.ai/setups/optimum-quanto.md): Simple PyTorch quantization backend.

## Evals & benchmarks

- [ThinkingBox](https://insidethemachine.ai/setups/thinkingbox.md): Runs agents against MCP tools and checks what they really did.
- [promptfoo](https://insidethemachine.ai/setups/promptfoo.md): Test and red-team prompts, agents and RAG.
- [OpenAI Evals](https://insidethemachine.ai/setups/openai-evals.md): Framework and registry for evaluating LLMs.
- [DeepEval](https://insidethemachine.ai/setups/deepeval.md): Unit-test style evaluation for LLM apps.
- [Ragas](https://insidethemachine.ai/setups/ragas.md): Metrics for evaluating RAG pipelines.
- [LM Evaluation Harness](https://insidethemachine.ai/setups/lm-evaluation-harness.md): The standard harness for academic LLM benchmarks.
- [Gorilla / BFCL](https://insidethemachine.ai/setups/gorilla-bfcl.md): Function-calling models and the Berkeley tool-use leaderboard.
- [OpenCompass](https://insidethemachine.ai/setups/opencompass.md): Large evaluation platform across many benchmarks.
- [SWE-bench](https://insidethemachine.ai/setups/swe-bench.md): Benchmark of real GitHub issues for coding agents.
- [simple-evals](https://insidethemachine.ai/setups/simple-evals.md): OpenAI's lightweight reference evals.
- [LMMs-Eval](https://insidethemachine.ai/setups/lmms-eval.md): Evaluation toolkit for multimodal models.
- [Inspect AI](https://insidethemachine.ai/setups/inspect-ai.md): UK AISI framework for LLM and agent evals.
- [Terminal-Bench](https://insidethemachine.ai/setups/terminal-bench.md): Benchmark of hard tasks in a real terminal.
- [LightEval](https://insidethemachine.ai/setups/lighteval.md): Hugging Face's multi-backend eval toolkit.

## Scaffolds & templates

- [AGENTS.md](https://insidethemachine.ai/setups/agents-md.md): A README written for AI agents. [runbook]
- [ClawHub](https://insidethemachine.ai/setups/clawhub.md): Skill and plugin registry for OpenClaw.
- [Superpowers](https://insidethemachine.ai/setups/superpowers.md): Agent skills library and dev methodology.
- [Anthropic Skills](https://insidethemachine.ai/setups/anthropic-skills.md): Anthropic's public Agent Skills examples.
- [Spec Kit](https://insidethemachine.ai/setups/spec-kit.md): Spec-driven development toolkit for agents.
- [OpenSpec](https://insidethemachine.ai/setups/openspec.md): Lightweight spec-driven workflow for coding agents.
- [Awesome Claude Code](https://insidethemachine.ai/setups/awesome-claude-code.md): Curated Claude Code commands, hooks and workflows.
- [BMAD Method](https://insidethemachine.ai/setups/bmad-method.md): Agile multi-agent development method.
- [Awesome Cursor Rules](https://insidethemachine.ai/setups/awesome-cursor-rules.md): Collection of agent rule files.
- [wshobson agents](https://insidethemachine.ai/setups/wshobson-agents.md): Multi-harness agent plugin marketplace.
- [Claude Code Templates](https://insidethemachine.ai/setups/claude-code-templates.md): CLI to install agent configs and templates.
- [Vercel Agent Skills](https://insidethemachine.ai/setups/vercel-agent-skills.md): Vercel's official agent skills.
- [Task Master](https://insidethemachine.ai/setups/task-master.md): Task-management system for coding agents.
- [SOUL.md (persona file)](https://insidethemachine.ai/setups/soul-md.md): A file that sets your agent's identity and voice.

## Observability

- [Langfuse](https://insidethemachine.ai/setups/langfuse.md): Open-source tracing and evals for LLM apps.
- [MLflow](https://insidethemachine.ai/setups/mlflow.md): ML and LLM lifecycle platform with tracing.
- [Opik](https://insidethemachine.ai/setups/opik.md): Trace, evaluate and monitor LLM apps.
- [Phoenix](https://insidethemachine.ai/setups/phoenix.md): Open-source LLM tracing and evaluation.
- [Evidently](https://insidethemachine.ai/setups/evidently.md): Evaluate and monitor ML and LLM systems.
- [OpenLLMetry](https://insidethemachine.ai/setups/openllmetry.md): OpenTelemetry instrumentation for LLM apps.
- [Helicone](https://insidethemachine.ai/setups/helicone.md): LLM observability via a one-line proxy.
- [AgentOps](https://insidethemachine.ai/setups/agentops.md): Monitoring and cost tracking for agents.
- [Agenta](https://insidethemachine.ai/setups/agenta.md): Prompt management, evals and observability.
- [Logfire](https://insidethemachine.ai/setups/logfire.md): Observability for LLM and agent systems.
- [OpenLIT](https://insidethemachine.ai/setups/openlit.md): OpenTelemetry-native tracing and guardrails.
- [Weave](https://insidethemachine.ai/setups/weave.md): W&B toolkit for tracing and evaluating AI apps.

## Hardware & system utilities

- [Apple container](https://insidethemachine.ai/setups/apple-container.md): Run Linux containers natively on a Mac.
- [Stats](https://insidethemachine.ai/setups/stats.md): macOS menu-bar system monitor.
- [llmfit](https://insidethemachine.ai/setups/llmfit.md): One command to see which models fit your hardware.
- [nvtop](https://insidethemachine.ai/setups/nvtop.md): GPU monitor for Apple, AMD, Intel and NVIDIA.
- [nvitop](https://insidethemachine.ai/setups/nvitop.md): Interactive NVIDIA GPU process viewer.
- [GPUStack](https://insidethemachine.ai/setups/gpustack.md): GPU cluster manager for model serving.
- [NVIDIA Container Toolkit](https://insidethemachine.ai/setups/nvidia-container-toolkit.md): Use NVIDIA GPUs inside containers.
- [mactop](https://insidethemachine.ai/setups/mactop.md): Apple Silicon CPU/GPU/power monitor in the terminal.
- [gguf-parser](https://insidethemachine.ai/setups/gguf-parser.md): Inspect GGUF files and estimate memory use.

## Optional

- [YouTube: Inside The Machine](https://www.youtube.com/@InsideTheMachine-s2u)