oMLX
Native macOS server for fast local AI model inference
Score
60
User Rating
3.0
Free/Trial Support
Not Supported
Starting Price
--
Last Updated
Sep 12, 2026
Overview
What is oMLX?
oMLX is a native macOS inference server for running local AI models on Apple Silicon using Apple's MLX framework. It provides paged SSD KV caching to reduce time-to-first-token for coding agents, continuous batching for higher throughput, and OpenAI- and Anthropic-compatible APIs. oMLX supports LLM, vision-language, embedding, and reranker models, along with tool calling, MCP integration, multi-model serving, and a native menu bar application with a web dashboard.
What is oMLX used for?
Top Features
- Paged SSD KV caching for faster responses and reduced time-to-first-token
- Continuous batching for concurrent request throughput
- OpenAI-compatible and Anthropic-compatible APIs
- Native macOS menu bar application
- Web dashboard for model management, chat, and real-time metrics
- Multi-model serving for LLM, VLM, embedding, and reranker models
- Tool calling and MCP integration
- Support for MLX-format models from Hugging Face
- LRU model eviction when memory is limited
- Automatic reuse of Hugging Face and LM Studio model caches
How to use oMLX?
Download the signed macOS DMG and drag oMLX to the Applications folder, then configure the model directory, start the server, and download or select an MLX-format model. Alternatively, install from source with Python 3.10+ using the provided Git commands and run omlx serve with a model directory. Connect Claude Code, OpenClaw, Cursor, or other compatible clients through the OpenAI or Anthropic API endpoints.
Alternative Tools
No alternative tools found
Pros & Cons
No Data
Use Cases
- Run local coding agents with faster responses on Apple Silicon Macs
- Use Claude Code, OpenClaw, or Cursor with a local AI backend
- Serve multiple local language and vision models through compatible APIs
- Build applications using local OpenAI-compatible or Anthropic-compatible inference
- Test tool calling and MCP workflows without relying on cloud inference
- Run private local chat, embedding, reranking, and multimodal workloads
User Reviews
3.0
out of 5
3.0
No reviews yet
Be the first to review this tool!