Audience
AI infrastructure engineers and developers seeking a fast, private local model inference and an OpenAI-compatible backend for applications and coding agents
About BaseRT
BaseRT is a high-performance LLM inference runtime for Apple Silicon that lets developers pull models from Hugging Face, chat with them locally, or serve an OpenAI-compatible API from one CLI. Accelerated by hand-written Metal kernels, it is designed to deliver fast prefill and decode performance on M-series Macs, with published benchmarks showing up to 6.4× faster prefill than llama.cpp, 3.9× faster than MLX, and up to 1.33× faster decode. The basert CLI handles model downloading, conversion, interactive chat, serving, completion, benchmarking, inspection, and bundle signing. Its server supports chat, completions, embeddings, transcription, tool calls, continuous batching, paged KV caching, and prefix caching, while supported models can process text, vision, and audio. BaseRT uses its own .base model format with Q2–Q8 affine quantization, optional AWQ calibration, and signed bundles, and can convert GGUF, Hugging Face, and MLX checkpoints.
Other Popular Alternatives & Related Software
DeepInfer
DeepInfer is an open-source inference framework for running DeepSeek language models on Linux systems with supported NVIDIA or Huawei Ascend accelerators. It provides offline inference and runtime components for model execution, optimized kernels, distributed parallelism, asynchronous batch scheduling, speculative decoding, deterministic generation, and structured outputs such as JSON Schema, regular expressions, and EBNF formats. The project documents DeepSeek-V4.1-Flash support and includes build instructions, configuration guidance, examples, and evaluation tools. It is intended for developers and infrastructure teams building or evaluating self-managed inference deployments. Its repository notes that production services may require additional KV-cache, prefill/decode disaggregation, and load-balancing components not included in this release.
Learn more about DeepInfer
EXAONE Deep
EXAONE Deep is a series of reasoning-enhanced language models developed by LG AI Research, featuring parameter sizes of 2.4 billion, 7.8 billion, and 32 billion. These models demonstrate superior capabilities in various reasoning tasks, including math and coding benchmarks. Notably, EXAONE Deep 2.4B outperforms other models of comparable size, EXAONE Deep 7.8B surpasses both open-weight models of similar scale and the proprietary reasoning model OpenAI o1-mini, and EXAONE Deep 32B shows competitive performance against leading open-weight models. The repository provides comprehensive documentation covering performance evaluations, quickstart guides for using EXAONE Deep models with Transformers, explanations of quantized EXAONE Deep weights in AWQ and GGUF formats, and instructions for running EXAONE Deep models locally using frameworks like llama.cpp and Ollama.
Learn more about EXAONE Deep
LMCache
LMCache is an open source Knowledge Delivery Network (KDN) designed as a caching layer for large language model serving that accelerates inference by reusing KV (key-value) caches across repeated or overlapping computations. It enables fast prompt caching, allowing LLMs to “prefill” recurring text only once and then reuse those stored KV caches, even in non-prefix positions, across multiple serving instances. This approach reduces time to first token, saves GPU cycles, and increases throughput in scenarios such as multi-round question answering or retrieval augmented generation. LMCache supports KV cache offloading (moving cache from GPU to CPU or disk), cache sharing across instances, and disaggregated prefill, which separates the prefill and decoding phases for resource efficiency. It is compatible with inference engines like vLLM and TGI and supports compressed storage, blending techniques to merge caches, and multiple backend storage options.
Learn more about LMCache
Muse Spark Open
Muse Spark Open is an open-weight, natively multimodal language model for developers and organizations building AI applications, coding systems, and agentic workflows. The released model has 816 billion total parameters, 42 billion active parameters, and a 1-million-token context window. Teams can download the model weights, run inference through supported engines such as vLLM, SGLang, TensorRT-LLM, Hugging Face Transformers, Apple MLX, llama.cpp, or Unsloth, and customize the model using PyTorch. It supports long-context reasoning, software engineering, coding, tool use, and multimodal inputs. Deployment options include cloud inference through listed API providers or self-hosting on compatible GPU systems; Meta documents a desktop setup using two Apple M5 Ultra Mac Studio computers. The model is released under the Muse Model License with limited commercial terms.
Learn more about Muse Spark Open
Company Information
Base Compute
Founded: 2026
Australia
www.basecompute.co/getbasert
Videos and Screen Captures
Other Useful Business Software
Ship Agents Faster
Gemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
Product Details
Platforms Supported
Mac
Training
Documentation
Support
Online
BaseRT Frequently Asked Questions
Q: What kinds of users and organization types does BaseRT work with?
BaseRT works with these users and organization types: Mid Size Business, Small Business, Enterprise, Freelance, Nonprofit, Government, and Startup.
Q: What languages does BaseRT support in their product?
BaseRT supports these languages: English.
Q: What kind of support options does BaseRT offer?
BaseRT offers support via online.
Q: What other applications or services does BaseRT integrate with?
BaseRT integrates with: Gemma 3, Gemma 4, Hugging Face, Llama 3.1, Llama 3.2, Mistral AI, OpenAI, Phi-3, Qwen3.5, Qwen3.6, and Qwen3.
Q: Does BaseRT have an API?
Yes, BaseRT has an API.
Q: What type of training does BaseRT provide?
BaseRT provides training in the form of documentation.
