API Reference

Relevant source files

• LLama.Examples/Examples/BatchedExecutorBoolQ.cs
• LLama/LLamaContext.cs
• LLama/LLamaWeights.cs
• LLama/Native/LLamaTokenDataArray.cs
• LLama/Native/NativeApi.cs
• LLama/Native/SafeLLamaContextHandle.cs
• LLama/Native/SafeLlamaModelHandle.cs
• LLama/Sampling/BaseSamplingPipeline.cs
• LLama/Sampling/DefaultSamplingPipeline.cs
• LLama/Sampling/GreedySamplingPipeline.cs
• LLama/Sampling/ISamplingPipeline.cs

This section provides comprehensive reference documentation for all public APIs in LLamaSharp. The library is organized into distinct abstraction layers, from high-level convenience APIs down to direct native interop. This page provides an overview of the API organization and directs you to detailed documentation for specific components.

For step-by-step usage examples, see Quick Start Guide. For architectural concepts behind these APIs, see Core Architecture.

---

API Organization

LLamaSharp exposes APIs at three distinct abstraction levels:

High-Level API - User-facing classes that provide safe, convenient interfaces:

• Model and context management (Core API Classes)
• Execution patterns (Executor API, Chat Sessions)
• Token sampling (Sampling API)
• Text embeddings and Reranking (Core API Classes)

Safe Handle Layer - Resource management wrappers:

• SafeLlamaModelHandle - Manages native model pointers with automatic cleanup LLama/Native/SafeLlamaModelHandle.cs15-17
• SafeLLamaContextHandle - Manages native context pointers with automatic cleanup LLama/Native/SafeLLamaContextHandle.cs13-15
• SafeMtmdModelHandle - Manages multimodal model resources.

Native API Layer - Direct P/Invoke declarations to llama.cpp (Native Interop API):

• Raw native functions in NativeApi class LLama/Native/NativeApi.cs11-12
• Native structure definitions (LLamaModelParams, LLamaContextParams, etc.)

---

Architecture Diagram

System Component Overview

The following diagram bridges the high-level managed classes with the low-level native handles and structures they wrap.

Sources: LLama/LLamaWeights.cs17-24 LLama/LLamaContext.cs18-42 LLama/Native/SafeLlamaModelHandle.cs15-17 LLama/Native/SafeLLamaContextHandle.cs13-15 LLama/Native/NativeApi.cs11-12

---

Core API Classes

The following table provides a quick reference to the main classes you'll work with:

Class	Purpose	Layer	Reference
LLamaWeights	Load and manage model weights	High-Level	Core API Classes
LLamaContext	Create inference contexts for generation	High-Level	Core API Classes
ILLamaExecutor	Interface for execution patterns	High-Level	Executor API
InteractiveExecutor	Stateful conversational inference	High-Level	Executor API
InstructExecutor	Stateful instruction-following inference	High-Level	Executor API
StatelessExecutor	One-shot inference without state	High-Level	Executor API
SafeLlamaModelHandle	Safe wrapper for native model pointers	Safe Handle	Native Interop API
SafeLLamaContextHandle	Safe wrapper for native context pointers	Safe Handle	Native Interop API
NativeApi	Direct P/Invoke declarations	Native	Native Interop API

Sources: LLama/LLamaWeights.cs17-19 LLama/LLamaContext.cs18-20 LLama/Native/SafeLlamaModelHandle.cs15-17 LLama/Native/SafeLLamaContextHandle.cs13-15

---

Object Lifecycle and Relationships

Inference Pipeline Lifecycle

This diagram illustrates the progression from loading weights to running a session, highlighting the code entities involved.

Typical Usage Pattern:

1. Load model weights with LLamaWeights.LoadFromFile() LLama/LLamaWeights.cs67-72 - creates SafeLlamaModelHandle internally.
2. Create inference context with weights.CreateContext() LLama/LLamaWeights.cs152-155 - creates SafeLLamaContextHandle internally LLama/LLamaContext.cs95
3. Create executor (e.g., InteractiveExecutor) wrapping the context.
4. Dispose in reverse order (executor → context → weights) or use using statements.

Sources: LLama/LLamaWeights.cs67-72 LLama/LLamaContext.cs84-98 LLama/LLamaWeights.cs152-155

---

Parameter Configuration Flow

Configuration flows from user-friendly interfaces through extension methods that perform memory pinning and conversion to native structures. See Configuration Reference for all available parameters.

Sources: LLama/LLamaWeights.cs67-72 LLama/LLamaContext.cs84-95 LLama/Native/SafeLlamaModelHandle.cs136-151 LLama/Native/SafeLLamaContextHandle.cs109-122

---

Finding the Right API

Use this guide to navigate to the appropriate reference section:

I want to...	Go to
Load a model or access model metadata	Core API Classes
Tokenize text or manage inference context	Core API Classes
Run conversational or instruction-based inference	Executor API
Control token selection (temperature, top-p, etc.)	Sampling API
Work with low-level native handles	Native Interop API

---

Common API Patterns

Resource Disposal

All native resources implement IDisposable. Always dispose or use using statements:

Sources: LLama/LLamaWeights.cs141-144 LLama/LLamaContext.cs19-20

Safe Handle Reference Counting

SafeLLamaContextHandle maintains a reference to its parent SafeLlamaModelHandle. This prevents the model from being disposed while contexts still reference it LLama/Native/SafeLLamaContextHandle.cs115-121

Sources: LLama/Native/SafeLLamaContextHandle.cs109-122 LLama/Native/SafeLLamaContextHandle.cs80-90

Async Operations

Most blocking operations have async variants:

• LLamaWeights.LoadFromFileAsync() - with cancellation and progress reporting LLama/LLamaWeights.cs83-138
• context.DecodeAsync() - async inference
• executor.InferAsync() - async generation with streaming

Sources: LLama/LLamaWeights.cs83-138

Tokenization

Tokenization is available at multiple levels with the same underlying implementation:

Sources: LLama/LLamaWeights.cs165-168 LLama/LLamaContext.cs107-110

---

Vocabulary Access

Special tokens and vocabulary information are accessible through the Vocab property:

The Vocabulary class provides access to all special tokens (BOS, EOS, etc.) and vocabulary metadata LLama/Native/SafeLlamaModelHandle.cs118-120

Sources: LLama/LLamaWeights.cs49 LLama/LLamaContext.cs75 LLama/Native/SafeLlamaModelHandle.cs118-120

---

State Persistence

Save and restore context state for resuming inference:

Stateful executors also support session files to warm up the KV cache via WithSessionFile, which internally uses NativeApi.llama_state_load_file LLama/Native/NativeApi.cs107-109

Sources: LLama/LLamaContext.cs128-171 LLama/Native/NativeApi.cs107-133

---

Performance and Threading

Thread Configuration

Control thread counts for generation and batch processing directly on the context or its native handle:

Sources: LLama/LLamaContext.cs51-65 LLama/Native/SafeLLamaContextHandle.cs45-58

---

Reference Sections

• Core API Classes - Detailed reference for LLamaWeights, LLamaContext, LLamaEmbedder, LLamaReranker, and fundamental types.
• Executor API - Reference for ILLamaExecutor, all executor implementations (Interactive, Instruct, Stateless, Batched), and ChatSession.
• Sampling API - Reference for ISamplingPipeline, grammar-constrained sampling, and specific sampler implementations.
• Native Interop API - Low-level reference for NativeApi (P/Invoke), SafeHandles hierarchy, and native parameter structures.

Sources: LLama/LLamaWeights.cs1-171 LLama/LLamaContext.cs1-589 LLama/Native/SafeLlamaModelHandle.cs1-199 LLama/Native/SafeLLamaContextHandle.cs1-180 LLama/Native/NativeApi.cs1-156