Local LMs

montez

Megathread for local and open source language and multimodal foundation models that can run on personal computers, phones, and other devices.

montez

From PrismML:

PrismML Bonsai 1-bit

Hugging Face: https://huggingface.co/prism-ml/Bonsai-8B-mlx-1bit, https://huggingface.co/prism-ml/Bonsai-4B-mlx-1bit, https://huggingface.co/prism-ml/Bonsai-1.7B-mlx-1bit
GitHub: https://github.com/PrismML-Eng/Bonsai-demo
Website: https://prismml.com
Announcement: https://prismml.com/news/bonsai-8b

Developer: PrismML
Released: March 2026
Parameters: 8B: 8.19B / 4B: 4.0B / 1.7B: 1.7B
Context: 8B: 65,536 / 4B: 32,768 / 1.7B: 32,768
Architecture: Qwen3 dense base, GQA, SwiGLU MLP, RoPE, RMSNorm; 8B: 36 layers, 32Q/8KV heads / 4B: 36 layers, 32Q/8KV heads / 1.7B: 28 layers, 16Q/8KV heads
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop
Formats: MLX 1-bit g128, GGUF Q1_0_g128
On disk: 8B: 1.28GB MLX, 1.15GB GGUF / 4B: 0.63GB MLX, 0.57GB GGUF / 1.7B: 0.27GB MLX, 0.24GB GGUF

montez

Liquid AI LFM2.5

Hugging Face: https://huggingface.co/collections/LiquidAI/lfm25-695a8eaf38eec2b5748a4868
Website: https://www.liquid.ai/blog/introducing-lfm2-5-the-next-generation-of-on-device-ai

Developer: Liquid AI
Released: Jan-Aug 2026, rolling family
Parameters: LFM2.5-230M: 230M / LFM2.5-350M: 350M / LFM2.5-1.2B-Thinking: 1.2B / LFM2.5-2.6B: 2.6B / LFM2.5-8B-A1B: 8B total, 1B active
Context: 230M: 128K / 350M/1.2B: 32K / 2.6B: 128K / 8B-A1B: 128K
Architecture: hybrid gated-convolution + grouped-query attention; dense for 230M/350M/1.2B/2.6B, MoE for 8B-A1B; 2.6B: 30 layers, 32Q/8KV, SwiGLU, RoPE
License: LFM Open License v1.0
Modalities: Text
Runs on: 230M/350M: Smartphone, Laptop, Edge device / 1.2B-Thinking: Smartphone, Laptop / 2.6B: Smartphone, Laptop / 8B-A1B: Laptop, 8GB+ unified memory
Formats: GGUF Q4_K_M, MLX 4-bit, ONNX
On disk: 230M: 153MB Q4_K_M / 350M: 229MB Q4_K_M / 1.2B-Thinking: 731MB Q4_K_M / 2.6B: 1.67GB Q4_K_M / 8B-A1B: 5.16GB Q4_K_M

montez

Apple Foundation Model 3 Core

Website: https://machinelearning.apple.com/research/introducing-third-generation-of-apple-foundation-models
Docs: https://developer.apple.com/documentation/FoundationModels

Developer: Apple
Released: June 2026
Parameters: 3B
Context: 4,096 tokens
Architecture: Dense
License: Proprietary, OS-bundled
Modalities: Text
Runs on: Smartphone, Laptop

montez

Apple Foundation Model 3 Core Advanced

Website: https://machinelearning.apple.com/research/introducing-third-generation-of-apple-foundation-models
Docs: https://developer.apple.com/documentation/FoundationModels

Developer: Apple
Released: June 2026
Parameters: 20B total, 1-4B active
Context: 4,096 tokens
Architecture: Sparse MoE with Instruction-Following Pruning
License: Proprietary, OS-bundled
Modalities: Text + Image + Audio
Runs on: Smartphone, Laptop, 12GB+ RAM

montez

IBM Granite 4.1 3B

Hugging Face: https://huggingface.co/ibm-granite/granite-4.1-3b
GitHub: https://github.com/ibm-granite/granite-4.1-language-models
Website: https://huggingface.co/blog/ibm-granite/granite-4-1

Developer: IBM
Released: April 2026
Parameters: 3B
Context: 131,072 tokens
Architecture: dense decoder-only transformer
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors BF16, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, MLX and Core ML via community conversion
On disk: 2.10GB Q4_K_M GGUF, 6.81GB BF16 safetensors

montez

Liquid AI LFM2.5-VL

Hugging Face: https://huggingface.co/LiquidAI/LFM2.5-VL-450M, https://huggingface.co/LiquidAI/LFM2.5-VL-3B
Website: https://www.liquid.ai/blog/lfm2-5-vl-3b

Developer: Liquid AI
Released: 450M: April 2026 / 3B: August 2026
Variants: 450M, 3B
Parameters: 450M / 3.1B
Context: 32,768 tokens
Architecture: hybrid gated-convolution + grouped-query attention LM backbone with SigLIP2 NaFlex vision encoder; 450M: LFM2.5-350M backbone, 86M vision encoder / 3B: LFM2.5-2.6B backbone, 30 layers, 32Q/8KV, 400M vision encoder
License: LFM Open License v1.0
Modalities: Text + Image
Runs on: 450M: Smartphone, Laptop, Edge device / 3B: Smartphone, Laptop, 3GB memory
Formats: GGUF, ONNX, MLX 4-bit/5-bit/6-bit/8-bit/bf16, safetensors
On disk: 450M: 229MB Q4_K_M / 3B: 1.67GB Q4_K_M, 6.25GB safetensors

montez

Liquid AI LFM2-24B-A2B

Hugging Face: https://huggingface.co/LiquidAI/LFM2-24B-A2B
GitHub: https://github.com/Liquid4All/cookbook
Website: https://www.liquid.ai/blog/lfm2-24b-a2b

Developer: Liquid AI
Released: February 2026
Parameters: 24B total, 2.3B active
Context: 32,768 tokens
Architecture: MoE, 40 layers: 30 conv + 10 attention, 64 experts, top-4 routing
License: LFM Open License v1.0
Modalities: Text
Runs on: Laptop, 14GB+ unified memory
Formats: GGUF Q4_K_M, Q5_K_M, Q6_K, safetensors, ONNX
On disk: 14.42GB Q4_K_M / 16.92GB Q5_K_M / 19.58GB Q6_K

montez

Qwen3.6

Hugging Face: https://huggingface.co/Qwen/Qwen3.6-27B, https://huggingface.co/Qwen/Qwen3.6-35B-A3B
Website: https://qwen.ai/blog?id=qwen3.6-27b

Developer: Alibaba Cloud, Qwen team
Released: April 2026
Parameters: 27B / 35B total, 3B active
Context: 262,144 tokens, extensible to 1,010,000
Architecture: Dense / MoE, 256 experts, 8 routed + 1 shared active
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: Laptop, 24GB+ unified memory minimum
Formats: safetensors BF16, safetensors FP8, GGUF and MLX via community conversion
On disk: 27B: 55.56GB BF16 safetensors / 35B-A3B: 71.9GB BF16 safetensors

montez

PrismML Bonsai Image 4B

Hugging Face: https://huggingface.co/prism-ml/bonsai-image-binary-4B-mlx-1bit, https://huggingface.co/prism-ml/bonsai-image-ternary-4B-mlx-2bit
GitHub: https://github.com/PrismML-Eng/Bonsai-Image-Demo
Website: https://prismml.com/news/bonsai-image-4b

Developer: PrismML
Released: May 2026
Variants: Binary, 1-bit / Ternary, 2-bit
Parameters: 4B
Architecture: MMDiT diffusion transformer, base architecture FLUX.2 Klein 4B, 25 blocks: 5 double-stream + 20 single-stream
License: Apache 2.0
Modalities: Text-to-Image
Runs on: Smartphone, Laptop
Formats: MLX 1-bit, MLX 2-bit, Gemlite 1-bit/2-bit for CUDA, safetensors
On disk: Binary: 0.93GB transformer / Ternary: 1.21GB transformer

montez

Google Gemma 4 Effective

Hugging Face: https://huggingface.co/collections/google/gemma-4
GitHub: https://github.com/google-deepmind/gemma
Website: https://deepmind.google/models/gemma/gemma-4/
Docs: https://ai.google.dev/gemma/docs/core/model_card_4

Developer: Google DeepMind
Released: April 2026
Variants: E2B, E4B
Parameters: E2B: 2.3B effective, 5.1B with embeddings / E4B: 4.5B effective, 8B with embeddings
Context: 128,000 tokens
Architecture: Dense, hybrid local sliding window + global attention, Per-Layer Embeddings for on-device efficiency
License: Apache 2.0
Modalities: Text + Image + Audio
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors, GGUF Q4_0 QAT
On disk: E2B: 3.35GB Q4_0 GGUF / E4B: 5.15GB Q4_0 GGUF

montez

Google Gemma 4

Hugging Face: https://huggingface.co/collections/google/gemma-4
GitHub: https://github.com/google-deepmind/gemma
Website: https://deepmind.google/models/gemma/gemma-4/
Docs: https://ai.google.dev/gemma/docs/core/model_card_4

Developer: Google DeepMind
Released: 26B A4B and 31B: April 2026 / 12B Unified: June 2026
Variants: 12B Unified, 26B A4B, 31B
Parameters: 12B Unified: 11.95B / 26B A4B: 25.2B total, 3.8B active / 31B: 30.7B
Context: 256,000 tokens
Architecture: 31B: Dense / 26B A4B: MoE, 8 active of 128 experts plus 1 shared / 12B: Unified encoder-free dense, multimodal input projected directly into the decoder
License: Apache 2.0
Modalities: 12B: Text + Image + Audio / 26B A4B and 31B: Text + Image
Runs on: Laptop, Desktop GPU
Formats: safetensors, GGUF Q4_0 QAT
On disk: 12B: 6.98GB Q4_0 GGUF / 26B A4B: 14.44GB Q4_0 GGUF / 31B: 17.65GB Q4_0 GGUF

montez

Agents-A1

Hugging Face: https://huggingface.co/InternScience/Agents-A1, https://huggingface.co/InternScience/Agents-A1-4B
GitHub: https://github.com/InternScience/Agents-A1
Website: https://internscience.github.io/Agents-A1/
Technical report: https://arxiv.org/abs/2606.30616

Developer: InternScience, Shanghai Artificial Intelligence Laboratory
Released: 35B-A3B: June 2026 / 4B: July 2026
Parameters: 35B total, 3B active / 4B dense
Context: 262,144 tokens
Architecture: Qwen3.5 base, hybrid linear + full attention (full attention every 4th layer); 35B-A3B: MoE, 40 layers, 256 experts, 8 routed + 1 shared active / 4B: dense, 32 layers
License: Apache 2.0
Modalities: Text + Image
Runs on: 35B-A3B: Laptop, 24GB+ unified memory / 4B: Smartphone, Laptop
Formats: safetensors, GGUF, MLX via community conversion
On disk: 35B-A3B: 19.7GB Q4_K_M / 4B: 2.5GB Q4_K_M, 4.2GB Q8_0

montez

Ornith-1.0

Hugging Face: https://huggingface.co/ornith-ai/Ornith-1.0-9B, https://huggingface.co/ornith-ai/Ornith-1.0-35B
GitHub: https://github.com/ornith-ai/Ornith-1
Website: https://ornith.ai/ornith_1_0.html

Developer: Ornith
Released: June 2026
Parameters: 9B dense / 35B total
Context: 262,144 tokens
Architecture: Qwen3.5 base; 9B: dense, 32 layers / 35B: MoE, 40 layers, 256 experts, 8 active
License: MIT
Modalities: Text + Image
Runs on: 9B: Smartphone, Laptop / 35B: Laptop, 24GB+ unified memory
Formats: GGUF, FP8, safetensors
On disk: 9B: 5.9GB Q4_K_M / 35B: 21.4GB Q4_K_M

montez

DeepGrove Maple-Preview

Hugging Face: https://huggingface.co/deepgrove/maple-preview
GitHub: https://github.com/deepgrove-ai/mlx-lm-deepgrove
Website: https://deepgrove.ai/modelcard/

Developer: DeepGrove
Released: August 2026
Parameters: 20B total, 1B active
Context: 131,072 tokens
Architecture: ternary-weight MoE, 24 layers, 256 experts, 8 active; 3:1 SWA-512:GA attention
License: MIT
Modalities: Text
Runs on: Mac mini M4
Formats: safetensors, MLX 2-bit
On disk: 5.31GB MLX 2-bit