Local Foundation Models
-
PrismML Bonsai 1-bit
Hugging Face: https://huggingface.co/prism-ml/Bonsai-8B-mlx-1bit, https://huggingface.co/prism-ml/Bonsai-4B-mlx-1bit, https://huggingface.co/prism-ml/Bonsai-1.7B-mlx-1bit
GitHub: https://github.com/PrismML-Eng/Bonsai-demo
Website: https://prismml.com
Announcement: https://prismml.com/news/bonsai-8bDeveloper: PrismML
Released: March 2026
Parameters: 8B: 8.19B / 4B: 4.0B / 1.7B: 1.7B
Context: 8B: 65,536 / 4B: 32,768 / 1.7B: 32,768
Architecture: Qwen3 dense base, GQA, SwiGLU MLP, RoPE, RMSNorm; 8B: 36 layers, 32Q/8KV heads / 4B: 36 layers, 32Q/8KV heads / 1.7B: 28 layers, 16Q/8KV heads
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop
Formats: MLX 1-bit g128, GGUF Q1_0_g128
On disk: 8B: 1.28GB MLX, 1.15GB GGUF / 4B: 0.63GB MLX, 0.57GB GGUF / 1.7B: 0.27GB MLX, 0.24GB GGUF -
Liquid AI LFM2.5
Hugging Face: https://huggingface.co/LiquidAI
Website: https://www.liquid.ai
Announcement: https://www.liquid.ai/blog/lfm2-5-230m, https://www.liquid.ai/blog/lfm2-5-350m-no-size-left-behind, https://www.liquid.ai/blog/lfm2-5-1-2b-thinking-on-device-reasoning-under-1gb, https://www.liquid.ai/blog/lfm2-5-2-6b, https://www.liquid.ai/blog/lfm2-5-8b-a1bDeveloper: Liquid AI
Released: Jan-Aug 2026, rolling family
Parameters: LFM2.5-230M: 230M / LFM2.5-350M: 350M / LFM2.5-1.2B-Thinking: 1.2B / LFM2.5-2.6B: 2.6B / LFM2.5-8B-A1B: 8B total, 1B active
Context: 230M: 128K / 350M/1.2B: 32K / 2.6B: 128K / 8B-A1B: 128K
Architecture: hybrid gated-convolution + grouped-query attention; dense for 230M/350M/1.2B/2.6B, MoE for 8B-A1B; 2.6B: 30 layers, 32Q/8KV, SwiGLU, RoPE
License: LFM Open License v1.0
Modalities: Text
Runs on: 230M/350M: Smartphone, Laptop, Edge device / 1.2B-Thinking: Smartphone, Laptop / 2.6B: Smartphone, Laptop / 8B-A1B: Laptop, 8GB+ unified memory
Formats: GGUF Q4_K_M, MLX 4-bit, ONNX
On disk: 230M: 153MB Q4_K_M / 350M: 250MB est. / 1.2B-Thinking: 720MB est. / 2.6B: 1.67GB Q4_K_M / 8B-A1B: 4.8GB est. -
Apple Foundation Model 3 Core
Docs: https://developer.apple.com/documentation/FoundationModels
Developer: Apple
Released: June 2026
Parameters: 3B
Context: 4,096 tokens
Architecture: Dense
License: Proprietary, OS-bundled
Modalities: Text
Runs on: Smartphone, Laptop -
Apple Foundation Model 3 Core Advanced
Docs: https://developer.apple.com/documentation/FoundationModels
Developer: Apple
Released: June 2026
Parameters: 20B total, 1-4B active
Context: 4,096 tokens
Architecture: Sparse MoE with Instruction-Following Pruning
License: Proprietary, OS-bundled
Modalities: Text + Image + Audio
Runs on: Smartphone, Laptop, 12GB+ RAM -
IBM Granite 4.1 3B
Hugging Face: https://huggingface.co/ibm-granite/granite-4.1-3b
GitHub: https://github.com/ibm-granite/granite-4.1-language-models
Announcement: https://huggingface.co/blog/ibm-granite/granite-4-1Developer: IBM
Released: April 2026
Parameters: 3B
Context: 131,072 tokens
Architecture: dense decoder-only transformer
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop, Edge device
Formats: GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, MLX/Core ML via conversion
On disk: 2.2GB Q4_K_M, estimated -
Liquid AI LFM2.5-VL
Hugging Face: https://huggingface.co/LiquidAI/LFM2.5-VL-450M, https://huggingface.co/LiquidAI/LFM2.5-VL-3B
Website: https://www.liquid.ai
Announcement: https://www.liquid.ai/blog/lfm2-5-vl-450m, https://www.liquid.ai/blog/lfm2-5-vl-3bDeveloper: Liquid AI
Released: 450M: April 2026 / 3B: August 2026
Variants: 450M, 3B
Parameters: 450M / 3.1B
Context: 32,768 tokens
Architecture: hybrid gated-convolution + grouped-query attention LM backbone with SigLIP2 NaFlex vision encoder; 450M: LFM2.5-350M backbone, 86M vision encoder / 3B: LFM2.5-2.6B backbone, 30 layers, 32Q/8KV, 400M vision encoder
License: LFM Open License v1.0
Modalities: Text + Vision, bounding box / object detection support
Runs on: 450M: Smartphone, Laptop, Edge device / 3B: Smartphone, Laptop, 3GB memory
Formats: GGUF, ONNX, MLX 4-bit/5-bit/6-bit/8-bit/bf16, safetensors
On disk: 450M: 270MB Q4, estimated / 3B: 1.67GB Q4_K_M, 6.25GB safetensors -
Liquid AI LFM2-24B-A2B
Hugging Face: https://huggingface.co/LiquidAI/LFM2-24B-A2B
GitHub: https://github.com/Liquid4All/cookbook
Website: https://www.liquid.ai
Announcement: https://www.liquid.ai/blog/lfm2-24b-a2bDeveloper: Liquid AI
Released: February 2026
Parameters: 24B total, 2.3B active
Context: 32,768 tokens
Architecture: MoE, 40 layers: 30 conv + 10 attention, 64 experts, top-4 routing
License: LFM Open License v1.0
Modalities: Text
Runs on: Laptop, 14GB+ unified memory minimum, 32GB+ recommended
Formats: GGUF Q4_K_M, Q5_K_M, Q6_K, safetensors, ONNX
On disk: 14.44GB Q4_K_M / 16.93GB Q5_K_M / 19.58GB Q6_K -
Qwen3.6
Hugging Face: https://huggingface.co/Qwen/Qwen3.6-27B, https://huggingface.co/Qwen/Qwen3.6-35B-A3B
GitHub: https://github.com/QwenLM/Qwen3.6
Website: https://qwen.ai/Developer: Alibaba Cloud, Qwen team
Released: April 2026
Parameters: 27B / 35B total, 3B active
Context: 262,144 tokens, extensible to 1,010,000
Architecture: Dense / MoE, 256 experts, 8 routed + 1 shared active
License: Apache 2.0
Modalities: Text + Vision
Runs on: Laptop, 24GB+ unified memory minimum
Formats: GGUF, MLX 4-bit, community quantizations
On disk: 27B: 16GB Q4_K_M, estimated / 35B-A3B: 21GB Q4_K_M, estimated -
PrismML Bonsai Image 4B
Hugging Face: https://huggingface.co/prism-ml/bonsai-image-binary-4B-mlx-1bit, https://huggingface.co/prism-ml/bonsai-image-ternary-4B-mlx-2bit
GitHub: https://github.com/PrismML-Eng/Bonsai-Image-Demo
Website: https://prismml.com
Announcement: https://prismml.com/news/bonsai-image-4bDeveloper: PrismML
Released: May 2026
Parameters: 4B, transformer trunk
Architecture: MMDiT diffusion transformer, base architecture FLUX.2 Klein 4B, 25 blocks: 5 double-stream + 20 single-stream
Variants: Binary, 1-bit / Ternary, 2-bit
License: Apache 2.0
Modalities: Text-to-Image
Runs on: Smartphone, Laptop
Formats: MLX 1-bit, MLX 2-bit, Gemlite 1-bit/2-bit for CUDA, safetensors
On disk: Binary: 0.93GB transformer, 3.42GB total deployment payload / Ternary: 1.21GB transformer -
Google Gemma 4 Effective
Hugging Face: https://huggingface.co/collections/google/gemma-4
GitHub: https://github.com/google-gemma
Website: https://ai.google.dev/gemma/docs/core
Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/Developer: Google DeepMind
Released: April 2026
Variants: E2B, E4B
Parameters: E2B: 2.3B effective, 5.1B with embeddings / E4B: 4.5B effective, 8B with embeddings
Context: 128,000 tokens
Architecture: Dense, hybrid local sliding window + global attention, Per-Layer Embeddings for on-device efficiency
License: Apache 2.0
Modalities: Text + Image + Audio
Runs on: Smartphone, Laptop, Edge device
Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face
On disk: E2B: 1.4GB Q4, estimated / E4B: 2.7GB Q4, estimated -
Google Gemma 4
Hugging Face: https://huggingface.co/collections/google/gemma-4
GitHub: https://github.com/google-gemma
Website: https://ai.google.dev/gemma/docs/core
Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/, https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12B/Developer: Google DeepMind
Released: 26B A4B and 31B: April 2026 / 12B Unified: June 2026
Variants: 12B Unified, 26B A4B, 31B
Parameters: 12B Unified: 11.95B / 26B A4B: 25.2B total, 3.8B active / 31B: 30.7B
Context: 256,000 tokens
Architecture: 31B: Dense / 26B A4B: MoE, 8 active of 128 experts plus 1 shared / 12B: Unified encoder-free dense, multimodal input projected directly into the decoder
License: Apache 2.0
Modalities: 12B: Text + Image + Audio / 26B A4B and 31B: Text + Image
Runs on: Laptop, consumer GPU/workstation class
Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face, Docker
On disk: 12B: ~7GB Q4 / 26B A4B: ~15GB Q4 / 31B: ~18GB Q4 -
NVIDIA Cosmos 3 Edge
Hugging Face: https://huggingface.co/nvidia/Cosmos3-Edge, https://huggingface.co/collections/nvidia/cosmos3
GitHub: https://github.com/nvidia/cosmos
Website: https://research.nvidia.com/labs/cosmos-lab/cosmos3/
White paper: https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdfDeveloper: NVIDIA
Released: July 2026
Variants: Cosmos3-Edge, Cosmos3-Edge-Policy-DROID
Parameters: 4B
Context: Reasoner: 256,000 tokens / Generator text input: 4,096 tokens
Architecture: Mixture-of-Transformers, two towers: an autoregressive transformer for text, a diffusion transformer for image/video/action generation
License: OpenMDW 1.1
Modalities: Text + Image + Video + Action trajectory in / Text + Image + Video + Action out
Formats: safetensors via Hugging Face, PyTorch (NVIDIA-proprietary inference only, BF16)
Runs on: NVIDIA consumer GPU (Linux), Jetson (Linux) -
Agents-A1
Hugging Face: https://huggingface.co/InternScience/Agents-A1, https://huggingface.co/InternScience/Agents-A1-4B
GitHub: https://github.com/InternScience/Agents-A1
X: https://x.com/intern_lm
Website: https://internscience.github.io/Agents-A1/
Technical report: https://arxiv.org/abs/2606.30616Developer: InternScience, Shanghai Artificial Intelligence Laboratory
Released: 35B-A3B: June 2026 / 4B: July 2026
Parameters: 35B total, 3B active / 4B dense
Context: 262,144 tokens
Architecture: Qwen3.5 base, hybrid linear + full attention (full attention every 4th layer); 35B-A3B: MoE, 40 layers, 256 experts, 8 routed + 1 shared active / 4B: dense, 32 layers
License: Apache 2.0
Modalities: Text + Vision
Runs on: 35B-A3B: Laptop, 24GB+ unified memory / 4B: Smartphone, Laptop
Formats: GGUF, MLX (community), safetensors
On disk: 35B-A3B: 19.7GB Q4_K_M / 4B: 2.5GB Q4_K_M, 4.2GB Q8_0 -
Ornith-1.0
Hugging Face: https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B, https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B
GitHub: https://github.com/deepreinforce-ai/Ornith-1
Website: https://deep-reinforce.com/ornith.htmlDeveloper: DeepReinforce
Released: June 2026
Parameters: 9B dense / 35B total, 8 of 256 experts active
Context: 262,144 tokens
Architecture: Qwen3.5 base; 9B: dense, 32 layers / 35B: MoE, 40 layers, 256 experts, 8 active
License: MIT
Modalities: Text + Vision
Runs on: 9B: Smartphone, Laptop / 35B: Laptop, 24GB+ unified memory
Formats: GGUF, FP8, safetensors
On disk: 9B: 5.9GB Q4_K_M / 35B: 21.4GB Q4_K_M -
DeepGrove Maple-Preview
Hugging Face: https://huggingface.co/deepgrove/maple-preview
GitHub: https://github.com/deepgrove-ai/mlx-lm-deepgrove
Website: https://deepgrove.ai/maple-previewDeveloper: DeepGrove
Released: August 2026
Parameters: 20B total, 1B active
Context: 131,072 tokens
Architecture: ternary-weight MoE, 24 layers, 256 experts, 8 active; 3:1 SWA-512:GA attention
License: MIT
Modalities: Text
Runs on: Mac mini M4
Formats: safetensors, MLX 2-bit
On disk: 5.31GB MLX 2-bit -
Nanbeige4.2-3B
Hugging Face: https://huggingface.co/Nanbeige/Nanbeige4.2-3B
Website: https://nanbeige.zhipin.com/
Technical report: https://arxiv.org/abs/2607.22083Developer: Nanbeige, BOSS Zhipin
Released: July 2026
Parameters: 4B total, 3B non-embedding
Context: 262,144 tokens
Architecture: dense Looped Transformer, LoopSplit, mHC with depth attention, concatenated n-gram embeddings
License: Apache 2.0
Modalities: Text
Runs on: Laptop
Formats: safetensors, GGUF Q4_K_M via conversion, MLX 4-bit
On disk: 3.30GB MLX 4-bit -
Fermion Research Neutrino
Hugging Face: https://huggingface.co/FermionResearch/Neutrino-8B, https://huggingface.co/FermionResearch/Neutrino-0.6B, https://huggingface.co/FermionResearch/Neutrino-0.6B-Chat
GitHub: https://github.com/fermionresearch/llama.cpp
Website: https://www.fermionresearch.com/models/neutrino-8b/
Announcement: https://www.fermionresearch.com/research/neutrino-8b/Developer: Fermion Research
Released: July 2026
Variants: 8B general purpose, 0.6B speculative-decoding draft, 0.6B-Chat conversational
Parameters: 8.19B / 596M
Context: 40,960 tokens
Architecture: ternary QAT with staged post-training; 8B: Qwen3-8B topology, 36-layer decoder-only transformer, SwiGLU, GQA 4:1, RoPE, RMSNorm / 0.6B: 28 layers, hidden 1,024, SwiGLU 3,072, GQA 16Q/8KV, per-head Q/K RMSNorm, tied int8 embeddings
License: Apache 2.0
Modalities: Text
Runs on: 8B: Laptop, 16GB unified memory / 0.6B: Laptop, CPU only
Formats: TRTC v4, custom-FV5 GGUF, MLX
On disk: 8B: 3.88GB TRTC v4 / 4.09GB custom-FV5 GGUF / 0.6B: 328MB TRTC v4, 238MB tv4z, 343MB GGUF -
Kakao Kanana-2
Hugging Face: https://huggingface.co/kakaocorp/kanana-2-3b-instruct, https://huggingface.co/kakaocorp/kanana-2-1.3b-instruct
Website: https://tech.kakao.com/posts/826Developer: Kakao, Kanana LLM
Released: July 2026
Variants: 3B, 1.3B
Parameters: 3B / 1.3B
Context: 32,768 tokens
Architecture: 3B: dense, pretrained from scratch, SFT + RL / 1.3B: cascade-pruned and distilled from 3B, sliding-window attention
License: Kanana Open License
Modalities: Text
Runs on: Smartphone, Laptop
Formats: safetensors, MLX 4-bit, 6-bit, 8-bit
On disk: 3B: 1.99GB MLX 4-bit -
Microsoft Fara1.5
Hugging Face: https://huggingface.co/microsoft/Fara1.5-4B, https://huggingface.co/microsoft/Fara1.5-9B, https://huggingface.co/microsoft/Fara1.5-27B
GitHub: https://github.com/microsoft/fara
Website: https://labs.ai.azure.com/innovations/fara1-5/Developer: Microsoft Research AI Frontiers
Released: May 2026
Variants: 4B, 9B, 27B
Parameters: 4B / 9B / 27B
Context: 262,144 tokens
Architecture: multimodal decoder-only LM, image + text to text
License: MIT
Modalities: Text + Image
Formats: safetensors
On disk: 4B: 9.08GB / 9B: 18.82GB / 27B: 54.71GB safetensors -
AMD Instella-MoE-16B-A3B-Think
Hugging Face: https://huggingface.co/amd/Instella-MoE-16B-A3B-Think
GitHub: https://github.com/AMD-AGI/Instella-MoE
Website: https://rocm.blogs.amd.com/artificial-intelligence/instella-moe/README.htmlDeveloper: AMD
Released: July 2026
Parameters: 16B total, 2.8B active
Context: 32,768 tokens
Architecture: decoder-only MoE, Gated Multi-head Latent Attention, FarSkip-Collective connectivity
License: Research RAIL
Modalities: Text
Formats: safetensors
On disk: 31.73GB safetensors
