Local LLMs
-
PrismML Bonsai 1-bit
Hugging Face: https://huggingface.co/prism-ml/Bonsai-8B-mlx-1bit, https://huggingface.co/prism-ml/Bonsai-4B-mlx-1bit, https://huggingface.co/prism-ml/Bonsai-1.7B-mlx-1bit
GitHub: https://github.com/PrismML-Eng/Bonsai-demo
Website: https://prismml.com
Announcement: https://prismml.com/news/bonsai-8bDeveloper: PrismML
Released: March 2026
Parameters: 8B: 8.19B / 4B: 4.0B / 1.7B: 1.7B
Context: 8B: 65,536 / 4B: 32,768 / 1.7B: 32,768
Architecture: Qwen3 dense base, GQA, SwiGLU MLP, RoPE, RMSNorm; 8B: 36 layers, 32Q/8KV heads / 4B: 36 layers, 32Q/8KV heads / 1.7B: 28 layers, 16Q/8KV heads
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop
Formats: MLX 1-bit g128, GGUF Q1_0_g128
On disk: 8B: 1.28GB MLX, 1.15GB GGUF / 4B: 0.63GB MLX, 0.57GB GGUF / 1.7B: 0.27GB MLX, 0.24GB GGUF -
Liquid AI LFM2.5
Hugging Face: https://huggingface.co/LiquidAI
Website: https://www.liquid.ai
Announcement: https://www.liquid.ai/blog/lfm2-5-230m, https://www.liquid.ai/blog/lfm2-5-350m-no-size-left-behind, https://www.liquid.ai/blog/lfm2-5-1-2b-thinking-on-device-reasoning-under-1gb, https://www.liquid.ai/blog/lfm2-5-8b-a1bDeveloper: Liquid AI
Released: Jan-Jun 2026, rolling family
Parameters: LFM2.5-230M: 230M / LFM2.5-350M: 350M / LFM2.5-1.2B-Thinking: 1.2B / LFM2.5-8B-A1B: 8B total, 1B active
Context: 230M/350M/1.2B: 32K / 8B-A1B: 128K
Architecture: hybrid gated-convolution + grouped-query attention; dense for 230M/350M/1.2B, MoE for 8B-A1B
License: LFM Open License v1.0
Modalities: Text
Runs on: 230M/350M: Smartphone, Laptop, Edge device / 1.2B-Thinking: Smartphone, Laptop / 8B-A1B: Laptop, 8GB+ unified memory
Formats: GGUF Q4_K_M, MLX 4-bit, ONNX
On disk: 230M: 153MB / 350M: 250MB est. / 1.2B-Thinking: 720MB est. / 8B-A1B: 4.8GB est. -
Apple Foundation Model 3 Core
Docs: https://developer.apple.com/documentation/FoundationModels
Developer: Apple
Released: June 2026, WWDC
Parameters: 3B
Context: 4,096 tokens
Architecture: Dense
License: Proprietary, OS-bundled
Modalities: Text
Runs on: Smartphone, Laptop -
Apple Foundation Model 3 Core Advanced
Framework: Apple Foundation Models
Docs: https://developer.apple.com/documentation/FoundationModelsDeveloper: Apple
Released: June 2026, WWDC
Parameters: 20B total, 1-4B active
Context: 4,096 tokens
Architecture: Sparse MoE with Instruction-Following Pruning
License: Proprietary, OS-bundled
Modalities: Text + Image + Audio
Runs on: Smartphone, Laptop, 12GB+ RAM -
IBM Granite 4.1 3B
Hugging Face: https://huggingface.co/ibm-granite/granite-4.1-3b
GitHub: https://github.com/ibm-granite/granite-4.1-language-models
Announcement: https://huggingface.co/blog/ibm-granite/granite-4-1Developer: IBM
Released: April 2026
Parameters: 3B
Context: 512,000 tokens
Architecture: dense decoder-only transformer
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, 2GB at 4-bit, estimated, Laptop, Edge device
Formats: GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, MLX/Core ML via conversion
On disk: 2.2GB Q4_K_M, estimated -
Liquid AI LFM2.5-VL-450M
Hugging Face: https://huggingface.co/LiquidAI/LFM2.5-VL-450M
Website: https://www.liquid.ai
Announcement: https://www.liquid.ai/blog/lfm2-5-vl-450mDeveloper: Liquid AI
Released: April 2026
Parameters: 450M
Context: 32,768 tokens
Architecture: hybrid gated-convolution + grouped-query attention LM backbone, LFM2.5-350M, with SigLIP2 NaFlex 86M vision encoder
License: LFM Open License v1.0
Modalities: Text + Vision, bounding box / object detection support
Runs on: Smartphone, Laptop, Edge device
Formats: GGUF, ONNX, MLX 4-bit/5-bit/6-bit/8-bit/bf16, safetensors
On disk: 270MB Q4, estimated -
Liquid AI LFM2-24B-A2B
Hugging Face: https://huggingface.co/LiquidAI/LFM2-24B-A2B
GitHub: https://github.com/Liquid4All/cookbook
Website: https://www.liquid.ai
Announcement: https://www.liquid.ai/blog/lfm2-24b-a2bDeveloper: Liquid AI
Released: February 2026
Parameters: 24B total, 2.3B active
Context: 32,768 tokens
Architecture: MoE, 40 layers: 30 conv + 10 attention, 64 experts, top-4 routing
License: LFM Open License v1.0
Modalities: Text
Runs on: Laptop, 14GB+ unified memory minimum, 32GB+ recommended
Formats: GGUF Q4_K_M, Q5_K_M, Q6_K, safetensors, ONNX
On disk: 14.44GB Q4_K_M / 16.93GB Q5_K_M / 19.58GB Q6_K -
Qwen3.6
Hugging Face: https://huggingface.co/Qwen/Qwen3.6-27B, https://huggingface.co/Qwen/Qwen3.6-35B-A3B
GitHub: https://github.com/QwenLM/Qwen3.6
Website: https://qwen.ai/Developer: Alibaba Cloud, Qwen team
Released: April 2026
Parameters: 27B / 35B total, 3B active
Context: 262,144 tokens, extensible to 1,010,000
Architecture: Dense / MoE, 256 experts, 8 routed + 1 shared active
License: Apache 2.0
Modalities: Text + Vision
Runs on: Laptop, 24GB+ unified memory minimum
Formats: GGUF, MLX 4-bit, community quantizations
On disk: 27B: 16GB Q4_K_M, estimated / 35B-A3B: 21GB Q4_K_M, estimated -
PrismML Bonsai Image 4B
Hugging Face: https://huggingface.co/prism-ml/bonsai-image-binary-4B-mlx-1bit, https://huggingface.co/prism-ml/bonsai-image-ternary-4B-mlx-2bit
GitHub: https://github.com/PrismML-Eng/Bonsai-Image-Demo
Website: https://prismml.com
Announcement: https://prismml.com/news/bonsai-image-4bDeveloper: PrismML
Released: May 2026
Parameters: 4B, transformer trunk
Architecture: MMDiT diffusion transformer, base architecture FLUX.2 Klein 4B, 25 blocks: 5 double-stream + 20 single-stream
Variants: Binary, 1-bit / Ternary, 2-bit
License: Apache 2.0
Modalities: Text-to-Image
Runs on: Smartphone, Laptop
Formats: MLX 1-bit, MLX 2-bit, Gemlite 1-bit/2-bit for CUDA, safetensors
On disk: Binary: 0.93GB transformer, 3.42GB total deployment payload / Ternary: 1.21GB transformer -
Google Gemma 4 Effective
Hugging Face: https://huggingface.co/collections/google/gemma-4
GitHub: https://github.com/google-gemma
Website: https://ai.google.dev/gemma/docs/core
Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/Developer: Google DeepMind
Released: April 2026
Variants: E2B, E4B
Parameters: E2B: 2.3B effective, 5.1B with embeddings / E4B: 4.5B effective, 8B with embeddings
Context: 128,000 tokens
Architecture: Dense, hybrid local sliding window + global attention, Per-Layer Embeddings for on-device efficiency
License: Apache 2.0
Modalities: Text + Image + Audio
Runs on: Smartphone, Laptop, Edge device
Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face
On disk: E2B: 1.4GB Q4, estimated / E4B: 2.7GB Q4, estimated -
Google Gemma 4
Hugging Face: https://huggingface.co/collections/google/gemma-4
GitHub: https://github.com/google-gemma
Website: https://ai.google.dev/gemma/docs/core
Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/, https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12B/Developer: Google DeepMind
Released: 26B A4B and 31B: April 2026 / 12B Unified: June 2026
Variants: 12B Unified, 26B A4B, 31B
Parameters: 12B Unified: 11.95B / 26B A4B: 25.2B total, 3.8B active / 31B: 30.7B
Context: 256,000 tokens
Architecture: 31B: Dense / 26B A4B: MoE, 8 active of 128 experts plus 1 shared / 12B: Unified encoder-free dense, multimodal input projected directly into the decoder
License: Apache 2.0
Modalities: 12B: Text + Image + Audio / 26B A4B and 31B: Text + Image
Runs on: Laptop, consumer GPU/workstation class
Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face, Docker
On disk: 12B: 7GB Q4, estimated / 26B A4B: 15GB Q4, estimated / 31B: 18GB Q4, estimated
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login