Local LMs

[montez]

Google Gemma 4 Effective

Hugging Face: https://huggingface.co/collections/google/gemma-4
GitHub: https://github.com/google-gemma
Website: https://ai.google.dev/gemma/docs/core
Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/

Developer: Google DeepMind
Released: April 2026
Variants: E2B, E4B
Parameters: E2B: 2.3B effective, 5.1B with embeddings / E4B: 4.5B effective, 8B with embeddings
Context: 128,000 tokens
Architecture: Dense, hybrid local sliding window + global attention, Per-Layer Embeddings for on-device efficiency
License: Apache 2.0
Modalities: Text + Image + Audio
Runs on: Smartphone, Laptop, Edge device
Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face
On disk: E2B: 1.4GB Q4, estimated / E4B: 2.7GB Q4, estimated

[montez]

Google Gemma 4

Hugging Face: https://huggingface.co/collections/google/gemma-4
GitHub: https://github.com/google-gemma
Website: https://ai.google.dev/gemma/docs/core
Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/, https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12B/

Developer: Google DeepMind
Released: 26B A4B and 31B: April 2026 / 12B Unified: June 2026
Variants: 12B Unified, 26B A4B, 31B
Parameters: 12B Unified: 11.95B / 26B A4B: 25.2B total, 3.8B active / 31B: 30.7B
Context: 256,000 tokens
Architecture: 31B: Dense / 26B A4B: MoE, 8 active of 128 experts plus 1 shared / 12B: Unified encoder-free dense, multimodal input projected directly into the decoder
License: Apache 2.0
Modalities: 12B: Text + Image + Audio / 26B A4B and 31B: Text + Image
Runs on: Laptop, consumer GPU/workstation class
Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face, Docker
On disk: 12B: ~7GB Q4 / 26B A4B: ~15GB Q4 / 31B: ~18GB Q4

[montez]

Agents-A1

Hugging Face: https://huggingface.co/InternScience/Agents-A1, https://huggingface.co/InternScience/Agents-A1-4B
GitHub: https://github.com/InternScience/Agents-A1
X: https://x.com/intern_lm
Website: https://internscience.github.io/Agents-A1/
Technical report: https://arxiv.org/abs/2606.30616

Developer: InternScience, Shanghai Artificial Intelligence Laboratory
Released: 35B-A3B: June 2026 / 4B: July 2026
Parameters: 35B total, 3B active / 4B dense
Context: 262,144 tokens
Architecture: Qwen3.5 base, hybrid linear + full attention (full attention every 4th layer); 35B-A3B: MoE, 40 layers, 256 experts, 8 routed + 1 shared active / 4B: dense, 32 layers
License: Apache 2.0
Modalities: Text + Vision
Runs on: 35B-A3B: Laptop, 24GB+ unified memory / 4B: Smartphone, Laptop
Formats: GGUF, MLX (community), safetensors
On disk: 35B-A3B: 19.7GB Q4_K_M / 4B: 2.5GB Q4_K_M, 4.2GB Q8_0

[montez]

Ornith-1.0

Hugging Face: https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B, https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B
GitHub: https://github.com/deepreinforce-ai/Ornith-1
Website: https://deep-reinforce.com/ornith.html

Developer: DeepReinforce
Released: June 2026
Parameters: 9B dense / 35B total, 8 of 256 experts active
Context: 262,144 tokens
Architecture: Qwen3.5 base; 9B: dense, 32 layers / 35B: MoE, 40 layers, 256 experts, 8 active
License: MIT
Modalities: Text + Vision
Runs on: 9B: Smartphone, Laptop / 35B: Laptop, 24GB+ unified memory
Formats: GGUF, FP8, safetensors
On disk: 9B: 5.9GB Q4_K_M / 35B: 21.4GB Q4_K_M

[montez]

DeepGrove Maple-Preview

Hugging Face: https://huggingface.co/deepgrove/maple-preview
GitHub: https://github.com/deepgrove-ai/mlx-lm-deepgrove
Website: https://deepgrove.ai/maple-preview

Developer: DeepGrove
Released: August 2026
Parameters: 20B total, 1B active
Context: 131,072 tokens
Architecture: ternary-weight MoE, 24 layers, 256 experts, 8 active; 3:1 SWA-512:GA attention
License: MIT
Modalities: Text
Runs on: Mac mini M4
Formats: safetensors, MLX 2-bit
On disk: 5.31GB MLX 2-bit

[montez]

Nanbeige4.2-3B

Hugging Face: https://huggingface.co/Nanbeige/Nanbeige4.2-3B
Website: https://nanbeige.zhipin.com/
Technical report: https://arxiv.org/abs/2607.22083

Developer: Nanbeige, BOSS Zhipin
Released: July 2026
Parameters: 4B total, 3B non-embedding
Context: 262,144 tokens
Architecture: dense Looped Transformer, LoopSplit, mHC with depth attention, concatenated n-gram embeddings
License: Apache 2.0
Modalities: Text
Runs on: Laptop
Formats: safetensors, GGUF Q4_K_M via conversion, MLX 4-bit
On disk: 3.30GB MLX 4-bit

[montez]

Fermion Research Neutrino

Hugging Face: https://huggingface.co/FermionResearch/Neutrino-8B, https://huggingface.co/FermionResearch/Neutrino-0.6B, https://huggingface.co/FermionResearch/Neutrino-0.6B-Chat
GitHub: https://github.com/fermionresearch/llama.cpp
Website: https://www.fermionresearch.com/models/neutrino-8b/
Announcement: https://www.fermionresearch.com/research/neutrino-8b/

Developer: Fermion Research
Released: July 2026
Variants: 8B general purpose, 0.6B speculative-decoding draft, 0.6B-Chat conversational
Parameters: 8.19B / 596M
Context: 40,960 tokens
Architecture: ternary QAT with staged post-training; 8B: Qwen3-8B topology, 36-layer decoder-only transformer, SwiGLU, GQA 4:1, RoPE, RMSNorm / 0.6B: 28 layers, hidden 1,024, SwiGLU 3,072, GQA 16Q/8KV, per-head Q/K RMSNorm, tied int8 embeddings
License: Apache 2.0
Modalities: Text
Runs on: 8B: Laptop, 16GB unified memory / 0.6B: Laptop, CPU only
Formats: TRTC v4, custom-FV5 GGUF, MLX
On disk: 8B: 3.88GB TRTC v4 / 4.09GB custom-FV5 GGUF / 0.6B: 328MB TRTC v4, 238MB tv4z, 343MB GGUF

[montez]

Kakao Kanana-2

Hugging Face: https://huggingface.co/kakaocorp/kanana-2-3b-instruct, https://huggingface.co/kakaocorp/kanana-2-1.3b-instruct
Website: https://tech.kakao.com/posts/826

Developer: Kakao, Kanana LLM
Released: July 2026
Variants: 3B, 1.3B
Parameters: 3B / 1.3B
Context: 32,768 tokens
Architecture: 3B: dense, pretrained from scratch, SFT + RL / 1.3B: cascade-pruned and distilled from 3B, sliding-window attention
License: Kanana Open License
Modalities: Text
Runs on: Smartphone, Laptop
Formats: safetensors, MLX 4-bit, 6-bit, 8-bit
On disk: 3B: 1.99GB MLX 4-bit

[montez]

Microsoft Fara1.5

Hugging Face: https://huggingface.co/microsoft/Fara1.5-4B, https://huggingface.co/microsoft/Fara1.5-9B, https://huggingface.co/microsoft/Fara1.5-27B
GitHub: https://github.com/microsoft/fara
Website: https://labs.ai.azure.com/innovations/fara1-5/

Developer: Microsoft Research AI Frontiers
Released: May 2026
Variants: 4B, 9B, 27B
Parameters: 4B / 9B / 27B
Context: 262,144 tokens
Architecture: multimodal decoder-only LM, image + text to text
License: MIT
Modalities: Text + Image
Formats: safetensors
On disk: 4B: 9.08GB / 9B: 18.82GB / 27B: 54.71GB safetensors

[montez]

AMD Instella-MoE-16B-A3B-Think

Hugging Face: https://huggingface.co/amd/Instella-MoE-16B-A3B-Think
GitHub: https://github.com/AMD-AGI/Instella-MoE
Website: https://rocm.blogs.amd.com/artificial-intelligence/instella-moe/README.html

Developer: AMD
Released: July 2026
Parameters: 16B total, 2.8B active
Context: 32,768 tokens
Architecture: decoder-only MoE, Gated Multi-head Latent Attention, FarSkip-Collective connectivity
License: Research RAIL
Modalities: Text
Formats: safetensors
On disk: 31.73GB safetensors

[montez]

Cisco Antares

Hugging Face: https://huggingface.co/fdtn-ai/antares-1b, https://huggingface.co/fdtn-ai/antares-350m
Website: https://cisco-foundation-ai.github.io/antares/
Announcement: https://cisco-foundation-ai.github.io/blogs/antares-beyond-vlocbench/

Developer: Cisco Foundation AI
Released: July 2026
Variants: 1b, 350m
Parameters: 1B / 350M
Architecture: fine-tuned from IBM Granite 4.0, GraniteMoEHybrid
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors
On disk: 1b: 3.67GB safetensors / 350m: 0.70GB safetensors

[montez]

AI9Stars G9v3-3B

Hugging Face: https://huggingface.co/ai9stars/G9v3-3B
GitHub: https://github.com/AI9Stars

Developer: AI9Stars
Parameters: ~3B
Context: 131,072 tokens
Architecture: dense causal LM, LlamaForCausalLM
License: Apache 2.0
Modalities: Text
Formats: safetensors
On disk: 5.99GB safetensors

[montez]

Poolside Laguna XS 2.1

Hugging Face: https://huggingface.co/poolside/Laguna-XS-2.1, https://huggingface.co/poolside/Laguna-XS-2.1-GGUF
Website: https://poolside.ai/blog/introducing-laguna-xs-2-1

Developer: Poolside
Released: July 2026
Variants: BF16, FP8, NVFP4, INT4; GGUF BF16, Q4_K_M
Parameters: 33B total, 3B active
Context: 262,144 tokens
Architecture: MoE, 40 layers: 10 global-attention + 30 sliding-window-attention; 256 experts + 1 shared expert
License: OpenMDW-1.1
Modalities: Text
Runs on: Mac with 36GB RAM
Formats: safetensors, GGUF
On disk: 20.27GB Q4_K_M GGUF / 66.89GB BF16 safetensors

[montez]

Meta Muse Glimmer-30B

Hugging Face: https://huggingface.co/meta-models/Muse-Glimmer-30B
Announcement: https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model
Technical report: https://research.meta.ai/static/muse-glimmer-methodology

Developer: Meta Superintelligence Lab
Released: August 2026
Parameters: 29.6B total, including perception encoder
Context: 131,072+ tokens
Architecture: Dense causal transformer with ViT-G/14 perception encoder; 52 layers, GQA, SwiGLU, RoPE
License: Apache 2.0
Modalities: Text + Image
Runs on: MacBook M4 Max/M5 Max, RTX 5090; 24GB+ memory with 4-bit weights
Formats: BF16 safetensors, 4-bit quantized weights
On disk: 17GB K-Quant

[montez]

webAI TwIL-LM

Hugging Face: https://huggingface.co/webAI-Official/TwIL-LM, https://huggingface.co/webAI-Official/TwIL-LM3
Website: https://www.webai.com/blog/webai-releases-twil-lm-a-family-of-formal-logic-models-that-outreason-a-120b-model-and-run-on-an-iphone

Developer: webAI Intelligence Lab
Released: August 2026
Variants: TwIL-LM 1.7B, TwIL-LM3 3B
Parameters: 1.7B: 1.78B total, 1.71B backbone + 72M LoRA / 3B: 3B
Context: 1.7B: 8,192 tokens / 3B: 65,536 tokens
Architecture: 1.7B: SmolLM2-1.7B-Instruct base, dense Llama, 24 layers, 32 heads, LoRA rank 64 SFT / 3B: SmolLM3-3B base, dense, 36 layers, GQA 16Q/4KV, NoPE every 4th layer; LoRA SFT, checkpoint fusion, WiSE-FT interpolation, GRPO reinforcement learning
License: webAI Non-Commercial License ver. 1.0
Modalities: Text
Runs on: 1.7B: Smartphone, Laptop / 3B: Laptop, 4GB VRAM or CPU
Formats: 1.7B: merged GGUF Q4_K_M, Q5_K_M, Q8_0, f16 / 3B: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16
On disk: 1.7B: 1.06GB Q4_K_M / 3B: 1.92GB Q4_K_M

[montez]

Ling 3.0

Hugging Face: https://huggingface.co/inclusionAI/Ling-3.0-flash, https://huggingface.co/inclusionAI/Ling-3.0-tiny, https://huggingface.co/inclusionAI/Ling-3.0-tiny-int4, https://huggingface.co/inclusionAI/Ling-3.0-tiny-fp8
GitHub: https://github.com/inclusionAI/Ling
X: https://x.com/AntLingAGI/status/2080351022028095681
Website: https://www.ant-ling.com/en
Docs: https://github.com/inclusionAI/ling-cookbook

Developer: Ant Group, InclusionAI
Released: July 2026
Variants: Ling-3.0-flash, Ling-3.0-tiny
Parameters: flash: 124B total, 5.1B active / tiny: 7.9B total, 1.3B active
Context: flash: 262,144 tokens / tiny: 131,072 tokens, 262,144 with YaRN
Architecture: BailingMoE hybrid; flash: linear KDA + MLA attention with sparse MoE / tiny: 3:1 KDA to MLA blocks, 128 routed experts, 8 routed + 1 shared active
License: MIT
Modalities: Text
Runs on: flash: NVIDIA DGX Spark / tiny: Laptop, 48GB unified memory
Formats: safetensors BF16, safetensors FP8, safetensors INT4, GGUF Q4_K_M
On disk: flash: 60.5GB Q4_K_M GGUF, 254.98GB BF16 safetensors / tiny: 5.81GB INT4, 8.41GB FP8, 15.79GB BF16 safetensors

[montez]

Qwen3.8

Hugging Face: https://huggingface.co/Qwen/Qwen3.8-27B, https://huggingface.co/Qwen/Qwen3.8-27B-FP8
GitHub: https://github.com/QwenLM/Qwen3
X: https://x.com/Alibaba_Qwen/status/2088280182356611304
Website: https://qwen.ai/blog?id=qwen3.8

Developer: Alibaba Cloud, Qwen team
Released: August 2026
Parameters: 27B
Context: 262,144 tokens
Architecture: Qwen3.5 foundation, dense native vision-language model; hybrid linear + full attention
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: Laptop, 24GB+ unified memory, estimated / Desktop GPU
Formats: safetensors BF16, safetensors FP8, GGUF and MLX community quantizations
On disk: 16.05GB MLX 4-bit / 30.87GB FP8 safetensors / 55.56GB BF16 safetensors

[montez]

NVIDIA Nemotron 3.5 Lightning

Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
GitHub: https://github.com/NVIDIA-NeMo/Nemotron
Website: https://build.nvidia.com/nvidia/nemotron-3.5-lightning-30b-a3b

Developer: NVIDIA
Released: August 2026
Variants: Instruct, Base, DSpark and DFlash speculative drafters
Parameters: 30B total, 3B active
Context: 1,048,576 tokens
Architecture: hybrid LatentMoE interleaving Mamba-2, MoE and attention; 52 layers, 128 routed experts, 6 active, 32Q/2KV heads, Multi-Token Prediction
License: OpenMDW License Agreement, version 1.1
Modalities: Text
Runs on: Desktop GPU, Edge device, DGX Spark; NVIDIA only
Formats: safetensors BF16, safetensors NVFP4, GGUF community conversion
On disk: 17.82GB NVFP4 safetensors / 31.58GB BF16 safetensors

[montez]

Google DiffusionGemma

Hugging Face: https://huggingface.co/google/diffusiongemma-26B-A4B-it
GitHub: https://github.com/google-gemma
Website: https://ai.google.dev/gemma/docs/diffusiongemma
Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/

Developer: Google DeepMind
Released: June 2026
Parameters: 25.2B total, 3.8B active
Context: 256,000 tokens
Architecture: discrete text diffusion on the Gemma 4 26B A4B MoE foundation; autoregressive encoder prefills the prompt into a KV cache, decoder applies bidirectional attention over a 256-token canvas, block-autoregressive multi-canvas sampling; 30 layers, 8 active of 128 experts plus 1 shared, 1,024 sliding window, 550M vision encoder
License: Apache 2.0
Modalities: Text + Image + Video in, Text out
Runs on: Desktop GPU, 18GB+ VRAM
Formats: safetensors
On disk: 51.65GB safetensors

[montez]

IBM Granite Swash

Hugging Face: https://huggingface.co/ibm-granite/granite-swash-2b, https://huggingface.co/ibm-granite/granite-swash-3b-a600m
GitHub: https://github.com/ibm-granite/granite-4.1-language-models

Developer: IBM Granite Team
Released: July 2026
Variants: SWASH-2B, SWASH-3B-A600M
Parameters: 2B / 3B total, 600M active
Context: 8,192 tokens
Architecture: sliding window attention with learnable per-head attention sinks, LSE-scaled; 2B: dense decoder-only, 24 layers, 7 full-attention + 17 sliding-window layers, window 128, GQA, SwiGLU, RoPE, RMSNorm / 3B-A600M: MoE, 28 layers, 48 experts, 4 routed active
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors
On disk: 2B: 4.29GB safetensors / 3B-A600M: 6.04GB safetensors