Local LMs
-
Nanbeige4.2-3B
Hugging Face: https://huggingface.co/Nanbeige/Nanbeige4.2-3B
GitHub: https://github.com/Nanbeige/sglang, https://github.com/Nanbeige/vllm
Technical report: https://arxiv.org/abs/2607.22083Developer: Nanbeige, BOSS Zhipin
Released: July 2026
Parameters: 4B total, 3B non-embedding
Context: 262,144 tokens
Architecture: dense Looped Transformer, LoopSplit, mHC with depth attention, concatenated n-gram embeddings
License: Apache 2.0
Modalities: Text
Runs on: Laptop
Formats: safetensors BF16, GGUF and MLX via community conversion
On disk: 8.34GB BF16 safetensors -
Fermion Research Neutrino
Hugging Face: https://huggingface.co/FermionResearch/Neutrino-8B, https://huggingface.co/FermionResearch/Neutrino-0.6B, https://huggingface.co/FermionResearch/Neutrino-0.6B-Chat
GitHub: https://github.com/fermionresearch/llama.cpp
Website: https://www.fermionresearch.com/models/neutrino-8b/Developer: Fermion Research
Released: July 2026
Variants: 8B, 0.6B (speculative-decoding draft), 0.6B-Chat
Parameters: 8.19B / 596M
Context: 40,960 tokens
Architecture: ternary weights; 8B: Qwen3-8B topology, 36 layers, GQA 4:1 / 0.6B: 28 layers, hidden 1,024, GQA 16Q/8KV
License: Apache 2.0
Modalities: Text
Runs on: 8B: Laptop, 16GB unified memory / 0.6B: Laptop, CPU only
Formats: TRTC v4, custom-FV5 GGUF, MLX
On disk: 8B: 3.88GB TRTC v4, 4.09GB GGUF / 0.6B: 328MB TRTC v4, 343MB GGUF -
Kakao Kanana-2
Hugging Face: https://huggingface.co/kakaocorp/kanana-2-3b-instruct, https://huggingface.co/kakaocorp/kanana-2-1.3b-instruct
Website: https://tech.kakao.com/posts/826Developer: Kakao, Kanana LLM
Released: July 2026
Variants: 3B, 1.3B
Parameters: 3B / 1.3B
Context: 32,768 tokens
Architecture: 3B: dense / 1.3B: pruned from 3B, sliding-window attention
License: Kanana Open License
Modalities: Text
Runs on: Smartphone, Laptop
Formats: safetensors BF16, MLX via community conversion
On disk: 3B: 7.02GB BF16 / 1.3B: 2.58GB BF16 -
Microsoft Fara1.5
Hugging Face: https://huggingface.co/microsoft/Fara1.5-4B, https://huggingface.co/microsoft/Fara1.5-9B, https://huggingface.co/microsoft/Fara1.5-27B
GitHub: https://github.com/microsoft/fara
Website: https://labs.ai.azure.com/innovations/fara1-5/
Technical report: https://arxiv.org/abs/2606.20785Developer: Microsoft Research AI Frontiers
Released: May 2026
Variants: 4B, 9B, 27B
Parameters: 4B / 9B / 27B
Context: 262,144 tokens
Architecture: Qwen3.5 base, multimodal decoder-only LM, image + text to text
License: MIT
Modalities: Text + Image
Formats: safetensors
On disk: 4B: 9.08GB / 9B: 18.82GB / 27B: 54.71GB safetensors -
AMD Instella-MoE-16B-A3B-Think
Hugging Face: https://huggingface.co/amd/Instella-MoE-16B-A3B-Think
GitHub: https://github.com/AMD-AGI/Instella-MoE
Website: https://rocm.blogs.amd.com/artificial-intelligence/instella-moe/README.htmlDeveloper: AMD
Released: July 2026
Parameters: 16B total, 2.8B active
Context: 32,768 tokens
Architecture: decoder-only MoE, Gated Multi-head Latent Attention, FarSkip-Collective connectivity
License: Research RAIL
Modalities: Text
Formats: safetensors
On disk: 31.73GB safetensors -
Cisco Antares
Hugging Face: https://huggingface.co/fdtn-ai/antares-1b, https://huggingface.co/fdtn-ai/antares-350m
Website: https://cisco-foundation-ai.github.io/antares/
Technical report: https://cisco-foundation-ai.github.io/antares/technical-report.pdfDeveloper: Cisco Foundation AI
Released: July 2026
Variants: 1b, 350m
Parameters: 1B / 350M
Architecture: fine-tuned from IBM Granite 4.0, GraniteMoEHybrid
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors
On disk: 1b: 3.67GB safetensors / 350m: 0.70GB safetensors -
Poolside Laguna XS 2.1
Hugging Face: https://huggingface.co/poolside/Laguna-XS-2.1, https://huggingface.co/poolside/Laguna-XS-2.1-GGUF
Website: https://poolside.ai/blog/introducing-laguna-xs-2-1Developer: Poolside
Released: July 2026
Variants: BF16, FP8, NVFP4, INT4, GGUF
Parameters: 33B total, 3B active
Context: 262,144 tokens
Architecture: MoE, 40 layers: 10 global-attention + 30 sliding-window-attention; 256 experts + 1 shared expert
License: OpenMDW-1.1
Modalities: Text
Runs on: Mac with 36GB RAM
Formats: safetensors, GGUF
On disk: 20.27GB Q4_K_M GGUF / 66.89GB BF16 safetensors -
Meta Muse Glimmer-30B
Hugging Face: https://huggingface.co/meta-models/Muse-Glimmer-30B
Website: https://dev.meta.ai/models/muse-glimmer
Technical report: https://research.meta.ai/static/muse-glimmer-methodologyDeveloper: Meta Superintelligence Lab
Released: August 2026
Parameters: 29.6B
Context: 131,072+ tokens
Architecture: Dense causal transformer with ViT-G/14 perception encoder; 52 layers, GQA, SwiGLU, RoPE
License: Apache 2.0
Modalities: Text + Image
Runs on: Laptop, Desktop GPU, 24GB+ memory with 4-bit weights
Formats: BF16 safetensors, 4-bit quantized weights
On disk: 17GB K-Quant -
webAI TwIL-LM
Hugging Face: https://huggingface.co/webAI-Official/TwIL-LM, https://huggingface.co/webAI-Official/TwIL-LM3
Website: https://www.webai.com/blog/webai-releases-twil-lm-a-family-of-formal-logic-models-that-outreason-a-120b-model-and-run-on-an-iphoneDeveloper: webAI Intelligence Lab
Released: August 2026
Variants: TwIL-LM 1.7B, TwIL-LM3 3B
Parameters: 1.7B: 1.78B / 3B: 3B
Context: 1.7B: 8,192 tokens / 3B: 65,536 tokens
Architecture: 1.7B: SmolLM2-1.7B-Instruct base, dense Llama, 24 layers, 32 heads / 3B: SmolLM3-3B base, dense, 36 layers, GQA 16Q/4KV, NoPE every 4th layer
License: webAI Non-Commercial License ver. 1.0
Modalities: Text
Runs on: 1.7B: Smartphone, Laptop / 3B: Laptop, 4GB VRAM or CPU
Formats: 1.7B: GGUF Q4_K_M, Q5_K_M, Q8_0, F16 / 3B: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16
On disk: 1.7B: 1.06GB Q4_K_M / 3B: 1.92GB Q4_K_M -
Ling 3.0
Hugging Face: https://huggingface.co/inclusionAI/Ling-3.0-flash, https://huggingface.co/inclusionAI/Ling-3.0-tiny, https://huggingface.co/inclusionAI/Ling-3.0-tiny-int4, https://huggingface.co/inclusionAI/Ling-3.0-tiny-fp8
GitHub: https://github.com/inclusionAI/Ling
X: https://x.com/AntLingAGI/status/2080351022028095681
Website: https://developer.ant-ling.com/en/blogs/ling-3.0-flash-release
Docs: https://github.com/inclusionAI/ling-cookbookDeveloper: Ant Group, InclusionAI
Released: July 2026
Variants: Ling-3.0-flash, Ling-3.0-tiny
Parameters: flash: 124B total, 5.1B active / tiny: 7.9B total, 1.3B active
Context: flash: 262,144 tokens / tiny: 131,072 tokens, 262,144 with YaRN
Architecture: BailingMoE hybrid; flash: linear KDA + MLA attention with sparse MoE / tiny: 3:1 KDA to MLA blocks, 128 routed experts, 8 routed + 1 shared active
License: MIT
Modalities: Text
Runs on: flash: NVIDIA DGX Spark / tiny: Laptop, 48GB unified memory
Formats: safetensors BF16, safetensors FP8, safetensors INT4, GGUF Q4_K_M
On disk: flash: 60.5GB Q4_K_M GGUF, 254.98GB BF16 safetensors / tiny: 5.81GB INT4, 8.41GB FP8, 15.79GB BF16 safetensors -
Qwen3.8
Hugging Face: https://huggingface.co/Qwen/Qwen3.8-27B, https://huggingface.co/Qwen/Qwen3.8-27B-FP8
GitHub: https://github.com/QwenLM/Qwen3.8
X: https://x.com/Alibaba_Qwen/status/2088280182356611304
Website: https://qwen.ai/blog?id=qwen3.8Developer: Alibaba Cloud, Qwen team
Released: August 2026
Parameters: 27B
Context: 262,144 tokens
Architecture: Qwen3.5 foundation, dense native vision-language model; hybrid linear + full attention
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: Laptop, 24GB+ unified memory / Desktop GPU
Formats: safetensors BF16, safetensors FP8, GGUF and MLX via community conversion
On disk: 30.87GB FP8 safetensors / 55.56GB BF16 safetensors -
NVIDIA Nemotron 3.5 Lightning
Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
GitHub: https://github.com/NVIDIA-NeMo/Nemotron
Website: https://build.nvidia.com/nvidia/nemotron-3.5-lightning-30b-a3bDeveloper: NVIDIA
Released: August 2026
Variants: Instruct, Base, DSpark (drafter), DFlash (drafter)
Parameters: 30B total, 3B active
Context: 1,048,576 tokens
Architecture: hybrid LatentMoE interleaving Mamba-2, MoE and attention; 52 layers, 128 routed experts, 6 active, 32Q/2KV heads, Multi-Token Prediction
License: OpenMDW License Agreement, version 1.1
Modalities: Text
Runs on: Desktop GPU, Edge device, DGX Spark; NVIDIA only
Formats: safetensors BF16, safetensors NVFP4, GGUF via community conversion
On disk: 17.82GB NVFP4 safetensors / 31.58GB BF16 safetensors -
Google DiffusionGemma
Hugging Face: https://huggingface.co/google/diffusiongemma-26B-A4B-it
GitHub: https://github.com/google-deepmind/gemma
Website: https://deepmind.google/models/gemma/diffusiongemma/
Docs: https://ai.google.dev/gemma/docs/diffusiongemmaDeveloper: Google DeepMind
Released: June 2026
Parameters: 25.2B total, 3.8B active
Context: 256,000 tokens
Architecture: discrete text diffusion on the Gemma 4 26B A4B MoE; autoregressive encoder prefills the KV cache, bidirectional decoder over a 256-token canvas, block-autoregressive sampling; 30 layers, 8 of 128 experts plus 1 shared, 550M vision encoder
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: Desktop GPU, 18GB+ VRAM
Formats: safetensors
On disk: 51.65GB safetensors -
IBM Granite Swash
Hugging Face: https://huggingface.co/ibm-granite/granite-swash-2b, https://huggingface.co/ibm-granite/granite-swash-3b-a600m
Developer: IBM Granite Team
Released: July 2026
Variants: SWASH-2B, SWASH-3B-A600M
Parameters: 2B / 3B total, 600M active
Context: 8,192 tokens
Architecture: sliding window attention with learnable attention sinks; 2B: dense decoder-only, 24 layers (7 full-attention, 17 sliding-window, window 128), GQA, SwiGLU, RoPE / 3B-A600M: MoE, 28 layers, 48 experts, 4 routed active
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors
On disk: 2B: 4.29GB safetensors / 3B-A600M: 6.04GB safetensors -
IBM Granite Vision 4.1
Hugging Face: https://huggingface.co/ibm-granite/granite-vision-4.1-4b, https://huggingface.co/ibm-granite/granite-vision-4.1-4b-GGUF
GitHub: https://github.com/ibm-granite/granite-vision-models
Website: https://research.ibm.com/blog/granite-4-1-ai-foundation-modelsDeveloper: IBM
Released: April 2026
Parameters: 4B total
Context: 131,072 tokens
Architecture: Granite 4.1 3B language model with rank-256 LoRA, SigLIP2 so400m vision encoder over 384x384 tiles, windowed Q-Former projectors compressing each 4x4 patch window to 2x2 tokens
License: Apache 2.0
Modalities: Text + Image
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, BF16
On disk: 2.10GB Q4_K_M plus 1.16GB f16 mmproj / 6.81GB bf16
