Local LMs
-
webAI TwIL-LM
Hugging Face: https://huggingface.co/webAI-Official/TwIL-LM, https://huggingface.co/webAI-Official/TwIL-LM3
Website: https://www.webai.com/blog/webai-releases-twil-lm-a-family-of-formal-logic-models-that-outreason-a-120b-model-and-run-on-an-iphoneDeveloper: webAI Intelligence Lab
Released: August 2026
Variants: TwIL-LM 1.7B, TwIL-LM3 3B
Parameters: 1.7B: 1.78B total, 1.71B backbone + 72M LoRA / 3B: 3B
Context: 1.7B: 8,192 tokens / 3B: 65,536 tokens
Architecture: 1.7B: SmolLM2-1.7B-Instruct base, dense Llama, 24 layers, 32 heads, LoRA rank 64 SFT / 3B: SmolLM3-3B base, dense, 36 layers, GQA 16Q/4KV, NoPE every 4th layer; LoRA SFT, checkpoint fusion, WiSE-FT interpolation, GRPO reinforcement learning
License: webAI Non-Commercial License ver. 1.0
Modalities: Text
Runs on: 1.7B: Smartphone, Laptop / 3B: Laptop, 4GB VRAM or CPU
Formats: 1.7B: merged GGUF Q4_K_M, Q5_K_M, Q8_0, f16 / 3B: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16
On disk: 1.7B: 1.06GB Q4_K_M / 3B: 1.92GB Q4_K_M -
Ling 3.0
Hugging Face: https://huggingface.co/inclusionAI/Ling-3.0-flash, https://huggingface.co/inclusionAI/Ling-3.0-tiny, https://huggingface.co/inclusionAI/Ling-3.0-tiny-int4, https://huggingface.co/inclusionAI/Ling-3.0-tiny-fp8
GitHub: https://github.com/inclusionAI/Ling
X: https://x.com/AntLingAGI/status/2080351022028095681
Website: https://www.ant-ling.com/en
Docs: https://github.com/inclusionAI/ling-cookbookDeveloper: Ant Group, InclusionAI
Released: July 2026
Variants: Ling-3.0-flash, Ling-3.0-tiny
Parameters: flash: 124B total, 5.1B active / tiny: 7.9B total, 1.3B active
Context: flash: 262,144 tokens / tiny: 131,072 tokens, 262,144 with YaRN
Architecture: BailingMoE hybrid; flash: linear KDA + MLA attention with sparse MoE / tiny: 3:1 KDA to MLA blocks, 128 routed experts, 8 routed + 1 shared active
License: MIT
Modalities: Text
Runs on: flash: NVIDIA DGX Spark / tiny: Laptop, 48GB unified memory
Formats: safetensors BF16, safetensors FP8, safetensors INT4, GGUF Q4_K_M
On disk: flash: 60.5GB Q4_K_M GGUF, 254.98GB BF16 safetensors / tiny: 5.81GB INT4, 8.41GB FP8, 15.79GB BF16 safetensors -
Qwen3.8
Hugging Face: https://huggingface.co/Qwen/Qwen3.8-27B, https://huggingface.co/Qwen/Qwen3.8-27B-FP8
GitHub: https://github.com/QwenLM/Qwen3
X: https://x.com/Alibaba_Qwen/status/2088280182356611304
Website: https://qwen.ai/blog?id=qwen3.8Developer: Alibaba Cloud, Qwen team
Released: August 2026
Parameters: 27B
Context: 262,144 tokens
Architecture: Qwen3.5 foundation, dense native vision-language model; hybrid linear + full attention
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: Laptop, 24GB+ unified memory, estimated / Desktop GPU
Formats: safetensors BF16, safetensors FP8, GGUF and MLX community quantizations
On disk: 16.05GB MLX 4-bit / 30.87GB FP8 safetensors / 55.56GB BF16 safetensors -
NVIDIA Nemotron 3.5 Lightning
Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
GitHub: https://github.com/NVIDIA-NeMo/Nemotron
Website: https://build.nvidia.com/nvidia/nemotron-3.5-lightning-30b-a3bDeveloper: NVIDIA
Released: August 2026
Variants: Instruct, Base, DSpark and DFlash speculative drafters
Parameters: 30B total, 3B active
Context: 1,048,576 tokens
Architecture: hybrid LatentMoE interleaving Mamba-2, MoE and attention; 52 layers, 128 routed experts, 6 active, 32Q/2KV heads, Multi-Token Prediction
License: OpenMDW License Agreement, version 1.1
Modalities: Text
Runs on: Desktop GPU, Edge device, DGX Spark; NVIDIA only
Formats: safetensors BF16, safetensors NVFP4, GGUF community conversion
On disk: 17.82GB NVFP4 safetensors / 31.58GB BF16 safetensors -
Google DiffusionGemma
Hugging Face: https://huggingface.co/google/diffusiongemma-26B-A4B-it
GitHub: https://github.com/google-gemma
Website: https://ai.google.dev/gemma/docs/diffusiongemma
Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/Developer: Google DeepMind
Released: June 2026
Parameters: 25.2B total, 3.8B active
Context: 256,000 tokens
Architecture: discrete text diffusion on the Gemma 4 26B A4B MoE foundation; autoregressive encoder prefills the prompt into a KV cache, decoder applies bidirectional attention over a 256-token canvas, block-autoregressive multi-canvas sampling; 30 layers, 8 active of 128 experts plus 1 shared, 1,024 sliding window, 550M vision encoder
License: Apache 2.0
Modalities: Text + Image + Video in, Text out
Runs on: Desktop GPU, 18GB+ VRAM
Formats: safetensors
On disk: 51.65GB safetensors -
IBM Granite Swash
Hugging Face: https://huggingface.co/ibm-granite/granite-swash-2b, https://huggingface.co/ibm-granite/granite-swash-3b-a600m
GitHub: https://github.com/ibm-granite/granite-4.1-language-modelsDeveloper: IBM Granite Team
Released: July 2026
Variants: SWASH-2B, SWASH-3B-A600M
Parameters: 2B / 3B total, 600M active
Context: 8,192 tokens
Architecture: sliding window attention with learnable per-head attention sinks, LSE-scaled; 2B: dense decoder-only, 24 layers, 7 full-attention + 17 sliding-window layers, window 128, GQA, SwiGLU, RoPE, RMSNorm / 3B-A600M: MoE, 28 layers, 48 experts, 4 routed active
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors
On disk: 2B: 4.29GB safetensors / 3B-A600M: 6.04GB safetensors -
IBM Granite Vision 4.1
Hugging Face: https://huggingface.co/ibm-granite/granite-vision-4.1-4b, https://huggingface.co/ibm-granite/granite-vision-4.1-4b-GGUF
GitHub: https://github.com/ibm-granite/granite-vision-models
Website: https://www.ibm.com/granite/docs/models/vision
Announcement: https://research.ibm.com/blog/granite-4-1-ai-foundation-modelsDeveloper: IBM
Released: April 2026
Parameters: 4B total, Granite 4.1 3B language model plus vision encoder and projectors
Context: 131,072 tokens
Architecture: SigLIP2 so400m patch16-384 vision encoder over 384x384 image tiles, windowed Q-Former projectors compressing each 4x4 patch window to 2x2 tokens, and a Granite 4.1 3B language model with rank-256 LoRA across all self-attention projections
License: Apache 2.0
Modalities: Text + Image
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, bf16, with f16 mmproj
On disk: 2.10GB Q4_K_M plus 1.16GB f16 mmproj / 6.81GB bf16 -
Microsoft Mage-VL
Hugging Face: https://huggingface.co/microsoft/Mage-VL
GitHub: https://github.com/microsoft/Mage
Website: https://microsoft.github.io/Mage/vl/
Technical report: https://arxiv.org/abs/2607.24904Developer: Microsoft Mage Team
Released: July 2026
Parameters: 4B
Context: 262,144 tokens
Architecture: Mage-ViT codec-native visual encoder trained from scratch, 24 layers, feeding a two-layer MLP projector into a Qwen3-4B-Instruct-2507 causal decoder; separate cognition gate for proactive streaming
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: Laptop, 12GB+ memory at BF16, estimated / Desktop GPU
Formats: safetensors, bundled streaming gate and neural codec
On disk: 9.48GB safetensors -
Cohere Labs North Micro Vision
Hugging Face: https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct
Technical report: https://huggingface.co/blog/CohereLabs/meet-north-micro-vision-instructDeveloper: Cohere Labs
Released: August 2026
Parameters: 2.4B total, 2B language model + 400M vision encoder
Context: 128,000 tokens, multimodal validated to 8,192
Architecture: custom native-resolution vision encoder with DeepStack patch embeddings injected into early decoder layers, projector, and the Command A+ style North Micro LLM: three sliding-window attention layers with RoPE plus one global layer without positional embeddings
License: Apache 2.0
Modalities: Text + Image
Runs on: Smartphone, Laptop, Edge device, with quantization
Formats: safetensors BF16, MLX 4-bit and 8-bit community conversions
On disk: 4.97GB BF16 safetensors -
Cactus Compute Needle 2
Hugging Face: https://huggingface.co/Cactus-Compute/needle2
GitHub: https://github.com/cactus-compute/needle
X: https://x.com/cactuscompute/status/2086865960669983035
Website: https://cactuscompute.com/needleDeveloper: Cactus Compute
Released: August 2026
Parameters: 45M
Context: 2,048 tokens
Architecture: Simple Attention Network, 27 layers, hidden 512, 8Q/4KV GQA, Hadamard MLP, engram sites, CQ2 quantization at 2.2 effective bits; byte-level grammar-constrained decoding and a tool-retrieval head
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Headset, Edge device, Microcontroller
Formats: cact single binary; ARM64, x86-64, ARMv7, RISC-V and WebAssembly builds
On disk: 13.7MB cact -
Syzygy Mach-1 Additive 35B
Hugging Face: https://huggingface.co/SyzygyResearch/Mach-1-Additive-35B
X: https://x.com/syzygyeng/status/2084350792841195992
Website: https://withsyzygy.com/mach-1
Docs: https://withsyzygy.com/docs/machDeveloper: Syzygy Research
Released: August 2026
Parameters: 35B total, 8 of 256 experts active
Context: 262,144 tokens
Architecture: Qwen3.5 MoE topology, 40 layers, 256 experts, 8 active, hybrid linear attention with full attention every 4th layer; additive 1.7-bit weights with no weight multiplication
License: Apache 2.0
Modalities: Text
Runs on: Laptop, 16GB+ unified memory; Apple Silicon only
Formats: packed 1.7-bit safetensors, MLX
On disk: 7.0GB -
OpenMOSS MOSS-VL
Hugging Face: https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0708-FP8, https://huggingface.co/OpenMOSS-Team/MOSS-VL-Realtime-FP8
GitHub: https://github.com/OpenMOSS/MOSS-VL
Website: https://openmoss.ai/MOSS-VL/
Technical report: https://arxiv.org/abs/2606.07639Developer: OpenMOSS Team
Released: August 2026
Variants: Instruct-0708, Realtime
Parameters: 11B
Context: 262,144 tokens
Architecture: unified cross-attention multimodal model, 48 language layers with 12 cross-attention layers, XRoPE 3D spatiotemporal positions, absolute frame timestamps for streaming video
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: Desktop GPU; Instruct: 24GB VRAM / Realtime: 26GB+ VRAM; NVIDIA only
Formats: FP8 compressed-tensors with BF16 cross-attention and vision, HQQ INT8 KV cache
On disk: 15.73GB FP8 safetensors -
IBM Granite 4.2
Hugging Face: https://huggingface.co/ibm-granite/granite-4.2-3b, https://huggingface.co/ibm-granite/granite-4.2-8b, https://huggingface.co/ibm-granite/granite-4.2-30b
GitHub: https://github.com/ibm-granite/granite-4.2-language-models
Website: https://www.ibm.com/granite/docs/models/granite4-2Developer: IBM
Released: August 2026
Variants: 3B / 8B / 30B
Parameters: 3B / 8B / 30B
Context: 3B and 8B: 131,072 tokens / 30B: 131,072 tokens native, 512,000 tokens extended
Architecture: dense decoder-only transformer; GQA, RoPE, SwiGLU and RMSNorm; native reasoning modes and tool calling
License: Apache 2.0
Modalities: Text
Formats: safetensors
On disk: 3B: 7.32GB BF16 / 8B: 17.58GB BF16 / 30B: 58.55GB BF16 -
FireRedTeam FireRedAudio
Hugging Face: https://huggingface.co/FireRedTeam/FireRedAudio
GitHub: https://github.com/FireRedTeam/FireRedAudio
Website: https://fireredteam.github.io/demos/fireredaudio/
Technical report: https://arxiv.org/abs/2608.24168Developer: FireRedTeam
Released: August 2026
Parameters: 9B
Architecture: shared 9B LLM; decoupled Audio Encoder for understanding and RedAE-Patch plus flow-matching DiT pathway for generation
License: Apache 2.0
Modalities: Text + Audio
Runs on: Desktop GPU, NVIDIA only
Formats: safetensors, PyTorch .pt
On disk: FireRedAudio: 21.23GB safetensors / RedAE decoder: 8.40GB PyTorch .pt / 29.63GB total, estimated -
Cohere Labs Tiny Aya L2-Thinker
Hugging Face: https://huggingface.co/CohereLabs/tiny-aya-l2-thinker
Technical report: https://arxiv.org/abs/2609.10445Developer: Cohere Labs
Released: September 2026
Parameters: 3.35B
Architecture: Cohere2 decoder-only transformer; supervised fine-tuning for in-language reasoning
License: CC-BY-NC-4.0
Modalities: Text
Runs on: Laptop
Formats: safetensors
On disk: 6.70GB BF16 safetensors, estimated -
InternLM Intern Lumina U2
Hugging Face: https://huggingface.co/internlm/InternLumina-U2
GitHub: https://github.com/InternLM/InternLumina-U2
Website: https://internlm.github.io/InternLumina-U2/Developer: Shanghai AI Laboratory / InternLM
Released: September 2026
Parameters: 16B total, 1B active
Architecture: LLaDA-2.0 MoE diffusion LLM backbone; 8-codebook fully-discrete AToken visual representation; spatial-parallel denoising plus codebook-depth autoregressive head
License: Apache 2.0
Modalities: Text + Image + Video + 3D
Runs on: Huawei Ascend NPU only
Formats: safetensors
On disk: 33.81GB Ascend safetensors, estimated -
Microsoft FrogNano
Hugging Face: https://huggingface.co/microsoft/FrogNano-4B-2609
GitHub: https://github.com/microsoft/FrogNano
Technical report: https://arxiv.org/abs/2609.07925Developer: Microsoft
Released: September 2026
Parameters: 4.66B
Context: 131K tokens in the evaluated configuration
Architecture: Qwen3.5-4B base; dense, 32 layers, hybrid Gated DeltaNet + gated attention; reinforcement-learning post-training on about 1,500 synthetic software-engineering tasks; repository-level coding agent run through the five-tool Leaf harness
License: MIT
Modalities: Text
Runs on: Laptop, 16GB+ unified memory
Formats: BF16 safetensors
On disk: 9.32GB BF16 -
H Company Holo4
Hugging Face: https://huggingface.co/Hcompany/Holo4-27B, https://huggingface.co/Hcompany/Holo4-35B-A3B, https://huggingface.co/Hcompany/Holotron4-30B-A3B
Website: https://hcompany.ai/newsroom/holo4Developer: H Company
Released: September 2026
Variants: Holo4-27B / Holo4-35B-A3B / Holotron4-30B-A3B
Parameters: 27B dense / 35B total, 3B active / 30B total, 3B active
Context: 262,144 tokens
Architecture: computer-use agent VLMs; 27B: Qwen3.8 base, dense, 64 layers / 35B-A3B: Qwen3.6 base, MoE, 40 layers, 256 experts, 8 routed active / Holotron4: NVIDIA Nemotron 3 Nano Omni base, hybrid Mamba + MoE + attention, 52 layers, 128 routed experts, 6 active; Holo4: supervised fine-tuning plus two merged reinforcement-learning LoRA experts
License: 27B: CC-BY-NC-4.0 / 35B-A3B: Apache 2.0 / Holotron4: NVIDIA Open Model Agreement
Modalities: Text + Image
Runs on: 27B: Laptop, 32GB+ unified memory / 35B-A3B: Laptop, 32GB+ unified memory / Holotron4: Workstation GPU, 48GB+ memory
Formats: 27B and 35B-A3B: BF16 safetensors, FP8, NVFP4, Q4_K_M GGUF with mmproj / Holotron4: BF16 safetensors, FP8
On disk: 27B: 16.88GB Q4_K_M GGUF, 54.71GB BF16 / 35B-A3B: 21.30GB Q4_K_M GGUF, 70.21GB BF16 / Holotron4: 35.20GB FP8, 66.03GB BF16 -
JetBrains Mellum2.1
Hugging Face: https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking, https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF, https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Base, https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Instruct, https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Thinking
Website: https://blog.jetbrains.com/ai/2026/10/mellum2-1-gets-to-work-a-fast-open-model-for-coding-agents/
Technical report: https://arxiv.org/abs/2605.31268Developer: JetBrains
Released: October 2026
Variants: Mellum2.1 Thinking / Mellum2 Base, Instruct, Thinking (June 2026)
Parameters: 12B total, 2.5B active
Context: 131,072 tokens
Architecture: MoE, 28 layers, 64 experts with 8 active, GQA with 32 Q and 4 KV heads, sliding window 1,024 on 3 of every 4 layers; Mellum2.1 keeps the Mellum2 architecture and adds reinforcement learning post-training
License: Apache 2.0
Modalities: Text
Runs on: Laptop, 16GB+ unified memory
Formats: safetensors BF16, GGUF (BF16, Q8_0, Q6_K, Q4_K_M, MXFP4_MOE)
On disk: Mellum2.1 Thinking: 8.07GB Q4_K_M, 24.31GB BF16 -
PrismML Bonsai 2 27B
Hugging Face: https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit, https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf
GitHub: https://github.com/PrismML-Eng/Bonsai-demo
Website: https://prismml.com/news/bonsai-2-27b
White paper: https://github.com/PrismML-Eng/Bonsai-demo/blob/main/bonsai-2-27b-whitepaper.pdfDeveloper: PrismML
Released: September 2026
Parameters: 27.36B total, 24.35B language + 2.54B embedding and LM head + 0.47B vision tower
Context: 262,144 tokens
Architecture: Qwen3.8-27B base; 64 blocks, 24Q/4KV heads, hybrid attention with about 75% linear and 25% full attention; ternary {-1, 0, +1} weights with FP16 group-wise scales at group size 128, 1.76 effective bits per weight
License: Apache 2.0
Modalities: Text + Image
Runs on: Laptop, Desktop GPU; Apple Silicon or NVIDIA GPU only
Formats: MLX 2-bit ternary, GGUF PTQ1_0, GGUF PQ2_0, optional mmproj vision pack
On disk: 5.95GB PTQ1_0 GGUF, 7.21GB PQ2_0 GGUF, 8.60GB MLX / 0.63GB mmproj vision pack
