Local LMs

[montez]

Microsoft Mage-VL

Hugging Face: https://huggingface.co/microsoft/Mage-VL
GitHub: https://github.com/microsoft/Mage
Website: https://microsoft.github.io/Mage/vl/
Technical report: https://arxiv.org/abs/2607.24904

Developer: Microsoft Mage Team
Released: July 2026
Parameters: 4B
Context: 262,144 tokens
Architecture: Mage-ViT codec-native visual encoder, 24 layers, with a two-layer MLP projector into a Qwen3-4B-Instruct-2507 causal decoder; separate cognition gate for proactive streaming
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: Laptop, 12GB+ memory / Desktop GPU
Formats: safetensors
On disk: 9.48GB safetensors

[montez]

Cohere Labs North Micro Vision

Hugging Face: https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct
Website: https://huggingface.co/blog/CohereLabs/meet-north-micro-vision-instruct
Technical report: https://huggingface.co/blog/CohereLabs/meet-north-micro-vision-instruct

Developer: Cohere Labs
Released: August 2026
Parameters: 2.4B
Context: 128,000 tokens, multimodal validated to 8,192
Architecture: native-resolution vision encoder (400M) with DeepStack patch embeddings injected into early decoder layers, projector, and Command A+ style North Micro LLM (2B): three sliding-window layers with RoPE per global layer without positional embeddings
License: Apache 2.0
Modalities: Text + Image
Runs on: Smartphone, Laptop, Edge device
Formats: safetensors BF16, MLX via community conversion
On disk: 4.97GB BF16 safetensors

[montez]

Cactus Compute Needle 2

Hugging Face: https://huggingface.co/Cactus-Compute/needle2
GitHub: https://github.com/cactus-compute/needle
X: https://x.com/cactuscompute/status/2086865960669983035
Website: https://cactuscompute.com/blog/needle-2

Developer: Cactus Compute
Released: August 2026
Parameters: 45M
Context: 2,048 tokens
Architecture: Simple Attention Network, 27 layers, hidden 512, 8Q/4KV GQA, Hadamard MLP, engram sites, tool-retrieval head
License: Apache 2.0
Modalities: Text
Runs on: Smartphone, Headset, Edge device, Microcontroller
Formats: cact single binary
On disk: 13.7MB cact

[montez]

Syzygy Mach-1 Additive 35B

Hugging Face: https://huggingface.co/SyzygyResearch/Mach-1-Additive-35B
X: https://x.com/syzygyeng/status/2084350792841195992
Website: https://withsyzygy.com/mach-1
Docs: https://withsyzygy.com/docs/mach

Developer: Syzygy Research
Released: August 2026
Parameters: 35B total
Context: 262,144 tokens
Architecture: Qwen3.5 MoE topology, 40 layers, 256 experts, 8 active, hybrid linear attention with full attention every 4th layer; additive 1.7-bit weights with no weight multiplication
License: Apache 2.0
Modalities: Text
Runs on: Laptop, 16GB+ unified memory; Apple Silicon only
Formats: packed 1.7-bit safetensors, MLX
On disk: 8.04GB packed safetensors

[montez]

OpenMOSS MOSS-VL

Hugging Face: https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0708-FP8, https://huggingface.co/OpenMOSS-Team/MOSS-VL-Realtime-FP8
GitHub: https://github.com/OpenMOSS/MOSS-VL
Website: https://openmoss.ai/MOSS-VL/
Technical report: https://arxiv.org/abs/2608.15045

Developer: OpenMOSS Team, Shanghai Innovation Institute
Released: August 2026
Variants: Instruct-0708, Realtime
Parameters: 11B
Context: 262,144 tokens
Architecture: unified cross-attention multimodal model, 48 language layers with 12 cross-attention layers, XRoPE 3D spatiotemporal positions, absolute frame timestamps for streaming video
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: Desktop GPU; Instruct: 24GB VRAM / Realtime: 26GB+ VRAM; NVIDIA only
Formats: safetensors FP8
On disk: 15.73GB FP8 safetensors

[montez]

IBM Granite 4.2

Hugging Face: https://huggingface.co/ibm-granite/granite-4.2-3b, https://huggingface.co/ibm-granite/granite-4.2-8b, https://huggingface.co/ibm-granite/granite-4.2-30b
GitHub: https://github.com/ibm-granite/granite-4.2-language-models
Website: https://research.ibm.com/blog/introducing-granite-4-2

Developer: IBM
Released: August 2026
Variants: 3B / 8B / 30B
Parameters: 3B / 8B / 30B
Context: 131,072 tokens native, 512,000 tokens extended
Architecture: dense decoder-only transformer; GQA, RoPE, SwiGLU and RMSNorm
License: Apache 2.0
Modalities: Text
Formats: safetensors
On disk: 3B: 7.32GB BF16 / 8B: 17.58GB BF16 / 30B: 58.55GB BF16

[montez]

FireRedTeam FireRedAudio

Hugging Face: https://huggingface.co/FireRedTeam/FireRedAudio
GitHub: https://github.com/FireRedTeam/FireRedAudio
Website: https://fireredteam.github.io/demos/fireredaudio/
Technical report: https://arxiv.org/abs/2608.24168

Developer: Xiaohongshu FireRed Team
Released: August 2026
Parameters: 9B
Architecture: shared 9B LLM; decoupled Audio Encoder for understanding and RedAE-Patch plus flow-matching DiT pathway for generation
License: Apache 2.0
Modalities: Text + Audio
Runs on: Desktop GPU, NVIDIA only
Formats: safetensors, PyTorch .pt
On disk: FireRedAudio: 21.23GB safetensors / RedAE decoder: 8.40GB PyTorch .pt

[montez]

Cohere Labs Tiny Aya L2-Thinker

Hugging Face: https://huggingface.co/CohereLabs/tiny-aya-l2-thinker
Website: https://cohere.com/blog/building-multilingual-bridges
Technical report: https://arxiv.org/abs/2609.10445

Developer: Cohere Labs
Released: September 2026
Parameters: 3.35B
Architecture: Cohere2 decoder-only transformer
License: CC-BY-NC-4.0
Modalities: Text
Runs on: Laptop
Formats: safetensors
On disk: 6.70GB BF16 safetensors, estimated

[montez]

InternLM Intern Lumina U2

Hugging Face: https://huggingface.co/internlm/InternLumina-U2
GitHub: https://github.com/InternLM/InternLumina-U2
Website: https://internlm.github.io/InternLumina-U2/
Technical report: https://arxiv.org/abs/2610.20818

Developer: Shanghai AI Laboratory / InternLM
Released: September 2026
Parameters: 16B total, 1B active
Architecture: LLaDA-2.0 MoE diffusion LLM backbone; 8-codebook fully-discrete AToken visual representation; spatial-parallel denoising plus codebook-depth autoregressive head
License: Apache 2.0
Modalities: Text + Image + Video + 3D
Runs on: NVIDIA GPU or Huawei Ascend NPU
Formats: safetensors
On disk: 33.82GB safetensors each for the NVIDIA and Ascend checkpoints

[montez]

Microsoft FrogNano

Hugging Face: https://huggingface.co/microsoft/FrogNano-4B-2609
GitHub: https://github.com/microsoft/FrogNano
Technical report: https://arxiv.org/abs/2609.07925

Developer: Microsoft
Released: September 2026
Parameters: 4.66B
Context: About 131K tokens
Architecture: Qwen3.5-4B base; dense, 32 layers, hybrid Gated DeltaNet + gated attention
License: MIT
Modalities: Text
Runs on: Laptop, 16GB+ unified memory
Formats: BF16 safetensors
On disk: 9.32GB BF16

[montez]

H Company Holo4

Hugging Face: https://huggingface.co/Hcompany/Holo4-27B, https://huggingface.co/Hcompany/Holo4-35B-A3B, https://huggingface.co/Hcompany/Holotron4-30B-A3B
Website: https://hcompany.ai/newsroom/holo4

Developer: H Company
Released: September 2026
Variants: Holo4-27B / Holo4-35B-A3B / Holotron4-30B-A3B
Parameters: 27B dense / 35B total, 3B active / 30B total, 3B active
Context: 262,144 tokens
Architecture: computer-use agent VLMs; Holo4-27B: Qwen3.8 base, dense / Holo4-35B-A3B: Qwen3.6 base, MoE / Holotron4: NVIDIA Nemotron 3 Nano Omni base, hybrid Mamba + MoE + attention
License: 27B: CC-BY-NC-4.0 / 35B-A3B: Apache 2.0 / Holotron4: NVIDIA Open Model Agreement
Modalities: Text + Image
Runs on: 27B and 35B-A3B: Laptop, 32GB+ unified memory / Holotron4: Workstation GPU, 48GB+ memory
Formats: 27B and 35B-A3B: BF16 safetensors, FP8, NVFP4, GGUF Q4_K_M / Holotron4: BF16 safetensors, FP8
On disk: 27B: 16.88GB Q4_K_M GGUF, 54.71GB BF16 / 35B-A3B: 21.30GB Q4_K_M GGUF, 70.21GB BF16 / Holotron4: 35.20GB FP8, 66.03GB BF16

[montez]

JetBrains Mellum2.1

Hugging Face: https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking, https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF, https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Base, https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Instruct, https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Thinking
Website: https://blog.jetbrains.com/ai/2026/10/mellum2-1-gets-to-work-a-fast-open-model-for-coding-agents/

Developer: JetBrains
Released: October 2026
Variants: Mellum2.1 Thinking / Mellum2 Base, Instruct, Thinking
Parameters: 12B total, 2.5B active
Context: 131,072 tokens
Architecture: MoE, 28 layers, 64 experts with 8 active, GQA, sliding-window attention on 3 of every 4 layers; Mellum2.1 keeps the Mellum2 architecture
License: Apache 2.0
Modalities: Text
Runs on: Laptop, 16GB+ unified memory
Formats: safetensors BF16, GGUF (BF16, Q8_0, Q6_K, Q4_K_M, MXFP4_MOE)
On disk: Mellum2.1 Thinking: 8.07GB Q4_K_M, 24.31GB BF16

[montez]

PrismML Bonsai 2 27B

Hugging Face: https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit, https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf
GitHub: https://github.com/PrismML-Eng/Bonsai-demo
Website: https://prismml.com/news/bonsai-2-27b
White paper: https://github.com/PrismML-Eng/Bonsai-demo/blob/main/bonsai-2-27b-whitepaper.pdf

Developer: PrismML
Released: September 2026
Parameters: 27.36B
Context: 262,144 tokens
Architecture: Qwen3.8-27B base; 64 blocks, hybrid attention with about 75% linear and 25% full; ternary {-1, 0, +1} weights with FP16 group scales, 1.76 effective bits per weight
License: Apache 2.0
Modalities: Text + Image
Runs on: Laptop, Desktop GPU; Apple Silicon or NVIDIA GPU only
Formats: MLX 2-bit ternary, GGUF PTQ1_0, GGUF PQ2_0, optional mmproj vision pack
On disk: 5.95GB PTQ1_0 GGUF, 7.21GB PQ2_0 GGUF, 8.60GB MLX / 0.63GB mmproj vision pack

[montez]

Swiss AI Apertus 1.5

Hugging Face: https://huggingface.co/swiss-ai/Apertus-v1.5-8B, https://huggingface.co/swiss-ai/Apertus-v1.5-70B
Website: https://www.apertus-ai.org/articles/2026-07-apertus-1-5
Technical report: https://arxiv.org/abs/2610.15542

Developer: Swiss AI Initiative (EPFL, ETH Zurich, CSCS)
Released: July 2026
Variants: 8B / 70B
Parameters: 8B / 70B
Context: 262,144 tokens
Architecture: decoder-only transformer with xIELU activation; continued pretraining of Apertus 1.0; optional thinking mode and tool calling
License: Apache 2.0
Modalities: Text + Image + Audio
Runs on: 8B: Laptop, Desktop GPU / 70B: Server-class hardware
Formats: safetensors
On disk: 8B: 18.40GB safetensors / 70B: 144.60GB safetensors

[montez]

Edge0

Hugging Face: https://huggingface.co/Edge0/Edge0-35B-A3B-preview, https://huggingface.co/Edge0/Edge0-8B-A1B-preview
GitHub: https://github.com/Edge0-AI/Edge0
Website: https://edge0.ai/research/edge0
Technical report: https://arxiv.org/abs/2609.18063

Developer: Edge0
Released: September 2026
Variants: Edge0-35B-A3B-preview / Edge0-8B-A1B-preview
Parameters: 35B total, 3B active / 8B total, 1B active
Context: 35B-A3B: 262,144 tokens / 8B-A1B: 131,072 tokens
Architecture: sparse MoE with int4 checkpoint, Recover-LoRA adapters and prerouter heads, experts streamed from SSD; 35B-A3B: Qwen3.6-35B-A3B base / 8B-A1B: Ling 3.0 tiny base
License: Apache 2.0
Modalities: Text
Runs on: 35B-A3B: Smartphone, 12GB+ RAM on Android / 8B-A1B: Smartphone, 8GB+ RAM on Android; edge0 engines only
Formats: MLX 4-bit safetensors, LoRA and prerouter adapter safetensors
On disk: 35B-A3B: 19.51GB int4 checkpoint plus 0.18GB adapters / 8B-A1B: 4.51GB int4 checkpoint plus 0.06GB adapters