Local LMs
-
H2O-Lightning-4B
Hugging Face: https://huggingface.co/h2oai/h2o-lightning-4b, https://huggingface.co/h2oai/h2o-lightning-4b-gguf
Website: https://h2o.ai/blog/2026/h2o-lightning-jevbench/Developer: H2O.ai
Released: October 2026
Parameters: 4B
Context: 40,960 tokens
Architecture: Qwen3.5-4B base, dense, 32 layers (8 full attention, 24 linear attention); decision model that scores every option of typed questions in one forward pass; causal plus bidirectional attention on the full-attention layers
License: Apache 2.0
Modalities: Text + Image in; probabilities per option out
Runs on: Laptop, Desktop GPU, 16GB+ unified memory or VRAM
Formats: safetensors BF16, GGUF BF16, Q8_0, Q5_K_M, Q4_K_M
On disk: 2.71GB Q4_K_M / 4.48GB Q8_0 / 9.32GB BF16 -
NVIDIA Nemotron Labs Dory1.0 4B
Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16, https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4, https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-MXFP8, https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF
Developer: NVIDIA
Released: October 2026
Parameters: 4B
Context: 262,144 tokens
Architecture: looped (latent-recurrent) normalized-GPT; a shared 20-layer recurrent block is applied 1, 2 or 4 times between 8 input and 8 output layers; interleaved sliding-window and full attention
License: OpenMDW License Agreement, version 1.1
Modalities: Text
Runs on: Laptop, Desktop GPU
Formats: safetensors BF16, safetensors MXFP8, safetensors NVFP4, GGUF Q5_K_M
On disk: 3.67GB Q5_K_M GGUF / 3.2GB NVFP4 / 4.75GB MXFP8 / 7.95GB BF16 -
NVIDIA Nemotron 3.5 Bolt
Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB, https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF, https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16
Website: https://research.nvidia.com/labs/nemotron/Nemotron-3.5-Bolt-24GB/
Technical report: https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-Technical-Report.pdfDeveloper: NVIDIA
Released: October 2026
Variants: 74B-A9B 24GB (2-bit) / 74B-A9B BF16
Parameters: 74B total, 9B active
Context: 262,144 tokens
Architecture: Mamba-2, MoE and attention hybrid LatentMoE, 88 layers (40 Mamba-2, 40 MoE, 8 attention), expert top-K 9; compressed from Nemotron 3.5 Super VL 120B-A12B; C-RADIO v4-H vision encoder
License: OpenMDW License Agreement, version 1.1
Modalities: Text + Image + Video
Runs on: Laptop, Desktop GPU, 24GB+ memory; NVIDIA only
Formats: safetensors 2-bit packed, GGUF with mmproj, safetensors BF16
On disk: 24.12GB safetensors / 22.02GB GGUF plus 1.73GB mmproj / 149.8GB BF16 -
IBM Granite Switch 4.1
Hugging Face: https://huggingface.co/ibm-granite/granite-switch-4.1-8b, https://huggingface.co/ibm-granite/granite-switch-4.1-30b
GitHub: https://github.com/generative-computing/granite-switch
Website: https://huggingface.co/blog/ibm-granite/granite-switch-4-1Developer: IBM
Released: October 2026
Variants: 8B / 30B
Parameters: 8B / 30B
Context: 131,072 tokens
Architecture: modular LLM composing a Granite 4.1 base model with activated LoRA adapters (Decision, Guardian Core) selected by control tokens, plus an embedded Granite Speech 5.0 TurboCTC 470M English speech encoder
License: Apache 2.0
Modalities: Text + Audio
Runs on: 8B: Desktop GPU / 30B: Workstation GPU
Formats: safetensors BF16
On disk: 8B: 17.56GB BF16 / 30B: 59.99GB BF16 -
Tencent Hy-Memory-30B-A3B
Hugging Face: https://huggingface.co/tencent/Hy-Memory-30B-A3B, https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8, https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF
GitHub: https://github.com/Tencent-Hunyuan/Hy-Memory
Technical report: https://arxiv.org/abs/2610.22196Developer: Tencent Hunyuan
Released: October 2026
Parameters: 30B total, 3B active
Context: 262,144 tokens
Architecture: Hunyuan Hy3 MoE, 48 layers with the first layer dense, 128 routed experts with 8 active plus 1 shared, GQA 32Q/4KV heads; long-term memory writer for the Mem0, MemOS and Hy-Memory frameworks
License: Apache 2.0
Modalities: Text
Runs on: Laptop, 32GB+ unified memory
Formats: safetensors BF16, safetensors FP8, GGUF Q4_K_M, Q6_K, Q8_0
On disk: 18.24GB Q4_K_M / 24.71GB Q6_K / 31.99GB Q8_0 / 30.59GB FP8 / 60.14GB BF16 -
OpenBMB MiniCPM-V 4.7
Hugging Face: https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B, https://huggingface.co/openbmb/MiniCPM-V-4.7-1B
GitHub: https://github.com/OpenBMB/MiniCPM-V
Website: https://openbmb.cn/en/model/minicpm-v-4-7
Docs: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/minicpm_v4dot7_en.mdDeveloper: OpenBMB, ModelBest
Released: October 2026
Variants: 35B-A3B / 1B
Parameters: 35B total, 3B active / 1B
Context: 262,144 tokens
Architecture: hybrid linear and full attention language backbone with full attention every 4th layer, 27-layer SigLIP2 vision encoder; 35B-A3B: MoE, 40 layers, 256 experts with 8 active / 1B: dense, 24 layers
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: 35B-A3B: Smartphone, Laptop, 32GB+ unified memory / 1B: Smartphone, Laptop, Edge device
Formats: safetensors BF16, FP8, int4, AWQ, GPTQ, GGUF
On disk: 35B-A3B: 21.17GB Q4_K_M GGUF, 70.56GB BF16 / 1B: 0.67GB Q4_K_M GGUF, 3.11GB BF16 -
Microsoft AesCode
Hugging Face: https://huggingface.co/microsoft/AesCode-8B, https://huggingface.co/microsoft/AesCode-32B
GitHub: https://github.com/microsoft/AesCode
Website: https://microsoft.github.io/AesCode/
Technical report: https://arxiv.org/abs/2610.18860Developer: Microsoft
Released: October 2026
Variants: 8B / 32B
Parameters: 8B / 32B
Context: 262,144 tokens
Architecture: 8B: Qwen3-VL-8B-Instruct base / 32B: Qwen3-VL-32B-Instruct base; generates editable HTML and CSS
License: Apache 2.0
Modalities: Text + Image
Runs on: 8B: Desktop GPU / 32B: Workstation GPU
Formats: safetensors BF16
On disk: 8B: 17.55GB BF16 / 32B: 66.73GB BF16 -
Xiaomi MiMo-V2.6-Distill-Qwen-9B
Hugging Face: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
Website: https://mimo.xiaomi.com/mimo-v2-6/article
Technical report: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL/blob/main/MiMo_V2_6_technical_report.pdfDeveloper: Xiaomi MiMo
Released: September 2026
Parameters: 9B
Context: 262,144 tokens
Architecture: Qwen3.5-9B base, hybrid linear and full attention with full attention every 4th layer, 32 layers, 27-layer vision encoder
License: MIT
Modalities: Text + Image
Runs on: Desktop GPU
Formats: safetensors BF16
On disk: 18.85GB BF16 -
Amazon ALoDLM
Hugging Face: https://huggingface.co/amazon/ALoDLM-8B, https://huggingface.co/amazon/ALoDLM-1.7B
GitHub: https://github.com/amazon-science/ALoDLM
Website: https://alo-dlm.github.io/
Technical report: https://arxiv.org/abs/2610.04198Developer: Amazon
Released: October 2026
Variants: 8B / 1.7B
Parameters: 8B / 1.7B
Architecture: adaptively looped masked diffusion language model initialized from Qwen3-8B and Qwen3-1.7B; shared transformer layers applied recurrently with a learned halting policy; 8B: 36 layers / 1.7B: 28 layers
License: CC-BY-NC-4.0
Modalities: Text
Runs on: 8B: Desktop GPU, 24GB+ VRAM / 1.7B: Laptop, 8GB+ VRAM; NVIDIA only
Formats: safetensors BF16
On disk: 8B: 16.4GB BF16 / 1.7B: 4.08GB BF16 -
Ai2 AstaBrief 8B
Hugging Face: https://huggingface.co/allenai/AstaBrief_8B, https://huggingface.co/allenai/AstaBrief_8B_SFT
Website: https://allenai.org/blog/astabriefDeveloper: Ai2
Released: October 2026
Variants: AstaBrief-8B (DPO) / AstaBrief-8B-SFT
Parameters: 8B
Context: 40,960 tokens
Architecture: Qwen3-8B base, 36 layers; turns a research question and retrieved literature excerpts into a cited scientific report
License: Apache 2.0
Modalities: Text
Runs on: Desktop GPU
Formats: safetensors BF16
On disk: 16.38GB BF16 -
IFM Jais 3
Hugging Face: https://huggingface.co/inception42/Jais-3-0.9B, https://huggingface.co/inception42/Jais-3-3.7B, https://huggingface.co/inception42/Jais-3-7B, https://huggingface.co/inception42/Jais-3-32B, https://huggingface.co/inception42/Jais-3-MoVA-36B-A4B
Website: https://ifm.ai/blog/jais3/Developer: Institute of Foundation Models (IFM), MBZUAI, and Inception42
Released: October 2026
Variants: 0.9B / 3.7B / 7B / 32B / MoVA-36B-A4B
Parameters: 0.9B / 3.7B / 7B / 32B / 36B total, 4B active
Context: 0.9B: 131,072 tokens / 3.7B, 7B, 32B and MoVA-36B-A4B: 524,288 tokens
Architecture: K2 Horizon base adapted to Arabic; 0.9B, 3.7B, 7B, 32B: dense decoder-only, GQA, 28 / 36 / 36 / 64 layers; MoVA-36B-A4B: MoE with Mixture-of-Values attention, 48 layers, 100 FFN experts with 8 per token plus 1 shared
License: Apache 2.0
Modalities: Text
Runs on: 0.9B: Smartphone / 3.7B and 7B: Smartphone, Laptop / 32B and MoVA-36B-A4B: Laptop, 32GB+ unified memory
Formats: safetensors BF16, safetensors FP8, GGUF (Q4_K_M, Q5_0, Q5_K_M, Q6_K, Q8_0, BF16), MLX 4-bit, 8-bit, bf16
On disk: 0.9B: 0.67GB Q4_K_M / 3.7B: 3.16GB Q4_K_M / 7B: 5.59GB Q4_K_M / 32B: 21.08GB Q4_K_M / MoVA-36B-A4B: 22.37GB Q4_K_M -
inclusionAI Ming-3.0-audio-tiny
Hugging Face: https://huggingface.co/inclusionAI/Ming-3.0-audio-tiny
GitHub: https://github.com/inclusionAI/Ming-3.0-audio-flashDeveloper: inclusionAI, Ant Group
Released: October 2026
Parameters: 9.3B total, 2.7B active
Context: 131,072 tokens
Architecture: native audio-language MoE; BailingMoE v3 language backbone with 128 experts, 32-layer audio encoder with projector, optional Talker for synthesized speech output
License: MIT
Modalities: Text + Audio
Runs on: Desktop GPU; 1 GPU for text output, 2 GPUs for synthesized speech
Formats: BF16 safetensors
On disk: 17.74GB BF16 -
inclusionAI Realtime-Venus
Hugging Face: https://huggingface.co/inclusionAI/Realtime-Venus
GitHub: https://github.com/inclusionAI/Realtime-Venus
Website: https://realtime-venus.github.io/
Technical report: https://arxiv.org/abs/2609.13814Developer: Venus Team, Ant Group, and Tsinghua University
Released: September 2026
Variants: Realtime-Venus-Omni / Realtime-Venus-Audio
Parameters: 9B each
Context: 40,960 tokens
Architecture: MiniCPM-o 4.5 base; Qwen3-8B language backbone, Whisper-Medium audio encoder, SigLIP2 visual encoder (Omni only), streaming flow-matching speech decoder; full-duplex streaming
License: Apache 2.0
Modalities: Omni: Text + Image + Video + Audio / Audio: Text + Audio
Runs on: Desktop GPU; NVIDIA only
Formats: BF16 safetensors
On disk: Omni: 20.08GB BF16 safetensors / Audio: 19.99GB BF16 safetensors -
Zhongguancun ZGCM-1-7B
Hugging Face: https://huggingface.co/zgcagi/ZGCM-1-7B
GitHub: https://github.com/zgcagi/ZGCM-1
Technical report: https://arxiv.org/abs/2609.13356Developer: Zhongguancun Academy and Zhongguancun Institute of Artificial Intelligence
Released: September 2026
Parameters: 7.39B
Context: 262,144 tokens
Architecture: trained from scratch; decoder-only dense, 32 layers, 27 gated sliding-window layers plus 5 global-attention layers, GQA
License: MIT
Modalities: Text
Runs on: Desktop GPU
Formats: BF16 safetensors
On disk: 14.79GB BF16 -
RWKV RWKV7-G1k
Hugging Face: https://huggingface.co/RWKV/RWKV7-G1k-1.5B-20260930, https://huggingface.co/RWKV/RWKV7-G1k-2.9B-20260930, https://huggingface.co/RWKV/RWKV7-G1k-7.2B-20260930, https://huggingface.co/RWKV/RWKV7-G1k-13.3B-20260930
GitHub: https://github.com/BlinkDL/RWKV-LM
Technical report: https://arxiv.org/abs/2503.14456Developer: RWKV (Bo Peng)
Released: September 2026
Variants: 1.5B / 2.9B / 7.2B / 13.3B
Parameters: 1.5B / 2.9B / 7.2B / 13.3B
Architecture: RWKV-7 Goose, attention-free recurrent model with a constant-size recurrent state; 24 / 32 / 32 / 61 layers
License: Apache 2.0
Modalities: Text
Runs on: 1.5B and 2.9B: Smartphone, Laptop / 7.2B: Desktop GPU / 13.3B: Workstation GPU
Formats: BF16 safetensors, PyTorch .pth
On disk: 1.5B: 3.06GB / 2.9B: 5.90GB / 7.2B: 14.40GB / 13.3B: 26.54GB BF16 safetensors
