Local LMs

[montez]

NII LLM-jp-4

Hugging Face: https://huggingface.co/llm-jp/llm-jp-4.1-8b-thinking, https://huggingface.co/llm-jp/llm-jp-4.1-8b-thinking-gguf, https://huggingface.co/llm-jp/llm-jp-4.1-32b-a3b-thinking, https://huggingface.co/llm-jp/llm-jp-4.1-32b-a3b-thinking-gguf, https://huggingface.co/llm-jp/llm-jp-4.1-33b-thinking, https://huggingface.co/llm-jp/llm-jp-4.1-33b-thinking-gguf, https://huggingface.co/llm-jp/llm-jp-4-vl-9b
GitHub: https://github.com/llm-jp/llm-jp-4-cookbook
Website: https://llm-jp.nii.ac.jp/blog/llm-jp-4-1/

Developer: National Institute of Informatics (LLMC, LLM-jp)
Released: April-September 2026, rolling family
Variants: LLM-jp-4 8B / 32B-A3B / 33B / VL 9B; LLM-jp-4.1 Thinking 8B / 32B-A3B / 33B
Parameters: 8B: 8.59B / 32B-A3B: 32.14B total, 3.83B active / 33B: 33.22B / VL 9B: 9.05B
Context: 65,536 tokens, text models
Architecture: 8B and 33B: dense Llama architecture / 32B-A3B: Qwen3-MoE architecture, 128 routed experts with 8 active / VL 9B: llm-jp-4-8b-thinking base, SigLIP 2 vision encoder
License: Apache 2.0
Modalities: 8B, 32B-A3B, 33B: Text / VL 9B: Text + Image
Runs on: 8B: Laptop / 32B-A3B and 33B: Laptop, 32GB+ unified memory / VL 9B: Laptop, 24GB+ unified memory
Formats: safetensors BF16, GGUF (BF16, Q4_K_M)
On disk: LLM-jp-4.1 8B: 5.50GB Q4_K_M GGUF / 32B-A3B: 21.52GB Q4_K_M GGUF / 33B: 20.41GB Q4_K_M GGUF / VL 9B: 18.11GB BF16

[montez]

Qwen-Image-2.1

Hugging Face: https://huggingface.co/Qwen/Qwen-Image-2.1, https://huggingface.co/Qwen/Qwen-Image-2.1-Turbo
GitHub: https://github.com/QwenLM/Qwen-Image-2.1
Website: https://qwen.ai/blog?id=qwen-image-2.1

Developer: Alibaba Cloud, Qwen team
Released: September 2026
Variants: Qwen-Image-2.1 / Qwen-Image-2.1-Turbo (8 steps)
Parameters: 7B visual generation component, plus Qwen3-VL 8B text encoder
Architecture: single-stream DiT, 32 layers, Qwen3-VL 8B text encoder, RGBA VAE with 16x spatial compression
License: Qwen Research License Agreement
Modalities: Text-to-Image + Image editing
Runs on: Desktop GPU, 24GB+ VRAM with model CPU offload
Formats: BF16 safetensors
On disk: Qwen-Image-2.1: 33.13GB safetensors / Turbo: 32.49GB safetensors

[montez]

Meta Muse Spark Open

Hugging Face: https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B, https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B-NVFP4
Website: https://dev.meta.ai/models/muse-spark-open

Developer: Meta Superintelligence Labs
Released: October 2026
Parameters: 816B total, 42B active
Context: 1,048,576 tokens
Architecture: sparse MoE multimodal transformer, 62 layers, 256 experts with 8 routed per token, GQA; 1.8B perception encoder plus audio encoder
License: Muse Model License
Modalities: Text + Image + Audio + Video
Runs on: Desktop, 2 Mac Studio M5 Ultra with 512GB unified memory each, or 4 with 256GB each (NVFP4, MLX) / Server, 4 NVIDIA Blackwell GPUs (NVFP4) or 8 (BF16)
Formats: BF16 safetensors, NVFP4 safetensors, GGUF via community conversion
On disk: BF16: 1.64TB safetensors / NVFP4: 533GB safetensors

[montez]

Cloudflare Clef

Hugging Face: https://huggingface.co/Cloudflare/clef, https://huggingface.co/Cloudflare/clef-flash, https://huggingface.co/Cloudflare/clef-omni
Website: https://blog.cloudflare.com/clef-faster-cheaper-multimodal

Developer: Cloudflare
Released: October 2026
Variants: Clef, Clef-Flash, Clef-Omni
Parameters: Clef: 27B / Clef-Flash: 9B / Clef-Omni: 30B total, 3B active
Architecture: decision models that score every option of typed questions in one forward pass; Clef: Qwen3.8-27B base / Clef-Flash: Qwen3.5-9B base / Clef-Omni: Qwen3-Omni-30B-A3B base, MoE; joint schema head with two-stage attention routing
License: Apache 2.0
Modalities: Clef and Clef-Flash: Text + Image + Video in / Clef-Omni: Text + Image + Audio + Video in; probabilities per option out
Runs on: Clef: Workstation GPU, 64GB+ memory / Clef-Flash: Desktop GPU, 24GB+ VRAM / Clef-Omni: Workstation GPU, 64GB+ memory
Formats: safetensors BF16
On disk: Clef: 54.97GB BF16 / Clef-Flash: 19.06GB BF16 / Clef-Omni: 70.74GB BF16

[montez]

Liquid AI d1

Hugging Face: https://huggingface.co/LiquidAI/d1-3B, https://huggingface.co/LiquidAI/d1-3B-GGUF, https://huggingface.co/LiquidAI/d1-omni-600M, https://huggingface.co/LiquidAI/d1-omni-600M-GGUF
Website: https://www.liquid.ai/blog/d1-open
Docs: https://docs.liquid.ai/lfm/models/decision-models

Developer: Liquid AI
Released: October 2026
Variants: d1-3B, d1-omni-600M
Parameters: d1-3B: 3.12B / d1-omni-600M: 587M
Context: d1-3B: 32,768 tokens / d1-omni-600M: 16,384 tokens
Architecture: decision models that return typed answers in one forward pass; d1-3B: LFM2.5-VL-3B base, hybrid convolution and attention decoder / d1-omni-600M: LFM2.5-Encoder-350M base, bidirectional encoder with vision and audio encoders
License: LFM Open License v1.0
Modalities: d1-3B: Text + Image in / d1-omni-600M: Text + Image or Text + Audio in; probabilities per option out
Runs on: d1-3B: Laptop, Desktop GPU, Edge device / d1-omni-600M: Smartphone, Laptop, Edge device
Formats: safetensors, GGUF (Q4_K_M to BF16), MLX, W8A8 INT8 (d1-3B)
On disk: d1-3B: 1.67GB Q4_K_M, 6.25GB BF16 / d1-omni-600M: 0.41GB Q8_0, 2.35GB F32

[montez]

H2O-Lightning-4B

Hugging Face: https://huggingface.co/h2oai/h2o-lightning-4b, https://huggingface.co/h2oai/h2o-lightning-4b-gguf
Website: https://h2o.ai/blog/2026/h2o-lightning-jevbench/

Developer: H2O.ai
Released: October 2026
Parameters: 4B
Context: 40,960 tokens
Architecture: Qwen3.5-4B base, dense, 32 layers (8 full attention, 24 linear attention); decision model that scores every option of typed questions in one forward pass; causal plus bidirectional attention on the full-attention layers
License: Apache 2.0
Modalities: Text + Image in; probabilities per option out
Runs on: Laptop, Desktop GPU, 16GB+ unified memory or VRAM
Formats: safetensors BF16, GGUF BF16, Q8_0, Q5_K_M, Q4_K_M
On disk: 2.71GB Q4_K_M / 4.48GB Q8_0 / 9.32GB BF16

[montez]

NVIDIA Nemotron Labs Dory1.0 4B

Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16, https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4, https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-MXFP8, https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF

Developer: NVIDIA
Released: October 2026
Parameters: 4B
Context: 262,144 tokens
Architecture: looped (latent-recurrent) normalized-GPT; a shared 20-layer recurrent block is applied 1, 2 or 4 times between 8 input and 8 output layers; interleaved sliding-window and full attention
License: OpenMDW License Agreement, version 1.1
Modalities: Text
Runs on: Laptop, Desktop GPU
Formats: safetensors BF16, safetensors MXFP8, safetensors NVFP4, GGUF Q5_K_M
On disk: 3.67GB Q5_K_M GGUF / 3.2GB NVFP4 / 4.75GB MXFP8 / 7.95GB BF16

[montez]

NVIDIA Nemotron 3.5 Bolt

Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB, https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF, https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16
Website: https://research.nvidia.com/labs/nemotron/Nemotron-3.5-Bolt-24GB/
Technical report: https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-Technical-Report.pdf

Developer: NVIDIA
Released: October 2026
Variants: 74B-A9B 24GB (2-bit) / 74B-A9B BF16
Parameters: 74B total, 9B active
Context: 262,144 tokens
Architecture: Mamba-2, MoE and attention hybrid LatentMoE, 88 layers (40 Mamba-2, 40 MoE, 8 attention), expert top-K 9; compressed from Nemotron 3.5 Super VL 120B-A12B; C-RADIO v4-H vision encoder
License: OpenMDW License Agreement, version 1.1
Modalities: Text + Image + Video
Runs on: Laptop, Desktop GPU, 24GB+ memory; NVIDIA only
Formats: safetensors 2-bit packed, GGUF with mmproj, safetensors BF16
On disk: 24.12GB safetensors / 22.02GB GGUF plus 1.73GB mmproj / 149.8GB BF16

[montez]

IBM Granite Switch 4.1

Hugging Face: https://huggingface.co/ibm-granite/granite-switch-4.1-8b, https://huggingface.co/ibm-granite/granite-switch-4.1-30b
GitHub: https://github.com/generative-computing/granite-switch
Website: https://huggingface.co/blog/ibm-granite/granite-switch-4-1

Developer: IBM
Released: October 2026
Variants: 8B / 30B
Parameters: 8B / 30B
Context: 131,072 tokens
Architecture: modular LLM composing a Granite 4.1 base model with activated LoRA adapters (Decision, Guardian Core) selected by control tokens, plus an embedded Granite Speech 5.0 TurboCTC 470M English speech encoder
License: Apache 2.0
Modalities: Text + Audio
Runs on: 8B: Desktop GPU / 30B: Workstation GPU
Formats: safetensors BF16
On disk: 8B: 17.56GB BF16 / 30B: 59.99GB BF16

[montez]

Tencent Hy-Memory-30B-A3B

Hugging Face: https://huggingface.co/tencent/Hy-Memory-30B-A3B, https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8, https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF
GitHub: https://github.com/Tencent-Hunyuan/Hy-Memory
Technical report: https://arxiv.org/abs/2610.22196

Developer: Tencent Hunyuan
Released: October 2026
Parameters: 30B total, 3B active
Context: 262,144 tokens
Architecture: Hunyuan Hy3 MoE, 48 layers with the first layer dense, 128 routed experts with 8 active plus 1 shared, GQA 32Q/4KV heads; long-term memory writer for the Mem0, MemOS and Hy-Memory frameworks
License: Apache 2.0
Modalities: Text
Runs on: Laptop, 32GB+ unified memory
Formats: safetensors BF16, safetensors FP8, GGUF Q4_K_M, Q6_K, Q8_0
On disk: 18.24GB Q4_K_M / 24.71GB Q6_K / 31.99GB Q8_0 / 30.59GB FP8 / 60.14GB BF16

[montez]

OpenBMB MiniCPM-V 4.7

Hugging Face: https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B, https://huggingface.co/openbmb/MiniCPM-V-4.7-1B
GitHub: https://github.com/OpenBMB/MiniCPM-V
Website: https://openbmb.cn/en/model/minicpm-v-4-7
Docs: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/minicpm_v4dot7_en.md

Developer: OpenBMB, ModelBest
Released: October 2026
Variants: 35B-A3B / 1B
Parameters: 35B total, 3B active / 1B
Context: 262,144 tokens
Architecture: hybrid linear and full attention language backbone with full attention every 4th layer, 27-layer SigLIP2 vision encoder; 35B-A3B: MoE, 40 layers, 256 experts with 8 active / 1B: dense, 24 layers
License: Apache 2.0
Modalities: Text + Image + Video
Runs on: 35B-A3B: Smartphone, Laptop, 32GB+ unified memory / 1B: Smartphone, Laptop, Edge device
Formats: safetensors BF16, FP8, int4, AWQ, GPTQ, GGUF
On disk: 35B-A3B: 21.17GB Q4_K_M GGUF, 70.56GB BF16 / 1B: 0.67GB Q4_K_M GGUF, 3.11GB BF16

[montez]

Microsoft AesCode

Hugging Face: https://huggingface.co/microsoft/AesCode-8B, https://huggingface.co/microsoft/AesCode-32B
GitHub: https://github.com/microsoft/AesCode
Website: https://microsoft.github.io/AesCode/
Technical report: https://arxiv.org/abs/2610.18860

Developer: Microsoft
Released: October 2026
Variants: 8B / 32B
Parameters: 8B / 32B
Context: 262,144 tokens
Architecture: 8B: Qwen3-VL-8B-Instruct base / 32B: Qwen3-VL-32B-Instruct base; generates editable HTML and CSS
License: Apache 2.0
Modalities: Text + Image
Runs on: 8B: Desktop GPU / 32B: Workstation GPU
Formats: safetensors BF16
On disk: 8B: 17.55GB BF16 / 32B: 66.73GB BF16

[montez]

Xiaomi MiMo-V2.6-Distill-Qwen-9B

Hugging Face: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
Website: https://mimo.xiaomi.com/mimo-v2-6/article
Technical report: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL/blob/main/MiMo_V2_6_technical_report.pdf

Developer: Xiaomi MiMo
Released: September 2026
Parameters: 9B
Context: 262,144 tokens
Architecture: Qwen3.5-9B base, hybrid linear and full attention with full attention every 4th layer, 32 layers, 27-layer vision encoder
License: MIT
Modalities: Text + Image
Runs on: Desktop GPU
Formats: safetensors BF16
On disk: 18.85GB BF16

[montez]

Amazon ALoDLM

Hugging Face: https://huggingface.co/amazon/ALoDLM-8B, https://huggingface.co/amazon/ALoDLM-1.7B
GitHub: https://github.com/amazon-science/ALoDLM
Website: https://alo-dlm.github.io/
Technical report: https://arxiv.org/abs/2610.04198

Developer: Amazon
Released: October 2026
Variants: 8B / 1.7B
Parameters: 8B / 1.7B
Architecture: adaptively looped masked diffusion language model initialized from Qwen3-8B and Qwen3-1.7B; shared transformer layers applied recurrently with a learned halting policy; 8B: 36 layers / 1.7B: 28 layers
License: CC-BY-NC-4.0
Modalities: Text
Runs on: 8B: Desktop GPU, 24GB+ VRAM / 1.7B: Laptop, 8GB+ VRAM; NVIDIA only
Formats: safetensors BF16
On disk: 8B: 16.4GB BF16 / 1.7B: 4.08GB BF16

[montez]

Ai2 AstaBrief 8B

Hugging Face: https://huggingface.co/allenai/AstaBrief_8B, https://huggingface.co/allenai/AstaBrief_8B_SFT
Website: https://allenai.org/blog/astabrief

Developer: Ai2
Released: October 2026
Variants: AstaBrief-8B (DPO) / AstaBrief-8B-SFT
Parameters: 8B
Context: 40,960 tokens
Architecture: Qwen3-8B base, 36 layers; turns a research question and retrieved literature excerpts into a cited scientific report
License: Apache 2.0
Modalities: Text
Runs on: Desktop GPU
Formats: safetensors BF16
On disk: 16.38GB BF16