Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
Menu
  1. Home
  2. AI & Software
  3. Local LMs

Local LMs

Scheduled Pinned Locked Moved AI & Software
llmfoundation models
53 Posts 1 Posters 2.3k Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • montezM Offline
    montezM Offline
    montez
    wrote on last edited by
    #22

    AI9Stars G9v3-3B

    Hugging Face: https://huggingface.co/ai9stars/G9v3-3B
    GitHub: https://github.com/AI9Stars

    Developer: AI9Stars
    Parameters: ~3B
    Context: 131,072 tokens
    Architecture: dense causal LM, LlamaForCausalLM
    License: Apache 2.0
    Modalities: Text
    Formats: safetensors
    On disk: 5.99GB safetensors

    1 Reply Last reply
    0
    • montezM Offline
      montezM Offline
      montez
      wrote on last edited by
      #23

      Poolside Laguna XS 2.1

      Hugging Face: https://huggingface.co/poolside/Laguna-XS-2.1, https://huggingface.co/poolside/Laguna-XS-2.1-GGUF
      Website: https://poolside.ai/blog/introducing-laguna-xs-2-1

      Developer: Poolside
      Released: July 2026
      Variants: BF16, FP8, NVFP4, INT4; GGUF BF16, Q4_K_M
      Parameters: 33B total, 3B active
      Context: 262,144 tokens
      Architecture: MoE, 40 layers: 10 global-attention + 30 sliding-window-attention; 256 experts + 1 shared expert
      License: OpenMDW-1.1
      Modalities: Text
      Runs on: Mac with 36GB RAM
      Formats: safetensors, GGUF
      On disk: 20.27GB Q4_K_M GGUF / 66.89GB BF16 safetensors

      1 Reply Last reply
      0
      • montezM Offline
        montezM Offline
        montez
        wrote on last edited by montez
        #24

        Meta Muse Glimmer-30B

        Hugging Face: https://huggingface.co/meta-models/Muse-Glimmer-30B
        Announcement: https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model
        Technical report: https://research.meta.ai/static/muse-glimmer-methodology

        Developer: Meta Superintelligence Lab
        Released: August 2026
        Parameters: 29.6B total, including perception encoder
        Context: 131,072+ tokens
        Architecture: Dense causal transformer with ViT-G/14 perception encoder; 52 layers, GQA, SwiGLU, RoPE
        License: Apache 2.0
        Modalities: Text + Image
        Runs on: MacBook M4 Max/M5 Max, RTX 5090; 24GB+ memory with 4-bit weights
        Formats: BF16 safetensors, 4-bit quantized weights
        On disk: 17GB K-Quant

        1 Reply Last reply
        0
        • montezM Offline
          montezM Offline
          montez
          wrote on last edited by
          #25

          webAI TwIL-LM

          Hugging Face: https://huggingface.co/webAI-Official/TwIL-LM, https://huggingface.co/webAI-Official/TwIL-LM3
          Website: https://www.webai.com/blog/webai-releases-twil-lm-a-family-of-formal-logic-models-that-outreason-a-120b-model-and-run-on-an-iphone

          Developer: webAI Intelligence Lab
          Released: August 2026
          Variants: TwIL-LM 1.7B, TwIL-LM3 3B
          Parameters: 1.7B: 1.78B total, 1.71B backbone + 72M LoRA / 3B: 3B
          Context: 1.7B: 8,192 tokens / 3B: 65,536 tokens
          Architecture: 1.7B: SmolLM2-1.7B-Instruct base, dense Llama, 24 layers, 32 heads, LoRA rank 64 SFT / 3B: SmolLM3-3B base, dense, 36 layers, GQA 16Q/4KV, NoPE every 4th layer; LoRA SFT, checkpoint fusion, WiSE-FT interpolation, GRPO reinforcement learning
          License: webAI Non-Commercial License ver. 1.0
          Modalities: Text
          Runs on: 1.7B: Smartphone, Laptop / 3B: Laptop, 4GB VRAM or CPU
          Formats: 1.7B: merged GGUF Q4_K_M, Q5_K_M, Q8_0, f16 / 3B: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16
          On disk: 1.7B: 1.06GB Q4_K_M / 3B: 1.92GB Q4_K_M

          1 Reply Last reply
          0
          • montezM Offline
            montezM Offline
            montez
            wrote on last edited by montez
            #26

            Ling 3.0

            Hugging Face: https://huggingface.co/inclusionAI/Ling-3.0-flash, https://huggingface.co/inclusionAI/Ling-3.0-tiny, https://huggingface.co/inclusionAI/Ling-3.0-tiny-int4, https://huggingface.co/inclusionAI/Ling-3.0-tiny-fp8
            GitHub: https://github.com/inclusionAI/Ling
            X: https://x.com/AntLingAGI/status/2080351022028095681
            Website: https://www.ant-ling.com/en
            Docs: https://github.com/inclusionAI/ling-cookbook

            Developer: Ant Group, InclusionAI
            Released: July 2026
            Variants: Ling-3.0-flash, Ling-3.0-tiny
            Parameters: flash: 124B total, 5.1B active / tiny: 7.9B total, 1.3B active
            Context: flash: 262,144 tokens / tiny: 131,072 tokens, 262,144 with YaRN
            Architecture: BailingMoE hybrid; flash: linear KDA + MLA attention with sparse MoE / tiny: 3:1 KDA to MLA blocks, 128 routed experts, 8 routed + 1 shared active
            License: MIT
            Modalities: Text
            Runs on: flash: NVIDIA DGX Spark / tiny: Laptop, 48GB unified memory
            Formats: safetensors BF16, safetensors FP8, safetensors INT4, GGUF Q4_K_M
            On disk: flash: 60.5GB Q4_K_M GGUF, 254.98GB BF16 safetensors / tiny: 5.81GB INT4, 8.41GB FP8, 15.79GB BF16 safetensors

            1 Reply Last reply
            0
            • montezM Offline
              montezM Offline
              montez
              wrote on last edited by montez
              #27

              Qwen3.8

              Hugging Face: https://huggingface.co/Qwen/Qwen3.8-27B, https://huggingface.co/Qwen/Qwen3.8-27B-FP8
              GitHub: https://github.com/QwenLM/Qwen3
              X: https://x.com/Alibaba_Qwen/status/2088280182356611304
              Website: https://qwen.ai/blog?id=qwen3.8

              Developer: Alibaba Cloud, Qwen team
              Released: August 2026
              Parameters: 27B
              Context: 262,144 tokens
              Architecture: Qwen3.5 foundation, dense native vision-language model; hybrid linear + full attention
              License: Apache 2.0
              Modalities: Text + Image + Video
              Runs on: Laptop, 24GB+ unified memory, estimated / Desktop GPU
              Formats: safetensors BF16, safetensors FP8, GGUF and MLX community quantizations
              On disk: 16.05GB MLX 4-bit / 30.87GB FP8 safetensors / 55.56GB BF16 safetensors

              1 Reply Last reply
              0
              • montezM Offline
                montezM Offline
                montez
                wrote on last edited by montez
                #28

                NVIDIA Nemotron 3.5 Lightning

                Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
                GitHub: https://github.com/NVIDIA-NeMo/Nemotron
                Website: https://build.nvidia.com/nvidia/nemotron-3.5-lightning-30b-a3b

                Developer: NVIDIA
                Released: August 2026
                Variants: Instruct, Base, DSpark and DFlash speculative drafters
                Parameters: 30B total, 3B active
                Context: 1,048,576 tokens
                Architecture: hybrid LatentMoE interleaving Mamba-2, MoE and attention; 52 layers, 128 routed experts, 6 active, 32Q/2KV heads, Multi-Token Prediction
                License: OpenMDW License Agreement, version 1.1
                Modalities: Text
                Runs on: Desktop GPU, Edge device, DGX Spark; NVIDIA only
                Formats: safetensors BF16, safetensors NVFP4, GGUF community conversion
                On disk: 17.82GB NVFP4 safetensors / 31.58GB BF16 safetensors

                1 Reply Last reply
                0
                • montezM Offline
                  montezM Offline
                  montez
                  wrote on last edited by
                  #29

                  Google DiffusionGemma

                  Hugging Face: https://huggingface.co/google/diffusiongemma-26B-A4B-it
                  GitHub: https://github.com/google-gemma
                  Website: https://ai.google.dev/gemma/docs/diffusiongemma
                  Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/

                  Developer: Google DeepMind
                  Released: June 2026
                  Parameters: 25.2B total, 3.8B active
                  Context: 256,000 tokens
                  Architecture: discrete text diffusion on the Gemma 4 26B A4B MoE foundation; autoregressive encoder prefills the prompt into a KV cache, decoder applies bidirectional attention over a 256-token canvas, block-autoregressive multi-canvas sampling; 30 layers, 8 active of 128 experts plus 1 shared, 1,024 sliding window, 550M vision encoder
                  License: Apache 2.0
                  Modalities: Text + Image + Video in, Text out
                  Runs on: Desktop GPU, 18GB+ VRAM
                  Formats: safetensors
                  On disk: 51.65GB safetensors

                  1 Reply Last reply
                  0
                  • montezM Offline
                    montezM Offline
                    montez
                    wrote on last edited by
                    #30

                    IBM Granite Swash

                    Hugging Face: https://huggingface.co/ibm-granite/granite-swash-2b, https://huggingface.co/ibm-granite/granite-swash-3b-a600m
                    GitHub: https://github.com/ibm-granite/granite-4.1-language-models

                    Developer: IBM Granite Team
                    Released: July 2026
                    Variants: SWASH-2B, SWASH-3B-A600M
                    Parameters: 2B / 3B total, 600M active
                    Context: 8,192 tokens
                    Architecture: sliding window attention with learnable per-head attention sinks, LSE-scaled; 2B: dense decoder-only, 24 layers, 7 full-attention + 17 sliding-window layers, window 128, GQA, SwiGLU, RoPE, RMSNorm / 3B-A600M: MoE, 28 layers, 48 experts, 4 routed active
                    License: Apache 2.0
                    Modalities: Text
                    Runs on: Smartphone, Laptop, Edge device
                    Formats: safetensors
                    On disk: 2B: 4.29GB safetensors / 3B-A600M: 6.04GB safetensors

                    1 Reply Last reply
                    0
                    • montezM Offline
                      montezM Offline
                      montez
                      wrote on last edited by
                      #31

                      IBM Granite Vision 4.1

                      Hugging Face: https://huggingface.co/ibm-granite/granite-vision-4.1-4b, https://huggingface.co/ibm-granite/granite-vision-4.1-4b-GGUF
                      GitHub: https://github.com/ibm-granite/granite-vision-models
                      Website: https://www.ibm.com/granite/docs/models/vision
                      Announcement: https://research.ibm.com/blog/granite-4-1-ai-foundation-models

                      Developer: IBM
                      Released: April 2026
                      Parameters: 4B total, Granite 4.1 3B language model plus vision encoder and projectors
                      Context: 131,072 tokens
                      Architecture: SigLIP2 so400m patch16-384 vision encoder over 384x384 image tiles, windowed Q-Former projectors compressing each 4x4 patch window to 2x2 tokens, and a Granite 4.1 3B language model with rank-256 LoRA across all self-attention projections
                      License: Apache 2.0
                      Modalities: Text + Image
                      Runs on: Smartphone, Laptop, Edge device
                      Formats: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, bf16, with f16 mmproj
                      On disk: 2.10GB Q4_K_M plus 1.16GB f16 mmproj / 6.81GB bf16

                      1 Reply Last reply
                      0
                      • montezM Offline
                        montezM Offline
                        montez
                        wrote on last edited by
                        #32

                        Microsoft Mage-VL

                        Hugging Face: https://huggingface.co/microsoft/Mage-VL
                        GitHub: https://github.com/microsoft/Mage
                        Website: https://microsoft.github.io/Mage/vl/
                        Technical report: https://arxiv.org/abs/2607.24904

                        Developer: Microsoft Mage Team
                        Released: July 2026
                        Parameters: 4B
                        Context: 262,144 tokens
                        Architecture: Mage-ViT codec-native visual encoder trained from scratch, 24 layers, feeding a two-layer MLP projector into a Qwen3-4B-Instruct-2507 causal decoder; separate cognition gate for proactive streaming
                        License: Apache 2.0
                        Modalities: Text + Image + Video
                        Runs on: Laptop, 12GB+ memory at BF16, estimated / Desktop GPU
                        Formats: safetensors, bundled streaming gate and neural codec
                        On disk: 9.48GB safetensors

                        1 Reply Last reply
                        0
                        • montezM Offline
                          montezM Offline
                          montez
                          wrote on last edited by
                          #33

                          Cohere Labs North Micro Vision

                          Hugging Face: https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct
                          Technical report: https://huggingface.co/blog/CohereLabs/meet-north-micro-vision-instruct

                          Developer: Cohere Labs
                          Released: August 2026
                          Parameters: 2.4B total, 2B language model + 400M vision encoder
                          Context: 128,000 tokens, multimodal validated to 8,192
                          Architecture: custom native-resolution vision encoder with DeepStack patch embeddings injected into early decoder layers, projector, and the Command A+ style North Micro LLM: three sliding-window attention layers with RoPE plus one global layer without positional embeddings
                          License: Apache 2.0
                          Modalities: Text + Image
                          Runs on: Smartphone, Laptop, Edge device, with quantization
                          Formats: safetensors BF16, MLX 4-bit and 8-bit community conversions
                          On disk: 4.97GB BF16 safetensors

                          1 Reply Last reply
                          0
                          • montezM Offline
                            montezM Offline
                            montez
                            wrote on last edited by
                            #34

                            Cactus Compute Needle 2

                            Hugging Face: https://huggingface.co/Cactus-Compute/needle2
                            GitHub: https://github.com/cactus-compute/needle
                            X: https://x.com/cactuscompute/status/2086865960669983035
                            Website: https://cactuscompute.com/needle

                            Developer: Cactus Compute
                            Released: August 2026
                            Parameters: 45M
                            Context: 2,048 tokens
                            Architecture: Simple Attention Network, 27 layers, hidden 512, 8Q/4KV GQA, Hadamard MLP, engram sites, CQ2 quantization at 2.2 effective bits; byte-level grammar-constrained decoding and a tool-retrieval head
                            License: Apache 2.0
                            Modalities: Text
                            Runs on: Smartphone, Headset, Edge device, Microcontroller
                            Formats: cact single binary; ARM64, x86-64, ARMv7, RISC-V and WebAssembly builds
                            On disk: 13.7MB cact

                            1 Reply Last reply
                            0
                            • montezM Offline
                              montezM Offline
                              montez
                              wrote on last edited by
                              #35

                              Syzygy Mach-1 Additive 35B

                              Hugging Face: https://huggingface.co/SyzygyResearch/Mach-1-Additive-35B
                              X: https://x.com/syzygyeng/status/2084350792841195992
                              Website: https://withsyzygy.com/mach-1
                              Docs: https://withsyzygy.com/docs/mach

                              Developer: Syzygy Research
                              Released: August 2026
                              Parameters: 35B total, 8 of 256 experts active
                              Context: 262,144 tokens
                              Architecture: Qwen3.5 MoE topology, 40 layers, 256 experts, 8 active, hybrid linear attention with full attention every 4th layer; additive 1.7-bit weights with no weight multiplication
                              License: Apache 2.0
                              Modalities: Text
                              Runs on: Laptop, 16GB+ unified memory; Apple Silicon only
                              Formats: packed 1.7-bit safetensors, MLX
                              On disk: 7.0GB

                              1 Reply Last reply
                              0
                              • montezM Offline
                                montezM Offline
                                montez
                                wrote on last edited by
                                #36

                                OpenMOSS MOSS-VL

                                Hugging Face: https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0708-FP8, https://huggingface.co/OpenMOSS-Team/MOSS-VL-Realtime-FP8
                                GitHub: https://github.com/OpenMOSS/MOSS-VL
                                Website: https://openmoss.ai/MOSS-VL/
                                Technical report: https://arxiv.org/abs/2606.07639

                                Developer: OpenMOSS Team
                                Released: August 2026
                                Variants: Instruct-0708, Realtime
                                Parameters: 11B
                                Context: 262,144 tokens
                                Architecture: unified cross-attention multimodal model, 48 language layers with 12 cross-attention layers, XRoPE 3D spatiotemporal positions, absolute frame timestamps for streaming video
                                License: Apache 2.0
                                Modalities: Text + Image + Video
                                Runs on: Desktop GPU; Instruct: 24GB VRAM / Realtime: 26GB+ VRAM; NVIDIA only
                                Formats: FP8 compressed-tensors with BF16 cross-attention and vision, HQQ INT8 KV cache
                                On disk: 15.73GB FP8 safetensors

                                1 Reply Last reply
                                0
                                • montezM Offline
                                  montezM Offline
                                  montez
                                  wrote last edited by
                                  #37

                                  IBM Granite 4.2

                                  Hugging Face: https://huggingface.co/ibm-granite/granite-4.2-3b, https://huggingface.co/ibm-granite/granite-4.2-8b, https://huggingface.co/ibm-granite/granite-4.2-30b
                                  GitHub: https://github.com/ibm-granite/granite-4.2-language-models
                                  Website: https://www.ibm.com/granite/docs/models/granite4-2

                                  Developer: IBM
                                  Released: August 2026
                                  Variants: 3B / 8B / 30B
                                  Parameters: 3B / 8B / 30B
                                  Context: 3B and 8B: 131,072 tokens / 30B: 131,072 tokens native, 512,000 tokens extended
                                  Architecture: dense decoder-only transformer; GQA, RoPE, SwiGLU and RMSNorm; native reasoning modes and tool calling
                                  License: Apache 2.0
                                  Modalities: Text
                                  Formats: safetensors
                                  On disk: 3B: 7.32GB BF16 / 8B: 17.58GB BF16 / 30B: 58.55GB BF16

                                  1 Reply Last reply
                                  0
                                  • montezM Offline
                                    montezM Offline
                                    montez
                                    wrote last edited by
                                    #38

                                    FireRedTeam FireRedAudio

                                    Hugging Face: https://huggingface.co/FireRedTeam/FireRedAudio
                                    GitHub: https://github.com/FireRedTeam/FireRedAudio
                                    Website: https://fireredteam.github.io/demos/fireredaudio/
                                    Technical report: https://arxiv.org/abs/2608.24168

                                    Developer: FireRedTeam
                                    Released: August 2026
                                    Parameters: 9B
                                    Architecture: shared 9B LLM; decoupled Audio Encoder for understanding and RedAE-Patch plus flow-matching DiT pathway for generation
                                    License: Apache 2.0
                                    Modalities: Text + Audio
                                    Runs on: Desktop GPU, NVIDIA only
                                    Formats: safetensors, PyTorch .pt
                                    On disk: FireRedAudio: 21.23GB safetensors / RedAE decoder: 8.40GB PyTorch .pt / 29.63GB total, estimated

                                    1 Reply Last reply
                                    0
                                    • montezM Offline
                                      montezM Offline
                                      montez
                                      wrote last edited by
                                      #39

                                      Cohere Labs Tiny Aya L2-Thinker

                                      Hugging Face: https://huggingface.co/CohereLabs/tiny-aya-l2-thinker
                                      Technical report: https://arxiv.org/abs/2609.10445

                                      Developer: Cohere Labs
                                      Released: September 2026
                                      Parameters: 3.35B
                                      Architecture: Cohere2 decoder-only transformer; supervised fine-tuning for in-language reasoning
                                      License: CC-BY-NC-4.0
                                      Modalities: Text
                                      Runs on: Laptop
                                      Formats: safetensors
                                      On disk: 6.70GB BF16 safetensors, estimated

                                      1 Reply Last reply
                                      0
                                      • montezM Offline
                                        montezM Offline
                                        montez
                                        wrote last edited by
                                        #40

                                        InternLM Intern Lumina U2

                                        Hugging Face: https://huggingface.co/internlm/InternLumina-U2
                                        GitHub: https://github.com/InternLM/InternLumina-U2
                                        Website: https://internlm.github.io/InternLumina-U2/

                                        Developer: Shanghai AI Laboratory / InternLM
                                        Released: September 2026
                                        Parameters: 16B total, 1B active
                                        Architecture: LLaDA-2.0 MoE diffusion LLM backbone; 8-codebook fully-discrete AToken visual representation; spatial-parallel denoising plus codebook-depth autoregressive head
                                        License: Apache 2.0
                                        Modalities: Text + Image + Video + 3D
                                        Runs on: Huawei Ascend NPU only
                                        Formats: safetensors
                                        On disk: 33.81GB Ascend safetensors, estimated

                                        1 Reply Last reply
                                        0
                                        • montezM Offline
                                          montezM Offline
                                          montez
                                          wrote last edited by
                                          #41

                                          Microsoft FrogNano

                                          Hugging Face: https://huggingface.co/microsoft/FrogNano-4B-2609
                                          GitHub: https://github.com/microsoft/FrogNano
                                          Technical report: https://arxiv.org/abs/2609.07925

                                          Developer: Microsoft
                                          Released: September 2026
                                          Parameters: 4.66B
                                          Context: 131K tokens in the evaluated configuration
                                          Architecture: Qwen3.5-4B base; dense, 32 layers, hybrid Gated DeltaNet + gated attention; reinforcement-learning post-training on about 1,500 synthetic software-engineering tasks; repository-level coding agent run through the five-tool Leaf harness
                                          License: MIT
                                          Modalities: Text
                                          Runs on: Laptop, 16GB+ unified memory
                                          Formats: BF16 safetensors
                                          On disk: 9.32GB BF16

                                          1 Reply Last reply
                                          0
                                          ↳

                                          OBJECTS Forum

                                          Join the conversation

                                          Create an account to return to your place in the thread, follow new replies, bookmark useful posts, and upvote contributions you value.

                                          Have something to add? Your perspective can make this thread better.

                                          Register Login
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes

                                          • Login
                                          • Don't have an account? Register
                                          • Login or register to search.
                                          • First post
                                            Last post
                                          • 0
                                            • Categories
                                            • Recent
                                            • Tags
                                            • Popular
                                            • Users