Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
Menu
  1. Home
  2. AI & Software
  3. Local LMs

Local LMs

Scheduled Pinned Locked Moved AI & Software
llmfoundation models
53 Posts 1 Posters 2.3k Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • montezM Offline
    montezM Offline
    montez
    wrote on last edited by montez
    #6

    IBM Granite 4.1 3B

    Hugging Face: https://huggingface.co/ibm-granite/granite-4.1-3b
    GitHub: https://github.com/ibm-granite/granite-4.1-language-models
    Announcement: https://huggingface.co/blog/ibm-granite/granite-4-1

    Developer: IBM
    Released: April 2026
    Parameters: 3B
    Context: 131,072 tokens
    Architecture: dense decoder-only transformer
    License: Apache 2.0
    Modalities: Text
    Runs on: Smartphone, Laptop, Edge device
    Formats: GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, MLX/Core ML via conversion
    On disk: 2.2GB Q4_K_M, estimated

    1 Reply Last reply
    0
    • montezM Offline
      montezM Offline
      montez
      wrote on last edited by montez
      #7

      Liquid AI LFM2.5-VL

      Hugging Face: https://huggingface.co/LiquidAI/LFM2.5-VL-450M, https://huggingface.co/LiquidAI/LFM2.5-VL-3B
      Website: https://www.liquid.ai
      Announcement: https://www.liquid.ai/blog/lfm2-5-vl-450m, https://www.liquid.ai/blog/lfm2-5-vl-3b

      Developer: Liquid AI
      Released: 450M: April 2026 / 3B: August 2026
      Variants: 450M, 3B
      Parameters: 450M / 3.1B
      Context: 32,768 tokens
      Architecture: hybrid gated-convolution + grouped-query attention LM backbone with SigLIP2 NaFlex vision encoder; 450M: LFM2.5-350M backbone, 86M vision encoder / 3B: LFM2.5-2.6B backbone, 30 layers, 32Q/8KV, 400M vision encoder
      License: LFM Open License v1.0
      Modalities: Text + Vision, bounding box / object detection support
      Runs on: 450M: Smartphone, Laptop, Edge device / 3B: Smartphone, Laptop, 3GB memory
      Formats: GGUF, ONNX, MLX 4-bit/5-bit/6-bit/8-bit/bf16, safetensors
      On disk: 450M: 270MB Q4, estimated / 3B: 1.67GB Q4_K_M, 6.25GB safetensors

      1 Reply Last reply
      0
      • montezM Offline
        montezM Offline
        montez
        wrote on last edited by
        #8

        Liquid AI LFM2-24B-A2B

        Hugging Face: https://huggingface.co/LiquidAI/LFM2-24B-A2B
        GitHub: https://github.com/Liquid4All/cookbook
        Website: https://www.liquid.ai
        Announcement: https://www.liquid.ai/blog/lfm2-24b-a2b

        Developer: Liquid AI
        Released: February 2026
        Parameters: 24B total, 2.3B active
        Context: 32,768 tokens
        Architecture: MoE, 40 layers: 30 conv + 10 attention, 64 experts, top-4 routing
        License: LFM Open License v1.0
        Modalities: Text
        Runs on: Laptop, 14GB+ unified memory minimum, 32GB+ recommended
        Formats: GGUF Q4_K_M, Q5_K_M, Q6_K, safetensors, ONNX
        On disk: 14.44GB Q4_K_M / 16.93GB Q5_K_M / 19.58GB Q6_K

        1 Reply Last reply
        0
        • montezM Offline
          montezM Offline
          montez
          wrote on last edited by
          #9

          Qwen3.6

          Hugging Face: https://huggingface.co/Qwen/Qwen3.6-27B, https://huggingface.co/Qwen/Qwen3.6-35B-A3B
          GitHub: https://github.com/QwenLM/Qwen3.6
          Website: https://qwen.ai/

          Developer: Alibaba Cloud, Qwen team
          Released: April 2026
          Parameters: 27B / 35B total, 3B active
          Context: 262,144 tokens, extensible to 1,010,000
          Architecture: Dense / MoE, 256 experts, 8 routed + 1 shared active
          License: Apache 2.0
          Modalities: Text + Vision
          Runs on: Laptop, 24GB+ unified memory minimum
          Formats: GGUF, MLX 4-bit, community quantizations
          On disk: 27B: 16GB Q4_K_M, estimated / 35B-A3B: 21GB Q4_K_M, estimated

          1 Reply Last reply
          0
          • montezM Offline
            montezM Offline
            montez
            wrote on last edited by
            #10

            PrismML Bonsai Image 4B

            Hugging Face: https://huggingface.co/prism-ml/bonsai-image-binary-4B-mlx-1bit, https://huggingface.co/prism-ml/bonsai-image-ternary-4B-mlx-2bit
            GitHub: https://github.com/PrismML-Eng/Bonsai-Image-Demo
            Website: https://prismml.com
            Announcement: https://prismml.com/news/bonsai-image-4b

            Developer: PrismML
            Released: May 2026
            Parameters: 4B, transformer trunk
            Architecture: MMDiT diffusion transformer, base architecture FLUX.2 Klein 4B, 25 blocks: 5 double-stream + 20 single-stream
            Variants: Binary, 1-bit / Ternary, 2-bit
            License: Apache 2.0
            Modalities: Text-to-Image
            Runs on: Smartphone, Laptop
            Formats: MLX 1-bit, MLX 2-bit, Gemlite 1-bit/2-bit for CUDA, safetensors
            On disk: Binary: 0.93GB transformer, 3.42GB total deployment payload / Ternary: 1.21GB transformer

            1 Reply Last reply
            0
            • montezM Offline
              montezM Offline
              montez
              wrote on last edited by
              #11

              Google Gemma 4 Effective

              Hugging Face: https://huggingface.co/collections/google/gemma-4
              GitHub: https://github.com/google-gemma
              Website: https://ai.google.dev/gemma/docs/core
              Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/

              Developer: Google DeepMind
              Released: April 2026
              Variants: E2B, E4B
              Parameters: E2B: 2.3B effective, 5.1B with embeddings / E4B: 4.5B effective, 8B with embeddings
              Context: 128,000 tokens
              Architecture: Dense, hybrid local sliding window + global attention, Per-Layer Embeddings for on-device efficiency
              License: Apache 2.0
              Modalities: Text + Image + Audio
              Runs on: Smartphone, Laptop, Edge device
              Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face
              On disk: E2B: 1.4GB Q4, estimated / E4B: 2.7GB Q4, estimated

              1 Reply Last reply
              0
              • montezM Offline
                montezM Offline
                montez
                wrote on last edited by montez
                #12

                Google Gemma 4

                Hugging Face: https://huggingface.co/collections/google/gemma-4
                GitHub: https://github.com/google-gemma
                Website: https://ai.google.dev/gemma/docs/core
                Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/, https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12B/

                Developer: Google DeepMind
                Released: 26B A4B and 31B: April 2026 / 12B Unified: June 2026
                Variants: 12B Unified, 26B A4B, 31B
                Parameters: 12B Unified: 11.95B / 26B A4B: 25.2B total, 3.8B active / 31B: 30.7B
                Context: 256,000 tokens
                Architecture: 31B: Dense / 26B A4B: MoE, 8 active of 128 experts plus 1 shared / 12B: Unified encoder-free dense, multimodal input projected directly into the decoder
                License: Apache 2.0
                Modalities: 12B: Text + Image + Audio / 26B A4B and 31B: Text + Image
                Runs on: Laptop, consumer GPU/workstation class
                Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face, Docker
                On disk: 12B: ~7GB Q4 / 26B A4B: ~15GB Q4 / 31B: ~18GB Q4

                1 Reply Last reply
                0
                • montezM Offline
                  montezM Offline
                  montez
                  wrote on last edited by montez
                  #13

                  Agents-A1

                  Hugging Face: https://huggingface.co/InternScience/Agents-A1, https://huggingface.co/InternScience/Agents-A1-4B
                  GitHub: https://github.com/InternScience/Agents-A1
                  X: https://x.com/intern_lm
                  Website: https://internscience.github.io/Agents-A1/
                  Technical report: https://arxiv.org/abs/2606.30616

                  Developer: InternScience, Shanghai Artificial Intelligence Laboratory
                  Released: 35B-A3B: June 2026 / 4B: July 2026
                  Parameters: 35B total, 3B active / 4B dense
                  Context: 262,144 tokens
                  Architecture: Qwen3.5 base, hybrid linear + full attention (full attention every 4th layer); 35B-A3B: MoE, 40 layers, 256 experts, 8 routed + 1 shared active / 4B: dense, 32 layers
                  License: Apache 2.0
                  Modalities: Text + Vision
                  Runs on: 35B-A3B: Laptop, 24GB+ unified memory / 4B: Smartphone, Laptop
                  Formats: GGUF, MLX (community), safetensors
                  On disk: 35B-A3B: 19.7GB Q4_K_M / 4B: 2.5GB Q4_K_M, 4.2GB Q8_0

                  1 Reply Last reply
                  0
                  • montezM Offline
                    montezM Offline
                    montez
                    wrote on last edited by
                    #14

                    Ornith-1.0

                    Hugging Face: https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B, https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B
                    GitHub: https://github.com/deepreinforce-ai/Ornith-1
                    Website: https://deep-reinforce.com/ornith.html

                    Developer: DeepReinforce
                    Released: June 2026
                    Parameters: 9B dense / 35B total, 8 of 256 experts active
                    Context: 262,144 tokens
                    Architecture: Qwen3.5 base; 9B: dense, 32 layers / 35B: MoE, 40 layers, 256 experts, 8 active
                    License: MIT
                    Modalities: Text + Vision
                    Runs on: 9B: Smartphone, Laptop / 35B: Laptop, 24GB+ unified memory
                    Formats: GGUF, FP8, safetensors
                    On disk: 9B: 5.9GB Q4_K_M / 35B: 21.4GB Q4_K_M

                    1 Reply Last reply
                    0
                    • montezM Offline
                      montezM Offline
                      montez
                      wrote on last edited by montez
                      #15

                      DeepGrove Maple-Preview

                      Hugging Face: https://huggingface.co/deepgrove/maple-preview
                      GitHub: https://github.com/deepgrove-ai/mlx-lm-deepgrove
                      Website: https://deepgrove.ai/maple-preview

                      Developer: DeepGrove
                      Released: August 2026
                      Parameters: 20B total, 1B active
                      Context: 131,072 tokens
                      Architecture: ternary-weight MoE, 24 layers, 256 experts, 8 active; 3:1 SWA-512:GA attention
                      License: MIT
                      Modalities: Text
                      Runs on: Mac mini M4
                      Formats: safetensors, MLX 2-bit
                      On disk: 5.31GB MLX 2-bit

                      1 Reply Last reply
                      0
                      • montezM Offline
                        montezM Offline
                        montez
                        wrote on last edited by montez
                        #16

                        Nanbeige4.2-3B

                        Hugging Face: https://huggingface.co/Nanbeige/Nanbeige4.2-3B
                        Website: https://nanbeige.zhipin.com/
                        Technical report: https://arxiv.org/abs/2607.22083

                        Developer: Nanbeige, BOSS Zhipin
                        Released: July 2026
                        Parameters: 4B total, 3B non-embedding
                        Context: 262,144 tokens
                        Architecture: dense Looped Transformer, LoopSplit, mHC with depth attention, concatenated n-gram embeddings
                        License: Apache 2.0
                        Modalities: Text
                        Runs on: Laptop
                        Formats: safetensors, GGUF Q4_K_M via conversion, MLX 4-bit
                        On disk: 3.30GB MLX 4-bit

                        1 Reply Last reply
                        0
                        • montezM Offline
                          montezM Offline
                          montez
                          wrote on last edited by montez
                          #17

                          Fermion Research Neutrino

                          Hugging Face: https://huggingface.co/FermionResearch/Neutrino-8B, https://huggingface.co/FermionResearch/Neutrino-0.6B, https://huggingface.co/FermionResearch/Neutrino-0.6B-Chat
                          GitHub: https://github.com/fermionresearch/llama.cpp
                          Website: https://www.fermionresearch.com/models/neutrino-8b/
                          Announcement: https://www.fermionresearch.com/research/neutrino-8b/

                          Developer: Fermion Research
                          Released: July 2026
                          Variants: 8B general purpose, 0.6B speculative-decoding draft, 0.6B-Chat conversational
                          Parameters: 8.19B / 596M
                          Context: 40,960 tokens
                          Architecture: ternary QAT with staged post-training; 8B: Qwen3-8B topology, 36-layer decoder-only transformer, SwiGLU, GQA 4:1, RoPE, RMSNorm / 0.6B: 28 layers, hidden 1,024, SwiGLU 3,072, GQA 16Q/8KV, per-head Q/K RMSNorm, tied int8 embeddings
                          License: Apache 2.0
                          Modalities: Text
                          Runs on: 8B: Laptop, 16GB unified memory / 0.6B: Laptop, CPU only
                          Formats: TRTC v4, custom-FV5 GGUF, MLX
                          On disk: 8B: 3.88GB TRTC v4 / 4.09GB custom-FV5 GGUF / 0.6B: 328MB TRTC v4, 238MB tv4z, 343MB GGUF

                          1 Reply Last reply
                          0
                          • montezM Offline
                            montezM Offline
                            montez
                            wrote on last edited by montez
                            #18

                            Kakao Kanana-2

                            Hugging Face: https://huggingface.co/kakaocorp/kanana-2-3b-instruct, https://huggingface.co/kakaocorp/kanana-2-1.3b-instruct
                            Website: https://tech.kakao.com/posts/826

                            Developer: Kakao, Kanana LLM
                            Released: July 2026
                            Variants: 3B, 1.3B
                            Parameters: 3B / 1.3B
                            Context: 32,768 tokens
                            Architecture: 3B: dense, pretrained from scratch, SFT + RL / 1.3B: cascade-pruned and distilled from 3B, sliding-window attention
                            License: Kanana Open License
                            Modalities: Text
                            Runs on: Smartphone, Laptop
                            Formats: safetensors, MLX 4-bit, 6-bit, 8-bit
                            On disk: 3B: 1.99GB MLX 4-bit

                            1 Reply Last reply
                            0
                            • montezM Offline
                              montezM Offline
                              montez
                              wrote on last edited by
                              #19

                              Microsoft Fara1.5

                              Hugging Face: https://huggingface.co/microsoft/Fara1.5-4B, https://huggingface.co/microsoft/Fara1.5-9B, https://huggingface.co/microsoft/Fara1.5-27B
                              GitHub: https://github.com/microsoft/fara
                              Website: https://labs.ai.azure.com/innovations/fara1-5/

                              Developer: Microsoft Research AI Frontiers
                              Released: May 2026
                              Variants: 4B, 9B, 27B
                              Parameters: 4B / 9B / 27B
                              Context: 262,144 tokens
                              Architecture: multimodal decoder-only LM, image + text to text
                              License: MIT
                              Modalities: Text + Image
                              Formats: safetensors
                              On disk: 4B: 9.08GB / 9B: 18.82GB / 27B: 54.71GB safetensors

                              1 Reply Last reply
                              0
                              • montezM Offline
                                montezM Offline
                                montez
                                wrote on last edited by
                                #20

                                AMD Instella-MoE-16B-A3B-Think

                                Hugging Face: https://huggingface.co/amd/Instella-MoE-16B-A3B-Think
                                GitHub: https://github.com/AMD-AGI/Instella-MoE
                                Website: https://rocm.blogs.amd.com/artificial-intelligence/instella-moe/README.html

                                Developer: AMD
                                Released: July 2026
                                Parameters: 16B total, 2.8B active
                                Context: 32,768 tokens
                                Architecture: decoder-only MoE, Gated Multi-head Latent Attention, FarSkip-Collective connectivity
                                License: Research RAIL
                                Modalities: Text
                                Formats: safetensors
                                On disk: 31.73GB safetensors

                                1 Reply Last reply
                                0
                                • montezM Offline
                                  montezM Offline
                                  montez
                                  wrote on last edited by montez
                                  #21

                                  Cisco Antares

                                  Hugging Face: https://huggingface.co/fdtn-ai/antares-1b, https://huggingface.co/fdtn-ai/antares-350m
                                  Website: https://cisco-foundation-ai.github.io/antares/
                                  Announcement: https://cisco-foundation-ai.github.io/blogs/antares-beyond-vlocbench/

                                  Developer: Cisco Foundation AI
                                  Released: July 2026
                                  Variants: 1b, 350m
                                  Parameters: 1B / 350M
                                  Architecture: fine-tuned from IBM Granite 4.0, GraniteMoEHybrid
                                  License: Apache 2.0
                                  Modalities: Text
                                  Runs on: Smartphone, Laptop, Edge device
                                  Formats: safetensors
                                  On disk: 1b: 3.67GB safetensors / 350m: 0.70GB safetensors

                                  1 Reply Last reply
                                  0
                                  • montezM Offline
                                    montezM Offline
                                    montez
                                    wrote on last edited by
                                    #22

                                    AI9Stars G9v3-3B

                                    Hugging Face: https://huggingface.co/ai9stars/G9v3-3B
                                    GitHub: https://github.com/AI9Stars

                                    Developer: AI9Stars
                                    Parameters: ~3B
                                    Context: 131,072 tokens
                                    Architecture: dense causal LM, LlamaForCausalLM
                                    License: Apache 2.0
                                    Modalities: Text
                                    Formats: safetensors
                                    On disk: 5.99GB safetensors

                                    1 Reply Last reply
                                    0
                                    • montezM Offline
                                      montezM Offline
                                      montez
                                      wrote on last edited by
                                      #23

                                      Poolside Laguna XS 2.1

                                      Hugging Face: https://huggingface.co/poolside/Laguna-XS-2.1, https://huggingface.co/poolside/Laguna-XS-2.1-GGUF
                                      Website: https://poolside.ai/blog/introducing-laguna-xs-2-1

                                      Developer: Poolside
                                      Released: July 2026
                                      Variants: BF16, FP8, NVFP4, INT4; GGUF BF16, Q4_K_M
                                      Parameters: 33B total, 3B active
                                      Context: 262,144 tokens
                                      Architecture: MoE, 40 layers: 10 global-attention + 30 sliding-window-attention; 256 experts + 1 shared expert
                                      License: OpenMDW-1.1
                                      Modalities: Text
                                      Runs on: Mac with 36GB RAM
                                      Formats: safetensors, GGUF
                                      On disk: 20.27GB Q4_K_M GGUF / 66.89GB BF16 safetensors

                                      1 Reply Last reply
                                      0
                                      • montezM Offline
                                        montezM Offline
                                        montez
                                        wrote on last edited by montez
                                        #24

                                        Meta Muse Glimmer-30B

                                        Hugging Face: https://huggingface.co/meta-models/Muse-Glimmer-30B
                                        Announcement: https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model
                                        Technical report: https://research.meta.ai/static/muse-glimmer-methodology

                                        Developer: Meta Superintelligence Lab
                                        Released: August 2026
                                        Parameters: 29.6B total, including perception encoder
                                        Context: 131,072+ tokens
                                        Architecture: Dense causal transformer with ViT-G/14 perception encoder; 52 layers, GQA, SwiGLU, RoPE
                                        License: Apache 2.0
                                        Modalities: Text + Image
                                        Runs on: MacBook M4 Max/M5 Max, RTX 5090; 24GB+ memory with 4-bit weights
                                        Formats: BF16 safetensors, 4-bit quantized weights
                                        On disk: 17GB K-Quant

                                        1 Reply Last reply
                                        0
                                        • montezM Offline
                                          montezM Offline
                                          montez
                                          wrote on last edited by
                                          #25

                                          webAI TwIL-LM

                                          Hugging Face: https://huggingface.co/webAI-Official/TwIL-LM, https://huggingface.co/webAI-Official/TwIL-LM3
                                          Website: https://www.webai.com/blog/webai-releases-twil-lm-a-family-of-formal-logic-models-that-outreason-a-120b-model-and-run-on-an-iphone

                                          Developer: webAI Intelligence Lab
                                          Released: August 2026
                                          Variants: TwIL-LM 1.7B, TwIL-LM3 3B
                                          Parameters: 1.7B: 1.78B total, 1.71B backbone + 72M LoRA / 3B: 3B
                                          Context: 1.7B: 8,192 tokens / 3B: 65,536 tokens
                                          Architecture: 1.7B: SmolLM2-1.7B-Instruct base, dense Llama, 24 layers, 32 heads, LoRA rank 64 SFT / 3B: SmolLM3-3B base, dense, 36 layers, GQA 16Q/4KV, NoPE every 4th layer; LoRA SFT, checkpoint fusion, WiSE-FT interpolation, GRPO reinforcement learning
                                          License: webAI Non-Commercial License ver. 1.0
                                          Modalities: Text
                                          Runs on: 1.7B: Smartphone, Laptop / 3B: Laptop, 4GB VRAM or CPU
                                          Formats: 1.7B: merged GGUF Q4_K_M, Q5_K_M, Q8_0, f16 / 3B: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16
                                          On disk: 1.7B: 1.06GB Q4_K_M / 3B: 1.92GB Q4_K_M

                                          1 Reply Last reply
                                          0
                                          ↳

                                          OBJECTS Forum

                                          Join the conversation

                                          Create an account to return to your place in the thread, follow new replies, bookmark useful posts, and upvote contributions you value.

                                          Have something to add? Your perspective can make this thread better.

                                          Register Login
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes

                                          • Login
                                          • Don't have an account? Register
                                          • Login or register to search.
                                          • First post
                                            Last post
                                          • 0
                                            • Categories
                                            • Recent
                                            • Tags
                                            • Popular
                                            • Users