Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
Menu
  1. Home
  2. AI & Software
  3. Local Foundation Models

Local Foundation Models

Scheduled Pinned Locked Moved AI & Software
llm
40 Posts 1 Posters 1.1k Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • montezM Offline
    montezM Offline
    montez
    wrote on last edited by
    #11

    Google Gemma 4 Effective

    Hugging Face: https://huggingface.co/collections/google/gemma-4
    GitHub: https://github.com/google-gemma
    Website: https://ai.google.dev/gemma/docs/core
    Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/

    Developer: Google DeepMind
    Released: April 2026
    Variants: E2B, E4B
    Parameters: E2B: 2.3B effective, 5.1B with embeddings / E4B: 4.5B effective, 8B with embeddings
    Context: 128,000 tokens
    Architecture: Dense, hybrid local sliding window + global attention, Per-Layer Embeddings for on-device efficiency
    License: Apache 2.0
    Modalities: Text + Image + Audio
    Runs on: Smartphone, Laptop, Edge device
    Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face
    On disk: E2B: 1.4GB Q4, estimated / E4B: 2.7GB Q4, estimated

    1 Reply Last reply
    0
    • montezM Offline
      montezM Offline
      montez
      wrote on last edited by montez
      #12

      Google Gemma 4

      Hugging Face: https://huggingface.co/collections/google/gemma-4
      GitHub: https://github.com/google-gemma
      Website: https://ai.google.dev/gemma/docs/core
      Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/, https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12B/

      Developer: Google DeepMind
      Released: 26B A4B and 31B: April 2026 / 12B Unified: June 2026
      Variants: 12B Unified, 26B A4B, 31B
      Parameters: 12B Unified: 11.95B / 26B A4B: 25.2B total, 3.8B active / 31B: 30.7B
      Context: 256,000 tokens
      Architecture: 31B: Dense / 26B A4B: MoE, 8 active of 128 experts plus 1 shared / 12B: Unified encoder-free dense, multimodal input projected directly into the decoder
      License: Apache 2.0
      Modalities: 12B: Text + Image + Audio / 26B A4B and 31B: Text + Image
      Runs on: Laptop, consumer GPU/workstation class
      Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face, Docker
      On disk: 12B: ~7GB Q4 / 26B A4B: ~15GB Q4 / 31B: ~18GB Q4

      1 Reply Last reply
      0
      • montezM Offline
        montezM Offline
        montez
        wrote on last edited by
        #13

        NVIDIA Cosmos 3 Edge

        Hugging Face: https://huggingface.co/nvidia/Cosmos3-Edge, https://huggingface.co/collections/nvidia/cosmos3
        GitHub: https://github.com/nvidia/cosmos
        Website: https://research.nvidia.com/labs/cosmos-lab/cosmos3/
        White paper: https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf

        Developer: NVIDIA
        Released: July 2026
        Variants: Cosmos3-Edge, Cosmos3-Edge-Policy-DROID
        Parameters: 4B
        Context: Reasoner: 256,000 tokens / Generator text input: 4,096 tokens
        Architecture: Mixture-of-Transformers, two towers: an autoregressive transformer for text, a diffusion transformer for image/video/action generation
        License: OpenMDW 1.1
        Modalities: Text + Image + Video + Action trajectory in / Text + Image + Video + Action out
        Formats: safetensors via Hugging Face, PyTorch (NVIDIA-proprietary inference only, BF16)
        Runs on: NVIDIA consumer GPU (Linux), Jetson (Linux)

        1 Reply Last reply
        0
        • montezM Offline
          montezM Offline
          montez
          wrote on last edited by montez
          #14

          Agents-A1

          Hugging Face: https://huggingface.co/InternScience/Agents-A1, https://huggingface.co/InternScience/Agents-A1-4B
          GitHub: https://github.com/InternScience/Agents-A1
          X: https://x.com/intern_lm
          Website: https://internscience.github.io/Agents-A1/
          Technical report: https://arxiv.org/abs/2606.30616

          Developer: InternScience, Shanghai Artificial Intelligence Laboratory
          Released: 35B-A3B: June 2026 / 4B: July 2026
          Parameters: 35B total, 3B active / 4B dense
          Context: 262,144 tokens
          Architecture: Qwen3.5 base, hybrid linear + full attention (full attention every 4th layer); 35B-A3B: MoE, 40 layers, 256 experts, 8 routed + 1 shared active / 4B: dense, 32 layers
          License: Apache 2.0
          Modalities: Text + Vision
          Runs on: 35B-A3B: Laptop, 24GB+ unified memory / 4B: Smartphone, Laptop
          Formats: GGUF, MLX (community), safetensors
          On disk: 35B-A3B: 19.7GB Q4_K_M / 4B: 2.5GB Q4_K_M, 4.2GB Q8_0

          1 Reply Last reply
          0
          • montezM Offline
            montezM Offline
            montez
            wrote on last edited by
            #15

            Ornith-1.0

            Hugging Face: https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B, https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B
            GitHub: https://github.com/deepreinforce-ai/Ornith-1
            Website: https://deep-reinforce.com/ornith.html

            Developer: DeepReinforce
            Released: June 2026
            Parameters: 9B dense / 35B total, 8 of 256 experts active
            Context: 262,144 tokens
            Architecture: Qwen3.5 base; 9B: dense, 32 layers / 35B: MoE, 40 layers, 256 experts, 8 active
            License: MIT
            Modalities: Text + Vision
            Runs on: 9B: Smartphone, Laptop / 35B: Laptop, 24GB+ unified memory
            Formats: GGUF, FP8, safetensors
            On disk: 9B: 5.9GB Q4_K_M / 35B: 21.4GB Q4_K_M

            1 Reply Last reply
            0
            • montezM Offline
              montezM Offline
              montez
              wrote last edited by montez
              #16

              DeepGrove Maple-Preview

              Hugging Face: https://huggingface.co/deepgrove/maple-preview
              GitHub: https://github.com/deepgrove-ai/mlx-lm-deepgrove
              Website: https://deepgrove.ai/maple-preview

              Developer: DeepGrove
              Released: August 2026
              Parameters: 20B total, 1B active
              Context: 131,072 tokens
              Architecture: ternary-weight MoE, 24 layers, 256 experts, 8 active; 3:1 SWA-512:GA attention
              License: MIT
              Modalities: Text
              Runs on: Mac mini M4
              Formats: safetensors, MLX 2-bit
              On disk: 5.31GB MLX 2-bit

              1 Reply Last reply
              0
              • montezM Offline
                montezM Offline
                montez
                wrote last edited by montez
                #17

                Nanbeige4.2-3B

                Hugging Face: https://huggingface.co/Nanbeige/Nanbeige4.2-3B
                Website: https://nanbeige.zhipin.com/
                Technical report: https://arxiv.org/abs/2607.22083

                Developer: Nanbeige, BOSS Zhipin
                Released: July 2026
                Parameters: 4B total, 3B non-embedding
                Context: 262,144 tokens
                Architecture: dense Looped Transformer, LoopSplit, mHC with depth attention, concatenated n-gram embeddings
                License: Apache 2.0
                Modalities: Text
                Runs on: Laptop
                Formats: safetensors, GGUF Q4_K_M via conversion, MLX 4-bit
                On disk: 3.30GB MLX 4-bit

                1 Reply Last reply
                0
                • montezM Offline
                  montezM Offline
                  montez
                  wrote last edited by montez
                  #18

                  Fermion Research Neutrino

                  Hugging Face: https://huggingface.co/FermionResearch/Neutrino-8B, https://huggingface.co/FermionResearch/Neutrino-0.6B, https://huggingface.co/FermionResearch/Neutrino-0.6B-Chat
                  GitHub: https://github.com/fermionresearch/llama.cpp
                  Website: https://www.fermionresearch.com/models/neutrino-8b/
                  Announcement: https://www.fermionresearch.com/research/neutrino-8b/

                  Developer: Fermion Research
                  Released: July 2026
                  Variants: 8B general purpose, 0.6B speculative-decoding draft, 0.6B-Chat conversational
                  Parameters: 8.19B / 596M
                  Context: 40,960 tokens
                  Architecture: ternary QAT with staged post-training; 8B: Qwen3-8B topology, 36-layer decoder-only transformer, SwiGLU, GQA 4:1, RoPE, RMSNorm / 0.6B: 28 layers, hidden 1,024, SwiGLU 3,072, GQA 16Q/8KV, per-head Q/K RMSNorm, tied int8 embeddings
                  License: Apache 2.0
                  Modalities: Text
                  Runs on: 8B: Laptop, 16GB unified memory / 0.6B: Laptop, CPU only
                  Formats: TRTC v4, custom-FV5 GGUF, MLX
                  On disk: 8B: 3.88GB TRTC v4 / 4.09GB custom-FV5 GGUF / 0.6B: 328MB TRTC v4, 238MB tv4z, 343MB GGUF

                  1 Reply Last reply
                  0
                  • montezM Offline
                    montezM Offline
                    montez
                    wrote last edited by montez
                    #19

                    Kakao Kanana-2

                    Hugging Face: https://huggingface.co/kakaocorp/kanana-2-3b-instruct, https://huggingface.co/kakaocorp/kanana-2-1.3b-instruct
                    Website: https://tech.kakao.com/posts/826

                    Developer: Kakao, Kanana LLM
                    Released: July 2026
                    Variants: 3B, 1.3B
                    Parameters: 3B / 1.3B
                    Context: 32,768 tokens
                    Architecture: 3B: dense, pretrained from scratch, SFT + RL / 1.3B: cascade-pruned and distilled from 3B, sliding-window attention
                    License: Kanana Open License
                    Modalities: Text
                    Runs on: Smartphone, Laptop
                    Formats: safetensors, MLX 4-bit, 6-bit, 8-bit
                    On disk: 3B: 1.99GB MLX 4-bit

                    1 Reply Last reply
                    0
                    • montezM Offline
                      montezM Offline
                      montez
                      wrote last edited by
                      #20

                      Microsoft Fara1.5

                      Hugging Face: https://huggingface.co/microsoft/Fara1.5-4B, https://huggingface.co/microsoft/Fara1.5-9B, https://huggingface.co/microsoft/Fara1.5-27B
                      GitHub: https://github.com/microsoft/fara
                      Website: https://labs.ai.azure.com/innovations/fara1-5/

                      Developer: Microsoft Research AI Frontiers
                      Released: May 2026
                      Variants: 4B, 9B, 27B
                      Parameters: 4B / 9B / 27B
                      Context: 262,144 tokens
                      Architecture: multimodal decoder-only LM, image + text to text
                      License: MIT
                      Modalities: Text + Image
                      Formats: safetensors
                      On disk: 4B: 9.08GB / 9B: 18.82GB / 27B: 54.71GB safetensors

                      1 Reply Last reply
                      0
                      • montezM Offline
                        montezM Offline
                        montez
                        wrote last edited by
                        #21

                        AMD Instella-MoE-16B-A3B-Think

                        Hugging Face: https://huggingface.co/amd/Instella-MoE-16B-A3B-Think
                        GitHub: https://github.com/AMD-AGI/Instella-MoE
                        Website: https://rocm.blogs.amd.com/artificial-intelligence/instella-moe/README.html

                        Developer: AMD
                        Released: July 2026
                        Parameters: 16B total, 2.8B active
                        Context: 32,768 tokens
                        Architecture: decoder-only MoE, Gated Multi-head Latent Attention, FarSkip-Collective connectivity
                        License: Research RAIL
                        Modalities: Text
                        Formats: safetensors
                        On disk: 31.73GB safetensors

                        1 Reply Last reply
                        0
                        • montezM Offline
                          montezM Offline
                          montez
                          wrote last edited by montez
                          #22

                          Cisco Antares

                          Hugging Face: https://huggingface.co/fdtn-ai/antares-1b, https://huggingface.co/fdtn-ai/antares-350m
                          Website: https://cisco-foundation-ai.github.io/antares/
                          Announcement: https://cisco-foundation-ai.github.io/blogs/antares-beyond-vlocbench/

                          Developer: Cisco Foundation AI
                          Released: July 2026
                          Variants: 1b, 350m
                          Parameters: 1B / 350M
                          Architecture: fine-tuned from IBM Granite 4.0, GraniteMoEHybrid
                          License: Apache 2.0
                          Modalities: Text
                          Runs on: Smartphone, Laptop, Edge device
                          Formats: safetensors
                          On disk: 1b: 3.67GB safetensors / 350m: 0.70GB safetensors

                          1 Reply Last reply
                          0
                          • montezM Offline
                            montezM Offline
                            montez
                            wrote last edited by
                            #23

                            AI9Stars G9v3-3B

                            Hugging Face: https://huggingface.co/ai9stars/G9v3-3B
                            GitHub: https://github.com/AI9Stars

                            Developer: AI9Stars
                            Parameters: ~3B
                            Context: 131,072 tokens
                            Architecture: dense causal LM, LlamaForCausalLM
                            License: Apache 2.0
                            Modalities: Text
                            Formats: safetensors
                            On disk: 5.99GB safetensors

                            1 Reply Last reply
                            0
                            • montezM Offline
                              montezM Offline
                              montez
                              wrote last edited by
                              #24

                              Tencent Hy-Embodied-RxBrain-1.0

                              Hugging Face: https://huggingface.co/tencent/Hy-Embodied-RxBrain-1.0
                              GitHub: https://github.com/Tencent-Hunyuan/Hy-Embodied-RxBrain-1.0
                              Website: https://tairos.tencent.com/openSourceModels/hy-embodied-rxbrain-1.0
                              Technical report: https://arxiv.org/abs/2607.14187

                              Developer: Tencent Robotics X, Futian Laboratory, Tencent Hy Team
                              Released: July 2026
                              Parameters: ~6.2B
                              Architecture: Unified Mixture-of-Transformers, modality-specific text, vision, and generation pathways
                              License: Apache 2.0
                              Modalities: Text + Image + Video
                              Runs on: NVIDIA GPU, CUDA 12.x, Linux recommended
                              Formats: safetensors
                              On disk: 12.42GB safetensors

                              1 Reply Last reply
                              0
                              • montezM Offline
                                montezM Offline
                                montez
                                wrote last edited by
                                #25

                                Poolside Laguna XS 2.1

                                Hugging Face: https://huggingface.co/poolside/Laguna-XS-2.1, https://huggingface.co/poolside/Laguna-XS-2.1-GGUF
                                Website: https://poolside.ai/blog/introducing-laguna-xs-2-1

                                Developer: Poolside
                                Released: July 2026
                                Variants: BF16, FP8, NVFP4, INT4; GGUF BF16, Q4_K_M
                                Parameters: 33B total, 3B active
                                Context: 262,144 tokens
                                Architecture: MoE, 40 layers: 10 global-attention + 30 sliding-window-attention; 256 experts + 1 shared expert
                                License: OpenMDW-1.1
                                Modalities: Text
                                Runs on: Mac with 36GB RAM
                                Formats: safetensors, GGUF
                                On disk: 20.27GB Q4_K_M GGUF / 66.89GB BF16 safetensors

                                1 Reply Last reply
                                0
                                • montezM Offline
                                  montezM Offline
                                  montez
                                  wrote last edited by montez
                                  #26

                                  Meta Muse Glimmer-30B

                                  Hugging Face: https://huggingface.co/meta-models/Muse-Glimmer-30B
                                  Announcement: https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model
                                  Technical report: https://research.meta.ai/static/muse-glimmer-methodology

                                  Developer: Meta Superintelligence Lab
                                  Released: August 2026
                                  Parameters: 29.6B total, including perception encoder
                                  Context: 131,072+ tokens
                                  Architecture: Dense causal transformer with ViT-G/14 perception encoder; 52 layers, GQA, SwiGLU, RoPE
                                  License: Apache 2.0
                                  Modalities: Text + Image
                                  Runs on: MacBook M4 Max/M5 Max, RTX 5090; 24GB+ memory with 4-bit weights
                                  Formats: BF16 safetensors, 4-bit quantized weights
                                  On disk: 17GB K-Quant

                                  1 Reply Last reply
                                  0
                                  • montezM Offline
                                    montezM Offline
                                    montez
                                    wrote last edited by
                                    #27

                                    webAI TwIL-LM

                                    Hugging Face: https://huggingface.co/webAI-Official/TwIL-LM, https://huggingface.co/webAI-Official/TwIL-LM3
                                    Website: https://www.webai.com/blog/webai-releases-twil-lm-a-family-of-formal-logic-models-that-outreason-a-120b-model-and-run-on-an-iphone

                                    Developer: webAI Intelligence Lab
                                    Released: August 2026
                                    Variants: TwIL-LM 1.7B, TwIL-LM3 3B
                                    Parameters: 1.7B: 1.78B total, 1.71B backbone + 72M LoRA / 3B: 3B
                                    Context: 1.7B: 8,192 tokens / 3B: 65,536 tokens
                                    Architecture: 1.7B: SmolLM2-1.7B-Instruct base, dense Llama, 24 layers, 32 heads, LoRA rank 64 SFT / 3B: SmolLM3-3B base, dense, 36 layers, GQA 16Q/4KV, NoPE every 4th layer; LoRA SFT, checkpoint fusion, WiSE-FT interpolation, GRPO reinforcement learning
                                    License: webAI Non-Commercial License ver. 1.0
                                    Modalities: Text
                                    Runs on: 1.7B: Smartphone, Laptop / 3B: Laptop, 4GB VRAM or CPU
                                    Formats: 1.7B: merged GGUF Q4_K_M, Q5_K_M, Q8_0, f16 / 3B: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16
                                    On disk: 1.7B: 1.06GB Q4_K_M / 3B: 1.92GB Q4_K_M

                                    1 Reply Last reply
                                    0
                                    • montezM Offline
                                      montezM Offline
                                      montez
                                      wrote last edited by montez
                                      #28

                                      Ling 3.0

                                      Hugging Face: https://huggingface.co/inclusionAI/Ling-3.0-flash, https://huggingface.co/inclusionAI/Ling-3.0-tiny, https://huggingface.co/inclusionAI/Ling-3.0-tiny-int4, https://huggingface.co/inclusionAI/Ling-3.0-tiny-fp8
                                      GitHub: https://github.com/inclusionAI/Ling
                                      X: https://x.com/AntLingAGI/status/2080351022028095681
                                      Website: https://www.ant-ling.com/en
                                      Docs: https://github.com/inclusionAI/ling-cookbook

                                      Developer: Ant Group, InclusionAI
                                      Released: July 2026
                                      Variants: Ling-3.0-flash, Ling-3.0-tiny
                                      Parameters: flash: 124B total, 5.1B active / tiny: 7.9B total, 1.3B active
                                      Context: flash: 262,144 tokens / tiny: 131,072 tokens, 262,144 with YaRN
                                      Architecture: BailingMoE hybrid; flash: linear KDA + MLA attention with sparse MoE / tiny: 3:1 KDA to MLA blocks, 128 routed experts, 8 routed + 1 shared active
                                      License: MIT
                                      Modalities: Text
                                      Runs on: flash: NVIDIA DGX Spark / tiny: Laptop, 48GB unified memory
                                      Formats: safetensors BF16, safetensors FP8, safetensors INT4, GGUF Q4_K_M
                                      On disk: flash: 60.5GB Q4_K_M GGUF, 254.98GB BF16 safetensors / tiny: 5.81GB INT4, 8.41GB FP8, 15.79GB BF16 safetensors

                                      1 Reply Last reply
                                      0
                                      • montezM Offline
                                        montezM Offline
                                        montez
                                        wrote last edited by montez
                                        #29

                                        Qwen3.8

                                        Hugging Face: https://huggingface.co/Qwen/Qwen3.8-27B, https://huggingface.co/Qwen/Qwen3.8-27B-FP8
                                        GitHub: https://github.com/QwenLM/Qwen3
                                        X: https://x.com/Alibaba_Qwen/status/2088280182356611304
                                        Website: https://qwen.ai/blog?id=qwen3.8

                                        Developer: Alibaba Cloud, Qwen team
                                        Released: August 2026
                                        Parameters: 27B
                                        Context: 262,144 tokens
                                        Architecture: Qwen3.5 foundation, dense native vision-language model; hybrid linear + full attention
                                        License: Apache 2.0
                                        Modalities: Text + Image + Video
                                        Runs on: Laptop, 24GB+ unified memory, estimated / Desktop GPU
                                        Formats: safetensors BF16, safetensors FP8, GGUF and MLX community quantizations
                                        On disk: 16.05GB MLX 4-bit / 30.87GB FP8 safetensors / 55.56GB BF16 safetensors

                                        1 Reply Last reply
                                        0
                                        • montezM Offline
                                          montezM Offline
                                          montez
                                          wrote last edited by montez
                                          #30

                                          NVIDIA Nemotron 3.5 Lightning

                                          Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
                                          GitHub: https://github.com/NVIDIA-NeMo/Nemotron
                                          Website: https://build.nvidia.com/nvidia/nemotron-3.5-lightning-30b-a3b

                                          Developer: NVIDIA
                                          Released: August 2026
                                          Variants: Instruct, Base, DSpark and DFlash speculative drafters
                                          Parameters: 30B total, 3B active
                                          Context: 1,048,576 tokens
                                          Architecture: hybrid LatentMoE interleaving Mamba-2, MoE and attention; 52 layers, 128 routed experts, 6 active, 32Q/2KV heads, Multi-Token Prediction
                                          License: OpenMDW License Agreement, version 1.1
                                          Modalities: Text
                                          Runs on: Desktop GPU, Edge device, DGX Spark; NVIDIA only
                                          Formats: safetensors BF16, safetensors NVFP4, GGUF community conversion
                                          On disk: 17.82GB NVFP4 safetensors / 31.58GB BF16 safetensors

                                          1 Reply Last reply
                                          0
                                          ↳

                                          OBJECTS Forum

                                          Join the conversation

                                          Create an account to return to your place in the thread, follow new replies, bookmark useful posts, and upvote contributions you value.

                                          Have something to add? Your perspective can make this thread better.

                                          Register Login
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes

                                          • Login
                                          • Don't have an account? Register
                                          • Login or register to search.
                                          • First post
                                            Last post
                                          • 0
                                            • Categories
                                            • Recent
                                            • Tags
                                            • Popular
                                            • Users