Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
Menu
  1. Home
  2. AI & Software
  3. Local Foundation Models

Local Foundation Models

Scheduled Pinned Locked Moved AI & Software
llm
40 Posts 1 Posters 1.1k Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • montezM Offline
    montezM Offline
    montez
    wrote on last edited by
    #10

    PrismML Bonsai Image 4B

    Hugging Face: https://huggingface.co/prism-ml/bonsai-image-binary-4B-mlx-1bit, https://huggingface.co/prism-ml/bonsai-image-ternary-4B-mlx-2bit
    GitHub: https://github.com/PrismML-Eng/Bonsai-Image-Demo
    Website: https://prismml.com
    Announcement: https://prismml.com/news/bonsai-image-4b

    Developer: PrismML
    Released: May 2026
    Parameters: 4B, transformer trunk
    Architecture: MMDiT diffusion transformer, base architecture FLUX.2 Klein 4B, 25 blocks: 5 double-stream + 20 single-stream
    Variants: Binary, 1-bit / Ternary, 2-bit
    License: Apache 2.0
    Modalities: Text-to-Image
    Runs on: Smartphone, Laptop
    Formats: MLX 1-bit, MLX 2-bit, Gemlite 1-bit/2-bit for CUDA, safetensors
    On disk: Binary: 0.93GB transformer, 3.42GB total deployment payload / Ternary: 1.21GB transformer

    1 Reply Last reply
    0
    • montezM Offline
      montezM Offline
      montez
      wrote on last edited by
      #11

      Google Gemma 4 Effective

      Hugging Face: https://huggingface.co/collections/google/gemma-4
      GitHub: https://github.com/google-gemma
      Website: https://ai.google.dev/gemma/docs/core
      Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/

      Developer: Google DeepMind
      Released: April 2026
      Variants: E2B, E4B
      Parameters: E2B: 2.3B effective, 5.1B with embeddings / E4B: 4.5B effective, 8B with embeddings
      Context: 128,000 tokens
      Architecture: Dense, hybrid local sliding window + global attention, Per-Layer Embeddings for on-device efficiency
      License: Apache 2.0
      Modalities: Text + Image + Audio
      Runs on: Smartphone, Laptop, Edge device
      Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face
      On disk: E2B: 1.4GB Q4, estimated / E4B: 2.7GB Q4, estimated

      1 Reply Last reply
      0
      • montezM Offline
        montezM Offline
        montez
        wrote on last edited by montez
        #12

        Google Gemma 4

        Hugging Face: https://huggingface.co/collections/google/gemma-4
        GitHub: https://github.com/google-gemma
        Website: https://ai.google.dev/gemma/docs/core
        Announcement: https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/, https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12B/

        Developer: Google DeepMind
        Released: 26B A4B and 31B: April 2026 / 12B Unified: June 2026
        Variants: 12B Unified, 26B A4B, 31B
        Parameters: 12B Unified: 11.95B / 26B A4B: 25.2B total, 3.8B active / 31B: 30.7B
        Context: 256,000 tokens
        Architecture: 31B: Dense / 26B A4B: MoE, 8 active of 128 experts plus 1 shared / 12B: Unified encoder-free dense, multimodal input projected directly into the decoder
        License: Apache 2.0
        Modalities: 12B: Text + Image + Audio / 26B A4B and 31B: Text + Image
        Runs on: Laptop, consumer GPU/workstation class
        Formats: GGUF via Ollama/LM Studio, safetensors via Hugging Face, Docker
        On disk: 12B: ~7GB Q4 / 26B A4B: ~15GB Q4 / 31B: ~18GB Q4

        1 Reply Last reply
        0
        • montezM Offline
          montezM Offline
          montez
          wrote on last edited by
          #13

          NVIDIA Cosmos 3 Edge

          Hugging Face: https://huggingface.co/nvidia/Cosmos3-Edge, https://huggingface.co/collections/nvidia/cosmos3
          GitHub: https://github.com/nvidia/cosmos
          Website: https://research.nvidia.com/labs/cosmos-lab/cosmos3/
          White paper: https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf

          Developer: NVIDIA
          Released: July 2026
          Variants: Cosmos3-Edge, Cosmos3-Edge-Policy-DROID
          Parameters: 4B
          Context: Reasoner: 256,000 tokens / Generator text input: 4,096 tokens
          Architecture: Mixture-of-Transformers, two towers: an autoregressive transformer for text, a diffusion transformer for image/video/action generation
          License: OpenMDW 1.1
          Modalities: Text + Image + Video + Action trajectory in / Text + Image + Video + Action out
          Formats: safetensors via Hugging Face, PyTorch (NVIDIA-proprietary inference only, BF16)
          Runs on: NVIDIA consumer GPU (Linux), Jetson (Linux)

          1 Reply Last reply
          0
          • montezM Offline
            montezM Offline
            montez
            wrote on last edited by montez
            #14

            Agents-A1

            Hugging Face: https://huggingface.co/InternScience/Agents-A1, https://huggingface.co/InternScience/Agents-A1-4B
            GitHub: https://github.com/InternScience/Agents-A1
            X: https://x.com/intern_lm
            Website: https://internscience.github.io/Agents-A1/
            Technical report: https://arxiv.org/abs/2606.30616

            Developer: InternScience, Shanghai Artificial Intelligence Laboratory
            Released: 35B-A3B: June 2026 / 4B: July 2026
            Parameters: 35B total, 3B active / 4B dense
            Context: 262,144 tokens
            Architecture: Qwen3.5 base, hybrid linear + full attention (full attention every 4th layer); 35B-A3B: MoE, 40 layers, 256 experts, 8 routed + 1 shared active / 4B: dense, 32 layers
            License: Apache 2.0
            Modalities: Text + Vision
            Runs on: 35B-A3B: Laptop, 24GB+ unified memory / 4B: Smartphone, Laptop
            Formats: GGUF, MLX (community), safetensors
            On disk: 35B-A3B: 19.7GB Q4_K_M / 4B: 2.5GB Q4_K_M, 4.2GB Q8_0

            1 Reply Last reply
            0
            • montezM Offline
              montezM Offline
              montez
              wrote on last edited by
              #15

              Ornith-1.0

              Hugging Face: https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B, https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B
              GitHub: https://github.com/deepreinforce-ai/Ornith-1
              Website: https://deep-reinforce.com/ornith.html

              Developer: DeepReinforce
              Released: June 2026
              Parameters: 9B dense / 35B total, 8 of 256 experts active
              Context: 262,144 tokens
              Architecture: Qwen3.5 base; 9B: dense, 32 layers / 35B: MoE, 40 layers, 256 experts, 8 active
              License: MIT
              Modalities: Text + Vision
              Runs on: 9B: Smartphone, Laptop / 35B: Laptop, 24GB+ unified memory
              Formats: GGUF, FP8, safetensors
              On disk: 9B: 5.9GB Q4_K_M / 35B: 21.4GB Q4_K_M

              1 Reply Last reply
              0
              • montezM Offline
                montezM Offline
                montez
                wrote last edited by montez
                #16

                DeepGrove Maple-Preview

                Hugging Face: https://huggingface.co/deepgrove/maple-preview
                GitHub: https://github.com/deepgrove-ai/mlx-lm-deepgrove
                Website: https://deepgrove.ai/maple-preview

                Developer: DeepGrove
                Released: August 2026
                Parameters: 20B total, 1B active
                Context: 131,072 tokens
                Architecture: ternary-weight MoE, 24 layers, 256 experts, 8 active; 3:1 SWA-512:GA attention
                License: MIT
                Modalities: Text
                Runs on: Mac mini M4
                Formats: safetensors, MLX 2-bit
                On disk: 5.31GB MLX 2-bit

                1 Reply Last reply
                0
                • montezM Offline
                  montezM Offline
                  montez
                  wrote last edited by montez
                  #17

                  Nanbeige4.2-3B

                  Hugging Face: https://huggingface.co/Nanbeige/Nanbeige4.2-3B
                  Website: https://nanbeige.zhipin.com/
                  Technical report: https://arxiv.org/abs/2607.22083

                  Developer: Nanbeige, BOSS Zhipin
                  Released: July 2026
                  Parameters: 4B total, 3B non-embedding
                  Context: 262,144 tokens
                  Architecture: dense Looped Transformer, LoopSplit, mHC with depth attention, concatenated n-gram embeddings
                  License: Apache 2.0
                  Modalities: Text
                  Runs on: Laptop
                  Formats: safetensors, GGUF Q4_K_M via conversion, MLX 4-bit
                  On disk: 3.30GB MLX 4-bit

                  1 Reply Last reply
                  0
                  • montezM Offline
                    montezM Offline
                    montez
                    wrote last edited by montez
                    #18

                    Fermion Research Neutrino

                    Hugging Face: https://huggingface.co/FermionResearch/Neutrino-8B, https://huggingface.co/FermionResearch/Neutrino-0.6B, https://huggingface.co/FermionResearch/Neutrino-0.6B-Chat
                    GitHub: https://github.com/fermionresearch/llama.cpp
                    Website: https://www.fermionresearch.com/models/neutrino-8b/
                    Announcement: https://www.fermionresearch.com/research/neutrino-8b/

                    Developer: Fermion Research
                    Released: July 2026
                    Variants: 8B general purpose, 0.6B speculative-decoding draft, 0.6B-Chat conversational
                    Parameters: 8.19B / 596M
                    Context: 40,960 tokens
                    Architecture: ternary QAT with staged post-training; 8B: Qwen3-8B topology, 36-layer decoder-only transformer, SwiGLU, GQA 4:1, RoPE, RMSNorm / 0.6B: 28 layers, hidden 1,024, SwiGLU 3,072, GQA 16Q/8KV, per-head Q/K RMSNorm, tied int8 embeddings
                    License: Apache 2.0
                    Modalities: Text
                    Runs on: 8B: Laptop, 16GB unified memory / 0.6B: Laptop, CPU only
                    Formats: TRTC v4, custom-FV5 GGUF, MLX
                    On disk: 8B: 3.88GB TRTC v4 / 4.09GB custom-FV5 GGUF / 0.6B: 328MB TRTC v4, 238MB tv4z, 343MB GGUF

                    1 Reply Last reply
                    0
                    • montezM Offline
                      montezM Offline
                      montez
                      wrote last edited by montez
                      #19

                      Kakao Kanana-2

                      Hugging Face: https://huggingface.co/kakaocorp/kanana-2-3b-instruct, https://huggingface.co/kakaocorp/kanana-2-1.3b-instruct
                      Website: https://tech.kakao.com/posts/826

                      Developer: Kakao, Kanana LLM
                      Released: July 2026
                      Variants: 3B, 1.3B
                      Parameters: 3B / 1.3B
                      Context: 32,768 tokens
                      Architecture: 3B: dense, pretrained from scratch, SFT + RL / 1.3B: cascade-pruned and distilled from 3B, sliding-window attention
                      License: Kanana Open License
                      Modalities: Text
                      Runs on: Smartphone, Laptop
                      Formats: safetensors, MLX 4-bit, 6-bit, 8-bit
                      On disk: 3B: 1.99GB MLX 4-bit

                      1 Reply Last reply
                      0
                      • montezM Offline
                        montezM Offline
                        montez
                        wrote last edited by
                        #20

                        Microsoft Fara1.5

                        Hugging Face: https://huggingface.co/microsoft/Fara1.5-4B, https://huggingface.co/microsoft/Fara1.5-9B, https://huggingface.co/microsoft/Fara1.5-27B
                        GitHub: https://github.com/microsoft/fara
                        Website: https://labs.ai.azure.com/innovations/fara1-5/

                        Developer: Microsoft Research AI Frontiers
                        Released: May 2026
                        Variants: 4B, 9B, 27B
                        Parameters: 4B / 9B / 27B
                        Context: 262,144 tokens
                        Architecture: multimodal decoder-only LM, image + text to text
                        License: MIT
                        Modalities: Text + Image
                        Formats: safetensors
                        On disk: 4B: 9.08GB / 9B: 18.82GB / 27B: 54.71GB safetensors

                        1 Reply Last reply
                        0
                        • montezM Offline
                          montezM Offline
                          montez
                          wrote last edited by
                          #21

                          AMD Instella-MoE-16B-A3B-Think

                          Hugging Face: https://huggingface.co/amd/Instella-MoE-16B-A3B-Think
                          GitHub: https://github.com/AMD-AGI/Instella-MoE
                          Website: https://rocm.blogs.amd.com/artificial-intelligence/instella-moe/README.html

                          Developer: AMD
                          Released: July 2026
                          Parameters: 16B total, 2.8B active
                          Context: 32,768 tokens
                          Architecture: decoder-only MoE, Gated Multi-head Latent Attention, FarSkip-Collective connectivity
                          License: Research RAIL
                          Modalities: Text
                          Formats: safetensors
                          On disk: 31.73GB safetensors

                          1 Reply Last reply
                          0
                          • montezM Offline
                            montezM Offline
                            montez
                            wrote last edited by montez
                            #22

                            Cisco Antares

                            Hugging Face: https://huggingface.co/fdtn-ai/antares-1b, https://huggingface.co/fdtn-ai/antares-350m
                            Website: https://cisco-foundation-ai.github.io/antares/
                            Announcement: https://cisco-foundation-ai.github.io/blogs/antares-beyond-vlocbench/

                            Developer: Cisco Foundation AI
                            Released: July 2026
                            Variants: 1b, 350m
                            Parameters: 1B / 350M
                            Architecture: fine-tuned from IBM Granite 4.0, GraniteMoEHybrid
                            License: Apache 2.0
                            Modalities: Text
                            Runs on: Smartphone, Laptop, Edge device
                            Formats: safetensors
                            On disk: 1b: 3.67GB safetensors / 350m: 0.70GB safetensors

                            1 Reply Last reply
                            0
                            • montezM Offline
                              montezM Offline
                              montez
                              wrote last edited by
                              #23

                              AI9Stars G9v3-3B

                              Hugging Face: https://huggingface.co/ai9stars/G9v3-3B
                              GitHub: https://github.com/AI9Stars

                              Developer: AI9Stars
                              Parameters: ~3B
                              Context: 131,072 tokens
                              Architecture: dense causal LM, LlamaForCausalLM
                              License: Apache 2.0
                              Modalities: Text
                              Formats: safetensors
                              On disk: 5.99GB safetensors

                              1 Reply Last reply
                              0
                              • montezM Offline
                                montezM Offline
                                montez
                                wrote last edited by
                                #24

                                Tencent Hy-Embodied-RxBrain-1.0

                                Hugging Face: https://huggingface.co/tencent/Hy-Embodied-RxBrain-1.0
                                GitHub: https://github.com/Tencent-Hunyuan/Hy-Embodied-RxBrain-1.0
                                Website: https://tairos.tencent.com/openSourceModels/hy-embodied-rxbrain-1.0
                                Technical report: https://arxiv.org/abs/2607.14187

                                Developer: Tencent Robotics X, Futian Laboratory, Tencent Hy Team
                                Released: July 2026
                                Parameters: ~6.2B
                                Architecture: Unified Mixture-of-Transformers, modality-specific text, vision, and generation pathways
                                License: Apache 2.0
                                Modalities: Text + Image + Video
                                Runs on: NVIDIA GPU, CUDA 12.x, Linux recommended
                                Formats: safetensors
                                On disk: 12.42GB safetensors

                                1 Reply Last reply
                                0
                                • montezM Offline
                                  montezM Offline
                                  montez
                                  wrote last edited by
                                  #25

                                  Poolside Laguna XS 2.1

                                  Hugging Face: https://huggingface.co/poolside/Laguna-XS-2.1, https://huggingface.co/poolside/Laguna-XS-2.1-GGUF
                                  Website: https://poolside.ai/blog/introducing-laguna-xs-2-1

                                  Developer: Poolside
                                  Released: July 2026
                                  Variants: BF16, FP8, NVFP4, INT4; GGUF BF16, Q4_K_M
                                  Parameters: 33B total, 3B active
                                  Context: 262,144 tokens
                                  Architecture: MoE, 40 layers: 10 global-attention + 30 sliding-window-attention; 256 experts + 1 shared expert
                                  License: OpenMDW-1.1
                                  Modalities: Text
                                  Runs on: Mac with 36GB RAM
                                  Formats: safetensors, GGUF
                                  On disk: 20.27GB Q4_K_M GGUF / 66.89GB BF16 safetensors

                                  1 Reply Last reply
                                  0
                                  • montezM Offline
                                    montezM Offline
                                    montez
                                    wrote last edited by montez
                                    #26

                                    Meta Muse Glimmer-30B

                                    Hugging Face: https://huggingface.co/meta-models/Muse-Glimmer-30B
                                    Announcement: https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model
                                    Technical report: https://research.meta.ai/static/muse-glimmer-methodology

                                    Developer: Meta Superintelligence Lab
                                    Released: August 2026
                                    Parameters: 29.6B total, including perception encoder
                                    Context: 131,072+ tokens
                                    Architecture: Dense causal transformer with ViT-G/14 perception encoder; 52 layers, GQA, SwiGLU, RoPE
                                    License: Apache 2.0
                                    Modalities: Text + Image
                                    Runs on: MacBook M4 Max/M5 Max, RTX 5090; 24GB+ memory with 4-bit weights
                                    Formats: BF16 safetensors, 4-bit quantized weights
                                    On disk: 17GB K-Quant

                                    1 Reply Last reply
                                    0
                                    • montezM Offline
                                      montezM Offline
                                      montez
                                      wrote last edited by
                                      #27

                                      webAI TwIL-LM

                                      Hugging Face: https://huggingface.co/webAI-Official/TwIL-LM, https://huggingface.co/webAI-Official/TwIL-LM3
                                      Website: https://www.webai.com/blog/webai-releases-twil-lm-a-family-of-formal-logic-models-that-outreason-a-120b-model-and-run-on-an-iphone

                                      Developer: webAI Intelligence Lab
                                      Released: August 2026
                                      Variants: TwIL-LM 1.7B, TwIL-LM3 3B
                                      Parameters: 1.7B: 1.78B total, 1.71B backbone + 72M LoRA / 3B: 3B
                                      Context: 1.7B: 8,192 tokens / 3B: 65,536 tokens
                                      Architecture: 1.7B: SmolLM2-1.7B-Instruct base, dense Llama, 24 layers, 32 heads, LoRA rank 64 SFT / 3B: SmolLM3-3B base, dense, 36 layers, GQA 16Q/4KV, NoPE every 4th layer; LoRA SFT, checkpoint fusion, WiSE-FT interpolation, GRPO reinforcement learning
                                      License: webAI Non-Commercial License ver. 1.0
                                      Modalities: Text
                                      Runs on: 1.7B: Smartphone, Laptop / 3B: Laptop, 4GB VRAM or CPU
                                      Formats: 1.7B: merged GGUF Q4_K_M, Q5_K_M, Q8_0, f16 / 3B: safetensors, GGUF Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16
                                      On disk: 1.7B: 1.06GB Q4_K_M / 3B: 1.92GB Q4_K_M

                                      1 Reply Last reply
                                      0
                                      • montezM Offline
                                        montezM Offline
                                        montez
                                        wrote last edited by montez
                                        #28

                                        Ling 3.0

                                        Hugging Face: https://huggingface.co/inclusionAI/Ling-3.0-flash, https://huggingface.co/inclusionAI/Ling-3.0-tiny, https://huggingface.co/inclusionAI/Ling-3.0-tiny-int4, https://huggingface.co/inclusionAI/Ling-3.0-tiny-fp8
                                        GitHub: https://github.com/inclusionAI/Ling
                                        X: https://x.com/AntLingAGI/status/2080351022028095681
                                        Website: https://www.ant-ling.com/en
                                        Docs: https://github.com/inclusionAI/ling-cookbook

                                        Developer: Ant Group, InclusionAI
                                        Released: July 2026
                                        Variants: Ling-3.0-flash, Ling-3.0-tiny
                                        Parameters: flash: 124B total, 5.1B active / tiny: 7.9B total, 1.3B active
                                        Context: flash: 262,144 tokens / tiny: 131,072 tokens, 262,144 with YaRN
                                        Architecture: BailingMoE hybrid; flash: linear KDA + MLA attention with sparse MoE / tiny: 3:1 KDA to MLA blocks, 128 routed experts, 8 routed + 1 shared active
                                        License: MIT
                                        Modalities: Text
                                        Runs on: flash: NVIDIA DGX Spark / tiny: Laptop, 48GB unified memory
                                        Formats: safetensors BF16, safetensors FP8, safetensors INT4, GGUF Q4_K_M
                                        On disk: flash: 60.5GB Q4_K_M GGUF, 254.98GB BF16 safetensors / tiny: 5.81GB INT4, 8.41GB FP8, 15.79GB BF16 safetensors

                                        1 Reply Last reply
                                        0
                                        • montezM Offline
                                          montezM Offline
                                          montez
                                          wrote last edited by montez
                                          #29

                                          Qwen3.8

                                          Hugging Face: https://huggingface.co/Qwen/Qwen3.8-27B, https://huggingface.co/Qwen/Qwen3.8-27B-FP8
                                          GitHub: https://github.com/QwenLM/Qwen3
                                          X: https://x.com/Alibaba_Qwen/status/2088280182356611304
                                          Website: https://qwen.ai/blog?id=qwen3.8

                                          Developer: Alibaba Cloud, Qwen team
                                          Released: August 2026
                                          Parameters: 27B
                                          Context: 262,144 tokens
                                          Architecture: Qwen3.5 foundation, dense native vision-language model; hybrid linear + full attention
                                          License: Apache 2.0
                                          Modalities: Text + Image + Video
                                          Runs on: Laptop, 24GB+ unified memory, estimated / Desktop GPU
                                          Formats: safetensors BF16, safetensors FP8, GGUF and MLX community quantizations
                                          On disk: 16.05GB MLX 4-bit / 30.87GB FP8 safetensors / 55.56GB BF16 safetensors

                                          1 Reply Last reply
                                          0
                                          ↳

                                          OBJECTS Forum

                                          Join the conversation

                                          Create an account to return to your place in the thread, follow new replies, bookmark useful posts, and upvote contributions you value.

                                          Have something to add? Your perspective can make this thread better.

                                          Register Login
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes

                                          • Login
                                          • Don't have an account? Register
                                          • Login or register to search.
                                          • First post
                                            Last post
                                          • 0
                                            • Categories
                                            • Recent
                                            • Tags
                                            • Popular
                                            • Users