Skip to content

Local LLM Tiering ($0.00 First)

gosec implements a 3-tier cognitive LLM hierarchy engineered to maximize cost efficiency and operational autonomy without sacrificing intelligence.

🧠 The 3-Tier Cognitive Architecture

flowchart TD
    Prompt["Developer / Agent Task Prompt"] --> Check{"Local Ollama Available?"}
    Check -->|Yes| Tier0["Tier 0/1: Local Ollama ($0.00)\nqwen3-coder:30b / deepseek-r1:14b"]
    Check -->|No / Busy| Tier2["Tier 2: Fast Cloud Default\nGemini 3.8 Flash"]
    Tier0 --> Complex{"Deep System Architecture\nor Complex Threat Model?"}
    Tier2 --> Complex
    Complex -->|Yes| Tier3["Tier 3: Deep Cognitive Tier (Restricted)\nGemini 3.8 Pro"]
    Complex -->|No| Output["Completed Code & Artifacts"]
    Tier3 --> Output
Hold "Alt" / "Option" to enable pan & zoom

1. Tier 0/1: Local Ollama ($0.00 Tier)

  • Target Host: http://127.0.0.1:11434
  • Recommended Models:
  • qwen3-coder:30b: Primary Go code generation, refactoring, and test synthesis.
  • deepseek-r1:14b: Deep reasoning, logic checks, AST evaluations, and code reviews.
  • llama3.3:70b: High-context comprehension and broad system inquiries.
  • Cost: $0.00 (runs entirely on local GPU or Apple Silicon unified memory).
  • Graceful Fallback: If the local Ollama daemon is offline or saturated, gosec routes the prompt immediately to Tier 2 with zero hard errors or interruptions.

2. Tier 2: Fast Cloud Default (Gemini 3.8 Flash)

  • Target Model: Gemini 3.8 Flash.
  • Role: High-speed code synthesis, large-scale MCP tool interactions, and automated CI triage.
  • Latency: Sub-second response time.

3. Tier 3: Deep Cognitive Tier (Gemini 3.8 Pro)

  • Target Model: Gemini 3.8 Pro.
  • Role: Strictly reserved for deep system architecture, cross-cutting ADR synthesis, and high-complexity security boundary analysis.

💰 FinOps & Token Economics

By routing routine boilerplate generation, unit test scaffolding, and AST evaluations to Tier 0/1 Ollama ($0.00), engineering teams reduce operational AI inference costs by up to 85% compared to purely cloud-hosted agent loops.

Task Type Assigned Tier Cost / 1k Tokens Typical Latency
Boilerplate & CRUD Generation Tier 0 (Local Ollama) $0.00 ~350ms
Unit Test Scaffolding Tier 0 (Local Ollama) $0.00 ~400ms
AST & Lint Evaluation Tier 1 (DeepSeek-R1) $0.00 ~600ms
Rapid MCP Tool Execution Tier 2 (Gemini Flash) ~$0.0001 ~250ms
Architectural Decision Records Tier 3 (Gemini Pro) ~$0.002 ~1200ms

⚙️ Configuration (config.yaml)

llm:
  provider: "ollama"
  endpoint: "http://127.0.0.1:11434"
  default_model: "qwen3-coder:30b"
  fallback_model: "deepseek-r1:14b"
  timeout_seconds: 60
  temperature: 0.0
  auto_fallback: true