freellmpool › providers › Groq

Free Groq API: the fast one

Groq's free API is the one developers reach for when latency matters. It runs models on custom LPU hardware that streams tokens noticeably faster than typical GPU inference. The current freellmpool routes include OpenAI's GPT-OSS 20B/120B, Qwen 3.6 27B, and Groq Compound. You get a key free at console.groq.com/keys and call an OpenAI-compatible endpoint at https://api.groq.com/openai/v1. The main constraint is the free daily/minute rate limit, which is where pooling Groq with other tiers via freellmpool helps.

What Groq's free tier is good for

Because generation is so fast, Groq is a great default for anything interactive: chat UIs, autocomplete, streaming agents, and tool-use loops where round-trip latency compounds. It's less suited to extremely long context windows or multimodal input — for big documents or vision, Gemini is a better free pick. Treat Groq as your low-latency workhorse and keep a higher-context provider in reserve.

Which free model to pick

Model	Use it for
openai/gpt-oss-120b	Strong reasoning when you can spare the budget
openai/gpt-oss-20b	Fast drafts, classification, and routing
qwen/qwen3.6-27b	General chat and multilingual work
groq/compound	Groq's compound system; lower free-plan daily limit

Get a key and call it

Sign in at console.groq.com/keys, create a key, and use the OpenAI-compatible route — most OpenAI SDKs work by only changing the base URL:

curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-oss-120b","messages":[{"role":"user","content":"Hi"}]}'

Limits and gotchas

• Free usage is capped by requests/minute, requests/day, and tokens/minute per model — the per-minute token cap is often what you hit first on long outputs. Check Groq's current rate-limit table; the values change.
• llama-3.1-8b-instant and llama-3.3-70b-versatile were shut down for Free and Developer plans on August 16, 2026. Groq's deprecation history is authoritative.
• qwen/qwen3.8-27b is cataloged as a disabled exact pin pending a credentialed canary; listing presence alone does not make it an automatic route.
• Some models are reasoning/“thinking” models and burn extra output tokens — set a generous max_tokens or you'll get truncated answers.

Pool Groq with other free tiers

Groq's speed is great until you hit the minute/day cap mid-task. freellmpool keeps Groq as a preferred provider and transparently fails over to Gemini, NVIDIA and others on a 429, so a single limit doesn't stall you:

pip install freellmpool
export GROQ_API_KEY=...                 # plus any other free keys
freellmpool ask -p groq "..."           # pin Groq when you want its speed
freellmpool ask "..."                   # or pool + fail over automatically
FREELLMPOOL_ROUTING=fast freellmpool ask "..."   # prefer the lowest-latency tier

See also Cerebras (fast inference with finite trial credit), best free LLM API gateway, and using multiple free LLM APIs together.

FAQ

Is the Groq API free?

Yes. Create a key at console.groq.com/keys and call the OpenAI-compatible endpoint at api.groq.com/openai/v1. Free usage is rate-limited per minute and per day; verify current numbers in the console.

Why is Groq so fast?

Groq runs inference on its own LPU hardware designed for sequential token generation, which gives much higher tokens-per-second than typical GPU serving — useful for interactive and streaming workloads.

What's the best free Groq model?

Use openai/gpt-oss-120b for harder reasoning, openai/gpt-oss-20b for speed, or qwen/qwen3.6-27b for a general Qwen route. Verify Groq's current model and rate-limit pages because availability changes.

Part of freellmpool (MIT, open source). Limits and model IDs change — check Groq's docs. Updated 2026-08-29.