freellmpool › guide

How to use multiple free LLM APIs together

To combine routes from several LLM providers, pool only the routes you can access behind one endpoint so each request fails over when a target is rate-limited. The open-source freellmpool catalogs 22 provider groups spanning recurring free tiers, keyless endpoints, finite trials, pin-only routes, and disabled candidates. It exposes 178 enabled chat routes through one OpenAI-compatible interface and can start without credentials when an enabled keyless route is available.

Why combine them

• Each free tier is small on its own; together they add up to real daily capacity.
• When one provider returns a 429, you want to fall through to the next automatically — not fail.
• You don't want to write and maintain glue code for each provider's SDK and limits.

Do it in two commands

pip install freellmpool
freellmpool ask "Explain the CAP theorem in one sentence."   # keyless if an enabled route is available

Add applicable provider credentials for more routes and capacity (terms vary):

export GROQ_API_KEY=...        # plus CEREBRAS_API_KEY, GEMINI_API_KEY, NVIDIA_API_KEY, ...
freellmpool providers          # shows what's configured
freellmpool benchmark          # times each provider so routing prefers the fastest

Use it from code or as a drop-in endpoint

from freellmpool import Pool
reply = Pool.from_default_config().ask("Write a haiku about sqlite")

# or run a proxy and keep your existing OpenAI code unchanged:
#   freellmpool proxy  →  OPENAI_BASE_URL=http://localhost:8080/v1

How routing works

freellmpool orders the providers you have access to least-used-first, then picks a least-used model inside that provider. This keeps large catalogs from getting extra traffic just because they expose more models. It tries candidates until one returns a non-empty result, and sets a rate-limited provider aside for a cooldown. freellmpool's local per-day counters reset at UTC midnight; upstream providers use their own limit and reset windows. Set FREELLMPOOL_ROUTING=fast to prefer the lowest-latency provider, or FREELLMPOOL_ROUTING=legacy (or model-fast) to restore per-model balancing.

FAQ

How do I use Groq and Cerebras and Gemini together?

Set each provider's free API key as an environment variable and let freellmpool pool them — it routes each request to one you have capacity on and fails over automatically. You can also pin a provider per call with -p.

Do I need an API key for every provider?

No. freellmpool can answer without credentials while at least one enabled keyless route is available. Add applicable credentials over time to expand accessible routes; not every cataloged provider is recurring-free.

Part of freellmpool (MIT, free, open source). Updated 2026-08-29.