Reduce your footprint

This page lists changes you can make once you have the number from the methodology. Each section covers one change, the order of magnitude of its effect, and the trade-off it carries.

Pick a smaller model when you can

This is the largest single lever. Energy scales roughly linearly with active parameters (see the methodology formula). A 7B-active model is ~10× lower energy per token than a 70B model.

When to use a smaller model:

• Classification, extraction, and structured-output tasks.
• High-volume background work (summarisation, tagging).
• Anything where the output is verified by a downstream system.

When not to:

• Tasks that the smaller model fails at and your application has to retry on a larger one anyway. Two failed cheap calls + one big call

  one big call.

A smaller model burns less energy per token, so pick a smaller explicit model when it can plausibly handle the request. Try it on your traffic and keep it if quality holds.

Cache prompts where the upstream supports it

Several providers offer prompt caching: a long system prompt sent repeatedly with different user messages is charged at a discount on the cached portion. Where supported, this cuts both cost and energy on the cached part.

In practice:

• Place stable instructions, examples, and reference material first in the messages array.
• Place the variable part (the user’s question) last.
• Keep the stable prefix above the upstream’s caching threshold (for example, ≥1024 tokens).

The dashboard’s per-transaction view shows cached_tokens when an upstream applied a cache hit.

Trim prompts

Every prompt token is processed on a GPU, so a shorter prompt uses less real compute, and it costs less. Our reported carbon figure will not show it, though: it follows the EcoLogits model, which counts generated tokens only and treats prompt processing as negligible (see prompt tokens). Trim prompts for the real saving and the cost; to move the reported number, bound the output (below).

• Drop preamble that doesn’t change the model’s behaviour.
• Drop few-shot examples that the model no longer needs.
• Compress reference material (use IDs instead of full descriptions when the model has been trained on them).

This compounds with prompt caching, since a shorter cached prefix is both cheaper and faster to cache.

Choose the cleaner region when residency permits

The grid intensity in a hydro/nuclear-heavy region is far lower than in a coal-heavy one. To pick a clean region explicitly, encode its UN/LOCODE as the last segment of the model ID:

{
  "model": "aws-bedrock/mistralai/ministral-3-3b-instruct/br-gru",
  "messages": [...]
}

Or name the model and let the router choose the route for it with auto/mistralai/mistral-large-2512: among the eligible routes it prefers EU-sovereign ones, then the lowest grid carbon intensity, so it picks the cleanest available route for that model without any extra flag. You keep the choice of model, and the router decides only where it runs.

Bound completion length

max_tokens lets you stop generation when “enough is enough”. For classification or extraction, set it to the actual answer length plus a small margin. The carbon savings are linear with the saved tokens.

Some prompts respond well to “Answer in one sentence.” instructions; others ignore them. Both are worth trying: the dashboard will show whether the average completion length actually came down.

Rate-limit your retries

A retry storm can multiply your footprint by 3–10× of the underlying call. Use exponential backoff with jitter on retries, cap the retry count, and never retry on a 4xx that is not a 408 timeout.

Memoise

If the same user asks the same question twice, an in-application cache returns the previous answer at zero gateway cost and no inference energy.

A few patterns that work:

• Hash the prompt (after normalisation) and cache the response by that hash.
• Cache lookup tables generated by the model (taxonomies, slot schemas) and refresh them on a schedule, not per-request.
• For chat, cache the last few responses in memory keyed by the full conversation; reuse when the user re-asks immediately.

Aggregate where you can

Many small completions cost more than one larger one with multiple items. Examples:

• Classify a batch of 20 items in a single request rather than 20 requests.
• Extract structured fields for a list of inputs in one structured- output call.

Watch out for context-window limits and for the cost of re-prompting when one item in the batch fails; sometimes individual calls are cheaper net.

Order of magnitude summary

These levers are independent, so their effects stack. Most teams start with the first two.

Confirm with the dashboard

After any of these changes, check the eco-impact widget on the dashboard for the same time window before/after. If the change you made should have reduced the per-1K-tokens carbon number and didn’t, check the dashboard’s transaction-detail page, which shows which model and provider actually served each request.