How to Rate Limit Your AI API Route (and Cap Your OpenAI Bill)

The short version

Auth first: only signed-in users can reach paid endpoints.
Rate limit per user: for example 10 requests a minute and 100 a day.
Cap the size: limit input length and maximum output tokens on every call.
Budget at the provider: a monthly spend limit and an alert, so a bug has a ceiling.

1. Why a hidden key still gets you a big bill

Your front end calls /api/chat, so the route's URL is in your public JavaScript. Anyone can copy that request into a script and replay it. Your server dutifully forwards each one to OpenAI with your key.

No secret leaked, but the effect on your bill is the same. The fix is to make the endpoint picky about who it serves and how much.

2. The four layers

Require a signed-in user

Check the session on the server before doing anything paid. Anonymous traffic gets a 401. See adding auth to API routes.

Rate limit by user ID

Limit by user rather than IP, since IPs are shared and easy to rotate. Add a daily cap on top of the per-minute one.

Cap request size

Reject oversized inputs and set a maximum output token count, so one request can't cost dollars.

Set a provider budget

Set a monthly budget and alert in your OpenAI or Anthropic project settings. This is your last line of defense against bugs, not just abuse.

3. A rate-limited route in Next.js

This example uses Upstash Redis for the counter, which works on serverless hosts where memory resets between requests. Swap in your own auth helper.

app/api/chat/route.ts

import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";
import OpenAI from "openai";
import { getUser } from "@/lib/auth"; // your auth helper

const perMinute = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(10, "1 m"),
  prefix: "chat:min",
});
const perDay = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.fixedWindow(100, "1 d"),
  prefix: "chat:day",
});

const openai = new OpenAI(); // reads OPENAI_API_KEY on the server
const MODEL = process.env.OPENAI_MODEL!; // pick your model

export async function POST(req: Request) {
  const user = await getUser(req);
  if (!user) return Response.json({ error: "Sign in first" }, { status: 401 });

  const [minute, day] = await Promise.all([
    perMinute.limit(user.id),
    perDay.limit(user.id),
  ]);
  if (!minute.success || !day.success)
    return Response.json({ error: "Slow down and try again soon" }, { status: 429 });

  const { message } = await req.json();
  if (typeof message !== "string" || message.length > 2000)
    return Response.json({ error: "Message is too long" }, { status: 400 });

  const completion = await openai.chat.completions.create({
    model: MODEL,
    max_tokens: 500, // caps the cost of one reply
    messages: [{ role: "user", content: message }],
  });

  return Response.json({ reply: completion.choices[0].message.content });
}

Why not count in memory? A plain JavaScript counter resets whenever a serverless function cold-starts and isn't shared across instances, so it barely limits anything. Use a shared store such as Redis, your database, or your host's built-in rate limiting.

4. Supabase Edge Functions and edge rules

Supabase Edge Functions (Lovable, Bolt): check the user's JWT at the top of the function, then count requests per user in a small table or Redis before calling the AI provider. Keep the AI key in Supabase secrets.
Cloudflare and Vercel both offer rate-limiting rules at the edge, which can block floods before they reach your code. They're a good extra layer, but per-user limits still belong in the route.
Bot checks such as Cloudflare Turnstile help on public, no-login endpoints like a free demo.

Before you ship

Paid routes return 401 to anonymous requests.
Per-user limits per minute and per day, stored somewhere shared.
Input length and max output tokens capped on every call.
Provider budget and alert set for every AI and paid API account.
A free or demo endpoint has a bot check and a strict global limit.

Check your live site in 60 seconds

ShipReady scans your deployed URL and the JavaScript visitors actually download: leaked keys, Supabase exposure, unprotected API routes, titles and share images. Secret findings unlock only after you verify you own the site.

Free scan, no signup, no code upload.

FAQ

What limits should I start with?

Something like 10 requests a minute and 100 a day per user covers normal use for most chat features. Watch real usage for a week, then adjust.

Can I rate limit by IP instead of user?

As a backup for logged-out pages, yes. As the main limit, no: offices and mobile networks share IPs, and attackers rotate them cheaply.

Is a provider budget enough on its own?

It stops the bleeding but also takes your app offline when it trips. Use it as the ceiling, with per-user limits doing the day-to-day work.

Keep reading

Sources: Upstash: Ratelimit SDK · Cloudflare: Turnstile