Smart Rate Limiting: the Fail-safe Your Bot Vendor Cannot Build for You

Your bot management platform will miss an attack eventually. This is the design we use for the thirty minutes between the miss and the fix: a forecast-driven, multi-dimensional rate limiter that runs inside your own stack, on libraries you already know. 01. Where the vendor stops and you start The call comes in at 01:12. Origin CPU is pinned, checkout latency is in the seconds, and the bot management dashboard you spent eleven months procuring is a wall of green. Nothing is being challenged, because as far as the vendor's models are concerned nothing unusual is happening. This is not a vendor failure story. Commercial bot platforms are good, and the ones we have used are better than anything a five-person security team is going to write. The problem is structural, and it comes down to three things that are true no matter whose logo is on the invoice. Detection needs data, and data needs time. When an attacker shows up with a fresh residential proxy pool and a headless browser that spoofs the environment down to the driver level, the vendor's classifier has never seen that combination before. It needs enough labelled traffic to separate the new pattern from your ordinary users, and then it needs to ship a signature to a global fleet. That is hours on a good day. Commercial scraping toolkits advertise very high bypass rates against the major vendors precisely because they win this race by default: they only need to work until the signature lands. Managed services are a queue, not a button. Every vendor's answer to a live bypass is the same: open a critical ticket, an analyst reads your logs, an analyst writes a static rule. It works. It also means your mitigation is sitting behind everyone else's mitigation during exactly the kind of internet-wide event that generates a hundred critical tickets, it is usually billed separately from your licence, and what comes back is a regex or a list of subnets that stops working the moment the attacker rotates payloads. Your best evidence cannot leave the building. This is the one people underestimate. Suppose the attack is gift-card validation, or loyalty-point draining. Your fraud team can identify those sessions in minutes, because they can join against account identifiers, transaction history and hashed email. That join is the signal. It is also personal data, and under GDPR or CCPA you are not pasting it into a support portal so that a third party can tune a rule. If the evidence has to stay inside your perimeter, the enforcement decision derived from it has to be made inside your perimeter too. THE POINT You are not replacing the vendor. You are covering the window between the moment an attack starts working and the moment somebody else's model catches up. That window is where the revenue loss, the origin outage and the incident review all live. 02. Why "rate limiting" got a bad name Say "rate limiting" to a security architect and you will get a tired look. They are thinking of the 1990s version: count requests per IP, block at 100 a minute. Against a botnet spread over 200,000 residential addresses, each one sending four requests a minute, that control does nothing at all. Every individual IP is a model citizen. They are right, and this is exactly why the bot management market exists. Two things are being conflated, though. One is the counter: how you accumulate requests and decide to shed load. That part is genuinely ancient and genuinely solved; a token bucket in Redis has been good enough for twenty years. The other is the key and the threshold: what you count, and what number counts as too many. That is where the old design fails. A single IP is a bad key against a distributed attacker, and a fixed threshold is a bad number against traffic that moves by a factor of six over the course of a day. Fix the key and the threshold and the same old counter becomes useful again. Concretely, "smart" here means three changes: The threshold is a forecast, not a constant. The system predicts what the next five minutes should look like, with an uncertainty band, and treats the top of that band as the ceiling. At 15:00 the ceiling might be 68,000 requests per minute. At 03:40 it might be 13,000. A flat threshold has to be set above the first number, which makes it blind to anything that happens overnight. The key is behavioural, not network-level. Instead of the client IP, we count against combinations of attributes the attacker has to keep constant to make the attack work: operating system, browser build, TLS fingerprint, header ordering, country. The rule is temporary and self-expiring. Anything the system creates lives for fifteen minutes unless renewed. Nothing accumulates in your WAF for three years until nobody remembers what it was for. The rest of this article is how we build that, with the code. If you want the short version of the design constraints before we start: it has to be readable at 3 a.m. by whoever is on call, it has to run on off-the-shelf libraries, it has to be cheap enough that nobody argues about the compute bill, and it has to be tunable without a retraining pipeline. Every design decision below falls out of one of those four. 03. What your traffic actually looks like Before any modelling, spend an afternoon looking at your own request counts. Everything in this design depends on one empirical fact, and you should verify it holds for you rather than taking my word for it: legitimate traffic is boring. It rises and falls on a daily cycle that repeats itself within a few percent, day after day, and the same is true of most slices you can cut out of it. Getting from raw logs to a time series Start with whatever your CDN or WAF ships you. We work from Parquet dumps of edge logs, one file per hour. The only columns that matter for this exercise are the timestamp and the behavioural attributes we might eventually want to count against. Bucketing raw edge logs into five-minute request counts. import pandas as pd COLS = ["ts", "host", "ua_os", "ua_browser", "ja4", "country", "edge_status"] logs = pd.read_parquet("s3://edge-logs/2026-05-14/", columns=COLS) logs["ts"] = pd.to_datetime(logs["ts"], utc=True) # One row per (5-minute bucket, hostname). size() just counts requests. counts = ( logs.set_index("ts") .groupby([pd.Grouper(freq="5min"), "host"]) .size() .rename("requests") .reset_index() ) print(counts.head()) # ts host requests # 0 2026-05-14 00:00:00+00:00 www.example.com 58_412 # 1 2026-05-14 00:00:00+00:00 api.example.com 31_204 Two details in that snippet are worth pausing on, because both have bitten us. Work in UTC and never in local time. If you bucket in a timezone that observes daylight saving, twice a year you get a 23-hour day and a 25-hour day. Every forecasting model you train on that history will learn a seasonal pattern with a discontinuity in it, and you will spend a morning wondering why the ceiling is wrong on the second Sunday in March. Keep storage and modelling in UTC; convert only for the dashboard humans read. Missing buckets are zeros, not gaps. A groupby only produces rows for buckets that had at least one request. If your API had no traffic at 04:15 you get no row, and a forecasting library will happily interpolate straight through the hole as though nothing happened. Reindex against a complete date range and fill with zero. Absent buckets carry information. Make them explicit. def densify(frame, freq="5min", value="requests"): """Return a gap-free series indexed at `freq`, with absent buckets set to 0.""" idx = pd.date_range(frame["ts"].min(), frame["ts"].max(), freq=freq, tz="UTC") return ( frame.set_index("ts")[value] .reindex(idx, fill_value=0) .rename_axis("ts") ) www = densify(counts[counts["host"] == "www.example.com"]) assert www.index.freq is not None and www.isna().sum() == 0 The shape you are going to model Plot a day of that series and you get the curve below. Traffic climbs from a predawn floor, plateaus through the afternoon, gets a second smaller bump in the evening, and drains overnight. The shaded band is what a forecasting model will later give us: the range the next observation should fall inside if nothing unusual is happening. Figure 1. A weekday of request volume in five-minute buckets, against a 99% forecast band. The band scales with the traffic underneath it, so it is thousands of requests wide at the afternoon plateau and only a few hundred wide at the overnight floor. That asymmetry is the whole trick: an extra 8,000 requests per minute at 04:00 is a screaming anomaly, and at 14:00 it is Tuesday. The same shape, sliced by dimension Now cut the same traffic by operating system, or browser, or country. Each slice roughly follows the overall curve, but not exactly. Windows traffic tends to be squarer and office-hours shaped. iOS peaks later in the evening. What matters is that each slice's share of the total is remarkably steady. Composition drifts slowly. Attacks move it fast. by_os = ( logs.set_index("ts") .groupby([pd.Grouper(freq="5min"), "ua_os"]) .size() .unstack(fill_value=0) ) share = by_os.div(by_os.sum(axis=1), axis=0) print(share.resample("1h").mean().round(3).iloc[:4, :5]) # Android Linux Mac OS X Windows iOS # ts # 2026-05-14 00:00+00:00 0.214 0.019 0.121 0.318 0.311 # 2026-05-14 01:00+00:00 0.219 0.018 0.118 0.309 0.319 # 2026-05-14 02:00+00:00 0.221 0.020 0.117 0.305 0.322 # 2026-05-14 03:00+00:00 0.217 0.019 0.120 0.311 0.318 Look at the Linux column: two percent of traffic, hour after hour, hardly moving. If it goes to nine percent inside ten minutes, you have not suddenly acquired 400,000 new Linux desktop users. Something automated arrived. That observation is the seed of the whole detection strategy, and the reason we bother modelling dimensions separately instead of only watching the total. PRACTICAL NOTE If your edge only gives you a raw User-Agent string, parse it once at ingest and store the parsed fields, rather than parsing at query time. We use ua-parser and cache the result keyed on the exact UA string. A production edge sees maybe 50,000 distinct UA strings a day against hundreds of millions of requests, so the cache hit rate is above 99.9% and parsing stops being a cost you think about. 04. Two kinds of bots, and which one this catches Malicious automation splits into two behavioural families, and it is important to be honest about which one a volume-based system can touch. FAMILY WHAT IT LOOKS LIKE IN THE DATA CAN VOLUME FORECASTING SEE IT? Low and slow Deliberately paced to sit inside human volume. A few requests per minute per identity, spread across tens of thousands of identities, often over days. Total volume never leaves the normal band. No. By construction it produces no volumetric signal. Catching it needs sequence and behaviour models: mouse dynamics, request-order entropy, session graphs. Different article, different system. Volumetric scraping, credential stuffing, scalping, layer-7 flood A vertical wall that ignores the circadian curve entirely. The attacker is optimising for throughput because their economics depend on finishing before you notice. Yes. This is what breaks your origin, and it is what the system below is built for. Some people find that scoping disappointing. I would argue the opposite. The volumetric family is the one that turns into an outage, a stock-out, or a status page update, and it is the one your vendor's slow path is most painful for. Solving one attack class completely beats solving four of them partially, especially for a component whose whole purpose is to be predictable during an incident. 05. The architecture on one page The system is two levels of detection plus an enforcement tier. The split exists for one reason: cost. Level 1 runs constantly and looks at a handful of series. Level 2 is expensive, looks at thousands of series, and only runs when Level 1 says something is wrong. Figure 2. Level 1 asks "is anything wrong?" against two or three total-volume series. Level 2 asks "who is doing it?" against thousands of sliced series, and only wakes up when Level 1 is sure. The rule builder is the only component allowed to touch the CDN. Data flow, in words: the edge writes per-minute counters into Redis and per-day Parquet into object storage. A nightly job trains the forecasting models on the Parquet history and writes the next 36 hours of predicted ceilings back into Redis. Every 60 seconds a small worker compares live counters against those precomputed ceilings, hands the result to the scoring engine, and goes back to sleep. Nothing on that hot path loads a model or does arithmetic more complicated than a division. 06. Level 1: forecasting the ceiling Level 1 answers one question every minute: is the amount of traffic we are getting right now consistent with the amount of traffic we should be getting right now? To answer it we need a prediction of "should", and around that prediction a band wide enough to absorb ordinary randomness. Why Prophet and not something fancier We use Prophet, the open-source forecasting library from Meta. If you have not used it: Prophet decomposes a time series into a slow-moving trend, a set of repeating seasonal cycles expressed as Fourier terms, and a list of named holiday effects. It then reports, for any future timestamp, a central estimate (yhat) and an uncertainty interval (yhat_lower, yhat_upper). We care almost exclusively about yhat_upper, because that is our ceiling. An LSTM or a temporal fusion transformer would very likely forecast this data a few percent more accurately. We do not use one, for reasons that have nothing to do with accuracy: You can read the model. When the ceiling looks wrong, you can plot the trend component, the weekly component and the daily component separately and see which one is lying. Try doing that with a 40-layer network at 03:00 while the incident channel is filling up. Tuning is declarative. Friday-night surge? Add a custom seasonality. Black Friday? Add a holiday with a window. It is two lines, not a feature-engineering pipeline. It costs nothing to run. Fitting 60 days of five-minute data takes under a minute on a laptop, and inference is a matrix multiply. There is no GPU in this architecture, and that is a feature when you are defending the budget. Cleaning the history first Here is the mistake almost everyone makes on the first attempt, us included. You train the model on the last 90 days of raw traffic. Those 90 days contain three bot attacks. The model dutifully learns that traffic on a Tuesday morning occasionally quadruples, widens its uncertainty band accordingly, and now sits there politely ignoring the next attack. Left alone, the system trains itself into uselessness: every attack you fail to exclude raises the ceiling that was supposed to catch the next one. So we scrub the training history before fitting. The method is deliberately crude, because a sophisticated outlier detector would just be another model to debug. Remove yesterday's attacks so they do not become tomorrow's normal. import numpy as np import pandas as pd def scrub_spikes(series, window=288, z=6.0, min_scale=25.0): """Replace obvious incident spikes with the local median. window : number of buckets in the rolling comparison (288 = one day of 5-min data) z : how many robust standard deviations above the local median counts as a spike """ med = series.rolling(window, center=True, min_periods=48).median() abs_dev = (series - med).abs() mad = abs_dev.rolling(window, center=True, min_periods=48).median() scale = (1.4826 * mad).clip(lower=min_scale) # MAD -> comparable to a std dev spike = (series - med) > z * scale cleaned = series.mask(spike, med).ffill().bfill() return cleaned, spike clean, flagged = scrub_spikes(www) print(f"scrubbed {flagged.sum()} of {len(www)} buckets " f"({flagged.sum() / len(www):.2%})") Plain-language version of what that does: for each bucket, look at the median of the day around it, then look at how far a typical bucket sits from that median. That second number, the median absolute deviation, is a measure of spread that a handful of enormous values cannot drag upward the way a standard deviation can. Anything sitting more than six of those units above the local median gets replaced by the median. The 1.4826 constant just rescales MAD so that, for normally distributed data, it lines up with a standard deviation and z=6 means what you expect it to mean. DO THIS TOO Keep a table of mitigation windows: every time the system enforces a rule, record the time range. Exclude those ranges from the next training run explicitly. The statistical scrub is your safety net for attacks you never noticed; the mitigation log is ground truth for the ones you did. Fitting the model One model per (hostname, window). Sixty days of history, about 40 seconds to fit. import joblib from prophet import Prophet def build_frame(series): # Prophet wants two columns, `ds` (timestamp, timezone-naive) and `y` (the value). # We model log1p(requests) so that the uncertainty band scales with traffic volume: # +/- 8% at the 03:00 floor is a much smaller absolute number than +/- 8% at the peak. return pd.DataFrame({ "ds": series.index.tz_convert("UTC").tz_localize(None), "y": np.log1p(series.to_numpy(dtype="float64")), }) def fit_window_model(series, holidays_country="US"): df = build_frame(series) model = Prophet( growth="flat", # request volume has no long-run linear trend; a flat # baseline stops Prophet extrapolating last month's # marketing campaign into next month's ceiling yearly_seasonality=False, # 60 days of history cannot learn a yearly cycle weekly_seasonality=True, # weekends genuinely differ daily_seasonality=False, # replaced below with a higher-resolution version seasonality_mode="additive",# additive in log space == multiplicative in real space interval_width=0.99, # the ceiling is the top of a 99% interval changepoint_prior_scale=0.03, # lower = stiffer trend, fewer bogus bends seasonality_prior_scale=8.0, ) # Prophet's default daily seasonality (fourier_order=4) is too smooth for five-minute # data: it rounds off the sharp 07:00 ramp and the 23:00 cliff. Twelve terms tracks # both without fitting noise. model.add_seasonality(name="daily", period=1, fourier_order=12) model.add_country_holidays(country_name=holidays_country) model.fit(df) return model model_5m = fit_window_model(clean) joblib.dump(model_5m, "models/www_5m.pkl") The parameters that actually change behaviour are interval_width, changepoint_prior_scale and the daily Fourier order. Everything else you can leave alone for a year. PARAMETER WHAT IT CONTROLS SYMPTOM IF IT IS WRONG interval_width How much of the historical variation the band is expected to contain. Too low: constant false alarms. Too high: the ceiling drifts so far above reality that a real attack fits underneath it. changepoint_prior_scale How eagerly the trend is allowed to bend. Too high: the trend chases a two-day traffic dip and the ceiling collapses with it. Too low: a genuine step change (a new market launch) takes a week to be absorbed. fourier_order (daily) How sharp a daily shape the model can represent. Too low: the band bulges around the morning ramp because the model cannot follow it. Too high: the model fits noise and the band gets uselessly tight. Do the arithmetic on interval_width before you pick it, because it sets your false alarm rate directly. At 0.99, roughly one percent of buckets fall outside the interval by chance, and only the upper half of those matter to us. Five-minute buckets give 288 observations a day, so expect around 1.4 spurious ceiling breaches per day, per series, in perfectly healthy traffic. Multiply by your number of hostnames. That number is why the next two sections exist: raw breaches are not alerts, and alerts are not actions. If you want to tune rather than guess, Prophet ships cross-validation. Run it once a quarter, not every night. Coverage is the metric to watch, not MAPE. from prophet.diagnostics import cross_validation, performance_metrics cv = cross_validation( model_5m, initial="45 days", # train on the first 45 days period="2 days", # then step the cutoff forward 2 days at a time horizon="12 hours", # and score predictions up to 12 hours ahead parallel="processes", ) metrics = performance_metrics(cv, rolling_window=0.1) print(metrics[["horizon", "mape", "coverage"]].head()) # `coverage` is the fraction of actuals that landed inside the interval. # With interval_width=0.99 you want to see roughly 0.99. If you see 0.93, # your band is too tight and you are about to spend a month chasing false positives. Precompute the ceilings; do not call predict() on the hot path A tempting design is to call model.predict() once a minute with the current timestamp. Do not. Loading a pickled Prophet model and predicting a single row takes tens of milliseconds and pins a Python process to a model file, which makes the detector awkward to scale horizontally and gives you a hard dependency on the model artifact being present in production. Since the forecast for 04:35 tomorrow does not depend on anything that happens between now and then, compute it in advance. The nightly job is the only thing that ever touches a model file. import json import redis R = redis.Redis(host="redis-rl", decode_responses=True) def publish_ceilings(model, start, periods, freq, key, ttl_hours=48): """Write the next `periods` forecast ceilings into a Redis hash.""" future = pd.DataFrame({"ds": pd.date_range(start, periods=periods, freq=freq)}) fc = model.predict(future) pipe = R.pipeline() for ts, yhat, upper in zip(fc["ds"], fc["yhat"], fc["yhat_upper"]): pipe.hset(key, ts.strftime("%Y-%m-%dT%H:%M"), json.dumps({ "expected": float(np.expm1(yhat)), # undo the log1p from training "ceiling": float(np.expm1(upper)), })) pipe.expire(key, ttl_hours * 3600) pipe.execute() # Nightly, 02:00 UTC: 36 hours of ceilings so a failed training run is survivable. publish_ceilings(model_5m, start=pd.Timestamp.utcnow().floor("h"), periods=36 * 12, freq="5min", key="rl:ceiling:www:5m") publish_ceilings(model_1h, start=pd.Timestamp.utcnow().floor("h"), periods=36, freq="1h", key="rl:ceiling:www:1h") Publishing 36 hours instead of 24 means a broken training run degrades into stale-but-usable ceilings rather than no ceilings at all. Alert on the age of the newest key, not just on the job exit code, because the failure mode that hurts is silent staleness. The detector itself is now trivial, which is the point: Two Redis reads and a division. This is the whole hot path. from dataclasses import dataclass @dataclass(frozen=True) class WindowVerdict: window: str # "5m" or "1h" observed: float expected: float ceiling: float @property def ratio(self) -> float: # How far past the ceiling we are. 1.0 == exactly at the ceiling. return self.observed / max(self.ceiling, 1.0) @property def breached(self) -> bool: return self.ratio > 1.0 def evaluate(host, window, now, live_count): slot = now.floor("5min" if window == "5m" else "h").strftime("%Y-%m-%dT%H:%M") raw = R.hget(f"rl:ceiling:{host}:{window}", slot) if raw is None: raise LookupError(f"no ceiling published for {host}/{window} at {slot}") fc = json.loads(raw) return WindowVerdict(window, live_count, fc["expected"], fc["ceiling"]) Two clocks are better than one Run this on a single window and you have to choose which failure you prefer. A five-minute window reacts fast and cries wolf: a link goes viral, a mobile app retries after a push notification, and there is your breach. An hourly window is calm and slow: an attack that starts at 03:05 has to sustain itself for half the hour before the hourly average moves enough to cross its own ceiling, and by then you have taken the damage. So we run both, on the same traffic, and combine the verdicts. Figure 3. The same incident through two lenses. The attack ramps up from 03:30; the blue five-minute line clears its ceiling at 03:40. The grey hourly average is still under its own (higher, smoother) ceiling for the rest of that hour and does not cross until after 04:00. Speed comes from the fast window, confidence from the slow one agreeing a while later. Speed when it is obvious, confirmation when it is not. def level1_state(five: WindowVerdict, hour: WindowVerdict, consecutive_5m: int): """Combine the two windows into one state, with a reason string for the log.""" if five.ratio >= 3.0: # Nothing organic triples the ceiling in five minutes. Do not wait for confirmation. return "ALERT", f"5m volume at {five.ratio:.1f}x ceiling" if five.breached and hour.breached: return "ALERT", "both windows above ceiling" if five.breached and consecutive_5m >= 3: return "ALERT", f"5m ceiling exceeded for {consecutive_5m} consecutive buckets" if five.breached or hour.breached: return "WATCH", f"single-window breach ({'5m' if five.breached else '1h'})" return "OK", "within forecast band" Three consecutive five-minute breaches is fifteen minutes of elevated traffic. Combined with the 1.4-per-day false breach rate computed earlier, the chance of three in a row happening by coincidence is negligible, while a real volumetric attack trips the ratio >= 3.0 branch inside two buckets anyway. The middle branch, both windows breached, catches the awkward shape in between: a sustained ramp that never spikes hard enough to look dramatic. 07. The scoring engine, or how not to page yourself at 3 a.m. Level 1 raising an ALERT is not a reason to change anything at the edge. Mitigation is not free: challenges cost conversion, throttles cost some users their session, and every rule you push is a change you now have to reason about during the incident. The scoring engine is the component that decides whether the anomaly is worth acting on, and it is where all the organisational knowledge lives. It combines four kinds of context. SIGNAL QUESTION IT ANSWERS EFFECT ON THE SCORE Infrastructure load Is this traffic actually hurting anything? Origin CPU, error rate, queue depth, database saturation. Strong positive when the fleet is in pain; strong suppressor when it is idle at 15% CPU. Business calendar Did marketing plan this? Flash sales, product drops, an email blast to nine million people. Large negative. A planned spike raises the bar rather than removing it entirely. Geography and routing Is the excess coming from markets we sell to? Moderate positive for volume from countries where you have no business. Cooldown state Have we just acted, and are we about to act again on the rebound? Suppressor. Prevents the loop where a mitigation ends, traffic returns, and the system immediately re-mitigates. A weighted sum, on purpose. Every term is arguable in a post-incident review. from dataclasses import dataclass def clamp(x, lo=0.0, hi=1.0): return max(lo, min(hi, x)) @dataclass class Context: origin_cpu: float # p95 CPU across the origin fleet, 0..1 origin_5xx_rate: float # share of origin responses that are 5xx, 0..1 upstream_p99_ms: float planned_event: bool # from the marketing calendar feed offmarket_share: float # share of current traffic from non-market countries, 0..1 def anomaly_score(state: str, ratio: float, ctx: Context) -> float: """0-100. Deliberately a weighted sum: an on-call engineer can do this in their head.""" score = {"OK": 0.0, "WATCH": 25.0, "ALERT": 55.0}[state] # Magnitude: 20% over the ceiling adds 2 points, 3x over adds the full 20. score += 20.0 * clamp((ratio - 1.0) / 2.0) # Infrastructure pain. Below 55% CPU this contributes nothing at all. score += 25.0 * clamp((ctx.origin_cpu - 0.55) / 0.35) score += 15.0 * clamp(ctx.origin_5xx_rate / 0.02) score += 10.0 * clamp((ctx.upstream_p99_ms - 800) / 1200) # Traffic from places we do not sell to is more suspicious per request. score += 10.0 * clamp(ctx.offmarket_share) if ctx.planned_event: score -= 30.0 return clamp(score, 0.0, 100.0) Note what the infrastructure term does. If both windows are screaming but origin CPU is at 15% and error rates are flat, the score lands around 60 and nothing happens beyond a notification. That is correct behaviour. A scraper politely reading your product catalogue at four times normal volume, while your servers shrug, is a business problem for a Monday, not a reason to start challenging real customers at midnight. Hysteresis, or why one threshold is never enough A single threshold produces flapping. The score crosses 70, you mitigate, traffic drops because you mitigated, the score falls to 65, you stop mitigating, traffic comes back, the score crosses 70 again. You have built an oscillator, and each cycle pushes a CDN rule change, which has its own propagation delay measured in tens of seconds. The fix is the same one your thermostat uses: separate the turn-on threshold from the turn-off threshold, and add a minimum dwell time. Three states and two thresholds. Every incident review starts by reading this log. ENTER = 70.0 # score needed to start mitigating EXIT = 40.0 # score we must fall below to stop MIN_DWELL_S = 600 # never mitigate for less than 10 minutes COOLDOWN_S = 1200 # after stopping, require a clearly higher score for 20 minutes RE_ENTER_PENALTY = 10.0 class Gatekeeper: def __init__(self): self.state = "IDLE" # IDLE -> MITIGATING -> COOLDOWN -> IDLE self.changed_at = 0.0 def update(self, score: float, now: float) -> str: elapsed = now - self.changed_at if self.state == "IDLE": if score >= ENTER: self._to("MITIGATING", now) elif self.state == "MITIGATING": if score = MIN_DWELL_S: self._to("COOLDOWN", now) elif self.state == "COOLDOWN": if score >= ENTER + RE_ENTER_PENALTY: self._to("MITIGATING", now) # it came straight back; act again elif elapsed >= COOLDOWN_S: self._to("IDLE", now) return self.state def _to(self, state, now): self.state, self.changed_at = state, now Log every transition with the score, the ratio and the full context object. When somebody asks in a review why the system throttled Chrome on Android for eleven minutes on a Thursday, you want to answer with a timestamped record rather than an opinion. 08. Level 2: finding out who is doing it The gatekeeper has decided something real is happening. The question changes from "is there an attack" to "which slice of traffic is the attack, and how narrowly can I describe it?" Get this wrong in the generous direction and you throttle a third of your customers. Get it wrong in the strict direction and the attacker slips through the gaps in your description. Why we cannot just model everything Cardinality is the number of distinct values a field takes. In web telemetry it multiplies unpleasantly. A realistic mid-size estate might see, in a day: 12 operating system families, and about 90 OS versions worth distinguishing ~350 browser build strings once you keep minor versions 1,500 to 4,000 distinct JA4 TLS fingerprints 90 countries with non-trivial volume Every combination of those is a potential rate-limiting key, and 90 × 350 × 2,000 × 90 is about 5.7 billion. Even restricting yourself to combinations that actually occur, you are looking at millions of series, most of which see three requests a week. Fitting and storing a forecasting model for each is absurd, and the marginal value of the millionth model is zero. The resolution is the two-level split. Level 2 models exist, but they only run inference when Level 1 and the gatekeeper have already agreed that something is wrong. That is a few dozen times a month rather than 1,440 times a day, so we can afford to be thorough when it happens. Which slices to look at We evaluate every single dimension independently, plus a deliberately short list of pairs. The pairs are not exhaustive; they are the ones where experience says the combination carries signal that neither field carries alone. Six single dimensions and four pairs. Resist the urge to add more. SINGLE = ["ua_os", "ua_browser", "ja4", "country", "asn", "header_hash"] # Pairs worth the extra sparsity. A TLS fingerprint plus an OS is powerful because # real browsers on real operating systems produce a small, stable set of combinations, # and automation tooling routinely produces impossible ones (a macOS User-Agent # presenting the TLS fingerprint of a Go HTTP client, for instance). PAIRS = [ ("ua_os", "ja4"), ("ua_browser", "ja4"), ("country", "ja4"), ("ua_os", "ua_browser"), ] def slice_counts(logs, keys, freq="5min"): return ( logs.set_index("ts") .groupby([pd.Grouper(freq=freq), *keys]) .size() .rename("requests") .reset_index() ) Attribution: who owns the excess This is the part that makes the rest tractable, and it is barely any code. We know the total observed volume and the total expected volume, so we know the excess. For each value of each dimension we also know its observed and expected volume. Divide one by the other and you get the share of the excess each value is responsible for. Excess attribution. Three lines that replace a dashboard and an analyst. def attribute_excess(observed: pd.Series, expected: pd.Series): """observed / expected: request counts indexed by dimension value, same index. Returns the share of the total excess attributable to each value, descending. """ surplus = (observed - expected).clip(lower=0) total_excess = max(surplus.sum(), 1.0) return (surplus / total_excess).sort_values(ascending=False) def top_contributors(shares, cover=0.8, max_terms=4): """Smallest set of values that together explain `cover` of the excess.""" running = shares.cumsum() keep = shares[running = 30: return prophet_ceiling(dim, "__other__", slot) return sparse_ceiling(history) The __other__ bucket matters more than it looks. Without it, an attacker who rotates through 4,000 rare User-Agent strings produces 4,000 individually unremarkable slices and no alert anywhere. By summing everything outside the top N into a single modelled series, that same behaviour shows up as the "other" bucket going from 1.2% of traffic to 22%, which is extremely visible. Two ways to build the micro-models Approach A: top N plus one Rank each dimension's values by 30-day volume, keep the top N (we use N=1,000 across all dimensions combined), fit an independent model for each, and sum the remainder into __other__. Simple, embarrassingly parallel, and every model is independently debuggable. The cost is 1,000 pickle files and a training job that takes about 40 minutes on eight cores. Approach B: one multi-series model The modern alternative is a single gradient-boosted model that forecasts many series at once, with the series identity as a feature. skforecast wraps this pattern around any scikit-learn regressor. One model object instead of a thousand. Pin the version. import numpy as np from lightgbm import LGBMRegressor from sklearn.preprocessing import StandardScaler from skforecast.recursive import ForecasterRecursiveMultiSeries # `wide` is one column per slice, one row per 5-minute bucket, zero-filled. wide = np.log1p(by_browser.asfreq("5min").fillna(0.0)) forecaster = ForecasterRecursiveMultiSeries( regressor=LGBMRegressor(n_estimators=500, learning_rate=0.05, max_depth=7, verbose=-1), lags=[1, 2, 3, 6, 12, # the last hour 288, 289, # same time yesterday 2016], # same time last week encoding="ordinal", # series identity as a categorical feature transformer_series=StandardScaler(), ) forecaster.fit(series=wide) pred = forecaster.predict_interval(steps=12, interval=[1, 99], n_boot=150) ceilings = np.expm1(pred.filter(like="upper_bound")) VERSION WARNING skforecast reorganised its public API in 0.14: the class we import above used to be ForecasterAutoregMultiSeries in skforecast.ForecasterAutoregMultiSeries. Pin the exact version in your lockfile and read the migration notes before upgrading, or your nightly training job will fail on a Sunday. TOP N + 1 MULTI-SERIES Cold start for a new slice No model until the next training run Handled immediately; the model generalises across series Debugging a bad ceiling Easy. Load one pickle, plot the components Harder. You are inspecting feature importances of a shared model Training time Scales linearly with N Roughly flat; one fit over a wide frame Artifacts to manage N model files One Who should pick it Teams who want to read the model at 3 a.m. Teams with an existing ML platform and versioned model registry We shipped Approach A first, on the grounds that the incident-time cost of not understanding your own defence is much higher than the ops cost of a thousand small files. Approach B is where we ended up once the pipeline had a year of operational history behind it. 09. Turning a finding into a rule Level 2 hands over a ranked list of candidate predicates with two numbers each: how much of the excess it covers, and how much normal traffic it touches. The rule builder picks one, decides what to do to it, and gives it an expiry date. The selection rule Prefer the scalpel. Reach for the sledgehammer only with the safety on. from dataclasses import dataclass, field MIN_COVERAGE = 0.60 # a rule must explain at least 60% of the excess MAX_BLAST = 0.05 # ...while touching at most 5% of normal traffic @dataclass class Candidate: predicate: dict # e.g. {"ja4": "t13d1516h2_8daaf6152771"} excess_covered: float # 0..1 historic_share: float # 0..1, this predicate's normal share of total traffic observed_rpm: float expected_rpm: float @property def leverage(self): # Excess caught per unit of collateral. Higher is better. return self.excess_covered / max(self.historic_share, 1e-4) def choose(candidates): surgical = [c for c in candidates if c.excess_covered >= MIN_COVERAGE and c.historic_share = MIN_COVERAGE] if broad: return min(broad, key=lambda c: c.historic_share), "challenge" # No single predicate explains it at all. This needs a human. return None, "escalate" The leverage ratio is doing the real work. In the incident printed above, the JA4 candidate scores 0.983 / 0.0021 = 468 and the browser candidate scores 0.955 / 0.061 = 16. Both clear the coverage bar, one is thirty times cheaper in collateral, and the choice needs no judgement call at three in the morning. The fallback path matters just as much. When an attacker randomises their TLS fingerprint per session, no narrow predicate covers the excess and the only description that fits is something broad like "Chrome on Windows". Blocking that would be self-harm. Challenging it is survivable: real browsers solve the challenge, most automation does not, and your origin gets to keep breathing. The limit writes itself Here is the quiet payoff of having built everything on forecasts. A traditional rate limit forces you to invent a number. We do not have to invent anything, because the model already told us what this slice should be doing right now. Set the limit at the expected rate plus headroom and you are, by construction, allowing every request the slice would have made if the attack were not happening. The rate limit is the forecast plus 25%. No magic constants. import hashlib, json, time NEVER_BLOCK = [ {"asn": 64500}, # our own corporate egress {"country": "US", "ua_os": "iOS"}, # primary market, primary app platform {"header_hash": "partner-integration-v3"}, ] def matches_protected(predicate): return any(all(predicate.get(k) == v for k, v in rule.items()) for rule in NEVER_BLOCK) def build_rule(candidate, action, now=None, ttl_s=900, headroom=1.25): now = now or time.time() if matches_protected(candidate.predicate) and action == "throttle": action = "challenge" # protected traffic is never hard-limited limit_rpm = max(60, int(candidate.expected_rpm * headroom)) rule = { "id": "srl-" + hashlib.sha1( json.dumps(candidate.predicate, sort_keys=True).encode() ).hexdigest()[:10], "match": candidate.predicate, "action": action, # throttle | challenge | log_only "limit": {"requests_per_minute": limit_rpm, "key": "client_ip"}, "expires_at": int(now + ttl_s), "evidence": { "excess_covered": round(candidate.excess_covered, 3), "historic_share": round(candidate.historic_share, 4), "observed_rpm": int(candidate.observed_rpm), "expected_rpm": int(candidate.expected_rpm), }, "created_by": "smart-rate-limit/level2", } return rule What actually gets POSTed to the CDN rule API. { "id": "srl-9f31c4ab02", "match": { "ja4": "t13d1516h2_8daaf6152771" }, "action": "throttle", "limit": { "requests_per_minute": 387, "key": "client_ip" }, "expires_at": 1778731320, "evidence": { "excess_covered": 0.983, "historic_share": 0.0021, "observed_rpm": 41490, "expected_rpm": 310 }, "created_by": "smart-rate-limit/level2" } Two properties of that payload are worth copying even if you take nothing else from this article. The id is a hash of the predicate, which makes rule creation idempotent: the same attack signature detected three ticks in a row updates one rule instead of creating three. And expires_at is mandatory. Every rule this system creates dies in fifteen minutes unless the detector renews it, which means the worst case of a bad rule is fifteen minutes of degraded experience for a slice of traffic, not a mystery entry in your WAF that somebody finds in 2029. Blast radius gating The last guard sits between the rule builder and the CDN, and it exists because the attribution maths can be fooled. If an attacker sends their flood using the single most common User-Agent string in your logs, the excess and the legitimate traffic land in the same bucket, and a naive selection will happily propose throttling forty percent of your customers. Above 20% of traffic, the system stops being allowed to act on its own. BLAST_WARN = 0.05 BLAST_HARD = 0.20 def gate(rule, candidate, notify, deploy): share = candidate.historic_share if share >= BLAST_HARD: notify.page( "smart-rate-limit refusing to self-deploy", f"candidate {candidate.predicate} would affect {share:.1%} of normal traffic", evidence=rule["evidence"], ) return "escalated" # a human decides. The system does not. if share >= BLAST_WARN: rule["action"] = "challenge" rule["limit"]["requests_per_minute"] = int( rule["limit"]["requests_per_minute"] * 2 ) notify.warn("deploying a broad rule in challenge mode", rule=rule) deploy.put(rule) return rule["action"] Pick your own numbers for those thresholds, but pick them in advance, in daylight, with the business in the room. The one thing you must not do is leave the decision to whoever happens to be on call at 03:00 with an origin at 100% CPU. 10. Enforcement at the edge Everything so far produces a JSON document. Something has to act on it. Where that happens depends on your estate, and the choice matters more for latency than for correctness. If your CDN exposes a rule API, use it. Enforcement at the edge means the attack traffic never reaches your infrastructure at all, which is the entire point when the origin is the thing falling over. Propagation is usually 10 to 60 seconds, so measure it and put the number in your runbook; the on-call engineer needs to know how long to wait before deciding the rule did not work. If you enforce at your own gateway, or you want a second line behind the CDN, the workhorse is a token bucket in Redis. A token bucket in thirty lines of Lua The mental model: each key owns a bucket that holds a certain number of tokens. Tokens refill at a steady rate up to a maximum (the burst size). Each request removes one token. No token, no request. A bucket allows short bursts, which matters because real browsers open six connections and fire a page's worth of requests at once, while still enforcing a long-run average rate. The reason this is Lua and not Python is atomicity. Redis runs an EVAL script as a single indivisible operation, so twenty gateway processes hitting the same bucket cannot interleave a read and a write and each conclude there was one token left. token_bucket.lua. Register it once with SCRIPT LOAD and call it by SHA. -- KEYS[1] : bucket key, e.g. rl:srl-9f31c4ab02:203.0.113.9 -- ARGV[1] : refill rate, tokens per second -- ARGV[2] : burst capacity, tokens -- ARGV[3] : now, unix seconds as a float -- ARGV[4] : cost of this request, normally 1 local rate = tonumber(ARGV[1]) local burst = tonumber(ARGV[2]) local now = tonumber(ARGV[3]) local cost = tonumber(ARGV[4]) local state = redis.call('HMGET', KEYS[1], 'tokens', 'ts') local tokens = tonumber(state[1]) local ts = tonumber(state[2]) if tokens == nil then -- first request for this key tokens = burst ts = now end -- Refill for the time that has passed, capped at the burst size. local elapsed = math.max(0, now - ts) tokens = math.min(burst, tokens + elapsed * rate) local allowed = 0 if tokens >= cost then tokens = tokens - cost allowed = 1 end redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now) -- Let idle buckets expire so memory tracks active clients, not lifetime clients. redis.call('EXPIRE', KEYS[1], math.ceil(burst / rate) + 60) -- retry_after: seconds until one token is available again local retry_after = 0 if allowed == 0 then retry_after = math.ceil((cost - tokens) / rate) end return {allowed, retry_after} The gateway side. One round trip per request. BUCKET = R.register_script(open("token_bucket.lua").read()) def check(rule, client_key, now=None): rate = rule["limit"]["requests_per_minute"] / 60.0 burst = max(10.0, rate * 10.0) # ten seconds of headroom allowed, retry_after = BUCKET( keys=[f"rl:{rule['id']}:{client_key}"], args=[rate, burst, now or time.time(), 1], ) return bool(allowed), int(retry_after) What to return when you say no Send 429 Too Many Requests with a Retry-After header, and keep the body small and cacheable. Three things people get wrong here: Do not return 403. It tells well-behaved clients, including your own mobile app and any partner integration, that retrying is pointless. 429 with Retry-After tells them exactly when to come back, and good clients honour it, which reduces your load further. Do not rate limit the challenge endpoint. If suspicious traffic is being routed to a CAPTCHA, the CAPTCHA has to be reachable, or you have built a very expensive way of returning errors. Emit a header you can see in the logs. We attach the rule id to every throttled response. Being able to run count by rule_id during an incident is worth the 30 bytes. If you sit behind Envoy, the same rules map onto the local rate limit filter with descriptors generated from the predicate, which is worth knowing if you would rather not put Redis on your request path: Same maths, enforced in the proxy rather than in Redis. # Envoy: descriptors generated from the rule predicate, one bucket each. rate_limits: - actions: - request_headers: { header_name: "x-ja4-fingerprint", descriptor_key: "ja4" } - remote_address: {} descriptors: - entries: - { key: "ja4", value: "t13d1516h2_8daaf6152771" } token_bucket: max_tokens: 65 tokens_per_fill: 65 fill_interval: 10s 11. Proving it works before it can hurt you A system that can throttle your customers should not be trusted on the strength of a design document, including this one. There are two things to do before enforcement is switched on, and both take weeks rather than days. Shadow mode Run the whole pipeline with action: "log_only". Rules are constructed, evidence is recorded, notifications fire, and the CDN is never touched. Leave it there for at least four weeks, and read every rule the system would have created. The number to watch is not accuracy, it is the count of proposed rules per week during normal operation. If shadow mode produces eleven rules a week and ten of them are marketing emails, a mobile app release, and a partner's nightly batch job, you do not have a detector, you have a nuisance. Fix the calendar feed and the thresholds until the noise floor is near zero, because on-call trust is spent once. Backtesting against real incidents Pull the raw counters for a day you know contained an attack, replay them through the detector, and measure how long it takes to fire. Time to mitigation is the number that justifies the project, so measure it honestly, including the CDN propagation delay. Replay is the only honest way to argue about time to mitigation. def replay(counts_5m, counts_1h, ceilings, incident_start, propagation_s=45): """Feed historical buckets through the live detector and report time to action.""" gate = Gatekeeper() consecutive = 0 first_action = None for ts, observed in counts_5m.items(): five = WindowVerdict("5m", observed, *ceilings.at(ts, "5m")) hour = WindowVerdict("1h", counts_1h.asof(ts), *ceilings.at(ts, "1h")) consecutive = consecutive + 1 if five.breached else 0 state, reason = level1_state(five, hour, consecutive) ctx = historical_context(ts) # replayed CPU, calendar, geo score = anomaly_score(state, five.ratio, ctx) if gate.update(score, ts.timestamp()) == "MITIGATING" and first_action is None: first_action = ts break if first_action is None: return {"detected": False} ttm = (first_action - incident_start).total_seconds() + propagation_s return {"detected": True, "first_action": first_action, "ttm_seconds": ttm} If you have no labelled incidents, or not enough of them, inject synthetic ones. Take a clean day, add a spike of known shape and size at a known time, and confirm the detector fires when you expect. Sweep the magnitude downward until it stops firing; that is your sensitivity floor, and it is a far more useful number than any accuracy percentage. Sweep magnitude against time of day. The results will surprise you at least once. def inject(series, start, duration_min, peak_multiplier, ramp_min=5): """Add a synthetic volumetric attack on top of a clean day of traffic.""" out = series.copy() idx = out.loc[start:start + pd.Timedelta(minutes=duration_min)].index ramp = np.clip(np.arange(len(idx)) / max(ramp_min // 5, 1), 0, 1) baseline = out.loc[idx].to_numpy() out.loc[idx] = baseline + baseline.mean() * (peak_multiplier - 1) * ramp return out for hour in (3, 9, 14, 21): # detection is much harder at 14:00 than at 03:00 for mult in (1.3, 1.5, 2.0, 3.0, 5.0): attacked = inject(clean_day, day_start + pd.Timedelta(hours=hour), 40, mult) print(hour, mult, replay(attacked, ...)["ttm_seconds"]) METRIC WHY IT MATTERS WHERE WE HOLD IT Time to mitigation, p50 and p90 The whole justification for the project Under 3 minutes end to end, including propagation Proposed rules per quiet week The on-call trust budget Below 1 Collateral share of deployed rules How many real users each action touches Median under 1% Sensitivity floor at the daily trough How small an attack still gets caught overnight Around 1.4x the ceiling Ceiling staleness Silent failure of the training job Alert at 26 hours Those right-hand values are ours and they depend heavily on traffic shape. Treat them as an example of what to measure, not as targets to copy. 12. What this system does not do Every honest design document has this section, and it is usually the most useful one. It cannot see low-and-slow automation. Stated up front, repeated here, because somebody will eventually present this system as the answer to credential stuffing in general. It answers volumetric credential stuffing. An attacker doing 30 login attempts a minute spread over 50,000 residential IPs produces no volume anomaly whatsoever, and you need session-level behavioural signals to catch them. Most of the fields are attacker-controlled. User-Agent is a string the client chooses. An attacker who reads this article will send the most common Chrome-on-Windows string in your logs and hide inside your largest bucket. That is exactly why the blast radius gate exists, and why TLS fingerprints and header-order hashes are worth the integration effort: they describe how the client actually speaks, not what it claims to be. They can be forged too, but forging them costs real engineering effort, and raising that cost is the whole game. It measures the layer you point it at. If your CDN serves 92% of requests from cache, edge request counts will barely move during an attack that is destroying your origin, because the attack is aimed at uncacheable paths. Model the traffic at the layer you are protecting: origin requests, or a specific route family, or authenticated endpoints only. Running one global model over everything is the most common way this design gets built badly. It cannot tell a flash crowd from a flood on volume alone. A genuinely viral moment looks like an attack in every dimension except intent. The calendar feed handles the planned version; the unplanned version is what the infrastructure-load term in the scoring engine is for, since a real crowd that is not hurting anything gets scored down. Some residual risk stays, and you accept it consciously. It will happily poison itself if you let it. Today's mitigated attack is tomorrow's training data. If the mitigation windows are not excluded from the next training run, the ceiling creeps upward after every incident until the system stops catching anything. Test this explicitly: retrain on a month containing three mitigated incidents, with and without exclusion, and compare the ceilings at the relevant times of day. 13. Where to start on Monday The full design took us the better part of two quarters, but the useful parts arrive much earlier than the finished system, and in this order: Counters and history. Per-minute request counts by hostname into Redis, daily Parquet into object storage, dimensions parsed at ingest. Nothing else works without this, and it is valuable on its own the first time somebody asks what traffic looked like last Tuesday. One Prophet model, one hostname, in shadow. Publish ceilings, plot them against actuals for a fortnight, and adjust interval_width until coverage matches what you configured. You now have a working anomaly detector, and it took a week. The second window and the scoring engine. This is what turns a noisy detector into something you would let near production. Wire in origin CPU first; it is the single highest value context signal. Level 2 attribution, still in shadow. Excess attribution over single dimensions only, no pairs, no micro-models. Print the table from section 08 into the incident channel whenever the gatekeeper fires. Analysts will start using it before you have automated anything, which is a good sign. Automated rules, log-only, then throttle. Turn on enforcement for one predicate type you trust most, usually TLS fingerprint, with a low blast radius cap. Widen slowly. Steps one through three are perhaps three weeks of work for one engineer who already knows pandas, and they cover the case that actually wakes you up. The rest is refinement. None of this replaces your bot management vendor, and if the pitch inside your organisation is that it might, you will lose. The pitch is that the vendor's slow path is measured in hours, your incident is measured in minutes, and the gap between those two numbers currently has nothing in it but a pager and an analyst reading a dashboard. That gap is worth about three weeks of engineering time, and it is one of the few security projects where you can put a defensible number on what it buys you.