Ransomware Recovery at the Filesystem Layer

In the last piece I wrote about closing the gap between "attacker gets in" and "attacker detonates" — using a kernel-level access record to see anomalous behavior in real time, and an append-only archive to shrink what's even reachable for a mass-encryption routine to find. That piece was about prevention and detection. This one is about the morning after. Because even with good detection, something eventually gets encrypted, deleted, or corrupted — a hot-tier file, a misconfigured share, a process that slipped past the anomaly threshold before it was throttled. And when that happens, the thing that actually determines how bad your day is isn't your defenses. It's your recovery process, and specifically, how long it takes. That number gets far less attention than it deserves. Two Numbers, One of Which Everyone Ignores Disaster recovery planning usually centers on Recovery Point Objective (RPO) — how much data you can afford to lose, expressed as a time window. Nightly backup, 24-hour RPO. Hourly snapshots, 1 hour RPO. It's a clean number, it's easy to put in a compliance document, and it's the number vendors lead with. The number that actually determines how long your business is degraded is Recovery Time Objective (RTO) — how long it takes to get back to a working state once you've decided to recover. RPO tells you how much you lost. RTO tells you how long you're down while you get it back. For a lot of organizations, RTO is the one nobody has actually measured, because nobody has run a full-scale restore outside of a tabletop exercise. That's a problem, because RTO doesn't scale the way people assume it does. Why "Restore Everything" Is the Default Traditional backup systems are usually organized around recovery points rather than individual filesystem events. Many can perform granular file restores, but the recovery workflow still begins by identifying the correct backup or snapshot and then resolving the affected files within it. Most backup architectures — nightly full-to-NAS, snapshot and replicate, tape rotation with a scheduled job — store data as a small number of large, opaque containers: a snapshot, a backup set, a tape written sequentially. That's a reasonable trade-off for the write path. It's a bad trade-off for the read path, because it means the backup system has no concept of "restore just the 40 files that were actually touched." It only knows how to restore the container. So in practice, incident response defaults to restoring far more than was actually damaged. Even when an investigation shows the blast radius was a few hundred files in two directories, the recovery runbook says "restore last night's snapshot" — because that's the unit the system understands. You either restore everything up to a point in time, accept everything since then is gone, or you spend investigator-hours trying to do a selective restore that the tooling wasn't built for. Two things fall out of that: Restore time scales with total dataset size, not incident size. A ransomware event that touched 0.1% of your files still triggers a restore workflow sized for 100% of your files, because that's the granularity the backup format offers. Determining "how far back" is itself slow and uncertain. Before you can restore anything, you have to figure out which backup predates the attack — and if the attacker spent two weeks in reconnaissance before detonating, some of your recent backups may already be contaminated. That determination is usually manual: cross referencing logs, sampling files, guessing conservatively and going back further than you probably need to. When the recovery workflow is snapshot centric, the operational process is tied to the recovery point and its storage layout even when only a small subset of files ultimately needs to be restored. With an addressable archive, the system can identify the affected files first and retrieve only those versions. Neither of these is a tooling bug so much as a structural consequence of treating backup as periodic snapshots of an undifferentiated blob. What Changes When Recovery Is Addressable The fix isn't a faster restore job. It's changing the unit of recovery from "the whole snapshot" to "the specific file, at a specific version, identified precisely." That requires two things to already be true, which is why this follows naturally from the access-record and archive design in the last article: A continuous, queryable record of what happened to which files, and when. Not a nightly log — an event-level record, so "what got touched between 03:14 and 03:41" is a query you can run in seconds, not a reconstruction project. A versioned, per file archive rather than a monolithic backup blob. If cold data is stored as individually addressable, checksummed units — rather than baked into one big sequential container — then restoring one file doesn't require touching, reading, or re-hydrating anything else. Put those together and recovery stops being "restore the container and hope" and becomes "query the record, resolve the affected paths, pull back just those versions." The rest of the filesystem — the overwhelming majority of it, usually — was never touched by the restore process at all, because it was never touched by the attack. This is the same shift that shows up any time you replace a full table scan with an index: the operation goes from being proportional to total data volume to being proportional to the size of the actual query result. Why Archive-Tier Files Recover Almost Instantly Ransomware doesn't pick targets randomly. Attackers doing reconnaissance tend to go after old, rarely touched files first, for reasons that make sense from their side: those files are less likely to have an active user notice something's wrong right away, less likely to be watched by an application-level integrity check, and disproportionately likely to be the kind of thing worth holding for ransom in the first place — financial records, contracts, years of accumulated compliance data. "Nobody has opened this directory since last year" is exactly the property that makes a file both a good target and a slow one to get caught touching. That behavior collides usefully with tiered storage. In an architecture with a hot tier and an archive tier — the kind described in the first piece, where cold data is pulled off the live filesystem into a versioned, checksummed archive and left behind as a lightweight stub — a file that's been sitting untouched for a year is, by definition, already off the hot tier. When a ransomware process accesses a stubbed file and the access follows the normal read/modify/write path, what's actually happening under the hood is: the read triggers the stub, the clean data streams back to the hot tier, and the file is modified and remains hot tier. The original bytes from a year ago were never modified in place. They're still sitting exactly where they were, checksummed and intact, because the archive format never allows in-place rewrites to begin with. The modified, encrypted copy now lives on the hot tier, and won't be re-archived until it meets your standard aging rules (e.g., 30, 60, or 90 days). Which means recovery for that file isn't really a restore. You simply delete the corrupted hot-tier version and issue a pointer update—telling the catalog to recreate the lightweight stub pointing back to the pristine cold-tier version. No bytes move, nothing has to be read back and rewritten, no volume has to be mounted. For the catalog lookup, recovery can be effectively immediate. The actual recovery time still depends on where the clean version lives and how much data has to be retrieved. That also matters on the impact side, not just the speed side. If a file hasn't been touched by an actual person in a year, rolling it back to last year's version inconveniences precisely no one. Nobody had it open, nobody was mid-edit, nothing downstream was depending on today's version existing. The rollback is invisible to the business in a way that rolling back an actively-used file never is. The Same Logic Covers Backup Targeting This also closes the loop on the scenario from the first article, where a patient attacker goes after the backup infrastructure itself before detonating. Because the archive tier inherently handles cold data, your actual backup system only needs to capture the active, hot-tier data (files that are live and haven't yet met your 30/60/90-day archiving rules). But what if the attacker encrypts those backup files? If your backup target is integrated with the filesystem and set to auto-archive, an attacker "targeting the backups" is mechanically just generating a new version. The old, clean backup copy is safely swept into the append-only archive, still checksummed and a pointer-update away. The massive advantage here is speed and size: because your backup is only capturing the hot tier—say, the 20TB of actively modified files rather than a full 1PB dataset—recovering the live data from that backup is incredibly fast. That makes the system substantially harder to destroy through ordinary file-level encryption—though it doesn't eliminate the need for offline or immutable copies. The Backup Bill You're Not Paying There's a cost dimension here that's easy to overlook. Most backup tooling is priced and provisioned against total data footprint — per terabyte, per agent, per host — because it has to treat everything the same way on the same schedule, whether or not any of it has been touched in years. A hot-tier/archive-tier split changes what "everything" means for that purpose. If the archive itself is independently durable and replicated, you don't necessarily need to duplicate the entire archive through a second conventional backup system. Instead, the backup strategy can focus on the active tier while the archive's own replication and integrity mechanisms provide durability for cold data. That's the difference between licensing, storing, and transferring 20TB versus 1PB. And it's not just a cost line: a "restore the backup" event now means moving 20TB instead of 1PB, which is the difference between a restore measured in minutes and one measured in days, for reasons that have nothing to do with cleverness and everything to do with simply not needing to move the other 98% of the data at all. A Generic Recovery Workflow Stripped of any specific tooling, a recovery process built on an addressable record and archive looks roughly like this: Establish the incident window. Use the access record to find the first anomalous event — first mass-open pattern, first unexpected process touching files it's never touched before, first burst of writes with a suspicious extension. This gives you a timestamp, not a guess. Enumerate the blast radius. Query the record for every file touched after that timestamp. This is a filtered list, not a full directory walk — usually a tiny fraction of total files. Resolve last-known-good version per file. For each affected path, look up the most recent version recorded *before* the incident window in the archive index. This can be performed with a python script. Stage what's needed to read it back. Depending on where cold data lives, this might mean nothing at all (disk- or cloud-backed archives), or it might mean pulling specific physical volumes back online (tape). Either way, you only need the subset of media that actually holds the affected files' clean versions — not your entire archive. Roll back file-by-file, verifying integrity as you go. This is where the hot-tier/archive-tier split pays off. For cold-tier data, the rollback is nearly instantaneous: you simply drop the encrypted hot-tier copy and reinstate the stub pointing to the pristine archive version. For data that was actively hot (and therefore hadn't met the archiving rules), you restore from your hot-tier backup. Because that backup is safely auto-archived itself, and because it only contains the tiny percentage of your total data that is actively hot, this restore takes a fraction of the usual time. Either way, files are checked against stored checksums so you aren't trading "encrypted garbage" for a "silently corrupted restore." Reconcile and audit. Confirm the restored set matches the enumerated blast radius, preserve the contaminated originals for forensics if needed, and close the loop with a record of exactly what was changed and when. None of these steps require guessing how far back to go. The record already told you. The Time Math, Roughly The honest way to frame the savings isn't "recovery is instant" — it's "recovery time stops scaling with total data volume and starts scaling with incident size," which for most real-world incidents is a very different number. Traditional snapshot restore Addressable, record-driven restore Unit of recovery Whole snapshot/backup set Hot Tier Backup + archive versions Time to determine scope Hours to days (log correlation, sampling) Seconds to minutes (indexed query) Time to restore a hot-tier file Scales with total dataset size Scales with blast-radius size Time to restore an archive-tier file Same as any other file — full restore path Effectively zero — a catalog pointer update, no data movement Data volume actually backed up / restored Full dataset (e.g., 1PB) Hot tier only (e.g., 20TB) — the archive tier is already self-durable Untouched data Re-read/re-verified as part of the restore Never touched Typical outcome for a contained incident Hours-to-days downtime, best case Minutes-to-hours, best case The gap widens as datasets grow. A snapshot restore of a few hundred gigabytes is annoying but tolerable. A snapshot restore of tens of terabytes, when only a few thousand files were actually affected, is where organizations end up paying a ransom simply because restoring cleanly would take longer than the extortion deadline What This Doesn't Fix Worth being direct about the limits, because this isn't a universal fix: Hot-tier data that hasn't been archived yet is still bound by whatever your normal backup cadence is. If a file was created and encrypted between backup runs and never made it into the versioned archive, this approach doesn't magically recover it. It shrinks the exposed window; it doesn't eliminate it. It doesn't replace an incident response plan. Knowing exactly what to restore doesn't help if nobody knows who has the authority to pull the trigger, or where the affected media physically is. It doesn't replace testing. A recovery process you've never run end-to-end is a hypothesis, not a plan. The value of query-driven recovery is that it's cheap enough to actually rehearse regularly, which most organizations don't do with full snapshot restores simply because of how disruptive and slow they are. Closing I'm biased here, since I work on this problem full time, but this is exactly the pattern we built into HuskHoard— an open-source (AGPL v3) active archive for Linux that pairs a fanotify-based event record with a queryable, versioned catalog, so recovery is a SQL query against known-clean file versions rather than a guess about which snapshot to trust. If you want to see the pattern above implemented end-to-end, including support for disk, cloud, and LTO/WORM tape targets, the source is on GitHub.