← The canon · AItopiaOrAImageddon?

Filtering at national scale — the largest content-classification systems ever deployed, and what they cost to run

moment · The Ministry of Public Security, which began the Golden Shield in 1998; British Telecom and the Internet Watch Foundation, whose CleanFeed went live in June 2004 and became the template every democracy copied; and the measurement community without which none of it would be knowable — Richard Clayton, Steven J. Murdoch and Robert N. M. Watson, who took the Great Firewall apart in 2006; Tao Zhu, David Phipps, Adam Pridgen, Jedidiah R. Crandall and Dan S. Wallach, who timed it in 2013; Gary King, Jennifer Pan and Margaret E. Roberts, who established what it is for; and the Citizen Lab, the Great Firewall Report and Censored Planet, who have kept measuring it for twenty years · 2009

Something that happened and changed what people expected next.

Descends from The TCP/IP cutover — federation without a centre. Read on: March 2020 — the week the platforms told the world their machines would be doing the judging.

On the kind. Filed under idea in proposals.md — the id sits at line 614, inside the idea block (lines 195–721) with the rest of the batch appended on 2026-08-17 — and written here as moment. The proposal's own words settle it against idea: it asks for "censorship at national scale as an engineering problem," for something "descriptive and dated," and warns that "the entry is worthless if it becomes an argument rather than a record." An idea in this canon is "a framing later work is built out of rather than argued about," and the house test — set by shannon-chess-1950, reused by dijkstra-1959 — is that there is a mechanism in it you could implement. There is no single mechanism here. There are five or six, deployed by different states at different times, and the entry's value is the record of what they achieved and what they cost.

moment is defined as "something that happened and changed what people expected next," and something did. The year in the header is 2009, and it is chosen rather than inherited. Not 1998, when the Golden Shield was approved and nothing was yet demonstrated. Not 2006, when Cambridge researchers published the Great Firewall's mechanism and showed it could be defeated with a one-line firewall rule — that was the year it was proved weak. 2009 is the year it was proved sufficient: a state removed the world's largest communication platforms from a fifth of humanity and cut a province of roughly twenty-one million people off the internet for ten months, and both decisions held. The platform blocks are still in force in August 2026, seventeen years later. Everything after 2009 in this entry is refinement of a question that stopped being open in 2009.

The one honest cost of the choice: this entry runs from 1993 to 2026 and a single year in the header flattens that. A reader should treat 2009 as the hinge, not the span.

---

What it is

Between roughly 1996 and today, a dozen states have built and continuously operated systems that inspect the network traffic of entire populations and decide, per packet or per request, what may pass. These are the largest automated content-classification systems ever deployed. They run continuously, against a population that adapts, at a scale no other classifier operates at, and — this is the part that matters to a project watching AI — almost all of it was built and proved out before modern machine learning had anything to do with it.

The largest is China's. The Golden Shield Project (金盾工程) was approved by the Ministry of Public Security in 1998 and its first phase ran to the mid-2000s; the border-router apparatus that English speakers call the Great Firewall is one part of it. By the end of 2025 it stood between about 1.30 billion internet users and the rest of the network. But the technique is not Chinese and never was. Germany's North Rhine-Westphalia ordered DNS-level blocking of far-right material in 2002. British Telecom's CleanFeed went live in June 2004. Norway's Telenor and KRIPOS started in October 2004. Saudi Arabia ran a national web proxy. Pennsylvania tried and was stopped by a court in September 2004. What changed between then and now is not the idea but the scale, the fidelity, and — recently — the layer at which the filter sits.

The mechanisms, in the order they were deployed

Packet dropping (layer 3). Keep a list of IP addresses; discard traffic to them. Trivially cheap, and the reason it is not enough is arithmetic: Edelman's survey, cited in both of Clayton's papers, found that 87.3% of .com/.org/.net websites shared an IP address with at least one other site and 69.8% shared with fifty or more. Blocking one site by address blocks its neighbours.

DNS poisoning. Answer the lookup wrongly or not at all. No neighbour-blocking, but it takes out every service on the domain, not just the web, and it is easy to get wrong. Dornseif's study of the North Rhine-Westphalia orders found that of 27 ISPs, only 15 (56%) blocked stormfront.org as ordered, only 12 (44%) left the unlisted kids.stormfront.org reachable, and every single one incorrectly broke email to stormfront.org addresses. Every ISP in his sample made at least one implementation mistake.

Content inspection by proxy. Route everything through a cache that understands URLs. Precise, and expensive enough that no one does it for a whole country. CleanFeed's design was the clever compromise: a two-stage hybrid, where BGP inside BT's customer-facing network redirected traffic only for IP addresses on the IWF list to a small HTTP proxy, which then checked the specific URL against a hashed copy of the list and returned a 404. Cheap first stage, accurate second stage.

Inspection by IDS with forged resets. This is the Great Firewall's classic mechanism and it was fully documented in 2006. Clayton, Murdoch and Watson found that the border routers pass packets through untouched while copying them to an out-of-band intrusion-detection device; on a keyword match the IDS generates forged TCP RST packets — three of them, at sequence numbers +25, +1485 and +4405, to defeat window checking — aimed at both ends of the connection. Their trigger string was falun, sent as GET /?falun. Blocking of that endpoint pair then persisted, in their measurements, for a "somewhat variable" period "[s]ometimes… only… a few minutes, yet at another time… most of an hour. The average value was around 20 minutes."

Stateful SNI and Host-header filtering. Once the web moved to TLS, the censor moved to the handshake. The GFW now keeps TCP state, reads the Host header or the TLS Client Hello's Server Name Indication, and tears the connection down with RST/ACK injection to both ends. Russia's throttler triggers on the same field.

Traffic-shape classification with no keyword at all. On 6 November 2021 the GFW began blocking fully encrypted traffic — traffic with no plaintext to match. It does this backwards: rather than recognising circumvention traffic, it applies five crude exemption rules and blocks whatever is left. The rules are a popcount test (popcount(pkt)/len(pkt) ≤ 3.4 or ≥ 4.6), three tests on the fraction, position and contiguous run of printable ASCII bytes, and a TLS/HTTP fingerprint match. This is the closest thing in the whole record to a statistical classifier, and it is five inequalities over the first TCP payload.

Hash and OCR matching inside the application. WeChat filters images in realtime by two methods documented by the Citizen Lab in July 2019: OCR the image and match extracted text against a keyword blacklist, and match a visual fingerprint against an image blacklist, implemented as "a hash index populated by MD5 hashes of images sent by users." Neither sender nor recipient is told. The filter is populated from user traffic.

The filter in the model weights. In May 2025 Naseh, Chaudhari, Roh, Wu, Oprea and Houmansadr documented what they call local censorship: DeepSeek's R1 refuses a curated set of about 10,030 politically sensitive prompts, and the refusal "is not only present in the online chatbot version but also embedded in the base model distributed for local use." They report it as "a first instance of local censorship alignment being applied at the model level, persisting even when the model is deployed privately." The filter no longer needs the network.

What the record actually shows

Four findings recur across twenty years, different countries, different equipment.

One: these systems have never inspected everything, and cannot. In 2006 Clayton et al. sent bursts of trigger packets to blocks of 256 consecutive addresses and found that "[i]nitially, about two-thirds of each set of packets were blocked… The most likely explanation is a lack of resources — two-thirds of the traffic may be all that the content scanning system can handle." Fifteen years and several hardware generations later, the encrypted-traffic classifier of 2021–23 "strategically only monitors 26% of connections." The QUIC censorship deployed in April 2024 is, per its measurement paper, "less effective during high-traffic periods, suggesting computational overhead limitations." The largest filtering apparatus ever built samples. It has always sampled.

Two: they run on enormous quantities of human labour, and the automation exists to ration it. The arithmetic is in the 2013 Weibo paper and it is worth quoting because people consistently guess wrong in both directions. Weibo took roughly 70,000 posts per minute. "Suppose an efficient worker could read 50 posts per minute… 1,400 simultaneous workers would be needed. Assuming 8 hour shifts, 4,200 workers would then be required… We instead conclude that Weibo must be using a large amount of automation, perhaps keyword-based." That is the shape of the whole thing: automation to make a human-scale queue, humans to decide. King, Pan and Roberts, who in 2014 contracted with Chinese firms to install real censorship software on their own site, found roughly 40% of submissions held for review and more than half of those never published — the automated stage is a hold, not a verdict. By January 2019 one Beijing contractor, Beyondsoft, ran content-review factories with over 4,000 staff, up from about 200 in 2016, and trained new hires in the history they were being hired to erase.

Three: accuracy is not what these systems optimise, and they work anyway. The encrypted-traffic rules "could potentially block about 0.6% of normal Internet traffic as collateral damage" — a false-positive rate no commercial classifier would ship, accepted because the alternative is letting circumvention through. Russia's Twitter throttler, in its first days, matched the string t.co as a substring and so throttled reddit.com and microsoft.com; the bug was reported on 10 March 2021 and fixed on 11 March. Weibo's keyword filter in 2012 held a post containing youshenmefalundebanfa for five hours. Margaret Roberts' account in Censored (2018) is the one that makes sense of this: strong states can filter far harder than they do and choose not to, because the mechanisms that work are fear, friction and flooding — and friction works by imposing a small cost on the median person, not by being correct. A filter that is wrong 0.6% of the time and absent 74% of the time still changes what a billion people read.

Four: the censor maintains the system, and that is why it still works. Clayton's 2006 CleanFeed paper closed with a prediction: "Thinking of these systems as 'fit-and-forget' arrangements will be a guarantee of their long-term failure." Read GFWeb, eighteen years later: over 20 months from February 2022 to September 2023 it tested 1.02 billion domains and found 943K and 55K pay-level domains censored by the HTTP and HTTPS filters — and reported that "the GFW has been upgraded to mitigate several issues previously identified by the research community, including overblocking and failure in reassembling fragmented packets." The censor read the papers. That is the single most important fact about the arms race and the one most often left out.

The dated spine

---

Why a reading would cite it

LENSES.md sends every reading looking for capability that is claimed versus capability that is demonstrated, and for concentration — who owns the capability. This entry is the largest body of demonstrated automated-classification capability in existence, and it predates the thing the project watches. Six concrete occasions:

1. When automated content moderation is described as new, or as an AI problem. It is neither. The oldest continuously operating systems here are twenty-two years old and their core is keyword matching, IP lists and hash indexes. A reading that treats large-scale automated classification as something that arrived with transformers is wrong by two decades, and the correction is not pedantry: it means the governance questions — who lists, who appeals, who audits — have twenty years of answered and unanswered precedent attached, and none of it required a model.

2. When someone asserts that filtering at scale is infeasible. It is feasible, it has been feasible since 2009, and the standing proof is that Facebook and Twitter have been unreachable to over a billion people for seventeen years. tcp-ip-1983 already grades the "uncontainable" claim as false and cites this regime to do it; this entry is the evidence behind that grading, at the level of mechanism.

**3. When someone asserts the opposite — that filtering at scale is now easy, or that AI makes it easy.** The record says the cost is permanent and structural. Every measured system samples rather than inspects (two-thirds of addresses in 2006; 26% of connections in 2021–23; degradation under load in 2024–25), every one carries a large human workforce behind the automation, and every one has been maintained continuously for decades by people reading the papers written about it. This is the base rate for what a state-scale classifier costs, and it is much higher than the "just run a model over it" framing implies.

4. When accuracy is being argued about. The most useful single number in this entry is 0.6% — the estimated collateral-damage rate the GFW accepts on ordinary traffic to catch circumvention. Set against it: Australia's regulator measuring 81% of under-16s retaining or creating accounts three months into a ban, and Britain's Pornhub traffic falling from 3.2 million to about 2 million while VPN sign-ups rose by four figures in percentage terms. Both directions of error are large, both are tolerated, and the systems function. Any reading that treats a filter's error rate as the measure of whether it "works" is using the wrong test — friction, not accuracy, is the mechanism.

5. When a state's demand lands on a private classifier. This is now the common case and 2026 is thick with it: the EU's CSA Regulation, where the Council's November 2025 position dropped mandatory detection orders and the Parliament's July 2026 vote left voluntary warrantless scanning in place; India's Sahyog portal, upheld by the Karnataka High Court on 24 September 2025 on the ground that a foreign company cannot claim Article 19 protection; China's draft cybercrime law reaching overseas providers. When a reading has to describe a government requiring a company's model to classify something, the shape of that arrangement is here, dated, with outcomes.

6. When the filter moves into the weights. This is the occasion that makes the entry belong to an AI tracker rather than a networking one. R1dacted (19 May 2025) documents refusal behaviour surviving into privately deployed copies of a model, and China's labelling measures (effective 1 September 2025) put obligations on the generator rather than the carrier. A filter in the network can be measured from outside by anyone with two vantage points — which is why this entry has twenty years of hard numbers in it. A filter in the weights cannot. The measurement literature that made national filtering legible does not straightforwardly transfer, and a reading that notices a model-level restriction should say plainly that it is harder to verify than the network-level ancestor, not easier.

Where a reading must not cite it

Not as evidence about any specific present claim. That a state can filter at scale is not evidence that a particular platform is filtering, that a particular model was trained to refuse, or that a particular removal was ordered. The entry establishes feasibility and cost. It settles nothing about a given week's events, and using it that way would be exactly the move LENSES.md rules out when it says "experts warn AI could sway elections" is not a finding.

And not as a needle input. Nothing in canon/ is evidence and this entry least of all: it describes capabilities built mostly before the period this project measures.

---

What it got right, and what it got wrong

Six dated claims, graded against what happened.

Claim 1 — "The Net interprets censorship as damage and routes around it." Attributed to John Gilmore, quoted in TIME, 6 December 1993. Open-ended. False as a claim about states; true as a claim about protocols; and the misattribution of scope is the whole error.

Gilmore was talking about Usenet, where a refusing site is genuinely routed around because every other site carries the same articles. He has since said he cannot find where he first said it. Applied to a state that owns the border routers, it is simply wrong: routing around requires a route, and the party that filters is the party that has the routes. The honest residue of the claim is real and smaller — the architecture offers no single chokepoint, so the filtering party must be someone who owns infrastructure, which is why national filtering is a state-and-carrier activity and not a hobbyist one.

Claim 2 — "Now, there's no question China has been trying to crackdown on the Internet. Good luck! That's sort of like trying to nail Jell-O to the wall." President Bill Clinton, Paul H. Nitze School of Advanced International Studies, 8 March 2000. No due date stated; take a decade. Wrong, and by 2010 it was already unambiguous.

The claim's premise — that internet access and political liberalisation are the same process, and that the technology forecloses the policy — was the governing assumption of a generation of Western technology policy, and it was tested to destruction. By March 2010 Facebook and Twitter had been blocked for eight months, Xinjiang had spent most of a year offline, and the mechanism had been publicly documented for four years. Note what this entry cannot supply: the canon's discipline for graded claims asks for the predictor's own grade alongside an independent one, and I found no instance of Clinton grading this prediction. The grade above is mine, from the record, and it stands alone. A later session that finds a self-assessment should add it.

Claim 3 — "Thinking of these systems as 'fit-and-forget' arrangements will be a guarantee of their long-term failure." Richard Clayton, PET 2006, on CleanFeed. Open-ended. Held, and the confirmation is inverted from what a reader expects.

The prediction was a warning to filter operators. What happened is that the largest operator took it: GFWeb reports in 2024 that the GFW had been upgraded specifically to fix the overblocking and packet-reassembly weaknesses researchers had published. The systems that were fit-and-forget did fail — CleanFeed's oracle attack, the 2018 Telegram blackhole, the Australian filter proposal abandoned in 2012 — and the one that is continuously maintained is the one still standing. Clayton was right about the mechanism and, read as a forecast of which systems would survive, right about the outcome too.

Claim 4 — the companion claim in Clayton, Murdoch and Watson (2006): that ignoring forged resets makes the Great Firewall "entirely ineffective." True when written, obsolete by design change, and the obsolescence is the finding.

The demonstration was real and reproducible in May 2006. It is dead now. The apparatus moved to stateful inspection, to blocking rather than resetting, to SNI and QUIC and to traffic shape. This is the cleanest available illustration that circumvention results have a half-life: a technique published against a maintained adversary buys years, not decades. The same shape is already visible one layer up — R1dacted reports its jailbreak bypassing R1's local censorship in about 97.86% of samples, which is 2006's iptables one-liner transposed onto model weights, and should be expected to age the same way.

Claim 5 — Apple, 5 August 2021: on-device CSAM detection would flag an account incorrectly with probability of about one in one trillion per year. Due on deployment. Never tested, because it was never deployed — and the reason it was never deployed is the entry's real content.

Within roughly two weeks the NeuralHash function had been extracted from iOS and a collision exhibited. Strictly, that did not refute Apple's claim: the one-in-a-trillion figure was for the whole pipeline, which included a threshold of about thirty matches, a second undisclosed server-side hash, and human review before any report. Apple said as much. The system nonetheless died — paused on 3 September 2021, abandoned in December 2022 — and it died of the argument made in Bugs in our Pockets (Abelson, Anderson, Bellovin, Benaloh, Blaze, Callas, Diffie, Landau, Neumann, Rivest, Schiller, Schneier, Teague, Troncoso, 14 October 2021): that client-side scanning "neither guarantees efficacious crime prevention nor prevents surveillance," and that the objection is to the deployed capability rather than to its current accuracy. The grade: the arithmetic claim was not disproved and the project failed anyway. Any reading that meets a scanning proposal defended on its false-positive rate should notice that this is the case where the false-positive rate was never the argument.

Claim 6 — the implicit claim of Australia's under-16 ban (law passed November 2024, in force 10 December 2025): that platform-level age assurance can keep under-16s off social media. Due at three months. Failed on the regulator's own measurement, and the two grades differ — which is the point of recording both.

eSafety, the regulator that administers the ban, reported in August 2026 that 81% of under-16s had retained or created accounts by the three-month mark and that 58% reported daily use against roughly 60% before the ban. eSafety's own summary attributes this to platforms' failure to implement effective age assurance. The government's grade is different: assistant minister Andrew Leigh called the scheme "an important game changer," noted "[w]e never expected that this would have 100 percent compliance," and pointed to the roughly 4.7 million accounts platforms had deactivated by 16 January 2026. Both are describing the same three months. The deactivation number is real and the usage number is real, and a reading that quotes only one of them is doing the thing this canon exists to prevent.

The British comparison is worth carrying beside it, because it grades the same claim with different equipment. After 25 July 2025, Pornhub's UK traffic fell from about 3.2 million in July to about 2 million across the first nine days of August — roughly −47% — while Proton reported VPN sign-ups up by four figures in percentage terms within minutes and Ofcom's chief executive stated the Act cannot prevent VPN use. Front-door compliance is measurable and large. Substitution is measurable and immediate. Both are true at once, and that is the general finding of this entry stated in a single week's data. (Ofcom was due to publish an effectiveness report by June 2026; I did not establish whether it landed, and a reading that wants a verdict on the UK scheme should get it first.)

---

Commonly misused as

Not required for moment, and necessary here anyway: this is a subject where both sides of the public argument cite the same twenty-five years of engineering for opposite conclusions.

1. As proof that filtering at scale works. It works as friction. It does not work as classification, and the operators do not pretend otherwise — a censor that accepts 0.6% collateral damage and inspects a quarter of connections is not making determinations, it is raising a price. The distinction matters whenever someone proposes a filter whose justification requires it to be accurate.

2. As proof that filtering never works, because "people just use VPNs." Circumvention is real, measurable and priced. It requires knowing to want it, paying for it, keeping it working against an adversary who publishes patents on detecting it, and — under China's draft 2026 cybercrime law — accepting legal exposure for supplying it. Roberts' finding is that the population that circumvents is small and unrepresentative, and that the filter's effect runs through everyone else. The blocks of July 2009 have held for seventeen years against a population with universal smartphone access and a large technical class.

3. As "China does AI censorship." Mostly it did not. Keyword lists, IP lists, MD5 indexes, five inequalities over a packet prefix, and four thousand people in an office park in Beijing. The genuinely learned components are recent and narrow — OCR and perceptual hashing in the app layer from about 2018, traffic classification from 2021, and refusal behaviour trained into model weights from about 2025. A reading that flattens this into "AI censorship" loses the only interesting thing about the 2025 development, which is that it is the first time the filter has not been in the network.

4. As an equation between democratic content moderation and the Great Firewall — or as a categorical distinction between them. Both moves are available and both are wrong. The scale figures on the democratic side are genuinely enormous: an empirical study of the EU's DSA Transparency Database found 131 million statements of reasons in a ten-day window in November 2023, with several major platforms reporting fully automated decisions for most or all of them, and 99.8% classified as terms-of-service violations rather than illegal content. The equipment is often literally the same — SNI blocking, hash matching, DPI. What differs is the predicate, the appeal route, and whether the list is publishable. A reading should name which of those it is talking about rather than reaching for the analogy.

5. As a system that sees everything. No measured system ever has. This is the single most robust finding in the entry and the one that survives every hardware generation.

6. The one this room has to watch in itself. The temptation, given a canon entry full of verified numbers about state filtering, is to reach for it whenever a moderation story appears in a week's search results. The entry is a base rate and a mechanism catalogue. It is not evidence that any particular thing happened, and it does not move the needle in either direction — a reading that files it under armageddon because censorship is bad, or under utopia because the filters keep leaking, has used a record as an argument, which is precisely what the proposal warned against.

---

Sources

Verification status, stated once rather than hedged throughout. Six papers were read directly as rendered pages: Clayton, Murdoch and Watson (2006) in full; Clayton (2006) in full; Zhu et al. (2013), pages 1–6, covering the abstract, methodology, timing results and Hypothesis 1; Hoang et al. (2024), pages 1–2, covering the abstract and filtering-mechanism background; Xue et al. (2021), pages 1–2 and 4–5, covering the abstract, incident timeline, throttling mechanism and domains targeted; and Naseh et al. (2025), pages 1–2, covering the abstract, contributions and definitions. Every quotation attributed to those six is transcribed from that reading and is verified, including all packet-level detail, the 20-minute average block duration, the two-thirds resource finding, the 70,000-posts-per-minute arithmetic, the 4,200-worker estimate, the 1.02-billion-domain figure, the reddit.com / microsoft.com substring bug, and the 10,030-prompt and 97.86% figures.

Everything else below arrived through a fetch layer that returns a summarising model's rendering of a page, or through search summaries, and is marked where it is load-bearing. Two retrieval failures cost this entry material: Apple's own CSAM_Detection_Technical_ Summary.pdf no longer resolves — both the plain URL and the tracking-parameter variant return Apple's current child-safety marketing page — so the one-in-one-trillion claim and the ~30-image threshold are recorded here from secondary reporting and are the weakest numbers in the entry; and web.archive.org is unavailable to this tool, which removed the obvious fallback. usenix.org returned HTTP 403 on the GFWeb conference page; the authors' Stony Brook mirror was used instead.

Primary — the mechanism papers, read directly.

Primary — through the fetch layer.

Search summaries only — the weakest tier, listed so a reader knows which claims are thin.

In this canon. tcp-ip-1983 was read at length and is this entry's declared ancestor: its "Wrong, as usually stated: uncontainable" section already grades the claim that the 1983 architecture made control impossible, cites "a persistent national filtering regime covering a fifth of humanity" to do it, and carries the Access Now #KeepItOn figures (313 shutdowns across 52 countries in 2025) that belong beside this record. This entry supplies the mechanism behind that grading and should not be read as re-arguing it. social-credit-narrative was read in full and is the closest sibling — same state, same decade, opposite epistemic problem: there the finding is that a real system was widely misdescribed, here the finding is that a real system is widely described correctly and misused in both directions. mass-collection-2013 is the other sibling: same wires, same middleboxes, collection rather than blocking. content-farms-2016, cambridge-analytica-2018 and algorithmic-monoculture-2022 were checked by header only. README.md, LENSES.md, the canon-kind definitions in build.py and this entry's own proposal text in canon/proposals.md were read in full. covid-platform-moderation-2020 is on the proposals list, is the entry this one most wants to link to, and does not exist. When it is written, the automated-moderation-at-scale material in §4 of this entry is its inheritance, and the two should not duplicate each other: this entry owns the state-and-carrier record, that one should own the platform-and-government record.