Rob's Notes 17: A Custom Filtered Future?
Are we ready for an even more fragmented online experience?
Yesterday, Meta announced a major shift in its content moderation strategy, moving away from broad automated enforcement to focus primarily on illegal and high-severity violations like terrorism, child exploitation, drugs, fraud, and scams. Under the new approach, Meta will only act on less severe policy violations after user reports, reduce content demotions, and require higher confidence levels before removing content.
Source: DALL-E
Over the past decade, platforms have invested heavily in a combination of automated and human moderation, attempting to keep out varying levels of actual (or possible) abuse. At the same time, governments globally have been ramping up their rules/enforcement on handling content issues (and implementing penalties, often based on GLOBAL and not just local revenue!) notably the Digital Services Act (DSA) in the EU but also their legal actions and inquiries.
For a while it has felt inevitable to me that amidst more red tape and requirements, we’d reach somewhat of a watershed moment where platforms start to retreat and shift their resources towards JUST what their imperfectly-informed external critics were requiring… That moment appears to have arrived.
Perhaps it is possible Meta decided it no longer had the apparent “luxury” of doing anything optional - even if that meant allowing some user harms in the process. Of course there is a danger though, that this action will make regulators even more eager to force Meta and other platforms to act on that potentially-harmful content absent them making additional progress.
That is why I think there’s a possibility we’ll see platforms fill this gap by putting the power of filtering directly into users’ hands.
A Default Set of Rules—Plus Custom Filters
In the near future, we’ll likely see a baseline set of rules in place to address mostly illegal content and content that pushes people off of platforms and reduces engagement. These might be industry-wide requirements or platform-specific policies that handle the worst of the worst (child exploitation, explicit illegal content, etc.). Once that baseline is enforced, though, there’s room for individuals to opt into more specialized filtering solutions that could be offered by platforms directly but also by third parties. In being able to quickly understand text, images and video, LLMs have recently made the costs of doing this come down by several orders of magnitude and still falling - meaning almost anyone could publish their own filter/model/algorithm.
If you like the App Store analogy, you could have a kind of “filter store,” where you can choose from any number of filters designed and curated by experts, nonprofits, or companies big and small. Instead of relying solely on a giant tech platform’s one-size-fits-all moderation, you’d subscribe to filters that match your preferences. Whether you’re sensitive to certain topics, want to avoid sensationalist headlines, or only want to see content rated for family viewing, these filters could be one-time or subscription purchases, or shared freely.
This will have obvious downsides - it’s already hard to know what’s happening inside of anyone’s feed as it is; it’s possible in a future personally-curated world that not even the platforms themselves would know or understand what a given user was seeing. It also has the same problems with fraud, abuse, and variable security and privacy protections that today’s App Stores have.
This concept of personalized filtering has long featured in science fiction including Neal Stephenson’s “Fall; or, Dodge in Hell” from 2019, which warns about echo chambers and the erosion of a shared reality. Back in 2015, I wrote about this topic as it pertained to blocking irrelevant ads and spam.
Is there a Market for “Safety”?
One of the most interesting possibilities here is the emergence of a market for filters, where individuals and organizations can produce custom sets of rules and algorithms. These “client-side” solutions would reframe trust and safety as a shared responsibility between platforms, which keep out the truly unlawful or harmful, and users, who can choose which additional filters align with their personal values and preferences. There are some places where this becomes trickier, like ads - because of the monetary incentives involved - but I could also see hybrid subscription/ad models emerging which give users a greater deal of control over type and frequency of ad as part of a future “filter market”.
Challenges and Criticisms
There are plenty of problems and complications with these ideas:
Technical challenges: Either these would live inside of APIs provided by a platform or on a device (or some combination). TBD how to actually make this happen efficiently given current infra. Phones have a lot of computing power but aren’t really set up YET to run powerful-enough models. There are privacy and security concerns too that could limit how much data is shared between users themselves let alone with platforms. .
Fragmentation: If users apply drastically different filters, do we lose a sense of shared reality? Will each of us be sealed into our own bubble?
Filter Quality & Transparency: Who sets the standards for a filter’s accuracy or fairness? Will competing filter vendors race to the bottom in terms of quality or subtle censorship?
Implementation: Platforms will still need to enforce a baseline. It’s unclear how easily third-party filters can slot into existing moderation technology or platform technologies whether through APIs or other mechanisms.
Regulatory Concerns: Governments might have an interest in ensuring certain content is universally blocked or displayed. Balancing user control with legal requirements will be complex.
Beyond Centralized Moderation
Despite these hurdles, the move toward personalized “client-side” moderation can enhance user agency. Rather than trusting a single algorithm designed by a handful of tech companies to shape our online experience, we become the curators of our digital worlds—subscribing to the sets of rules and filters that best reflect our needs.
And as filters become more intelligent—with AI that learns from your behaviors in real time—they won’t just block the negative but also enhance the positive. Just as spam filters transitioned from basic blocklists to complex predictive models, future client-side safety tools could highlight beneficial conversations, resources, and opportunities tailored to each user’s unique perspective.
Platforms will likely still bear the responsibility of enforcing core rules against illegal content, but real innovation may be possible in user-driven, client-side filters that give each of us the power to choose how we want to engage with the internet.