Families can now set simple moderation rules for a child: block links, and flag a list of words an admin would rather know about. It’s intentionally modest, and it runs in an unusual place — on the device, against the already-decrypted message, never on our servers.
Where the checking happens
Most moderation is server-side, because that’s where most services can read your messages. Ours can’t, by design, so server-side checking was never an option that fit. Instead, the rules live as an encrypted policy: an admin’s word and link rules are encrypted and stored as opaque ciphertext, the same as any other content. A child’s device fetches and decrypts that policy and does the checking locally, at the moment a message is read or sent, where the plaintext already exists.
When something matches, the admin gets an alert that deep-links straight to the message in question. What our servers record is deliberately threadbare: that an event happened, as a set of booleans, with none of the content attached. The Safety view an admin sees is reconstructed on their own device from the conversation they can already read — not from a log we keep.
A guardrail, not a lock
It’s worth being plain about what client-side moderation can and can’t be. Because the check runs on the device, a determined, technical teenager could in principle work around it. We’re comfortable with that, because this isn’t anti-tamper security against an adversary — it’s a guardrail for a young child who isn’t reading fluently yet, the digital equivalent of putting the bleach on a high shelf. It catches the accidental and the impulsive, which is most of what actually happens, and it does so without us reading anyone’s messages to manage it.
There’s a gentler cousin to this, too: instead of blocking a word outright, an admin can set words that simply ask a child to slow down and reconsider before sending. We gave that its own write-up in take a moment, because it’s a different idea — a nudge rather than a barrier.
Some honest limits
The rules are simple word and link matching, not a model that understands meaning, so they’ll miss cleverly-spelled things and occasionally flag something innocent. They’re per child, set by an admin, and they only protect conversations on devices that family manages. And they share the architectural limit of our encryption model, which we describe in full on the privacy page.
How this fits with everything else
Because moderation reads decrypted content on the device, it acts on a voice note’s transcript once one exists, and on a photo’s caption the same way. It sits beside admin oversight and active hours in the family settings; the moderation feature page has the details.
Try Shoal, and tell us what it’s catching, and missing, at [email protected].