AI Spam Filtering for Forms: What to Ask Before You Trust It
Form backends are adding AI classification of submissions. Asking a model beats keyword rules, but a model is never unsure out loud. Here is the failure mode, and the three questions to ask any vendor shipping it.
The ShipMyForm team
· 8 min read
Before you turn on AI spam filtering, ask one question: what does it do when it is unsure? A language model reading your submissions will beat keyword rules at deciding what a message is. It will not tell you when it does not know, because a classifier always returns an answer. If that answer becomes a verdict, you lose real enquiries and nothing on your dashboard says so.
Full disclosure: ShipMyForm is our product, and the failure below is ours. We shipped it, found it in production in October 2026, and fixed it on 30 October. Forminit figures were read off their pricing page on 2 November 2026, the morning the feature appeared.
What just shipped
Forminit — the service formerly called Getform, renamed in January 2026 — added AI Classify to every plan. A classify step asks a model up to ten questions about each submission, and later steps in the same workflow can route it, mark it as spam, or change its status based on the answers. The free plan allows one step per workflow; Pro and Business allow up to five.
It is a good feature, and the design is more honest than most: the questions are yours, so you can see what is being asked. We are not writing this to argue against it. We are writing it because we built the same idea, ran it across every customer, and learned something the marketing for this category does not mention.
Why asking a model really is better
The case that breaks keyword filtering is the one where spam and a real enquiry use identical words.
"Hi, I noticed your site isn't ranking for some obvious terms and wanted to ask about your SEO."
That is either a cold pitch or a prospective customer, and no amount of rule-writing separates them, because the difference is not in the vocabulary. It is in whether the sender wants something from you or is offering you money. A model reads that. A keyword list cannot, which is why keyword lists end up tuned until they either leak or eat your enquiries.
So on the question what is this message, classification is a genuine improvement and we would not go back. The problem is a different question.
A model is never unsure out loud
Ask a classifier whether something is spam and it returns a score. Ask it about a two-word message with no content and it still returns a score — a confident-looking number produced by a model that has nothing to go on.
Serious classifiers handle this with two thresholds rather than one: a low bar, below which something is clearly fine, and a high bar, above which it is clearly spam. Ours sit near zero and near 0.78, chosen so that calling something spam is right 99 times out of 100. The gap between those two bars is not a tie waiting to be broken. It is the model saying I do not know, and it is the single most important output it produces.
What happens to that gap is the whole ballgame. There are only three options:
- Resolve it with something slower. Send the uncertain cases to a bigger model, or a second signal, off the request path.
- Hand it to a human as a third outcome. Deliver it, but marked.
- Force it into one of the two hard labels. Pick a side and move on.
The third is the cheap one, it is invisible from outside, and it is what we did.
A spam filter with exactly two outcomes has no way to express uncertainty. Every genuinely ambiguous submission is being forced into "fine" or "spam", and you cannot audit the second bucket, because you never see what is in it.
What it cost us
Our classifier runs inline so the visitor is not kept waiting, and the uncertain cases are supposed to be re-examined afterwards by a language model. When that second pass did not run, the uncertainty band was resolved by splitting it down the middle and taking the nearest hard label.
Split a band that ends at 0.78 and the midpoint is around 0.41. So a submission the model had explicitly declined to judge was being filed as spam on a score roughly half of what our own calibration demanded. The 99%-precision figure we had measured did not apply to those decisions at all; it had been thrown away at exactly the point it was needed.
Then a second mistake turned a bug into a silent one. We re-checked submissions we had let through, and never submissions we had filtered. Read that back: the only class of error we never looked at was the one that loses real mail. A false positive, once made, was permanent and invisible by construction.
You could see the result without knowing any of this. A customer's dashboard read 225 spam, 138 delivered, 2 flagged over a month. A filter catching more than half of everything, with almost nothing in the middle, is not a filter that is working hard. It is a filter that has stopped reading messages.
Short, information-free messages — "test", "hello", "asdf" — give a content classifier nothing to work with, so they land in the uncertain band every time. Whatever your system does with that band, it does to you first. If you are debugging a form and your own submissions vanish, you have found the answer to the question at the top of this page.
What we changed
Three fixes, in the order they mattered.
A guess is now a different outcome from a verdict. When the uncertainty band is resolved by splitting it, the submission is marked flagged rather than spam. It is delivered, it is visible, and it carries a note saying the classifier did not actually decide. The owner sees it and judges it, which is the correct place for that judgement.
The review path opens both ways. Uncertain cases are re-examined whether we let them through or filtered them. This is the fix we should have had first, because it is the one that makes the other errors findable rather than theoretical.
The reason is recorded, not just the score. Every filtered submission now stores which layer decided — blocklist, heuristic, rate, classifier — and the specific signals behind it. A verdict you cannot interrogate is one you have to take on faith, and we had been asking customers to take ours on faith. Being straight about the state of this one: the recording is live for everyone, but the panel that displays it is switched on for a handful of accounts while we find out whether it reads as useful or as noise. It is not yet something we can tell you to go and look at.
One note on what is quoted above. The high bar is in this article because the arithmetic does not work without it, and the arithmetic is the point. The rest of the configuration — the low bar, the signals that feed the score, which checks run inline and which run after — is not here, because that is tuning information for anyone who wants to sit just under the bar rather than something that helps you choose a form backend. We would rather say where the line is than quietly leave the omission unexplained.
The three questions
These work on any form backend adding AI classification, ours included.
1. What happens when it is unsure? You are listening for a third outcome. "Deliver" and "spam" alone means every ambiguous case is being forced, and the forcing is invisible. A vendor that can describe a flagged or held state has thought about this.
2. Who pays for the uncertainty? If an uncertain submission is filtered, the cost lands on you and on the person who wrote to you, and neither of you is told. If it is delivered and marked, the cost is a few seconds of your attention. Those are very different products.
3. How does a wrongly filtered message get back, and how would you know to look? This is the question that separates a considered feature from a demo. Ask whether filtered submissions are re-examined, whether you can see the reason, and what in the interface would ever prompt you to check. If the answer is "review your spam folder", the honest translation is that false positives are your problem and you will be finding them by accident.
A high accuracy number answers none of these. Accuracy averages the two mistakes together, and you do not experience them equally: a spam message that gets through costs you a second, and a filtered enquiry can cost you a customer you never knew existed. Ask for precision on the filtered class, and ask what happens in the middle.
Where this leaves the choice
AI classification of form submissions is the right direction and we expect every form backend to have it within a year. Forminit shipping it is a reasonable thing to buy, and asking your own questions about each submission is a genuinely nice design.
Just do not read "AI-powered" as "solved". The hard part was never judging an obvious message. It is what the system does in the gap where it has no idea, and that is a product decision, not a model capability. We got it wrong for weeks with a correctly calibrated classifier and a 99% precision figure we had actually measured. The calibration was fine. We were discarding it precisely where it mattered.
For what it is worth on our side: uncertain submissions now arrive flagged and delivered rather than filtered, and the reason behind every filtered one is recorded even where the panel showing it is still being trialled. If you are debugging a filter that is eating real mail — ours or anyone's — when legitimate submissions are marked spam is the triage guide.
Frequently asked questions
- Is AI spam filtering better than keyword rules for form submissions?
- For deciding what a message is, yes, and by a wide margin. Keyword rules cannot tell a genuine enquiry mentioning SEO services from a pitch for SEO services, because the words are the same and only the intent differs. A language model reads intent, so it gets that case right. What it does not improve is what happens when the answer is uncertain, and that is where the losses come from. Treat the model as better at judging and no better at knowing when it should not.
- Why did an AI spam filter mark a real enquiry as spam?
- Most often because the submission fell in the band where the classifier was genuinely undecided and something resolved that band into a hard label anyway. Calibrated classifiers have two thresholds — a low bar below which something is clearly fine and a high bar above which it is clearly spam — and the gap between them is uncertainty, not a tie to be broken. Short, information-free messages land there constantly, which is why your own test submissions are the most likely thing to be filtered.
- What should I ask a form backend about its AI spam filtering?
- Three things. What happens when the model is unsure — is there a third outcome between deliver and filter, or only two? Who pays for that uncertainty, you or the person who filled in the form? And how does a wrongly filtered message get back to you, and how would you know to go looking? A vendor that can answer the third question has thought about false positives; one that only quotes an accuracy number has not.
- Does a high accuracy number mean an AI filter is safe to trust?
- No, because accuracy averages over both kinds of mistake and you do not experience them equally. Letting one spam message through costs you a second of annoyance. Filtering one real enquiry can cost a customer, and you will never know it happened. Ask for precision on the filtered class and for what the filter does when it is unsure; a single accuracy figure hides exactly the error you care about.
- How do I check whether my form backend is over-filtering?
- Compare the size of the spam bucket against the delivered one and look for a middle category. A filter reporting far more spam than delivered submissions, with almost nothing in between, is either under real attack or has stopped reading messages and started recognising senders. If there is no middle category at all, every uncertain case is being forced into one of two buckets, and the one you cannot audit is the filtered one.
Related guides
Why Real Form Submissions Get Marked as Spam (and How to Get Them Back)
Everyone tunes a form filter by asking how much spam it caught. The number that matters is the one you cannot see — the enquiries that never arrived. Here is what causes them, including a mistake we shipped ourselves.
Why Your Contact Form Suddenly Started Getting Spam
It is not personal and it is not targeted: a crawler found a <form> tag. Here is how that happens, why it starts on an ordinary Tuesday, and what each kind of spam is actually trying to get out of you.
Why Your Contact Form Emails Land in Spam (and How to Fix It)
Most form notifications land in spam for one reason: the From address is the person who filled the form. Here's why that fails authentication, and what to send instead.