You Do Not Have an Automation Problem. You Have a Data Problem.
Your automation fails because your data is inconsistent. Validate at entry, normalise first, route bad records to review. The tool is rarely the problem.
You Do Not Have an Automation Problem. You Have a Data Problem.
The automation keeps failing. You have rebuilt it three times. You have tried Zapier, then Make.com, then n8n. Each one works in testing and breaks in production. The logs show an error in the field mapping step.
The automation is not your problem.
Your data is inconsistent. Phone numbers in six different formats. Email addresses with trailing spaces. Company names that sometimes have “Ltd” and sometimes do not. Date fields that are sometimes a formatted string and sometimes a Unix timestamp and occasionally a blank.
No automation can reliably process data that is not reliably structured. The problem is upstream, not in the tool.
Why This Mistake Is So Common
When an automation fails, the instinct is to fix the automation. The error appears in the automation. The automation is where you are working. Naturally, that is where you look for the fix.
But the automation is only doing what the data allows it to do. If the data is inconsistent, the automation will be inconsistent. Fixing the mapping step addresses the symptom. Fixing the data source addresses the cause.
The businesses that build reliable automation infrastructure do two things that most automation builders skip: they validate data at entry and they enforce consistency at the source.
The Data Problems That Break the Most Automations
Inconsistent field formats. A CRM where phone numbers are stored as “07700 900123”, “+447700900123”, “07700900123”, and “7700900123”. Every format is the same number. Every format breaks downstream automations differently.
Mixed data types. A spreadsheet where a “revenue” column contains numbers (“50000”), currency strings (“£50,000”), and text descriptions (“N/A” for no revenue). An automation that tries to sum this column will fail on the strings or produce wrong results.
Empty required fields. Records where a field that downstream processing depends on is simply blank. Not null, not a placeholder, just absent. The automation encounters it, errors, and the whole run fails.
Duplicate records. The same customer in your CRM three times under slightly different names. An automation that processes CRM records processes that customer three times. The problem looks like an automation problem. It is a data hygiene problem.
Encoding issues. Names with accented characters (résumé, Müller, São Paulo) that break when passed through systems that do not handle UTF-8 consistently. The character corruption happens in the data layer, not the automation layer.
Fixing the Source, Not the Symptom
Validate at entry. If data enters your systems through a form, validate it before it is saved. Enforce phone number format at the point of collection. Strip leading and trailing spaces from email addresses automatically. Require date fields to use a consistent format.
Ask your AI: “Can you add input validation to my form before it submits? Phone numbers should match [your required format]. Email addresses should be stripped of spaces. All text fields should be trimmed. Show an error message if validation fails.”
Normalise before automating. If you are inheriting inconsistent historical data, run a normalisation step before your automation. A Make.com or n8n workflow that cleans data before the main processing workflow is worth building once.
Ask your AI: “Can you write a script that normalises the following fields in my database: phone numbers to the E.164 format, email addresses stripped of spaces and lowercased, company names with ‘Ltd’ removed and replaced with ‘Ltd.’ consistently? Run it once on existing data and then as a nightly cleanup on new records.”
Build defensive automations. Your automation should handle the data it will actually receive, not the data you hope it will receive. Add a validation step at the start of every automation that checks data shape and routes bad data to a review queue rather than failing the entire run.
The Data Quality Checklist Before You Build an Automation
| Check | What It Reveals |
|---|---|
| Sample 100 records from the source | See actual format variation before building |
| Check for nulls in required fields | Find records that will cause errors |
| Check numeric fields for non-numeric values | Find strings masquerading as numbers |
| Check date fields for format consistency | Find mixed formats before they break processing |
| Check for duplicate records | Find deduplication work needed before automation |
This takes thirty minutes and prevents hours of debugging after launch.
Frequently Asked Questions
How do I find data quality issues before they break my automation?
Run a sample query or export from your data source before building the automation. Look at 100 to 200 records manually. The inconsistencies that will break your automation almost always appear in this sample.
Should I clean historical data or just ensure new data is clean?
Both, in order. New data should be validated at entry so it comes in clean. Historical data should be normalised before automation runs on it, or the automation should handle historical format variation explicitly.
What is the best way to handle bad data in an automation?
Route it to a review queue rather than failing the run. Add an IF condition at the start of your workflow: if required fields are missing or wrong type, write the record to a “needs review” table and continue with the next record. This keeps the automation running while flagging problems for human review.
At what point should I hire a data engineer rather than fixing this myself?
When the data inconsistency comes from multiple source systems that have been operating independently for years and fixing it requires transformation logic more complex than a script. At that scale, a data pipeline built by someone who specialises in this is more reliable than a patchwork of automation fixes.
The One Thing to Remember
Automation reveals data quality problems; it does not create them. If your automation fails inconsistently, inspect a sample of the data before rebuilding the automation. Validate at the point of entry. Normalise historical data before running automations on it. Build defensively so bad data is routed to a review queue rather than failing the run. The automation is almost never the problem.
Want your data processing workflows running reliably even on inconsistent inputs? → Snapdock
New here? These might help: n8n didn’t break. It succeeded at doing the wrong thing. → You didn’t ship an automation. You adopted something that dies quietly. →