Disclaimer: Data Contracts are one of the concepts of transformation tools for large organisations. The following description may contain details of the solution that will not fit every company.
In every large company, sooner or later, the question arises: is our data ready for AI?
And then, even faster, the answer comes: we have everything in Data Lake.
And just as a muddy pond (or Data… Swamp) does not make anyone a master swimmer, so the accumulation of terabytes of data does not bring us even a step closer to the value it was supposed to bring.
This is the moment when digital transformation ceases to be a marketing folder and begins to resemble an attempt to synchronize an orchestra where everyone is playing something different and no one knows the notes.
Chaos ensues
Data changes without warning, reporting teams hunt for errors like detectives in a B-grade crime thriller, and developers look away, saying, “It’s not ours, we just write it to the database.”
This is where Data Contract comes in. It may seem like a small, innocent JSON, but it is the foundation for civilizing data in organizations.
What is a Data Contract and why should you take it seriously?
A Data Contract is a kind of social contract between the team that creates data (e.g., the application team) and the team that uses it (e.g., analysts, reporting, AI/ML). It is a declaration: this is what the data I provide you with will look like, and this is what you can expect from it.
It is not just another piece of documentation that no one reads. A good contract:
- describes the data structure (what fields there are, what types they have, whether they are mandatory),
- indicates acceptable changes (e.g., you can add a field, but you cannot delete an existing one without warning),
- contains quality expectations (does the data have to be available every day at 8:00 a.m.?),
- works automatically - as code that can be tested and validated.
And all this in a language that not only a machine understands, but also a human being. At least one who knows what true, false means and why a comma in the wrong place can blow up the universe.
Data Contract in practice - what JSON looks like
For those who fear technology more than Excel with macros, here is a friendly example. Imagine that the system registers users. We want to determine how this data should look - and not change it from one day to the next without warning:
{
"name": "user_registration_event",
"version": "1.0.0",
"schema": {
"user_id": { "type": "string", "required": true },
"email": { "type": "string", "format": "email", "required": true },
"age": { "type": "integer", "minimum": 18, "maximum": 120 },
"created_at": { "type": "string", "format": "date-time", "required": true },
"source": { "type": "string", "enum": ["web", "mobile", "api"], "required": true }
},
"quality_expectations": {
"completeness": 0.99,
"freshness": "24h",
"uniqueness": ["user_id"]
},
"owners": ["product_team", "data_engineering"]
}Such a file is not just a decoration - it can be validated automatically, it can run tests, it can block the implementation of changes that break something. It is a controlled contract, not a fragile arrangement based on trust and penguin memes.
What Data Contract is not - Feature Store and Data Catalog won’t do the job
Many people confuse these concepts. And for good reason - in the tool market, data is like coffee in a hipster café: each one is different, each one is “specialty,” each one is from Ethiopia. But not everything that smells like a schema is a contract.
Data Catalog - a table of contents without an editor
Data Catalog is like Google Docs for companies: it shows what we have, where it is, and who has touched it.
Great for exploration, poor for control. It does not enforce quality or protect against changes. It offers knowledge, but no guarantees. You could say it’s like a museum guide, but without information about which exhibits are authentic.
Feature Store - a product for AI, not a source of truth
Feature Store is used to train AI models - it takes data that should already be correct and transforms it into something usable for algorithms.
It does not protect data sources. It does not validate semantics. It assumes that data “just is” - and if it is not there, “someone will take care of it.” Usually, no one does.
Data Contract - guardian at the source
Data Contract works where it really matters - at the point of entry into the system. It blocks bad data before it spreads throughout the organization. It is prevention, not detection. And that is why it is so powerful.
Three layers/types of contract
In an ideal world, data is like IKEA - every part fits together, everything is labeled, and the assembly instructions fit on one page. In reality, we have more of a junk room: shelves from Germany, screws from Canada, and someone even threw in a French dictionary.
That is why today more and more companies are approaching Data Contracts in layers/types - because it is not enough to describe what is in the data, you also need to explain what it means and who is responsible for it. Three types of contracts, three different levels of control:
1. Schema Contract - form and types, i.e. “what is here”
Describes the data structure: what columns are there, what types do they have (string, integer, datetime), are they mandatory, what values are allowed (enum, range). Schema Contracts are like grammar rules. They do not say what a sentence means, but they guarantee that the syntax is correct.
{
"field": "user_id",
"type": "string",
"required": true
}2. Semantics Contract - meaning that is not obvious
Is age the current age or the age at registration? Is customer_type = VIP a premium customer or just a friend of the boss? Semantics Contract answers the question: what does this data mean in the business world? It protects against “it works to my eye.”
{
"field": "user_id",
"meaning": "a unique identifier assigned to the user upon registration",
"values": "UUID character string, e.g. '550e8400-e29b-41d4-a716-44665544000'"
}3. Metadata Contract - context, availability, and owners
Is the data updated daily at 7:00 a.m. or ad hoc after lunch? Who is responsible for its accuracy? When was it last changed? Metadata Contract is an operational agreement: owners, SLA, update frequency.
{
"field": "user_id",
"owner": "data_platform_team",
"refresh_frequency": "24h",
"last_updated": "2025-04-24T07:00:00Z"
}Data Contracts - what’s new on the front and is this the only way forward?
Disclaimer: I am not a technical person (so if you are, have mercy on me). I just like to read a lot on various topics, including tidbits like the ones below.
There would be no transformation without fashion. And in the world of data, trends change faster than corporate WiFi passwords. Data Contracts have been a hot topic for the past 2-3 years - at conferences, in posts by data leaders, and on slides presented by consultants in expensive suits. But what is actually happening on the front lines?
- Contracts as code - or schema as code
More and more teams are integrating contracts directly into application code repositories. The data schema, written as JSON Schema or Protobuf, lives alongside the application, and every change goes through a review process - just like code.
DevOps patterns are finally making their way into the world of data, and what used to be “thrown into the database” now undergoes testing, versioning, and auditing.
- Automatic validation - the contract as a build guard
Tools that automatically validate data entering the system against the contract before it reaches the downstream are gaining popularity. So if someone suddenly decides to change the date format from YYYY-MM-DD to DD.MM.YYYY, the pipeline doesn’t explode… because it doesn’t let such nonsense pass. It’s shift-left for data: better to block at the source than catch errors in production and blame the analysts.
- Multi-layered contracts - multiple points of view
(I covered that one above)
The “multi-contract” approach is also becoming a trend: one technical contract (i.e., what the payload looks like), a second semantic contract (i.e., what the data means), and a third operational contract (i.e., who is responsible for when the data should be available and how often it changes). In short, it is not enough to say that a field exists - you also have to say why it exists.
- Data Mesh + Contract = distributed responsibility with control
(I really encourage you to learn more about Data Mesh - I love this idea)
In companies moving towards Data Mesh (i.e., “each team manages its data as a product”), Data Contracts are essential. They allow each domain team to create and manage their own contracts without waiting for a central BI team or approval from the Grand Data Architect.
Are there alternatives to Data Contracts? Naturally
- Contract testing without schemas - or “if something breaks, we’ll notice.” It works, but it’s too late.
- Observability - monitoring quality and anomalies. Great as support, but without prevention.
- Soft governance - a relaxed contract: you declare, but you don’t enforce anything. It works where trust between teams is not yet a luxury.
For management: data is not oil, it’s infrastructure. And someone has to maintain it
We know what management likes to hear: “Data is the new oil,” “AI-ready by 2027,” “Insights at scale.”
Sounds good. Except that in practice, it often looks like this: data is incomplete, outdated, incomprehensible. And at the end of the table, someone added a column called “debug” and everyone pretends that nothing happened.
Data Contract is not a technology - it’s a contract that makes sense
It’s not about a new platform. It’s about someone being able to stand in front of the board and say, “This data is reliable because we know where it comes from, how it changes, and who is responsible for it.”
ROI? Peace of mind, predictability, and no embarrassment
It’s not about spectacular savings. It’s about not wasting hours putting out fires that could have been predicted. We don’t have to explain KPIs that suddenly went haywire. And finally, we stop operating in “someone will fix it someday, probably” mode.
Strategically? It’s the foundation of a scalable organization
Without Data Contracts, you can’t build AI, automated reporting, or a data-driven culture. It’s like building a city without a master plan. It’s exciting at first. Then you get traffic jams.
Don’t start a revolution with a data lake, start with the question: who is responsible for it?
Transformation doesn’t start with migration to the cloud or installing new fancy AI tool on every computer. It starts with an email: “Hey, can you tell me what the status field really means?”
If you can’t figure it out, maybe it’s time for a contract.
But that’s just the beginning. Because a contract is not a goal - it’s a tool. A tool that enables you to:
- build trust between teams,
- avoid costly misunderstandings,
- design systems that don’t break with every change,
- and create an organization where data is not only accessible but also understandable and usable.
Above all, a contract is an invitation to maturity. To move away from chaotic data collection in favor of conscious information management. This is what we need if we want to talk about real digital transformation, not just exchange buzzwords on slides.