← All casesCase 0916 min

Case 09 · Lake, warehouse, lakehouse

One has a bouncer on the door. One lets everybody in.

Three words that get used loosely and explained badly. Underneath them are two real questions, and once you can answer those, the rest is marketing.

Two questions

Forget the words for a minute.

You will hear three names thrown around: warehouse, lake, lakehouse. People will tell you one is modern and one is old. That is not a useful way in.

There are really only two questions underneath, and the three names are just three different answers to them.

When do you check the data is the right shape? On the way in, or when somebody finally reads it?
What happens when one row needs to change? Because the answer is not the same, and it is not obvious.

The whole of this case is those two questions. Everything else — the product names, the diagrams with arrows — falls out of them.

Case 01 covered why a warehouse is quick at adding up a column. This is a different thing: not how the data is read, but how it gets in and how it changes.

Question one

Is there somebody on the door?

A partner sends you data every night. Some nights it is perfect. Some nights a column is missing. Or an amount turns up as text. Or they have added a field and told nobody.

A warehouse has a bouncer. You tell it the shape in advance. Anything that does not match is usually turned away at the door.

More precisely: an analytical warehouse generally defines and checks a table’s shape when data is loaded. That is what people mean by schema on write. Treat that as a useful mental model, not a hard rule. Modern warehouses can hold semi-structured data such as JSON. Many also have loose staging areas where anything can land first.

A lake usually has no bouncer on that door. Files land in object storage without one central table shape being enforced as they arrive, and the structure is worked out when somebody reads them.

Say this one carefully, because it is widely misunderstood. It does not mean a lake has no schema, no checks and no rules. It means nothing is enforcing one table shape at the moment of landing. Plenty of lakes have strict validation in the loading jobs, a catalog describing every folder, and well-governed tables built on top. That is a choice each team makes, not something the storage decides for them.

Both can find the problem. One is set up to find it tonight.

The two names for this are schema on write and schema on read. Neither helps much on first hearing. Check at the door, or check when you read — that is all they are describing, and either can be made stricter than its name suggests.

Try it

Five files turn up tonight. Two of them are a mess.

Same five files, same night. Switch the door on and off and watch what gets in, and more importantly, when you find out about the problems.

somebody checking at the door
—files that landed
—turned away
—problems you know about

What each one costs

Neither answer is free.

The bouncer

Nothing bad gets in

Every row in there is the right shape. Anybody can query it without checking anything first.

The cost: somebody is awake tonight fixing a rejected file. And you cannot keep anything you have not already planned for — the new field the partner added gets dropped on the floor.

No bouncer

Nothing is lost

Every byte they sent is kept, including the field nobody has thought of a use for yet. Nobody is woken up.

The cost: the mess is still in there. Every person who reads it has to deal with it, separately, forever, and the first few will not know it is there.

That is a real trade and reasonable people land on both sides of it. What is not reasonable is pretending one side has no cost, which is how most of these articles read.

What happens if nobody minds the lake

It becomes a swamp.

This is the word people use and it is the right one.

Two years of landing everything, and nobody wrote down what any of it was. There are four thousand folders now. Some are duplicates. Some were a one-off experiment in 2024. Nobody knows which ones anything depends on.

Keeping it is cheap. Understanding it is not. Storage costs almost nothing. That is exactly why nobody deletes anything, or asks whether it is needed.
The rot is invisible until you need something. Nothing errors. The folders just sit there. You find out the day somebody asks a question you cannot answer.
The fix is not technical. It is Case 06 again — somebody’s name on each thing, and a note saying what it is. A lake without that is not storage. It is a loft.

Worth saying plainly: a lake going bad is not a failure of the technology. It is what happens when “we will sort it out later” meets two years.

Question two

Why changing one row is awkward.

A lake is a pile of files sitting in cloud storage. That is genuinely all it is.

And the data files in it are usually Parquet. Those are not built for changing one row. A database can reach into a page on disk and edit a few bytes in place. A Parquet file in object storage is not designed to be poked at that way.

So changing one row normally means one of two things. Rewrite the file that row lives in. Or write a new file, and record somewhere that it replaces part of the old one.

So think about what that means for something completely ordinary. A customer moves house. One row, one field, in a table of eight million rows.

In a warehouse that is one small update. In a lake it is a problem.

The next page lets you do it and watch roughly what each one has to shift to make it happen.

One row changes

Customer C-77 moved to Pune. Go on, update her.

Eight million rows, spread across four files of about 500 MB each. Her row is in the second one. Pick where this table lives and make the change.

Illustrative example. Real sizes and times depend on your storage, network, file format, compression, partitioning, query engine, cluster size and what else is running.

where the table lives
the list of files that make up this table—
—written to change one row
—how long it takes
—what readers see meanwhile
Press Change her city and watch what has to move.

The actual invention

A lakehouse is a list of which files count.

That is it. That is the whole invention, and it is far less magical than the word suggests.

A plain lake says: this table is whatever files are in this folder. A lakehouse keeps a small list beside it, saying which files make up the table right now. The folder can hold other things too. Only what is on the list counts.

That is the intuition, and it is worth holding on to. Now the fuller version. What sits beside the files is a metadata and transaction layer. The thing that provides it is called a table format, and there are three you will meet: Delta Lake, Apache Iceberg and Apache Hudi.

object storage │ ├── data file 1 parquet ├── data file 2 parquet ├── data file 3 parquet ├── data file 4 parquet └── metadata / transaction log │ ├── which files are currently part of the table ├── the table’s schema ├── earlier versions of the table └── a record of every change

The files on their own are just files. The metadata tells a query engine which physical files make up the table right now. That is the difference between a folder and a table.

Each of the three table formats does this its own way, and none of them is literally one plain text list. “A list of which files count” is the right picture to carry; just do not expect to open one and find it.

to change a rowWrite a small new file with the change in it.

Nothing existing is touched. The big file stays exactly where it is.

thenWrite a new version of the list.

The new list says: files one, three, four, and the new small one. File two is no longer part of this table.

and that is the clever bitReaders follow the list.

Until the list changes they see the old table, complete and correct. The moment it changes they see the new one, complete and correct. There is no in-between.

That last point is what people mean when they say a lakehouse gives you transactions on a pile of files. Nobody ever sees a half-finished change, because the switch is one small write at the end.

It also means yesterday’s version is still described, so you can read yesterday’s table back exactly. That is a useful thing to have the morning after a bad load.

Why the metadata matters

Nobody ever sees a half-finished table.

Once there is a record of which files count, four useful promises become possible. People bundle them under one word, ACID, which is an acronym and not an explanation. Here they are using the change you just made.

All or nothingHer new city appears completely, or not at all. There is no state where half the change landed.
Still a valid tableAfterwards the table still has the shape it is supposed to have. The change could not quietly break it.
Readers are not caught mid-wayAnyone querying during the write sees the whole old table. Then the whole new one. Never a mixture.
Once done, it stays doneAfter the change is recorded, it survives. A crash a second later does not undo it.

And it is one small write that does all of this. The new data file can take as long as it likes; nothing changes for anybody until the metadata moves from one version to the next.

before after version 10 version 11 → file A → file A → file B → file B → file C → file C′ the replacement → file D → file D

Notice what is not in that picture: file C being deleted. Old files and old versions usually stay for a while. That is how you read the table as it was last Tuesday, or undo this morning’s bad load. They get cleaned up later, on whatever schedule you set.

Reading an earlier version has a name: time travel. It is a side effect of keeping the history, not a separate feature somebody had to build.

The harder version

Now try deleting a person.

Changing one row was awkward. Removing somebody entirely is the same problem with the volume turned up, and you have met it already — in Case 06, Asha asked to be deleted.

Her rows are not in one convenient place. She has been ordering for three years, so she is scattered across hundreds of files.

A plain lake

You do it by hand

find every file with her in it
read each one
write it out without her
swap the new file in
repeat, a few hundred times

Somebody writes a script. There is no safety net anywhere in it. If it stops halfway you are left with some files rewritten, some not, and nothing recording which is which.

A lakehouse table

It is an actual operation

delete from audit_events
where customer_id = 'C-77'

The engine works out which files hold her, writes replacements without her, and moves the metadata to the new version in one step. Anyone reading sees the table before, or after.

That is not a small convenience. Delete and update became real operations on data sitting in object storage, instead of a hand-written job somebody has to babysit. It is one of the main reasons table formats took over from raw lakes — and the reason has as much to do with the law as with engineering. A legal request to remove somebody is not something you want to answer with a script and a hope.

And the trap, which sounds clever for about ten seconds. “Let us store it as CSV instead, then we can edit it.” You cannot. A CSV sitting in object storage is exactly as hard to edit in place as a Parquet file — the limitation is the storage, not the format. What you would actually have done is thrown away your compression and your ability to read one column without reading the rest. You get nothing, and you pay for it on every query from then on.

What the files are

Why everyone stores Parquet.

One thing has been sitting in the background of this whole case: the files themselves are usually Parquet, not CSV. That is worth two minutes.

A CSV file

One row after another

A-1,Mumbai,120
A-2,Pune,80
A-3,Delhi,150

To add up the amounts you have to walk past every city and every order id to get at them. Fine for a small file. Painful for a billion rows.

A Parquet file

Each column kept together

A-1 | A-2 | A-3
Mumbai | Pune | Delhi
120 | 80 | 150

Ask for the amounts and only the amounts get read. Repeated values sit next to each other, so they squash down well. This is Case 01 again, now as a file on disk.

So a lakehouse table is really three things stacked up, and none of them is new on its own:

cheap object storage somewhere to keep files + columnar data files parquet, so reads are cheap + a metadata / transaction layer so the files behave like a table ════════════════════════════════ a lakehouse table

How it got here

A lakehouse is not just a tidier folder.

Read in order, the whole thing is one question being answered step by step.

Raw files — whatever the sources sent, in whatever shape.
↓
A data lake — all of it kept cheaply in object storage.
↓
“How do we make these files behave like a table?”
↓
A table format — Delta Lake, Iceberg or Hudi.
↓
Metadata and a transaction log — which files count, what shape they are, what changed and when.
↓
Which buys you — safe updates, schema changes, and reading the table as it was.
↓
A lakehouse table

Every step there is answering a problem the step above it had. That is why the word exists, and it is a better thing to remember than any product name.

What it looks like in practice

The same folders, read two ways.

Say your company lands operational data every day. In storage it looks like this either way:

s3://company-data/audit-events/ ├── day=2026-09-28/ ├── day=2026-09-29/ └── day=2026-09-30/

As a plain lake, those are folders with files in them. Anything reading it has to be told where to look and what it will find.

As a lakehouse table, the same files plus a metadata layer become one thing called audit_events, which you can query by name.

The naive worry

“Does it read everything?”

select * from audit_events
where day = '2026-09-30'

It feels like that query has to open three days of files and throw two away.

What actually happens

It skips most of it

metadata → these files are current
folders  → only day=2026-09-30
stats    → and only these parts

The metadata says which files are part of the table. The folder layout rules out two days without opening them. Per-file notes rule out more.

Skipping whole folders because of how they are named is called partition pruning. Skipping files because of the notes kept about them is the block-skipping from Case 01. Same trick, one level up.

The old arrangement

Why people ended up running both.

For years the normal setup was one of each. Raw files land in the lake. A job cleans them and loads the result into the warehouse. Analysts use the warehouse. Data scientists use the lake.

It works. It also quietly charges you for things nobody puts in a budget.

You pay to store it twice. The same orders exist in both, in two shapes.
You pay to move it, every day, forever. And that job is one more thing that can fail at 3am.
The two drift apart. Somebody fixes a definition in one and forgets the other. Now there are two revenue numbers and a long meeting. That is the problem from Case 04, arriving by a different road.

A lakehouse is an attempt to have one copy instead of two. That is the pitch, and it is a reasonable one. Whether it is worth changing a working setup for is a different question, and usually the answer is no.

Choosing

Which one, and honestly.

There is no winner here. There is a fit between what you have and what you need, so the useful thing is a list of conditions rather than a verdict.

A warehouse may be enough

When all of this is true

The data is mostly structured. SQL analytics is the main thing people do with it. Strong, central governance matters. You would rather somebody else ran the platform. And the amount you query fits the way that platform charges.

A lake or lakehouse may earn its place

When any of this is true

Very large volumes of files need cheap storage. Raw or semi-structured data has to be kept. Several different engines need to read the same data. Open file and table formats matter to you. Or you need schema changes, time travel and safe updates directly on object storage.

✓Starting from nothing today. A lakehouse is a reasonable default. The table formats are mature enough to be boring, which is the quality you want in storage.
✕You already have a warehouse and nobody is complaining. Moving costs months. Make sure you can name the problem you are solving before you start.
Workload, governance, cost, who else needs to read it, and who has to run it.

Those five things decide it. Not which architecture is newer, and not which one reads better in a design document.

Side by side

The shape of the three.

A conceptual comparison, not a product one. Individual products often support more than their column suggests, and they change every year.

data warehousedata lakelakehouse
Where it livesstorage the warehouse managesobject storage, as filesobject storage, as files
The shapeusually set and checked when loadingoften worked out when readingdefined and tracked by the table format
Raw or odd datapossible, depends on the producta natural fita natural fit
Transactionsstrong, built inusually limited at the raw file leveltable-level, via the table format
Changing a rowhandled efficiently by the engineusually expensive — rewrite the filenew or replacement files, plus a metadata update
Reading an older versiondepends on the productnot there by defaulta common table-format feature
Open storageusually not the pointcentral to the ideacentral to the idea
Names you will hearSnowflake, BigQuery, RedshiftS3, GCS, ADLS plus filesDelta Lake, Iceberg, Hudi

Three things people say

And why two of them are wrong.

“A lake is cheaper.” The storage is cheaper. That is one line on the bill. What goes up is the compute to make sense of raw files, every time somebody reads them. And the people you need to build those jobs and keep fixing them. Plenty of companies moved to a lake to save money and spent more.
“You have to pick one.” Plenty of large companies ran both for years, and it worked. The argument for a lakehouse is not that two systems cannot work — it is that two copies of the same data cost money and drift apart.
“A lake is more flexible, so it is better.” It is more flexible, and that is exactly the cost. Flexible means nothing is checked, so every reader checks it themselves or nobody does. In an interview, saying this without naming the trade is the thing that marks you out as having read about it rather than used it.

If you get asked

“Would you use a lake or a warehouse here?”

Do not answer with a preference. Answer with questions, then a choice, then what it costs you.

Ask what the data looks like. Mostly tables, or a mix of logs and files and things nobody has classified? That alone decides most of it.

Ask who reads it. Analysts writing SQL against known tables want a warehouse. People experimenting with raw events want the raw events.

Then say when the checking happens. That you would check the shape at the door if the data is well understood, and later if you need to keep everything. That one sentence is the whole topic and almost nobody says it.

Then mention changing a row. Parquet files are not built for editing one row in place. An update usually means rewriting that file, or writing a replacement and updating the table’s metadata. That is why formats which keep a list of current files exist at all. This is the part that sounds like experience.

Finish with the trade, out loud. There is no answer here that costs nothing, and saying so is the answer they are listening for.

Two questions

Did it land?

In the usual warehouse mental model, when is a table’s shape mainly checked?

What turns files sitting in object storage into a managed table?

Why does changing one row in a plain data lake cost so much?

Your playbook · Case 09

Eight things to keep

  1. It comes down to two questionsWhen do you check the shape, and what happens when one row changes. The three names are mostly answers to those.
  2. A warehouse checks at the doorNothing bad gets in, somebody is awake fixing the rejected file, and anything unplanned is dropped.
  3. A lake usually checks when you readNothing is lost and nobody is woken up. The mess is still in there — unless you add the checks yourself, which you can.
  4. Data files resist row editsParquet is not built for changing a row in place, so one change means rewriting that file or writing a replacement. Everything else follows from this.
  5. A lakehouse adds metadata over the filesIt tracks which files are currently the table. Write the change, then move to the next version. Readers see one whole version or the next, never a half-finished one.
  6. Cheap storage is not a cheap systemWhat goes up is the compute to read raw files and the people to maintain the jobs.
  7. Two systems means two copies that driftStoring it twice, moving it nightly, and eventually two different answers to the same question.
  8. Fit, not fashionWorkload, governance, cost, who else must read it, and who has to run it. Those five decide it — not which one is newer.

Your review

How was this case?

Loading ratings…

Next · Case 10

You asked for one day.

Case 10 is about why a one-day question reads five years of data, and the two layout choices that stop it.

Open Case 10 →← Case 08