← All casesCase 0614 min

Case 06 · Governance and compliance

You took the names out. Everybody is still identifiable.

This topic sounds like meetings and paperwork. It is not. Two hard problems live inside it. How do you hide a person so they stay hidden? And how do you find every copy of them on the day they ask to be deleted?

The junkyard

A thousand tables and nobody to ask.

Somebody messages you: can you send me the shipping delays data? Perfectly reasonable request. You open the warehouse. Eleven hundred tables, with names like ship_evt_v3, pay_ref_2021, usr_master_bkp and stg_merge_tmp.

Three questions, and you cannot answer any of them. Which one is the right table? One of them has a field called px_ref. What is that for? And is anyone still here who knows?

Nothing is missing. Everything is unusable.

Nothing failed here. No pipeline broke. No number is wrong. The warehouse just grew faster than people could remember it. One day it stopped being useful and became a pile.

The habits that stop this are called governance. None of them are exciting.

And once those tables hold real people, something else turns up. The law. That part is not your choice.

Two words, two questions

One is a habit. One is the law.

Governance

Can our own people use this?

Can someone find the right table? Work out what the columns mean? Know whether to trust it? Know who to ask? Nobody outside the company checks any of this. You do it so you do not end up with the pile.

Compliance

Are we allowed to?

Once the data is about actual people, laws apply. What you may keep. Who may look. What you must do when someone asks you to stop. You do not get to decide these. Breaking them costs money.

People teach these two as one subject, which confuses everyone, because only one of them is your choice. Here is the link worth keeping. Good governance is what makes the law possible to follow. You cannot remove someone from everywhere if nobody knows where everything is.

What governance means in practice

Four questions someone must be able to answer.

Where is it? A list you can search. What exists, what is in it, where it lives. People call this a catalog. Not a wiki page somebody wrote once and left. You type “shipping” and get the real tables.
What does this column mean? px_ref holds the payment company’s own reference number. Nobody can work that out from the name. So they stop trying and start guessing. One guess about one column, and the figure in the board meeting is wrong.
Who owns it? A name. One real person who is answerable for that table being right.
Who may read it? Not everybody, and that is a governance decision long before it becomes a security setting.

All four are dull. All four decide one thing. Do people use your tables, or give up and build their own copy? Giving up is how a company ends up with eleven hundred tables.

The one that does the work

Only one of the four looks after itself.

Catalogs go out of date. So do the definitions. An owner does not, because an owner is a person, and people get asked things.

In practice it is dull. Every table that matters has one name against it. That person answers when the returns team asks what is inside. Not a team inbox. Not a ticket queue. A person.

You do not get documents out of this. You get answers in ten minutes instead of guesses. And when something is wrong, one person’s job is to care.

Watch out for a table owned by “the data team”. That means nobody owns it. When a group owns something, everyone assumes someone else has it. The tables that cause the most trouble later are always the ones with no name on them.

The part that surprises people

Take away the name, the email and the phone number. See if that is enough.

Eight customers. Pick who is looking at the table, or set the columns yourself. The counter watches one thing. How many of these eight rows still point at one real person?

—rows that still point at one person
—smallest group of lookalikes
—columns hidden

Why that happens

The name was never the risky part.

Everyone assumes the risky columns are names, emails and card numbers. Hide those three and the job is done. Try it in the table above. Hide all three, and every single row still points at one person.

Here is why. What gives a person away is the mix, not any one column. A city on its own tells you nothing. A birth date on its own tells you nothing. A gender on its own tells you nothing. Put all three in the same row and you usually have one exact human being.

No single column is personal. The row is.

So you cannot check this one column at a time. Look at the whole row and ask one question. How many other people in this table look exactly like this one? If the answer is none, that row is still a person.

And the fix is usually not more hiding. It is less detail. A birth year instead of a birth date. A state instead of a pincode. Keep going until nobody is standing on their own.

There is no word you need to learn here. You need the habit of checking. A table can pass a column-by-column review and still be a list of named people.

Three laws, in plain words

Nobody expects you to be a lawyer. They do expect you to build this.

GDPR
Europe
People get rights over the data a company holds about them. Two of those become engineering work. Send me a copy of all of it, and remove me. Consent has to be real and clear, not buried in a tick box.
CCPA
California
Much the same idea, with one extra part that matters. People can say do not sell or share my data. For you that means a flag that has to travel with the data everywhere it goes.
HIPAA
US health data
Medical records, held to a tighter standard because of what gets out if they leak. Not an email address. Someone’s diagnosis. Very few people get access, and every look is recorded.

Each of these is a sentence on a page until somebody writes the code underneath it. A right to be removed is worth nothing until an engineer can find every copy of one customer, across a dozen systems, and take her out of all of them. Building that is the job. It is the next chapter.

The request

“Remove me from your systems.” Go on, then.

Asha has asked to be deleted. You have thirty days. Here is everywhere she exists. Click each one to deal with it. Three of them are not deletions at all, and those three are the ones teams get wrong.

0 of 12places handled
—still carrying her
compliant?as things stand
One row in one database is where most people stop. Start clicking and see how far that actually gets you.

The nuance nobody mentions

Half of these are not deletions.

The first thought is DELETE FROM customers. It is wrong in two separate ways.

First, you are often not allowed to delete the order. In most countries, tax rules say a record of a sale must be kept for years.

So two laws pull opposite ways. You do not pick one. You keep the order and cut its link to the person. The sale still happened, for the right amount, on the right day. It is just no longer attached to anybody.

Deleting

The row goes

customers
support_tickets
marketing list

Rows that exist only to describe her. Nothing else needs them, so they go completely.

Anonymising

The row stays, she does not

orders        → customer_id null
fct_orders    → key replaced
analytics     → token only

Rows that other things depend on — revenue, totals, history. Keep the row, cut the thread back to the person.

Second, some places cannot be changed at all. A backup is a frozen copy on purpose. That is what it is for. Reach into last Tuesday’s copy and edit one person, and it is not a backup any more.

What you do instead

For the three you cannot simply delete.

Backups. You do not edit them. You write down that she must not be brought back. You let the backup get old and drop off on its own. If anything is restored before then, you delete her again straight away. Regulators are fine with that. What they are not fine with is nobody having thought about it.
Anything you sent to another company. The email tool, the support system, the analytics supplier. Their copy is still yours to answer for, and asking them is your job. This is why knowing what leaves your building belongs in governance, not in a contract folder.
A model trained on her data. Deleting the row does not untrain anything. You write it down, and her effect goes at the next retrain. Worth saying in an interview, because hardly anyone does.

And under all of it: write down that you did it. From the outside, a deletion you cannot prove looks the same as one that never happened.

That is the quiet argument for this whole chapter. Knowing where everything was, before anyone asked, turns a two-week hunt into a twenty-minute job.

Why it is worth your attention

This is the clearest line between junior and senior.

A junior engineer moves data from one place to another and it works. That is a real skill. It is most of what the job advert asks for.

A senior engineer finishes the same pipeline, then asks four questions nobody asked for. Can someone else find this table? Does anyone know what the columns mean? Whose name is on it? And can the wrong people see what is inside?

Same pipeline. Completely different engineer.

None of those four are hard. They are invisible when you do them and very obvious when you do not. That is why people skip them, and why doing them gets you noticed.

If you get asked

“Someone asks to be deleted. Walk me through it.”

The thin answer is one sentence about a delete statement. What is really being asked is this. Do you know how many copies of one person a company collects without meaning to?

Start by finding, not deleting. Say you would first work out everywhere that person exists. Production, the warehouse, exports, logs, caches, backups, and anything sent to another company. Then say what makes that list possible: a catalog and lineage, instead of memory.

Then split deleting from anonymising. Orders stay, because the law says they must. The link to the person is what goes. That one sentence puts you ahead of most people in the queue, because it shows you know two laws can pull opposite ways.

Then be straight about backups. You cannot edit them. You mark her, and you let them expire. Saying that calmly sounds like someone who has done it.

Then say you would log it. Proof matters as much as the act.

If there is time, add the other half. Hide personal data, and give people only the access they need. Then fewer places hold the real thing, and every one of these requests gets smaller.

Two questions

Did it land?

You remove name, email and phone from an analytics table. Is it anonymous?

Someone requests removal, but accounting rules say her orders must be kept for seven years. What now?

Your playbook · Case 06

Nine things to keep

  1. Governance is what makes compliance possibleYou cannot remove someone from everywhere if nobody knows where everything is.
  2. Ownership is one name, not a teamA table owned by “the data team” is owned by nobody. Those are the ones that hurt you later.
  3. The name was never the identifying partCity plus birth date plus gender is usually one human being. Harmless columns are only harmless alone.
  4. Check the row, not the columnAsk how many other people look exactly like this one. If the answer is none, you have anonymised nothing.
  5. Less precision beats more maskingA birth year instead of a birth date. A state instead of a pincode. Keeps the data useful, stops it pointing at one person.
  6. Raw personal data does not belong in test environmentsThat is where the most people have the loosest access and nobody is watching.
  7. Deleting is only half of forgettingSome rows have to survive because of other laws. Keep the row, cut the thread back to the person.
  8. Know what left the buildingCopies held by other companies are still yours to answer for, and asking them is your job.
  9. Write down that you did itA deletion you cannot prove looks exactly like one that never happened.

Your review

How was this case?

Loading ratings…

Next · Case 07

Run it twice. Get the same answer.

Case 07 is about idempotency — why re-running a failed pipeline is terrifying in most systems, and the handful of habits that make it boring.

Open Case 07 →← Case 05