How to Not Get Burned by Security in Data Engineering

Don't Let Security Bite You - My Advice for Data Engineers

Photo by Matthew Menendez on Unsplash

Security is an afterthought. Read that part twice so it sinks in. That’s the truth of it.

It’s something folks in the wild don’t pay too much attention to on the way through. I am, of course, generalising here, but I’ve been around long enough to see how many corners get cut and how many folks say, “How we will tie up security at the end” — they never do. It gets kicked down the road until something goes very wrong.

And something will go wrong — eventually.

That’s the thing, things go wrong very fast when you take the shortcuts with security.

I’ve seen some wild things, believe you me. The stuff that keeps you up at night.

Access is granted on the fly, process goes out the window, and you’re stuck there wondering, if I lower the permission on this, what’s going to break? and who is going to throw their toys out the cot? Do any online tutorial and security is breezed over with a little side note of how you should only give the permission you need — but in this course, we will just give full access. Two thumbs up, that’s a great intro to security for you.

I come from DBA land, and I come from banking, and there, dear reader, shit gets real when you cut corners. All I’s must be dotted and T’s crossed, or you will be nailed. I’ve been through my fair share of audits and it ain’t fun. I’ve seen people get called out and escorted out of buildings.

Here’s lesson numero uno of security:

IT’S ALWAYS YOUR RESPONSIBILITY.