Why Data Engineers Must Balance AI Use with Hands-On Problem Solving
Embracing AI Without Losing Critical Skills
Here’s what usually happens when you want to know something.
You phone a friend? Lol, no — we aren’t cavemen.
You Google it.
You look for answers on the web, search through a handful of useful sites, skim a few, and 9 times out of 10 (if you work in tech), you end up on some wizard’s answer on Stack Overflow.
Yes, the answer might not be 100%, but you grab the code or solution, rejig it, give it a shake, and try to make it work for you.
Ba da bing, ba da boom — you have an answer or the start of one. You’re on a roll and on your way.
That’s how it used to be, anyway.
The Future of Troubleshooting
Nowadays and in the near future, it seems the first port of call will be ChatGPT (or your favorite LLM). We now look at ChatGPT as if it’s the gospel, this all-knowing “thing” with all the answers.
There is nothing wrong with hitting ChatGPT for the answer; a wise person uses all the resources at their disposal to get to the answer. But these LLMs are tools, much like a spade that you pick up when you need it and put down when you don’t.
Problem is, folks aren’t putting it down.
People are hitting ChatGPT for absolutely everything now. Every tiny problem, every small issue is thrown at AI to solve. All the thinking is now done by ChatGPT and not you.
It’s not our fault either; every company in the world is sprinkling AI on everything, even toothbrushes. There’s a real risk here. We are on the road to becoming the fat blob people from WALL-E. Of that, I’m sure.
Don’t believe me here’s what’s at stake:
Critical thinking derailment — ✅
Misinformation — ✅
Dependency for decision-making — ✅
Skill degradation — ✅
If that doesn’t scare you, nothing will.
I guess this situation is like someone who smokes. They know it’s bad for them: the box is covered in warnings, yet they do it anyway. Same goes for social media: thousands of people, articles, and studies are telling you it’s bad, don’t do it, but people don’t listen and do it anyway.
Tell someone not to do something, and they will do it.
There’s no stopping it now; it’s out there, no going back. The only option is to get on board, even if you don’t want to.
Now here’s the problem that scares me
What happens if you don’t have that tool? What if one day some kind of meteor destroys the AI gremlins off the face of the earth, and you don’t have your safety net to think for you anymore? Your little AI buddy isn’t there to hold your hand and answer all your questions. What then?
You see this play out already. ChatGPT has issues, people freak and go to Gemini for their AI fix. If this is you, you’re probably in trouble — big — you’re too reliant on it.
Time for a story…
When I was a DBA back in the olden days, a DBA joined our team. He plonked himself down on the chair next to me on his first day, took a long sip of his coffee, started up SQL Server Management Studio, and froze. The look of sheer terror crossed his face (think deer in headlights sort of thing).
I was puzzled.
Being the happy little helper I am, I asked him what was up. “Where’s SQL Prompt?” he demanded, flustered, like the whole world was about to end.
“We don’t have it here,” I told the dude.
Long story short, he refused to work without it. Meetings were had, toys thrown out of the cot — something about how he can’t do his job without it — bla bla bla.
All that was going through my head was “Alarm bells.” In the end, the company coughed up the money, bought him his little SQL Prompt license, and all was well in his world. But in my world, that was a wake-up call — a big one!
Never become too reliant on some tool, because one day you might not have it.
To quote Iron Man:
“If you’re nothing without this suit, then you shouldn’t have it”
Therein lies the truth.
In the wild west days of data (when everyone was figuring this sh*t out), if you didn’t know something, well, you pulled up your sleeves and tried it. Tested it yourself to see if it worked. You would (God forbid) figure it out. You would try this, try that, fail, repeat, and you know what? You would learn.
Don’t get me wrong here. I’m not saying don’t use ChatGPT — go for it, make your monthly subscription offering to the AI gods, BUT don’t get too comfortable with it. You have to find a happy balance.
Look at it as a Junior Data Engineer: throw simple things at it, but do the hard things yourself. Understand what it’s telling you. Therein lies the answer. You need to do the work — that is the key. That is where you learn.
If you are throwing every little thing at ChatGPT, you’re in trouble. You are nothing but a monkey pushing buttons. You are snowplowing your way headfirst off a cliff, and you will one day be replaced.
That's the truth of it.
The best thing you can do is this: If you want to know something and you don’t know how to do it, try and figure it out first! Then research — use ChatGPT or Google or phone a friend or slog it out yourself, but you have to do the heavy lifting yourself. Understand the problem and find a solution. Figure out why it works or doesn’t work. Why one way is good and another is terrible.
The best way to learn is to do, not to copy and paste. The average Joe copies and pastes; you, dear reader, are smarter than that. You learn by trying things!
That’s how experimentation works. Trial and error. You’ve got to give things a shot yourself. Build the thing, test the thing, experiment! Stop asking ChatGPT or anyone else before you’ve given it a go yourself.
You will learn more.
Thanks for reading! I send this email out twice a week. If you would like to receive it, join your fellow data peeps and subscribe below to get my latest articles.
👉 If you enjoy reading this post, feel free to share it! Or feel free to click the ❤️ button on this post so more people can discover it on Substack 🙏
🚀 Want to get in touch? feel free to find me on LinkedIn.
🎯 Want to read more of my articles, find me on Medium.
✍🏼 Write Content for Art of Data Engineering Publication.