7 Surprising Habits That Make Data Engineers Unstoppable

How to Thrive in Data Engineering

Aug 23, 2024

True story. The world is getting busier, not quieter.

If you think you have time, you don’t.

Procrastination and distraction are all around, and what’s worse, we all know it. Yet the Netflix subscriptions keep getting paid, the Instagrams get scrolled, and no one seems bothered.

Things are moving fast, way too fast.

No one can keep up. It’s depressing but true.

I tell you all this because the data industry is filled with people wanting to learn and improve. They feel stuck or lost or, worse — overwhelmed.

I’ve run into tons of people who want this or want that but lack the drive to see it through.

“I want to be a Data Engineer, but I don’t know where to start.” And that’s where they leave it, like saying the words will somehow make it happen.

“I want to learn SQL, but I can’t find the time.” I’ve heard that one thousands of times. It’s normally around the start of the year when I start getting messages from people asking for advice on where to start when they want to get into data, only for the novelty to wear off a few weeks later.

They have good intentions, I’ll say that much. They sign up for newsletters, buy courses and tutorials too. But the list of uncompleted and half-assed attempts piles up. Folks happily drop who knows how much money on boot camps and books, only for the books to remain unopened and the courses stuck at 5% complete or never started.

It’s my belief that when things get hard or uncomfortable, people quit. Then a couple of months will pass by, they start to feel stuck again, and the cycle repeats itself.

I tell you all this because everyone feels this way. Everyone can relate. Don’t be fooled by the ‘top voices’ out there who make it look easy — it’s not. They’ve been through this, guaranteed! (We all do).

The only difference is the greats take charge and stop the cycle. You can too.

There is no roadmap…

The first thing you need to get square is this: there is no blueprint or roadmap. You cannot follow someone else’s steps and think it will work for you. Sure, you can try, but you are you and not them. I suggest you follow as best you can, take what works, and apply it to your circumstances and your way of working. That’s the best you can do. Your path is going to be different. I wish it was copy and paste, but it’s not.

Here are some lessons learn in my career, some were hard earned others I’ve picked up long the way from helpful people.

1. Increase the Urgency Dial

Working with a sense of urgency is a secret weapon. Why? Most people coast. Most people do just enough to get by, and that is where they draw the line. If you can turn your urgency dial up a few notches and work with a higher level of intensity than the average Joe, then you will start to stand out and get shit done. Most people think they have loads of time. They don’t. If you want to learn, grow, and get things done, light a fire under your ass and do it like you’re running out of time.

2. Back Yourself to Figure it Out

If you run into a problem and your first thought is to ask for help, you’re in BIG trouble. This game rewards the self-sufficient. Data is a no spoon-feeding zone. If you need hand-holding and constant guidance, then the road ahead will be rocky. Someone out there has had the same problem, error, or issue. It’s a matter of finding ways to solve dilemmas and figuring it out. Back yourself to find a solution or, at the very least, try things. We live in a world of near-endless resources — use them.

3. Find an Hour Everyday

Make time. If something is important to you, you will find the time. If it’s not, you won’t. It’s really that simple. If you want to learn, up-skill, build something, create something, or master something badly enough, then you have to find an hour every day. No excuses.

The caveat is it will come at a price (this is where people stumble). It will mean giving up time, comfort, or distractions. You will, however, gain progress, growth, and the satisfaction of investing in yourself. It’s learning to play the long game. It’s all about choice.

4. Do the Hard Things First

If you do the hard things first, the easy will take care of itself. I’ve applied this to nearly every aspect of my working life, and it’s served me well. Most people avoid doing the hard things because they are hard.

More often than not, the “hard” is only hard because you think it is. In reality, when you get stuck in, you will see it’s actually quite easy. Hard is where the learning is. Hard is where you will grow. If easy is always your default, you will never progress. Lean into the hard things, and the easy stuff will follow.

5. Be the Light for Others to Follow

I spent years looking out for number one. Then I realized when you help others, you help yourself. When you lift others up, you, in fact, lift yourself up. Helping others, teaching others, guiding others is now the name of the game for me. The more you give, the more you grow. Be a light for others to follow. This, in turn, has a ripple effect that can cascade across a team or an organization. When that happens, you have a culture. Then you have magic.

6. Easy isn’t Always Simple

“It’s easy” is the curse of the data engineer. I’ve seen those two words burn people more than a few times. They’ve burned me many a time. Sometimes tasks can seem straightforward (on paper), but when you get down to it, they can quickly spiral out of control into complex solutions or, worse, become massively over-engineered. This tends to happen when you wing it or lack requirements because you are guessing and making things up as you go. Plan well and accept that nothing is easy in this job — it’s about the balance between simplicity and effectiveness.

7. Be a Learner

I thought naively that once I’d left school, that would be that. Fast forward some twenty-odd years, and I can safely say the learning never stops.

Be a perpetual learner. Never, ever stop.

It’s one of the many blessings of this job. There is always something to learn, and there is always another level. The trick is getting good at picking up new things. The only way I know how is by doing. You’ve got to learn and apply. You need to stay curious. Follow industry trends. Read people’s articles. Expose yourself to different thinking and different approaches. Medium and Substack are gold mines of clever, everyday people doing some interesting things in data. Find them.

📩 Think this is valuable? Share it with someone who might benefit.

Thanks for reading! I send this email weekly. If you would like to receive it, join other readers and subscribe below to get my latest articles.

👉 If you enjoyed reading this post, and think this is valuable? Share it with someone who might benefit. Or click the ❤️ button on this post so more people can discover it on Substack 🙏

🚀 Want to get in touch? feel free to find me on LinkedIn.

🎯 Want to read more of my articles, find me on Medium.

✍🏼 Write Content for Art of Data Engineering Publication.