So I landed a data engineering role and I was buzzing to get started. Finally, I was going to test my scripting skills in a business environment, I thought to myself.
My focus was simple too; write better and more efficient scripts and write them faster.
After 6 months my perspectives have changed quite a bit, although I’m well on my way to reaching my initial goal, I’ve had to unlearn and pick up some new behaviour.
The following is my attempt to articulate some of the lessons learned, some quite the hard way. So here goes, in no particular order:
I had read countless blogs posts on data engineering that stressed its importance but it never really quite dawned till I found myself on the job.
I quickly found that the data engineering role meant, you’re going to be at the receiving end of the majority of the data that is used by the company, data really really familiar to them.
I know the temptation is to jump right into the coding but do spend some time to get familiar with the operations of the various departments.
Learn the terminologies, understand what the abbreviations represent, identify the metrics and KPIs that will be thrown around you.
I was a little over-ambitious when I began, I set off to try to understand the entire code-base no matter the cost.
A noble cause yes but reality slowly dawned on me that business wasn’t going to slow down for me to catch up. And I ended up over-exerting time and energy trying to satisfy my curiosity which was still painfully short.
I learned a key lesson though, focus on the business use case rather than the fine details of the code. Every project folder, script or query addresses a specific use case and wading through the code base with that as a guide can help you quickly make sense of the code little by little. Then over time as you complete more tasks, you build a better intuition of the code.
So take it slow, you’re not expected to have the entire codebase in your head. Just focus on knowing enough so you know where to look.
It doesn’t mean you’re dumb!
This hurt me one time when I completed a ticket late because I felt it was my responsibility to figure out any confusion by myself.
After failing on this several times I learned the approach of typing a quick summary after each project meeting and sharing it with the stakeholder.
Aim to seek clarity before undertaking any assigned task. Engage the task issuer as soon as soon can be, jump on a quick call if you have to. It really shows you’re paying attention and you will deliver on tasks much much faster.
So brace yourself to be naive and painfully clueless at times. Don’t try to understand everything on your own, ask ask and ask!
When I started, I put an unhealthy amount of pressure on myself to prove my worth. So much so that anytime I made a mistake or do something terribly wrong, I began to doubt my competence.
Things happen yes just don’t beat yourself over it!
Expect to make mistakes every once in a while but focus on owning them. I found that this was a much faster approach to making fewer mistakes overall.
Owning your mistake means alerting your team as soon as possible and providing solutions to fix it, so it can be corrected.
Saying: “it is my fault” is not that hard and you quickly become a more trustworthy person in your team.
I struggled with this initially, I just wanted to work in my small corner and produce the needed output, but I quickly found that my outputs as a data engineer depended on the inputs of several others and so it wasn’t that prudent.
Giving periodic updates to your stakeholders is one of the surest proofs that you’re working.
Communicate blockers too, you do not only yourself but your team and sometimes the larger business a disservice if you fail to communicate blockers in a timely manner.
This was hard for me because I saw it as defeat if I communicated my blockers but yeah things caught up to me fast. So I developed a system for asking for help:
Well, I’m getting better at this, but yeah, communicate, communicate communicate.
June 2022 will mark my 3rd active working year since I graduated. So I still think it’s the early days in my career. There’s a great deal more I can know, and I feel I’m only getting started.
During my culture fit interview session, I stumbled on a question on work-life balance. I hadn’t considered this before. So I didn’t really know what an ideal working environment looked like. So this lesson was perhaps the most important lesson I had to learn: Know thyself!!
Capture the feedback that will be communicated to you, take them in your stride and keep on moving.
Learn about yourself, be intentional about identifying what works for you; Maybe you’d find that your day gets messed up if you respond to emails early morning, Or that you thrive with minimal supervision than micromanaging, Or that you tend to do great work when there are no distractions around.
Whatever it is, learn about yourself and use that to plan your working day.
Over time you’re going to build a great intuition, which would help inform your next career step.
Finally, I really think it is a long road to mastery, and it takes time to build competency. So slow down, browse through some programming memes and laugh at yourself every once in a while.
PS: If you want a quick intro to Data Engineering do check this post by a good colleague.
Hope you enjoyed this.
Cheers!!