One Month Into Public Learning Data Engineer: Here's What I Didn't Write

I went to someone on LinkedIn I'd never met, asking what he thought was the best way to do data engineering, and whether certifications were worth pursuing. Turns out he had read one of my articles. So instead of giving me a new answer, he gave me back my own. He said: “You're on the right track. You're following a 12-month guide, stick to it. Don't take suggestions from anyone, it might confuse you.”

I read that message twice. Because two weeks before that, I had almost talked myself out of the very road I had committed to in public.

Let me support you.

It's been a month since I published my first article on this journey, “From Data Analyst to Data Engineer: My 12-Month Self-Study Guide.” Since then I've written three more, each one about something I've built. ETL pipeline from scratch. That pipeline is production ready with SQLite and idempotency. It was then edited with GitHub Actions, which turned into a portability tutorial before it became a programming tutorial.

Those four articles are a good summary of what I did this month. It's not a fair record of what the month really felt like. So this one is different. No code, no navigation. Just parts that didn't fit anywhere else.

The program said one thing, the moon did another

The road map put things in perspective. SQL first, Python, Git, Spark, Airflow, Databricks. It's neat. What follows. The kind of program that looks good on a Notion board.

I didn't follow it. Not really. Rather than going one step at a time in the stack, I built one small pipe and kept pushing it until it broke in new ways. SQL was born. Python appeared. Git appeared. But not in the way I had planned, and not because I planned it that way. They came because the pipeline wanted them.

I thought that meant I was off track. Now I think it means that the plan was a start, not a contract. The road map got me moving. The pipeline determined exactly what I needed to read next.

The walls were never about tools

If you read the construction articles, there is a pattern that I didn't call out at the time. Every wall I hit had a technical fix and a non-technical course sitting underneath it.

• Idempotency wasn't really about SQLite. It was about learning not to trust my own idea that “it worked once” means it “will continue to work.”
• Persistence wasn't really about Google Drive. It was about realizing that my work could disappear when I closed a tab, and that I had already lost a file once before without realizing how much I was about to lose.
• Portability was never about GitHub Actions. It was about finding out that one hard-coded method had silently made my entire pipeline depend on a single point, without me ever intentionally deciding that.

None of these are coding lessons. They think the lessons that happened came from the code. That sounds like the real shape of data engineering so far, at least from where I'm standing. Tools are how lessons are realized. It's not the lessons themselves.

The part that the construction articles didn't show

Here's what they didn't show you. Somewhere in the middle of this month, the energy coming from the people watching started to wane. Not significantly. There was silence. I stopped feeling the pressure I wrote about in the second article, the one where strangers reached out and were happy to follow. That pressure never went away because I stopped caring. Dim because the ultimate goal, to find a high-paying data engineering role, began to feel distant. So far that it stopped motivating and became tiring to even think about it.

It didn't help that my real job was very busy, and that I had to learn Laravel to work on top of everything else. Three to four hours a day sounds reasonable until you're exhausted from your day's work and the thing waiting for you at home is another stack you're not used to.

There were places where I seriously considered slowing down. What kept me at the desk was no longer the idea of ​​work. It was less than that. I kept thinking about the one person out there who was exactly where I was in May, stuck somewhere between math and engineering without a clear map, who might be waiting for the next article. Helping that one person was more motivating than the abstract vision of my future role. I've also made a point of looking for something truly challenging to fill my free time, instead of simple, mindless things that fill time without leaving anything to be desired. Even on days work felt far away, at least I wasn't wasting hours.

It's almost out of the way

I also have to be honest about what I called before I started this journey. In my first article I admitted that I have shiny object syndrome, jumping between design, animation, marketing, IT, and data now, before any of them have a chance to stick. I said that I would have to be deliberate in order to avoid that here as well.

It happens anyway. Halfway through the month I fell down a rabbit hole about certifications, whether I needed one, which one, or if I was on the right track at all. That led to seriously considering a pivot towards AI engineering instead, because at the time it looked shiny and in demand. I had to stop carefully and remind myself of the way I had already chosen, and I had already told myself the audience I had committed to.

Which brings me back to that message.

The stranger who gave me my advice

That LinkedIn message I mentioned earlier came after that shock. I would reach out to someone for advice on the best way to do data engineering, half asking for permission to guess, half wondering aloud if a certification would help. Instead he told me that he had read my guide story, and that the answer to my question was in my writing. Stick to the plan. Stop gathering ideas from strangers, because too many ideas just confuse you.

There is something almost comical about being reprimanded on your show by someone who only knows that show because I published it. But that's all doing this in the community. Accountability does not only come from the people watching you build. Sometimes it comes back around and gives you your words right when you need them.

What I learned is actually how I learn

Another observation this month has nothing to do with motivation and everything to do with how I work. I used to think I needed to build something big to learn anything. A real project, many weeks, something that is a portfolio right from the start. That thinking almost killed my momentum more than once, because big projects are easy to start and easy to stop halfway through.

Substitutes are small. A pipe that I can build and tear down in a weekend teaches me as much as a multi-week incremental project, and I actually finish it. As we enter month two, I'm trading the idea of ​​one large building for a series of smaller ones. Small projects, scaled to fit a 9 to 5 and a Laravel deadline, each aimed at a specific idea rather than an entire line of resumes.

Where that leaves me

If you had asked me on day one what success looked like, I would have said the highest paying role, full stop. A month in, that's still somewhere in the picture, but it's not what makes me open my laptop after a long day at work.

What keeps me there now is less, and honestly more sustainable. I like to build things. I like to write about what breaks. And every time someone tells me an article that really helps them find their next step, that does more for me than the thought of a future job offer.

The road map is still a guide. I'm still following it, cron job and all. But I start the second month with a clear sense of what exactly will carry me through the next eleven, and it's not what I thought it would be when I wrote that first article.

This is part of my ongoing series documenting my transition from systems analyst to data engineer. If you've been following along, thank you. If this is your first article in the series, the previous ones are linked above.

Connect with me on LinkedIn, YouTube, and Twitter.