I was irritable and grumpy this week - my energy levels were plummeting, my spirits were low.
I could not handle some difficulties at work: there was a stubborn programming problem I was not grasping and it really bothered me. Then, to make matters worse, it turned out that one of my code deployments from around two weeks ago contained a lurking bug, that was finally noticed on Friday. It meant that more than two weeks worth of client data was incomplete; financial ramifications were unclear but looked significant.

What bothers me is that I clearly remember making that particular code change, and being aware that it’s a high-risk one. Therefore, I made every precautionary step I could think of: I tested it ad nauseum with unit tests, manual tests and integration tests. My work was peer reviewed, and the PR was merged. To be safe, I took an iterative approach, and I’ve written the code so that it enabled a canary release: I enabled it first for our developer testing account, where it seemed like everything is fine. Then, I deployed it to a single production client and monitored the logs closely - it all looked good. So I deployed it to all clients and kept an eye on it - it still looked good.

Then, Friday afternoon, a client notified us that some of their data is not being processed. After some investigation, it turned out that despite all the precautions I have taken, I have left out one line of code, which meant that a particular keyword argument was not passed to a function - which caused a subtle, sneaky bug.

A fix was quickly deployed and the data was backfilled. However, the bug still caused some client impact, which made me feel real bad.

This incident bothered me a lot and I couldn’t seem to get it out of my head at night. I felt like there is nothing to learn from this, as I took every safety measure I could think of and I still screwed up. I realised I needed some kind of shock to my system to shake me out of this mindset. I decided to go for a really long run.

I ran to Victoria Park, along the Regents Canal. Once there, I felt I could still go on, so I went ahead and ran to Hackney Marshes, where I did a large lap. Pushing my boundaries, I started running back home via Hackney Wick, along the river Lee and the Regents Canal.
Overall, I did about 25 kilometers. I felt exhausted and happy.

During the long run, I managed to get some things straight in my head and I developed a better mindset. I realised that I make mistakes simply because I work. And sometimes I make big mistakes which have an large impact on others - which suggests that what I do has some kind of effect on others to start with.

I really want to have a meaningful role, an impact, and responsibility in this world - so I better be prepared for a lot of mistakes.