The Home Doctor - Practical Medicine for Every Household - is a 304 page doctor written and approved guide on how to manage most health situations when help is not on the way.
Doesn't sound like your fault. Making the source of income not backed-up and tested routinely (which is not a big effort), eventually cause a fuck-up, doesn't matter the source. Crappy environment, glad you've left.
100% not your fault, especially as a junior. The company failed on every level here, both in terms of management and technical implementation. You were lucky to leave. I hope you ended up somewhere better run after this.
It's a bummer to hear this. You learned a lesson worth, "millions in revenue" and they let you leave. They let someone with that profound experience leave. They clearly didn't learn the lesson intended for them. Every dev that has been around for a while and worked on something sufficiently large has a story like this. I'm reasonably sure you'll tell it to many people in the future, and you'll remember it fondly. You earned this story, and I'll bet you never blow something up quite like this again. Companies will be lucky to have you.
This was a really sad read. I was expecting a conclusion with some lessons learned and maybe the company helping you in some way. You were a victim of sloppy process and a toxic environment that turns a silly human error into catastrophic and irreversible failure. You didn’t fire yourself, you rightfully quit. Hope you came out ok.
Great read, and just to mirror what others have said that is 100% not on you. You were not paid to make sure the company had structures in place to prevent this, which they absolutely should have, and it was obviously not your responsibility. It's so easy to jump straight to the root-cause, and ignore all the layers that should have prevented or mitigated the issue afterwards, but it's a very flawed way of thinking. There will always be root causes you cannot predict, and so approaching reliability as a series of layers (like they do in for example the airline industry) is a much healthier approach I think.
The failure is hardly yours to own outright. The process itself was begging for something like this to happen.
If you didn't do it, it would be eventually be someone else.
If you're all working directly against the production database, it's inevitable that something like this would have happened. At the very least, there should have been some restrictions in place regarding what operations what users could do on the database.
The founders should have apologised to the entire company AND openly absolve you from any guilt - as they created the environment that led to the failure. Clear cut.
I'm really sorry you had to go through this. However, it's clear you learned a lot from it. We all did.
Many people in IT industry has this kind of horror stories in their background. It changes you.
It is also worth to keep this in mind:
Many of such horror stories could have been prevented with better onboarding and security practices, better backup practices, better sandboxing, better everything...
But above all this, thanks and congratulations for staying honest, speaking up and working hard to help, understand the consequences and learn.
You definitely have a great professional career ahead, I'm sure.
Doesn't sound like your fault. Making the source of income not backed-up and tested routinely (which is not a big effort), eventually cause a fuck-up, doesn't matter the source. Crappy environment, glad you've left.
100% not your fault, especially as a junior. The company failed on every level here, both in terms of management and technical implementation. You were lucky to leave. I hope you ended up somewhere better run after this.
It's a bummer to hear this. You learned a lesson worth, "millions in revenue" and they let you leave. They let someone with that profound experience leave. They clearly didn't learn the lesson intended for them. Every dev that has been around for a while and worked on something sufficiently large has a story like this. I'm reasonably sure you'll tell it to many people in the future, and you'll remember it fondly. You earned this story, and I'll bet you never blow something up quite like this again. Companies will be lucky to have you.
This was a really sad read. I was expecting a conclusion with some lessons learned and maybe the company helping you in some way. You were a victim of sloppy process and a toxic environment that turns a silly human error into catastrophic and irreversible failure. You didn’t fire yourself, you rightfully quit. Hope you came out ok.
https://www.youtube.com/watch?v=KjJdN3GmzYg&t=307s
Great read, and just to mirror what others have said that is 100% not on you. You were not paid to make sure the company had structures in place to prevent this, which they absolutely should have, and it was obviously not your responsibility. It's so easy to jump straight to the root-cause, and ignore all the layers that should have prevented or mitigated the issue afterwards, but it's a very flawed way of thinking. There will always be root causes you cannot predict, and so approaching reliability as a series of layers (like they do in for example the airline industry) is a much healthier approach I think.
The failure is hardly yours to own outright. The process itself was begging for something like this to happen.
If you didn't do it, it would be eventually be someone else.
If you're all working directly against the production database, it's inevitable that something like this would have happened. At the very least, there should have been some restrictions in place regarding what operations what users could do on the database.
The founders should have apologised to the entire company AND openly absolve you from any guilt - as they created the environment that led to the failure. Clear cut.
Sounds like the founders were incompetent, that's unfortunate but not very uncommon. I hope you've found healthier workplaces since!
I'm really sorry you had to go through this. However, it's clear you learned a lot from it. We all did.
Many people in IT industry has this kind of horror stories in their background. It changes you.
It is also worth to keep this in mind:
Many of such horror stories could have been prevented with better onboarding and security practices, better backup practices, better sandboxing, better everything...
But above all this, thanks and congratulations for staying honest, speaking up and working hard to help, understand the consequences and learn.
You definitely have a great professional career ahead, I'm sure.