"There was an emergency and power was temporarily cut to the card reader terminals, and now none of them work. We need to get to the server room to reboot the access control server, but it's behind a card-only door."
"How much do we want to get in?"
"Very much."
"Stand back."And that's how I ended up busting down the server room's door.
I'm currently working with a client that doesn't have a health endpoint or any kind of monitoring on their new API . They say monitoring isn't needed because it will never go down.
Naturally it went down on day two. They still haven't added any "unnecessary" monitoring, insisting that it will never go down.
If you never know when it goes down to you it never goes down
Think smarter, not harder
Schrödinger's status
This kind of hubris may be an opportunity for contractual fuckery...
I never wrote an api that had a health system. Could you help me understand why that matters and how that helps ?
Just have an endpoint in your API (like
/health) that doesn’t do anything but return “ok”.So if your database goes down, your filesystem is full, etc, that endpoint will always return “ok” with HTTP 200. That way you can setup a ping monitoring service that will trigger an alarm if the process itself is down.
You of course need more pinging for the database server etc. But at least you know which service is down instead of “the whole website is down and we don’t know which parts”.
That's because you don't write bugs. Health check are only needed when you're planning on having bugs in your system. Instead of doing monitoring I prefer to spend a bit more time fixing all the bugs and then my systems never break so no monitoring is needed. Of course the downside is lack of job security. They can fire me and the system will just continue running forever, no support needed. If you add some bugs they cannot fire you because someone needs to keep fixing the broken system.
Source (with alt-text on site): https://xkcd.com/705/
Thanks every couple of months I see an xkcd and think haven't read that in a while, then I get to go catch up. Today was a good day
Haha! Same!
People argue about repost bots. I'm happy when I see a comic I haven't thought about in years get upvotes and more people discover it.
after several years it's fine, but not two days after the initial post
I do the exact same thing
That's why you have failover; you never have to fight terrorists to keep your contractual uptime agreements!
tf2 engineer was just a sysadmin trying to create a good failover
Haha lol
"Now I have an access point, ho ho ho!"
Time for shenanigans
