Fiber provider is absolutely shitting the bed today [Services Degraded]
Fiber provider is absolutely shitting the bed today [Services Degraded]
This is the second time today I've had to cut over to the backup WAN (which is much slower). Fiber provider is royally screwing the pooch today, and I currently have no ETA. The only silver lining is it's a widespread issue, so hopefully they're inclined to address the problem in a timely manner.
To relieve congestion on the backup connection, I've temporarily disabled external access to pictrs. All requests (GET or POST) to /pictrs
will return a 404
response until the main connection is back online.
Sorry for the inconvenience. Please direct all rage at the shit fiber provider I foolishly "upgraded" to.
Status as of 5:17PM:
Service | Status | Notes |
---|---|---|
Lemmy ActivityPub | Up (Degraded) | Running on backup WAN |
Lemmy API | Up (Degraded) | Backup WAN |
Pict-rs | Down | Intentionally scuttled to save bandwidth |
Tesseract Media Proxy | Down | Intentionally scuttled |
Matrix Server | Up (Degraded) | Backup WAN |
Matrix Web Client | Up (Degraded) | Backup WAN |
DNS | Up (Degraded) | Operationally up, but no backup servers available |
SMTP | Up (Degraded) | Backup WAN |
Update: 4:30 PM: Tried to call and re-activate my account on my old provider since I own the equipment, and it's all still hooked up. I can even still access my old account on the web portal. Should be easy, right? Fucking wrong! They said they couldn't activate it until they sent me equipment, I returned it, and then called back to say I want to use my own. What the actual fuck?? I'm literally waving money at you, you're still charging me a $100 "install" fee (where you have to do nothing), and all you have to do is take it (and my equipment serial number). Fuck my life (and fuck those idiots, too).
Update 5:17 PM: We remain on backup WAN. ETA for fiber restoration is within 24 hours, but I'm not holding my breath. Like, how do you even fuck up this badly? (the whole goddamned state is affected including business customers)
Update 7:39 PM: Primary connection is back online (for now?). Services have successfully failed back over.
Service | Status | Notes |
---|---|---|
Lemmy ActivityPub | Up | - |
Lemmy API | Up | - |
Pict-rs | Up | - |
Tesseract Media Proxy | Up | - |
Matrix Server | Up | - |
Matrix Web Client | Up | - |
DNS | Up | - |
SMTP | Up | - |