[talk] FB outage

[Warner Losh]

The rumors from my side. I've not verified them all (some are in the
twitter feed). We'll see how reliable this is...

But, Facebook does all their own tooling, and it's all down.
Even internal chat. So they are talking directly via cell phones (system to
lookup cell numbers, also down)
The only way that can provision the bgp routes is via the console of the
routers at this point...
... but the internal system to get credentials is also down.
They also need to get someone in front of the console to bring things back
online...
Their tools to coordinate and meet to come up with a plan, also down....

Single point of failure in the extreme.


On Mon, Oct 4, 2021 at 3:20 PM Michael W. Lucas <mwlucas at michaelwlucas.com>
wrote:

>
> <blink> <blink>
>
> That's it. Twitter is the next to go.
>
> ==ml
>
>
> On Mon, Oct 04, 2021 at 04:15:39PM -0400, George Rosamond wrote:
> > One of the cooler things we can all learn from the Facebook/Instagram/WA
> > outage is some technical innovations most of you probably missed.
> >
> > https://nitter.eu/briankrebs/status/1445077617426718725
> >
> > Apparently, there are no RFCs for it, but DNS Global Routing Tables is
> > the place to be. Move over blockchain.
> >
> > g
> >
> > _______________________________________________
> > talk mailing list
> > talk at lists.nycbug.org
> > http://lists.nycbug.org:8080/mailman/listinfo/talk
>
> --
> Michael W. Lucas        https://mwl.io/
> author of: Absolute OpenBSD, SSH Mastery, git commit murder,
>  Absolute FreeBSD, Immortal Clay, Prohibition Orcs, etc, etc, etc...
> ### New books: TLS Mastery, the Networknomicon, $ git sync murder ###
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org:8080/mailman/listinfo/talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nycbug.org/pipermail/talk/attachments/20211004/c1bde62e/attachment.htm>