[AusNOG] Global DNS yuck?

[Luke Thompson]

cPanel also failed to plan for the expiry, so we're seeing workarounds 
then revocations (oops, that didn't work - etc). Still no real headway 
after 12~ hours.

The root cert expiry was a long time coming, though if you check Twitter 
it seems like it's caught many out.

Cheers,

Luke Thompson
Operations Manager


On 1/10/21 12:40 pm, Lachlan Gilmour wrote:
> I believe it is related to the Lets Encrypt root cert that expired 
> overnight.
>
> I've seen quite a few older devices today having issues accessing 
> sites using lets encrypt certs.
>
> More info on the issue can be found here: 
> https://docs.certifytheweb.com/docs/kb/kb-202109-letsencrypt/ 
> <https://docs.certifytheweb.com/docs/kb/kb-202109-letsencrypt/>
>
> On Fri, Oct 1, 2021 at 12:36 PM Mark Andrews <marka at isc.org 
> <mailto:marka at isc.org>> wrote:
>
>     More correctly they had working DNSSEC deployed
>     (https://dnsviz.net/d/slack.com/YVXX_g/dnssec/
>     <https://dnsviz.net/d/slack.com/YVXX_g/dnssec/>) and then pulled
>     both the DS records for slack.com <http://slack.com> and the
>     DNSSEC records in slack.com <http://slack.com> AT THE SAME TIME
>     resulting in DNSSEC validation failures. Cached DS records said
>     slack.com <http://slack.com> is signed but the answers from the
>     slack.com <http://slack.com> servers where missing the DNSSEC
>     records. They failed to wait for the DS records to expire from DNS
>     caches before removing the DNSSEC records in slack.com
>     <http://slack.com>. Failure to wait for unsigned responses to
>     clear caches before publishing DS records can also cause issues
>     with multiple levels of caching.
>
>     > On 1 Oct 2021, at 08:23, Scott Howard <scott at doc.net.au
>     <mailto:scott at doc.net.au>> wrote:
>     >
>     > They broke (and subsequently fixed) their DNSSEC configuration
>     many hours ago, but it was broken long enough to get cached by
>     some servers for up to 24 hours so some users are still having
>     issues connecting.
>     >
>     > Short of the classic "have your ISP clear their DNS cache" not
>     much anyone can do except wait it out...
>     >
>     > https://status.slack.com/2021-09/06c1e17de93e7dc2
>     <https://status.slack.com/2021-09/06c1e17de93e7dc2>
>     >
>     >   Scott
>     >
>     >
>     > On Thu, Sep 30, 2021 at 3:19 PM Andrew Yager <andrew at rwts.com.au
>     <mailto:andrew at rwts.com.au>> wrote:
>     > Hi,
>     >
>     > Slack is down and finding a few other (non slack) services etc
>     being broken seemingly with DNS things. Anyone know what’s going on?
>     >
>     > A
>     > _______________________________________________
>     > AusNOG mailing list
>     > AusNOG at lists.ausnog.net <mailto:AusNOG at lists.ausnog.net>
>     > http://lists.ausnog.net/mailman/listinfo/ausnog
>     <http://lists.ausnog.net/mailman/listinfo/ausnog>
>     > _______________________________________________
>     > AusNOG mailing list
>     > AusNOG at lists.ausnog.net <mailto:AusNOG at lists.ausnog.net>
>     > http://lists.ausnog.net/mailman/listinfo/ausnog
>     <http://lists.ausnog.net/mailman/listinfo/ausnog>
>
>     -- 
>     Mark Andrews, ISC
>     1 Seymour St., Dundas Valley, NSW 2117, Australia
>     PHONE: +61 2 9871 4742              INTERNET: marka at isc.org
>     <mailto:marka at isc.org>
>
>     _______________________________________________
>     AusNOG mailing list
>     AusNOG at lists.ausnog.net <mailto:AusNOG at lists.ausnog.net>
>     http://lists.ausnog.net/mailman/listinfo/ausnog
>     <http://lists.ausnog.net/mailman/listinfo/ausnog>
>
>
>
> -- 
>
> Lachlan Gilmour
>
> w 	: surfpacific.com.au <https://surfpacific.com.au/>
> p 	: +61 7 5571 1161 <tel:+61755711161>
> f 	: +61 7 5676 6652
> e 	: lachlan.gilmour at surfpacific.com.au 
> <mailto:lachlan.gilmour at surfpacific.com.au>
> a 	
>
> : Suite 30307, Level 3, Tower 3 Southport Central Commercial,
>   9 Lawson Street, Southport, Queensland 4215, Australia.
>
> <http://remote.surfpacific.com/>
> ------------------------------------------------------------------------
> *Legal Notice:* If this email message is received by other than the 
> named addressee(s), then the recipient is requested immediately to 
> notify us and delete the email from the recipient’s computer memory 
> and to destroy all hard and other copies of it. Privilege is not 
> waived or lost by reason of a mistaken delivery or transmission to 
> other than the addressee. Please
>
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20211001/a3488a97/attachment.html>