[AusNOG] [patchmanagement] ntp server issues today... strange one... clutching at straws but just in case

Fri Feb 1 13:31:50 EST 2019

As someone who runs a few Stratum 1 and 2 servers, pool.ntp.org checks my
servers every 15 minutes or so, and if they're more than 50ms out, they're
de-prioritized, until they're removed. It's all completely automatic.

Example: https://www.ntppool.org/scores/203.4.240.10

I think this is more an issue on your end, honestly.

--Rob

On Fri, 1 Feb 2019 at 12:17, Roy Adams <roy at racs.com.au> wrote:

> Single PDC site... Hit the PDC every 30 mins or so - forward, back,
> forward, back etc
> then of course started changing time on all domain members shortly after -
> each time.
> Problem is it was not instant for all members and the AD-integrated
> Synology NAS....
> Backups broke, complaints from 20 users randomly every 30 mins until
> isolated.
>
> Cluster 3.au.pool.ntp.org has been fine since 3.39pm Brisbane time
> yesterday.
> I'll just ignore the 0. for now and wait for someone @ ntp.org to spot it
> I think
>
> Could just be specific to win2008r2Sp1 - who knows.
> AU Admins, you have been warned :)
> Enuf of my time wasted on it
>
> Thanks for all the comments and PM's
> I have actually picked up a lot of tips from you all - many thanks
>
>
> Kindly,
>
> ROY ADAMS* | *P 07 3040 5010  | Web: http://www.racs.com.au/ | Wiki:
> https://ex.racs.com.au:444/ | eMail: mailto:roy at racs.com.au
> <roy at racs.com.au>
> Please never upgrade to the latest Windows 10 - You don’t need the hassle,
> and I don’t need the work.
> If you think it's expensive to hire a professional to do the job, wait
> until you hire an amateur - Red Adair.
> Life is a journey through a series of adventures.. Live them, love them,
> hate them, but never give up on your dreams, desires, and goals.
>
>
>
> On Fri, 1 Feb 2019 at 00:54, Joseph Daly <JDaly at arrowstreetcapital.com>
> wrote:
>
>> One small thing and this is probably just the wording of the email.
>>
>>
>>
>>
>>
>> *I always use the below config for domain controllers:*
>>
>> *sc config W32Time start= auto & net start W32Time*
>>
>> *w32tm /config /manualpeerlist:"0.au.pool.ntp.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=U499vWTuQrHOdlAPlmDHrA-rgZbLYU7PaXpE2Kd48eM&e=>
>> 2.au.pool.ntp.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__2.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PRGGsc1Vf_jVqorVPZnTpw7JvXoa49lzKAVZTXF0gUs&e=>
>> 3.au.pool.ntp.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__3.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=B3Zd-DIP0O9MfiOMufSpJ84RI0He4iXoMtlIv_CdbB0&e=>"
>> /syncfromflags:manual /reliable:yes /update & w32tm /resync /nowait*
>>
>>
>>
>> All DCs or just your PDC emulator? Usually the other DCs sync from the
>> PDC emulator.
>>
>>
>>
>>
>>
>> *From:* Roy Adams <roy at racs.com.au>
>> *Sent:* Thursday, January 31, 2019 1:33 AM
>> *To:* Patch Management Mailing List <
>> patchmanagement at listserv.patchmanagement.org>
>> *Subject:* Re:[patchmanagement] [AusNOG] ntp server issues today...
>> strange one... clutching at straws but just in case
>>
>>
>>
>> Thanks for the PM's offering ideas
>>
>> I am tempted to set it back to 0. to debug the offending ntp pool IP, but
>> it was breaking all the backups among other things due to AD sync being
>> more than 5 mins out.
>>
>>
>>
>> I always use the below config for domain controllers:
>>
>> sc config W32Time start= auto & net start W32Time
>>
>> w32tm /config /manualpeerlist:"0.au.pool.ntp.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=U499vWTuQrHOdlAPlmDHrA-rgZbLYU7PaXpE2Kd48eM&e=>
>> 2.au.pool.ntp.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__2.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PRGGsc1Vf_jVqorVPZnTpw7JvXoa49lzKAVZTXF0gUs&e=>
>> 3.au.pool.ntp.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__3.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=B3Zd-DIP0O9MfiOMufSpJ84RI0He4iXoMtlIv_CdbB0&e=>"
>> /syncfromflags:manual /reliable:yes /update & w32tm /resync /nowait
>>
>>
>>
>> One of the replies noted that linux sanity checks by getting ntp time
>> from 4 servers - I wish MS were that smart.
>>
>> Clearly MS are not using all the configured servers, I suspect they are
>> purely for failover like a DNS client.
>>
>>
>>
>> I have just changed this site to:
>>
>> w32tm /config /manualpeerlist:"3.au.pool.ntp.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__3.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=B3Zd-DIP0O9MfiOMufSpJ84RI0He4iXoMtlIv_CdbB0&e=>"
>> /syncfromflags:manual /reliable:yes /update & w32tm /resync /nowait
>>
>> So far so good.. still stable
>>
>>
>>
>> All Domain members of course sync to the DC
>>
>> I am not seeing this on any other sites.. all sites are cookie cutter for
>> me
>>
>>
>>
>>
>>
>> event logs confirm ONLY the change... not the server IP :(
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T01:47:11.254922100Z from
>> ‎2019‎-‎01‎-‎31T02:18:29.514800000Z.
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T01:47:11.254000000Z from
>> ‎2019‎-‎01‎-‎31T01:47:11.254922100Z.
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T03:43:51.747000000Z from
>> ‎2019‎-‎01‎-‎31T03:12:32.312621000Z.
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T03:36:17.703840400Z from
>> ‎2019‎-‎01‎-‎31T04:07:36.105000000Z.
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T03:36:17.703000000Z from
>> ‎2019‎-‎01‎-‎31T03:36:17.703840400Z.
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T05:41:23.075000000Z from
>> ‎2019‎-‎01‎-‎31T05:10:04.617935900Z.
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T06:01:12.107000000Z from
>> ‎2019‎-‎01‎-‎31T06:01:12.107000000Z.
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T05:30:09.707385800Z from
>> ‎2019‎-‎01‎-‎31T06:01:28.112628100Z.
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T05:30:09.707000000Z from
>> ‎2019‎-‎01‎-‎31T05:30:09.707385800Z.
>>
>> The system time has changed to ‎2019‎-‎01‎-‎31T05:39:51.770000000Z from
>> ‎2019‎-‎01‎-‎31T05:39:51.770276000Z.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Kindly,
>>
>>
>>
>> ROY ADAMS* | *P 07 3040 5010  | Web: http://www.racs.com.au/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.racs.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=ZRKXJDH4ciRXiwcDhbLIwFHIvgqzrytOtvGja-WyEso&e=>
>>  | Wiki: https://ex.racs.com.au:444/
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__ex.racs.com.au-3A444_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=DsV_izPAHTAwk-h02V5W_v5P98BrMy1Ul7Kol0HTTmc&e=> |
>> eMail: mailto:roy at racs.com.au <roy at racs.com.au>
>>
>> Please never upgrade to the latest Windows 10 - You don’t need the
>> hassle, and I don’t need the work.
>> If you think it's expensive to hire a professional to do the job, wait
>> until you hire an amateur - Red Adair.
>>
>> Life is a journey through a series of adventures.. Live them, love them,
>> hate them, but never give up on your dreams, desires, and goals.
>>
>>
>>
>>
>>
>>
>>
>> On Thu, 31 Jan 2019 at 16:13, Nick Stallman <nick at agentpoint.com> wrote:
>>
>> Do you know which server specifically? The ntp pools hand out random NTP
>> server IPs, it's not a fixed server.
>>
>> I'm not a Windows server admin, but this would likely be why Linux
>> connects to ~4 NTP servers so it can disregard dodgy servers.
>>
>> On 31/1/19 5:09 pm, Roy Adams wrote:
>>
>> Hi All, I have a domain controller *seemingly* receiving bad time info
>> today from 0.au.pool.ntp.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=U499vWTuQrHOdlAPlmDHrA-rgZbLYU7PaXpE2Kd48eM&e=>
>>
>> Issuing this confirmed the time was flapping forward 30 mins, then 30
>> mins later back to normal:
>>
>> w32tm /query /status
>>
>> It confirmed the above ntp server as the server that supplied the bad
>> (then good, then bad, then good etc) time
>>
>> I have now changed the DC to pull instead from 3.au.pool.ntp.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=vF1MfzyyrPVr436Gt8h40rxV1qMJ68GEs4Gu9MqWD-k&e=>
>> .
>>
>> 1 hour has passed and so far so good.
>>
>>
>>
>> Cannot say I have ever seen anything like this...
>>
>> It's only occurring on one site on a windows2008r2sp1 domain controller.
>>
>> The DC in turn relays this updated time to all domain members of course.
>>
>> Anyone else had time issues on any sites today in Aus?
>>
>>
>>
>>
>>
>> Kindly,
>>
>>
>>
>> ROY ADAMS* | *P 07 3040 5010  | Web: http://www.racs.com.au/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.racs.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=ZRKXJDH4ciRXiwcDhbLIwFHIvgqzrytOtvGja-WyEso&e=>
>>  | Wiki: https://ex.racs.com.au:444/
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__ex.racs.com.au-3A444_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=DsV_izPAHTAwk-h02V5W_v5P98BrMy1Ul7Kol0HTTmc&e=> |
>> eMail: mailto:roy at racs.com.au <roy at racs.com.au>
>>
>> Please never upgrade to the latest Windows 10 - You don’t need the
>> hassle, and I don’t need the work.
>> If you think it's expensive to hire a professional to do the job, wait
>> until you hire an amateur - Red Adair.
>>
>> Life is a journey through a series of adventures.. Live them, love them,
>> hate them, but never give up on your dreams, desires, and goals.
>>
>>
>>
>>
>>
>> _______________________________________________
>>
>> AusNOG mailing list
>>
>> AusNOG at lists.ausnog.net
>>
>> http://lists.ausnog.net/mailman/listinfo/ausnog <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ausnog.net_mailman_listinfo_ausnog&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=FeOndl6vwYUzDu74O11bqvYM6U3YN9aOiq9rAI3KKvw&e=>
>>
>> --
>>
>> *Nick Stallman*
>>
>> *Technical Director*
>>
>> nick at agentpoint.com
>>
>> 02 8039 6820 <0280396820>
>>
>> www.agentpoint.com.au
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>>
>>
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>>
>>
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>>
>> Level 3, 100 Harris Street, Pyrmont NSW 2009
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>>
>>
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>>
>>
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>>
>> Arrowstreet Capital,LP-DISCLAIMER:
>> ==============================
>> This email message and its attachments are being sent by Arrowstreet
>> Capital, Limited Partnership and are confidential and proprietary. If you
>> are not the intended recipient, please notify us immediately by replying to
>> this message and destroy all copies of this message and any attachments.
>> Thank you.
>>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20190201/f33aa25b/attachment.html>