[AusNOG] [patchmanagement] ntp server issues today... strange one... clutching at straws but just in case

Fri Feb 1 13:17:04 EST 2019

Single PDC site... Hit the PDC every 30 mins or so - forward, back,
forward, back etc
then of course started changing time on all domain members shortly after -
each time.
Problem is it was not instant for all members and the AD-integrated
Synology NAS....
Backups broke, complaints from 20 users randomly every 30 mins until
isolated.

Cluster 3.au.pool.ntp.org has been fine since 3.39pm Brisbane time
yesterday.
I'll just ignore the 0. for now and wait for someone @ ntp.org to spot it I
think

Could just be specific to win2008r2Sp1 - who knows.
AU Admins, you have been warned :)
Enuf of my time wasted on it

Thanks for all the comments and PM's
I have actually picked up a lot of tips from you all - many thanks

Kindly,

ROY ADAMS* | *P 07 3040 5010  | Web: http://www.racs.com.au/ | Wiki:
https://ex.racs.com.au:444/ | eMail: mailto:roy at racs.com.au
<roy at racs.com.au>
Please never upgrade to the latest Windows 10 - You don’t need the hassle,
and I don’t need the work.
If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur - Red Adair.
Life is a journey through a series of adventures.. Live them, love them,
hate them, but never give up on your dreams, desires, and goals.

On Fri, 1 Feb 2019 at 00:54, Joseph Daly <JDaly at arrowstreetcapital.com>
wrote:

> One small thing and this is probably just the wording of the email.
>
>
>
>
>
> *I always use the below config for domain controllers:*
>
> *sc config W32Time start= auto & net start W32Time*
>
> *w32tm /config /manualpeerlist:"0.au.pool.ntp.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=U499vWTuQrHOdlAPlmDHrA-rgZbLYU7PaXpE2Kd48eM&e=>
> 2.au.pool.ntp.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__2.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PRGGsc1Vf_jVqorVPZnTpw7JvXoa49lzKAVZTXF0gUs&e=>
> 3.au.pool.ntp.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__3.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=B3Zd-DIP0O9MfiOMufSpJ84RI0He4iXoMtlIv_CdbB0&e=>"
> /syncfromflags:manual /reliable:yes /update & w32tm /resync /nowait*
>
>
>
> All DCs or just your PDC emulator? Usually the other DCs sync from the PDC
> emulator.
>
>
>
>
>
> *From:* Roy Adams <roy at racs.com.au>
> *Sent:* Thursday, January 31, 2019 1:33 AM
> *To:* Patch Management Mailing List <
> patchmanagement at listserv.patchmanagement.org>
> *Subject:* Re:[patchmanagement] [AusNOG] ntp server issues today...
> strange one... clutching at straws but just in case
>
>
>
> Thanks for the PM's offering ideas
>
> I am tempted to set it back to 0. to debug the offending ntp pool IP, but
> it was breaking all the backups among other things due to AD sync being
> more than 5 mins out.
>
>
>
> I always use the below config for domain controllers:
>
> sc config W32Time start= auto & net start W32Time
>
> w32tm /config /manualpeerlist:"0.au.pool.ntp.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=U499vWTuQrHOdlAPlmDHrA-rgZbLYU7PaXpE2Kd48eM&e=>
> 2.au.pool.ntp.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__2.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PRGGsc1Vf_jVqorVPZnTpw7JvXoa49lzKAVZTXF0gUs&e=>
> 3.au.pool.ntp.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__3.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=B3Zd-DIP0O9MfiOMufSpJ84RI0He4iXoMtlIv_CdbB0&e=>"
> /syncfromflags:manual /reliable:yes /update & w32tm /resync /nowait
>
>
>
> One of the replies noted that linux sanity checks by getting ntp time from
> 4 servers - I wish MS were that smart.
>
> Clearly MS are not using all the configured servers, I suspect they are
> purely for failover like a DNS client.
>
>
>
> I have just changed this site to:
>
> w32tm /config /manualpeerlist:"3.au.pool.ntp.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__3.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=B3Zd-DIP0O9MfiOMufSpJ84RI0He4iXoMtlIv_CdbB0&e=>"
> /syncfromflags:manual /reliable:yes /update & w32tm /resync /nowait
>
> So far so good.. still stable
>
>
>
> All Domain members of course sync to the DC
>
> I am not seeing this on any other sites.. all sites are cookie cutter for
> me
>
>
>
>
>
> event logs confirm ONLY the change... not the server IP :(
>
> The system time has changed to ‎2019‎-‎01‎-‎31T01:47:11.254922100Z from
> ‎2019‎-‎01‎-‎31T02:18:29.514800000Z.
>
> The system time has changed to ‎2019‎-‎01‎-‎31T01:47:11.254000000Z from
> ‎2019‎-‎01‎-‎31T01:47:11.254922100Z.
>
> The system time has changed to ‎2019‎-‎01‎-‎31T03:43:51.747000000Z from
> ‎2019‎-‎01‎-‎31T03:12:32.312621000Z.
>
> The system time has changed to ‎2019‎-‎01‎-‎31T03:36:17.703840400Z from
> ‎2019‎-‎01‎-‎31T04:07:36.105000000Z.
>
> The system time has changed to ‎2019‎-‎01‎-‎31T03:36:17.703000000Z from
> ‎2019‎-‎01‎-‎31T03:36:17.703840400Z.
>
> The system time has changed to ‎2019‎-‎01‎-‎31T05:41:23.075000000Z from
> ‎2019‎-‎01‎-‎31T05:10:04.617935900Z.
>
> The system time has changed to ‎2019‎-‎01‎-‎31T06:01:12.107000000Z from
> ‎2019‎-‎01‎-‎31T06:01:12.107000000Z.
>
> The system time has changed to ‎2019‎-‎01‎-‎31T05:30:09.707385800Z from
> ‎2019‎-‎01‎-‎31T06:01:28.112628100Z.
>
> The system time has changed to ‎2019‎-‎01‎-‎31T05:30:09.707000000Z from
> ‎2019‎-‎01‎-‎31T05:30:09.707385800Z.
>
> The system time has changed to ‎2019‎-‎01‎-‎31T05:39:51.770000000Z from
> ‎2019‎-‎01‎-‎31T05:39:51.770276000Z.
>
>
>
>
>
>
>
>
>
>
>
> Kindly,
>
>
>
> ROY ADAMS* | *P 07 3040 5010  | Web: http://www.racs.com.au/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.racs.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=ZRKXJDH4ciRXiwcDhbLIwFHIvgqzrytOtvGja-WyEso&e=>
>  | Wiki: https://ex.racs.com.au:444/
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__ex.racs.com.au-3A444_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=DsV_izPAHTAwk-h02V5W_v5P98BrMy1Ul7Kol0HTTmc&e=> |
> eMail: mailto:roy at racs.com.au <roy at racs.com.au>
>
> Please never upgrade to the latest Windows 10 - You don’t need the hassle,
> and I don’t need the work.
> If you think it's expensive to hire a professional to do the job, wait
> until you hire an amateur - Red Adair.
>
> Life is a journey through a series of adventures.. Live them, love them,
> hate them, but never give up on your dreams, desires, and goals.
>
>
>
>
>
>
>
> On Thu, 31 Jan 2019 at 16:13, Nick Stallman <nick at agentpoint.com> wrote:
>
> Do you know which server specifically? The ntp pools hand out random NTP
> server IPs, it's not a fixed server.
>
> I'm not a Windows server admin, but this would likely be why Linux
> connects to ~4 NTP servers so it can disregard dodgy servers.
>
> On 31/1/19 5:09 pm, Roy Adams wrote:
>
> Hi All, I have a domain controller *seemingly* receiving bad time info
> today from 0.au.pool.ntp.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=U499vWTuQrHOdlAPlmDHrA-rgZbLYU7PaXpE2Kd48eM&e=>
>
> Issuing this confirmed the time was flapping forward 30 mins, then 30 mins
> later back to normal:
>
> w32tm /query /status
>
> It confirmed the above ntp server as the server that supplied the bad
> (then good, then bad, then good etc) time
>
> I have now changed the DC to pull instead from 3.au.pool.ntp.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=vF1MfzyyrPVr436Gt8h40rxV1qMJ68GEs4Gu9MqWD-k&e=>
> .
>
> 1 hour has passed and so far so good.
>
>
>
> Cannot say I have ever seen anything like this...
>
> It's only occurring on one site on a windows2008r2sp1 domain controller.
>
> The DC in turn relays this updated time to all domain members of course.
>
> Anyone else had time issues on any sites today in Aus?
>
>
>
>
>
> Kindly,
>
>
>
> ROY ADAMS* | *P 07 3040 5010  | Web: http://www.racs.com.au/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.racs.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=ZRKXJDH4ciRXiwcDhbLIwFHIvgqzrytOtvGja-WyEso&e=>
>  | Wiki: https://ex.racs.com.au:444/
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__ex.racs.com.au-3A444_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=DsV_izPAHTAwk-h02V5W_v5P98BrMy1Ul7Kol0HTTmc&e=> |
> eMail: mailto:roy at racs.com.au <roy at racs.com.au>
>
> Please never upgrade to the latest Windows 10 - You don’t need the hassle,
> and I don’t need the work.
> If you think it's expensive to hire a professional to do the job, wait
> until you hire an amateur - Red Adair.
>
> Life is a journey through a series of adventures.. Live them, love them,
> hate them, but never give up on your dreams, desires, and goals.
>
>
>
>
>
> _______________________________________________
>
> AusNOG mailing list
>
> AusNOG at lists.ausnog.net
>
> http://lists.ausnog.net/mailman/listinfo/ausnog <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ausnog.net_mailman_listinfo_ausnog&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=FeOndl6vwYUzDu74O11bqvYM6U3YN9aOiq9rAI3KKvw&e=>
>
> --
>
> *Nick Stallman*
>
> *Technical Director*
>
> nick at agentpoint.com
>
> 02 8039 6820 <0280396820>
>
> www.agentpoint.com.au
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>
> Level 3, 100 Harris Street, Pyrmont NSW 2009
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=>
>
> Arrowstreet Capital,LP-DISCLAIMER:
> ==============================
> This email message and its attachments are being sent by Arrowstreet
> Capital, Limited Partnership and are confidential and proprietary. If you
> are not the intended recipient, please notify us immediately by replying to
> this message and destroy all copies of this message and any attachments.
> Thank you.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20190201/a66ee67b/attachment.html>