[AusNOG] 10GBs

Thu Nov 9 10:24:16 EST 2017

You're over-thinking the costs associated with it.

My work with 10Gbps fibre was for a small company (200 staff in Australia,
5000 globally).  We used it locally within DCs, and also between DCs, so I
was working with SR and LR optics (and multi-mode and single-mode
respectively).

You can pick up a fibre scope and cleaning kit for < $500.

If you're talking about work within a server room/datacentre, you may also
want to consider 10Gbps over copper (either CAT6 or Twinax).

If you stick with fibre, you also need to make sure that you're running the
right type of cable.  Single-mode is for long range or extended range
(generally between DCs), multi-mode is for short range (within a server
room/data centre).

Fibre isn't expensive - but nor is it "simple" or for people who don't
understand what they're doing with it.  You can beat that aspect by simply
doing lots of reading (and asking questions - you'll get plenty of help
here - off list if you wish. :)

On 9 November 2017 at 09:59, Burt Mascareigne <Burt at stormnetwork.com.au>
wrote:

> Hey All
>
>
>
> No, we have not done optical loss testing.  We’re a IaaS and thought
> instead of Etherchanneling 4 x 1GB, we’d go crazy and try 10GB.
>
>
>
> We’re not a multimillion $ setup, it is something we wanted to dabble in,
> as all the documents seem to suggest Plug and Play.  I’m not really sure
> anymore, it seems AusNog is just for huge companies only.
>
>
>
> Just to clarify, it is in Intel Card with Cisco SFPs connecting to a ZyXEL
> off MM with a 3meter range.
>
>
>
> Based on the feed back:
>
>
>
> Intel don’t play nice with anything but Intel, but it “should” work.
>
> It’s probably one of the tech cleaning the fibre endpoints with their
> fingers
>
> And no one has even mentioned ZyXEL, which speaks volumes…
>
>
>
> We keep seeing “Mellanox” everywhere.  Looking at going end to end with
> them, new MM cables etc.
>
>
>
> Thanks all, it was not quite plug and play as I thought it would be, but
> we’re learning.
>
>
>
>
>
>
>
> Regards,
>
>
>
>
>
>
> *Burt Mascareigne Mobile* 0414 450 962   *Office* (02) 9965 5422
> *Address* Level 19, 1 O’Connell Street, Sydney NSW 2000
> <https://maps.google.com/?q=Level+19,+1+O%E2%80%99Connell+Street,+Sydney+NSW+2000&entry=gmail&source=g>
> *Web* http://www.stormnetwork.com.au
> [image: 01-300x70]
>
>
>
>
>
> *From:* AusNOG [mailto:ausnog-bounces at lists.ausnog.net] *On Behalf Of *Robert
> Hudson
> *Sent:* Thursday, 9 November 2017 9:42 AM
> *To:* Mike Taylor <mtaylor at totalteam.co.nz>
> *Cc:* ausnog at lists.ausnog.net
> *Subject:* Re: [AusNOG] 10GBs
>
>
>
> Grab an optical fibre scope while you're at it. It's amazing how dirty
> your cable ends and recepticals can actually be.
>
>
>
> And for the love of $deity, make sure nobody's done something as stupid
> (been there, done that - I had an excuse because I was new to working with
> fibre, the major carrier datacentre operators less so - yes, they did it
> too) as plugging SC cables into SCA recepticals or vice versa.
>
>
>
> Have you done optical loss testing? Some network gear can tell you
> directly what your tx and rx power levels are.
>
>
>
> On 9 Nov. 2017 7:03 am, "Mike Taylor" <mtaylor at totalteam.co.nz> wrote:
>
> So, yeah, Layer1 first, apply your fibre cleaning tools to the fibres,
> SFPs, patch leads etc.
>
> 10gig gets very very twitchy if the cables and SFPs aren't cleaned
> properly...
>
> Better still, swap out all the patch leads for brand new ones and see what
> happens...
>
> Mike
>
>
>
> Mike Taylor
>
> The Total Team
>
>
>
> On 08/11/17 15:59, Robert Hudson wrote:
>
> There is no trial and error. Start at Layer 1, and work up the stack.
>
>
>
> Correct setup and good logging should resolve it.
>
>
>
> There are lots of potential issues. You can cut some of them down
> immediately by selecting hardware on the hypervisor supplier's HCL - they
> should have good drivers for the 10Gbps NICs at that point.
>
>
>
> Speed/duplex, frame size, etc, all need to be consistent across the entire
> environment. Get wireshark or an equivalent involved (using a mirror port
> if necessary).
>
>
>
> If your hardware and software are all under full support, involve your
> vendors in the issue.
>
>
>
> On 8 Nov. 2017 1:23 pm, "Burt Mascareigne" <Burt at stormnetwork.com.au>
> wrote:
>
> Hi Noggers
>
>
>
> We’re having “fun” with 10GB NICS, Switches and SFPs.
>
>
>
> We can’t work out which is causing drop out, packets.. we are somewhat
> certain packets are vanishing into the Ether..
>
>
>
> We don’t know which is causing what.
>
>
>
> I’m not after troubleshooting, I am curious what people’s view on this.
>
>
>
> Do you buy 1 Vendor for all your 10GBs, A-Z, is there some hidden rule
> about mixing vendors, is this all trial and error (and very expensive at
> best)?
>
>
>
> Any recommendations on set ups?  Preferred Vendors, Vender to avoid like
> the plague.
>
>
>
> We’re talking Servers to Switch kind of thing.  Not SANs etc.  Good ol’
> HyperVisor to the switch.
>
>
>
> Thanks
>
>
>
>
>
>
>
>
>
>
>
> Regards,
>
>
>
>
>
>
> *Burt Mascareigne Mobile* 0414 450 962   *Office* (02) 9965 5422
> *Address* Level 19, 1 O’Connell Street, Sydney NSW 2000
> <https://maps.google.com/?q=Level+19,+1+O%E2%80%99Connell+Street,+Sydney+NSW+2000&entry=gmail&source=g>
> *Web* http://www.stormnetwork.com.au
> [image: 01-300x70]
>
>
>
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
>
> _______________________________________________
>
> AusNOG mailing list
>
> AusNOG at lists.ausnog.net
>
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
>
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20171109/12f19d83/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 5873 bytes
Desc: not available
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20171109/12f19d83/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 10463 bytes
Desc: not available
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20171109/12f19d83/attachment-0001.png>