ThousandEyes is a network monitoring company who’s shining a light on the darkened portion of the network path you don’t own. Rather than the Internet appearing to an enterprise as a generic cloud where magic happens, ThousandEyes looks inside the cloud, revealing details about how your enterprise gets to remote services.
For example, most network engineers know that when traffic leaves their network, it will traverse a series of Internet routers before arriving at its destination. A simple traceroute reveals this, although a traceroute doesn’t reveal too much more.
synack-rmbp15:~ ebanks$ traceroute 8.8.4.4 traceroute to 8.8.4.4 (8.8.4.4), 64 hops max, 52 byte packets 1 192.168.0.1 (192.168.95.1) 1.981 ms 4.092 ms 4.748 ms 2 10.0.254.1 (10.95.254.1) 0.952 ms 0.913 ms 0.886 ms 3 0.0.0.0 (0.0.0.0) 8.672 ms 8.571 ms 7.923 ms 4 static-65-175-142-3.cpe.metrocast.net (65.175.142.3) 8.42ms 21.53ms 11.87ms 5 tbd-65-175-128-2.metrocast.net (65.175.128.2) 13.309ms 15.735ms 11.637ms 6 te0-7-0-9.ccr21.bos01.atlas.cogentco.com (38.140.12.129) 15.051 ms 7 be2094.ccr21.jfk02.atlas.cogentco.com(154.54.30.13) 18.25ms 17.92ms 18.15ms 8 be2150.mpd21.dca01.atlas.cogentco.com (154.54.31.129) 24.612 ms 23.398 ms 9 be2176.ccr41.iad02.atlas.cogentco.com (154.54.41.53) 25.508 ms 10 38.88.214.50 (38.88.214.50) 24.577 ms 25.252 ms 23.913 ms 11 209.85.252.46 (209.85.252.46) 37.398 ms 12 72.14.236.152 (72.14.236.152) 29.569 ms 13 66.249.95.229 (66.249.95.229) 35.674 ms 14 72.14.234.67 (72.14.234.67) 32.628 ms 15 * * * 16 google-public-dns-b.google.com (8.8.4.4) 35.929 ms 33.106 ms 31.828 ms synack-rmbp15:~ ebanks$
An MTR trace does a little bit better, providing more per-hop information that plain old traceroute does. It’s possible to run MTR for several cycles using the -c option, but even then the information gathered only shows what’s going on at that specific range of time.
synack-rmbp15:~ ebanks$ sudo /usr/local/sbin/mtr -r 8.8.4.4 HOST: synack-rmbp15.local Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.0.1 0.0% 10 9.7 15.9 2.8 39.3 13.7 2.|-- 10.0.254.1 0.0% 10 1.2 1.2 1.1 1.3 0.1 3.|-- ??? 0.0% 10 8.0 9.1 7.6 12.1 1.3 4.|-- static-65-175-142-3.cpe.m 0.0% 10 15.1 13.6 8.2 19.9 4.2 5.|-- tbd-65-175-128-2.metrocas 0.0% 10 17.8 15.8 11.2 20.3 3.6 6.|-- te0-7-0-19.ccr21.bos01.at 0.0% 10 11.7 12.1 11.3 13.6 0.7 7.|-- be2095.mpd21.jfk02.atlas. 0.0% 10 18.2 18.1 16.8 20.1 1.0 8.|-- be2150.mpd21.dca01.atlas. 0.0% 10 25.6 25.1 24.2 26.1 0.7 9.|-- be2113.ccr41.iad02.atlas. 0.0% 10 25.3 29.0 24.3 58.1 10.3 10.|-- 38.88.214.50 0.0% 10 24.3 24.5 23.8 26.3 0.7 11.|-- 209.85.252.46 0.0% 10 24.5 31.3 24.5 41.4 6.1 12.|-- 72.14.236.98 0.0% 10 25.5 26.7 25.4 30.9 1.7 13.|-- 72.14.235.12 0.0% 10 33.4 34.7 32.6 40.6 2.6 14.|-- 209.85.250.199 0.0% 10 55.8 48.0 32.6 72.5 13.9 15.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 16.|-- google-public-dns-b.googl 0.0% 10 34.4 32.9 31.6 34.4 0.9 synack-rmbp15:~ ebanks$
ThousandEyes provides substantially more information about path than simple tools like traceroute or MTR can. For instance, the image below (somewhat obfuscated) shows a great deal more data about each hop, along with loss, latency, jitter, and even available bandwidth. I am specifically looking at historical latency in this chart, and have selected one specific hop about which I want more information – the 38.140.12.114 router at Cogent.
ThousandEyes offers insight into the Internet’s ever-changing BGP infrastructure as well. There’s 30 days worth of history with all of the ThousandEyes monitoring, so it’s possible to go back in time to discover why your enterprise might have had trouble reaching a cloud application at a specific point in time. Alternatively, you might be able to discover why your customer had trouble reaching your Internet-facing application.
In the view below, I’ve zoomed into a specific point in time where reachability to a node ThousandEyes was monitoring dropped. Looking at the BGP path visualization for that point in time, I’m able to see that traffic from the New York area traversing ASNs 20225 and 13536 underwent a path change on the way to the destination ASN 40715. So, it’s possible that during the path change that some users had trouble accessing the resource being monitored (located in 67.154.188.0/22) in ASN 40715. Note that although I can’t illustrate it in a static image, the lines offer more information when you hover over them.
More ThousandEyes Capabilities
- Active network probing. ThousandEyes sees shortcomings in traditional path tracing tools, so they built some of their own tools to probe the networks. They have created a packet crafting library built from scratch in C. This allows them to set any bit they like, designing a packet in the precise way they want, not relying on the OS kernel for any part of the packet generation. There’s even a CLI version of these tools, accessible only to customers.
- Voice troubleshooting capabilities. The most recent iteration of ThousandEyes offer VoIP performance management. Different from most ThousandEyes tests, voice testing requires an agent on both sides of the conversation. Why? Because voice testing is predominantly UDP, meaning there’s no acknowledgement of received packets. The only way to know if test traffic was received and the characteristics of the path is if there’s a second agent to catch it. The results of these tests include an industry-standard MOS computation. ThousandEyes also claims that they can tell, per-hop, when a DSCP value is changed and what it’s been changed to.
More ThousandEyes Use Cases
With a tool like ThousandEyes, there are a number of other possible use cases. Here’s a few that came up as I was listening to them present recently.
- Monitoring cloud providers. It’s possible to generate ticket with the data ThousandEyes uncovered and offer it freely to the provider.
- Monitoring key customer networks. In my world, I have several large “high touch” customer that will notice if there’s a performance or connectivity problem.
- Monitoring site-to-site call quality across a WAN. It’s possible for call quality to be splendid at several sites, but not others. While Cisco IP SLA can take quality measurements and lots of tools can provide a MOS score, many solutions lack the historical element ThousandEyes provide.
- Allowing key customers to monitor your network. The idea here is that an business could provide a ThousandEyes agent to a key customer, and take measurements from the customer’s perspective. This could be very useful in nailing down last-mile problem, eliminating finger-pointing.
- Estimating bandwidth between two points. iPerf is a free tool often used to measure bandwidth between two sites. ThousandEyes can do this for you without hammering the pipe into oblivion.
- Planning a VoIP infrastructure. When adding voice services to an existing network, finding out baseline performance is a key part of the project. ThousandEyes can help here.
Details and more technical information aimed at network engineers and operators about ThousandEyes can be found in the Tech Field Day videos below, and others from Networking Field Day 8.
Note that I am a ThousandEyes customer, and ThousandEyes is also one of my customers. You might like to read my disclosures page.