Intel Takes Vyatta to 10Gig

December 18, 2009 at 12:37 pm 12 comments

“You fell victim to one of the classic blunders – the most famous of which is, ‘Never get involved a land war in Asia.'”
– Vizzini, “The Princess Bride”

Intel just published a very exciting Router benchmark using Vyatta that will prove to have historical ramifications.  It’s worth stepping back for a moment to put it in the proper context.

By now we all know that open-source is the most wildly disruptive force to ever hit software, and that Linux holds the highest credit for starting it all.  But in the jolting earthquakes and aftershocks of this adoption, it’s easy to forget the single greatest reason why Linux achieved such broad initial adoption.

It was the hardware.

Before Linux, Unix was only available on vendor-specific hardware.  Sun’s offering was Solaris on their SPARC-based servers, IBM offered AIX on their PowerPC-based servers, HP gave you HP-UX on their PA-RISC-based servers, etc.  Everywhere you looked, Unix was trapped in proprietary hardware.

Then along came Linux which gave the world Unix functionality — on the x86 architecture. This had two initial benefits:
1.   A radically lower hardware cost model
2.  A phenomenal new price/performance curve to ride.

Almost overnight Linux/x86 systems took out departmental installations.  As Intel continued its inexorable drive, Linux quickly moved into the datacenter.  A lot of people think Linux hurt Microsoft the most; the truth is, Linux cleaned out the Unix server market like Michael Moore at a free buffet.  And superior price/performance delivered by x86 servers was the true enabler.  As a wise old man once told me, “When it comes to standards-based platforms, never bet against the world’s largest semiconductor company.”

That’s the way it all went down.  And now, repeating history, Intel has published an earth-shattering benchmark using Vyatta on a single-CPU Nehalem server.  The result?  Line-rate 20-Gigabit bi-directional networking performance from a class of servers widely available from Dell, HP, IBM etc — servers that cost less than $5,000. This is precisely the hardware dynamic that powered the adoption of Linux.

So why don’t the existing networking vendors jump on board?  It’s simple:  It would destroy their business as they know it — they make money by offering small, slow boxes and upselling the customer into large, expensive boxes.  That’s why 10-Gigabit in the proprietary network vendor model is a $100,000 expense proposition.

Even the equity researchers are starting to understand just how powerful this open-source / x86 dynamic is in networking.  Wells Fargo Securities’ Equity Research Department recently wrote, “Based on independent tests, Vyatta software consistently meets or exceeds comparable closed-source routers in performance and throughput, typically at a much lower cost-basis (due to economic advantages of open source and commodity hardware).”

That’s exactly the kind of thing penned by analysts about Linux/x86 circa 2000.   Stay tuned… the revolution is upon us.


Entry filed under: Uncategorized.

The Open-Source Business: What’s Tomorrow’s Dominance Worth Today? Global Telecom Goes Virtual With Vyatta

12 Comments Add your own

  • 1. Azhrei  |  December 20, 2009 at 8:14 pm

    Hmm. I could’ve sworn I was running Unix SVR4 on Dell hardware prior to Linux. In fact, I seem to recall doing some text compression work in… oh, about Nov 1990? And SVR4 had been available on x86 hardware for a couple years at that point. Linux wasn’t distributed on comp.sources.unix until 2Q1991.

    • 2. Kelly Herrell  |  December 21, 2009 at 8:47 am

      You’re correct; SVR4 had an x86 version that far back. My point is not about available ports, but price/performance: x86 hadn’t blasted through the P/P of Sparc et al until the mid/late ’90s. It was sometime around 1999 that Morgan Stanley published their benchmark results that compared $650K worth of Sparc / Solaris gear to $50K worth of x86 / Linux gear and concluded they would start shifting to Linux… and it was 2001 when Gerstner from IBM stood up and proclaimed they were putting $1B behind Linux. So while you could get Unix on x86 earlier, you wouldn’t get Big Branded- and App-Supporting Unix (Solaris, HP-UX, AIX) on x86 until after the perfect storm of Linux had already hit full force.

  • 3. Open  |  December 20, 2009 at 11:05 pm

    Line rate 10G at 1024 Byte packets isn’t that impressive. And a Cisco 4948-10GE (which would significantly outperform these benchmarks) is not a $100,000 expense. Try $10,000.

    Maybe tune down the hyperbole? It will make you look better in the end.

    • 4. Kelly Herrell  |  December 21, 2009 at 8:28 am

      Thanks for the comment. Yes, line rate 10G is common for a switch like the Cat 4948 to which you refer — but routers are a totally different animal in terms of product class and cost. To get 10G speed for routing/firewall/VPN the Cisco comparison is the ASR1000. By the time you get the base chassis, software and module upgrades for the extra services, and multiple 10G cards the price rolls up into the $100K range.

      • 5. Curtis Lim  |  June 17, 2010 at 9:08 pm

        Bit late to the show, but not sure how you define line rate…

        Cisco and every other NEM defines line rate as 64B packets plus the ether frame overhead: meaning 10 Gbps line rate forwarding = ~14.6M packets per second (No Services). Sorry, vyatta is nowhere near this, let alone 20 Gbps bi-directional (implying egress services).

        This should not be surprising, x86 processors were not build for packet processing. Highly multi-cored, multi-threaded processors are. As one becomes the other (add more cores, add more memory interfaces), of course routers will become the domain of OTS systems.

    • 6. Someone  |  December 22, 2009 at 2:21 pm

      You say using 1Kbyte packets at wire speed is not impressive? Have you noticed that the lower the packet size, more work has to be done to open and analyze each packet? Compare this to a Jumbo frame of 9K and you have 9 times the work to open and decode header and have lower net result transmission cause less data is being sent per packet.

      So I AM IMPRESSED that it could decode that much packets and still have power to do other things. All this in ROUTING is defintely impressive.

  • 7. Tom  |  January 3, 2010 at 2:09 pm

    I think that L2-7 line speed is something that is achievable today with commoditized COTS components. (the math says it is possible)

    Routing is also something that is becoming “simple” from a memory/Flops perspective.

    and I am waiting until 3-4Q 2010 * PCI-e 3.0 * and SATA III with SSD

    “PCIe 2.0 delivers 5 GT/s but employed an 8b/10b encoding scheme which took 20 percent overhead on the overall raw bit rate. By removing the requirement for the 8b/10b encoding scheme, and replacing it with a 128b/130b encoding scheme with only ~1.5 percent overhead,[17] PCIe 3.0’s 8 GT/s bit rate effectively delivers double PCIe 2.0 bandwidth. According to an official press release by PCI-SIG on 8 August 2007:” –

    PCI-e 3.0 will make whats challanging/impressive today … common place … at least thats how I see it.


  • 8. AC  |  January 6, 2010 at 9:50 pm

    This is where Vyatta really shines. They are competing with Cisco’s mid to lower end where Cisco’s routing is mostly handled in software. So when you have a bleeding edge x86 blade vs. an aging product line with an older processor from Cisco it is clear who is going to have better performance. Couple this with ~65% margins for Cisco and free software from Vyatta and the winner is extremely clear. For those that will respond that you pay for Vyatta support, compare that to the extraordinary costs for Cisco TAC on top of the cost of the router.

  • 9. Edward Dore  |  January 12, 2010 at 3:30 am

    How well does this scale when adding a second quad-core processor with the additional 10Gbps interfaces to feed it?
    I assume there are diminishing returns given that from my quick once over of the whitepaper it says “Another
    processor could have been installed to perform other services” and is also absent as an option from the new 3500 series of appliances.
    I would be interested to know how well this would perform with an additional pair of 10Gbps interfaces and a second quad-core processor.

    The series of 3500 appliances looks to be based on the Dell R710. Do you have any info on how the PCI-E slots are wired up to the chipset on these (i.e. are they direct or do they do through any switch chips?)

    • 10. Kelly Herrell  |  January 12, 2010 at 5:27 pm

      You will continue to scale performance by adding cores, but it’s not linear with our current software release. Expect to see that change significantly this year; huge benefits via massive parallelism are certainly achievable.

      The R710 is a Tylersburg chipset capable of supporting 32 lanes of PCIe into the IOH which is connected to the processors directly using QPI.

  • 11. Networking’s Software Era « Kelly Herrell's Blog  |  February 3, 2010 at 12:19 pm

    […] of news over just the past two months: The world’s dominant chip vendor is promoting an awesome benchmark using Vyatta, large networking vendors are partnering with us, a huge telecom is preparing a massive Vyatta […]

  • 12. Vincent JARDIN  |  March 31, 2010 at 6:20 am


    Yes, it demonstrates that multicore CPUs + Linux can change the deals. However, if we check the packet per second, it is still low compared to other switch routers made of NPUs, FPGAs, or ASICs.

    The real number is not the Gbps, but the Mpps that you can scale up with Linux.

    In fact, as demonstrated on this forum,, we have seen that with Linux (same for any other OS), you cannot make it with Linux kernel stack. The performance is limited and does not scale with the number of cores.
    However, do not throw away Linux! Keep Linux (and Vyatta) on some cores and on the other cores, run a Fast Path that will scale up with 10s of Mpps for 64 bytes packets.

    Moreover, this Fast Path saves many cycles to Linux that can run more application.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed

Vyatta Tweets

Error: Twitter did not respond. Please wait a few minutes and refresh this page.


Top Posts

%d bloggers like this: