2 Comments

Oracle acquires Xsigo – an Infiniband bet?

 By: David Lenrow Posted: Aug. 03, 2012

Oracle Buys Xsigo

Tom Nolle has done a great job of explaining that Oracle’s announced acquisition of  Xsigo is an Infiniband deal and that Xigo’s version of virtual networking has nothing to do with the kind of network virtualization that Nicira and clones are doing, nor does it have anything to do with SDN, OpenFlow or any other hot buzzwords in networking (http://blog.cimicorp.com/?p=861).

What this is about is a bet by Oracle that Infiniband is going to be a good foundation for building data center networks to support virtualized workloads and/or whatever cloud is  (Larry Ellison has repeatedly demonstrated a personal Infiniband fetish in public appearances in recent years). It’s no secret that the exa-whatever series of high performance DC boxes from Oracle use Infiniband as the fabric within the cabinet.  In the short to medium term one could make a good case that the best and most “virtual data center” ready fabrics are based on infiniband.  To put it simply, Infiniband has better I/O integration that Ethernet. This is because  Infiniband includes deep support for the critical data center storage protocols that are essential to decent IO performance in distributed computing and hardware offload of critical protocol and payload processing functions that otherwise become a drain on server CPUs and add big time latency.  Some day the emerging Ethernet fabrics (QFabric, FabricPath, VCS, TRILL, SPB)  may deliver these capabilities (RDMA anyone?) and more. If that were to happen, the conventional wisdom is that Ethernet, which benefits from much higher production volumes (orders of magnitude)  and a much larger R&D budget across the ecosystem, will  ultimately eclipse Infiniband, killing this technology.

The wildcard is how long it takes all of this to play out. The performance and scale/density of the enterprise and cloud provider data centers requires solutions that can deliver loss-less storage networking, differentiated QoS, and whole-fabric management models now. If infiniband has that capability today (I don’t have personal experience to say that it does or doesn’t) during the 10Gbs Server and switch refresh that is ocurring now and will be huge for the next few years, there will be a large number of cases where Infiniband will be technically the best solution. Less clear is whether that Infiniband solution will be economically competitive and justifiable in IT budgets.  Network virtualization, in the encapsulation/overlay form, could be supported by the Ethernet interfaces exposed by Infiniband fabrics.

The real challenges here for Oracle will be positioning and building credibility as a supplier of cloud-scale computing solutions. Oracle clearly has designs on this space, but it’s not clear that performance to date and existing products are getting them to the “hockey stick”. System and fabric-wide networking capabilities based on tight integration of Infiniband adapters into server designs could be a powerful competitive weapon for Oracle and that’s what this acquisition offers in the best case.  I think that, like everything else in networking, Infiniband will get sucked into the Ethernet vortex and the volume/cost economics will kill it. Oracle has bet they can win a race with that macro-trend and I think it  looks like a pretty good bet for the next few years. However, I’m still professionally focused on pushing the rate of introduction for a sufficient Ethernet fabric solution and am comfortable betting on the growth rate of that network industry segment for the next few years.

Checkout more from our Guest Bloggers:

Like our Content? Join the SDNCentral Mailing List

  • This field is for validation purposes and should be left unchanged.

About the Author

.

David Lenrow

David Lenrow was trained as a computer scientist and has spent more than 20 years driving innovation… More

Connect with the AuthorWebsiteLinkedIn

  1. Harry Quackenboss says:
    August 3, 2012 at 12:00 pm

    David,

    Insightful and timely piece. A couple of additional comments.

    In addition to Oracle having championed InfiniBand as an interconnect for Oracle RAC (clustered databases), Sun had a significant investment in InfiniBand.

    Given that both Oracle and Sun were interested in InfiniBand, and while widely used in HPC clusters, why has it never achieved significant mainstream traction in the data centers?

    One part of the answer is that Cisco acquired Topspin Communications a company which pioneered InfiniBand as a virtualization technology for storage and computing in mainstream IT applications. Topspin at the time supported gateways for Ethernet and Fibre Channel, which were based on FPGAs, which were expensive and too slow for 8 Gigabit/sec Fibre Channel and 10 Gigabit Ethernet, and after the acquisition, Cisco chose not to invest in ASIC designs, nor did they choose to put marketing muscle behind InfiniBand, and eventually wound down the Topspin-based InfiniBand line of business. The other InfiniBand vendors chose to concentrate on HPC use cases.

    The second point is that for general-purpose IP-based applications, at the time data centers started deploying 10GbE, InfinBand fabrics have higher end-end latency than comparable Ethernet, because of the latency in the data path from the TCP/IP application interface down through the InfiniBand adapter was high. Coupled with hype that FCoE (Fibre Channel Over Ethernet) was going to be the universal interconnect. For a variety of reasons, enterprises that were using Fibre Channel are continuing to expand those Storage Area Networks.

    The question for the future is that large data centers hosting virtual machines operated by AWS, RackSpace, etc. for the most part aren’t using Fibre Channel-based SANs. Based on many forecasts, including one published by Cisco in late 2011, the large data center operators are going to account for an increasing share of servers, and therefore, associated server networking equipment. So far, I haven’t seen any evidence that InfiniBand-based fabrics are under consideration.

    However, to the extent that InfiniBand compute fabrics achieve the promise that early evangelists predicted almost a decade ago, they can legitimately be called SDN solutions. If you look at the management/control of InfiniBand fabrics in HPC, they are comprised of an InfiniBand Controller, which programs the forwarding tables in a bunch of relatively dumb but fast InfiniBand switches. That model is a pretty good analog for an OpenFlow Controller and a bunch of OpenFlow switches.

  2. ggreenma     says:
    August 8, 2012 at 6:56 am

    Agreed! InfiniBand is absolutely related to the SDN buzz. Much of the work being done in the Ethernet world to create virtualized fabrics is bolting on additional capabilities that InfiniBand has had built-in since day one. This includes the ability to do centralized forwarding decisions, creation of virtual networks, lossless traffic forwarding and hardware-based QOS.

    The real issue is that building a virtualized network fabric for the data center that provides real value to the customer, integrates with the customers applications/management infrastructure and isn’t ridiculously complex is an extremely non-trivial exercise. The purchase of Nicira by VMWare is very similar to the purchase of xsigo by Oracle. These major industry players understand that you can’t expect your customers to go buy all the piece parts and make them work together. There are very few customers capable or willing to do this. This purchase by Oracle will further their lead in deploying SDN-like solutions for their high end solutions.

Leave a Reply

You must be logged in to post a comment.

Login


MODAL