By Aniket Khosla, Vice President Product Management at Spirent https://www.spirent.com/
Although much of the hype has been centred on 5G, the power of the new emerging mobile standard – and what comes next – will be intrinsically dependent on the core connectivity within underlying global networks. 400G Ethernet (GbE) is now rolling out mostly HPC and large hyperscalers data centres. While 800G Ethernet is waiting in the wings.
Although 5G gets a lot of the headlines, it is the hyperscalers that are really driving the 400G evolution and they are also really driving the bleeding edge of 800G. In combination with Network Equipment Manufacturer (NEMs), the industry is driving the standards much quicker than we’ve ever seen before. Hyperscalers are experimenting while carriers are following in their wake, as they are more focused on price points and are waiting for a point of acceptance from a technology perspective and the change in optics before they start considering migrating to 400G. At the moment, most carriers have not even moved to up to 200G so 400G is still pretty far out for many.
The reason why the 400G market is so exciting is that it has unlocked more competition. The 100G silicon space was basically dominated by a single supplier. In one way, this was good for the market in terms of allowing seamless adoption but stifled innovation to some degree. But when 400G arrived, a lot of the big silicon providers and NEMs started to realise that they could not afford to be limited to a single supplier. 400G has seen a blossoming of both new start-ups and existing participants getting back into the silicon market – players like Cisco, Marvel and others now see an opportunity in 400G where they had abdicated 100G.
The hyperscalers are not going to slow down, because the silicon suppliers can’t keep up. Their business model is based on pushing the boundaries in terms of the different types of technologies that their customers want to consume. Their focus is on looking at a lower cost per bit and they are prepared to explore new options where they can see a clear advantage. As the 400G ecosystem becomes a lot more crowded, the diversity of the ecosystem is good for the customers and for continued innovation.
Another fundamental shift is within the underlying serial lane technology that is not just impacting 400G but will have a role to play in the adoption of 800G. Standard 100G QSFP28 transceivers do not transmit a single lane of 100Gbps, but instead reach this rate via four lanes of 25Gbps. This became mainstream and gave us a path from 25G to 100G using these fundamental 25G lane building blocks.
The single electrical lane of 100G serial is now going mainstream. Whether it’s consumed as 100G, 200G, 400G or 800G is less important – the fact that we now have a fundamental building block will provide stability to the market for the next decade. The 200G market that is currently based around 50G lanes times 4 is already slowing down as potential buyers see the rapid evolution of 100G serial lane-based silicon and finished products – which is bringing into question the longevity of any 25G lane-based solution.
The 400G space is still small – somewhere in a single digit percentage range of the overall market, and there are two issues holding back adoption. The first is cost, 400g is still expensive when comparing cost-per-bit and the more complex optic situation that makes it harder to reach mainstream adoption. It is also evident that many potential customers are waiting to see what happens with the evolution of 800G before committing to 400G. However, 400G could experience sharp growth over the next couple of years – with 200G storage an early use case and these applications will endure. However, the incredibly low cost of 100G means that any provider looking to upgrade its network will gravitate to 100G as first choice now due to its evident value and the expansive ecosystem of white box switches and network operating systems within the 100G ecosystem. The economics are changing – but there is still hesitation from early adopters at this point in the technology evolution.
The underlying technologies between 100G using 25G lanes and 400G using 100G lanes are fundamentally different in the way they interconnect. 400G require mandatory Forward Error Correction (FEC) and so the optic and cable ecosystem are different. There is some amount of compatibility between 400G and older 100G transceivers – when running in a pseudo ‘100g mode’ – but this means that the constraints within the ecosystem are also different. One of these is around the fibre plant and the old debate of competing optical transceiver form factors. On one side is QSFP-DD (Quad Small Form Factor – Double Density) and on the other is OSFP (Octal Small Form Factor). In the 100G space, QSFP clearly won one. But now, that same debate is starting again in 400G and 800G as in some very early testing OSFP at 800G seems to be performing better in key areas such as size and power consumption. The jury is still out however on QSFP-DD vs OSFP.
As 400G products start entering the market – and volumes grow, it is clear that adoption will follow. Although 800G is still going through standards ratification, there is enough maturity for a surge in testing. This is focused around the fundamental 100G lanes and every major silicon vendor alongside a few innovators is ramping up testing to be able to get 800G products to market as IEEE ratification processes conclude. Hyperscalers are also starting the ramp up testing of 100G serial-based solutions.
Yet, one of the biggest changes is how closely aligned the different parts of the networking ecosystem are operating within 100G serial lane products irrespective of whether these are 200G, 400G or 800G variants. This is most surprising at the 800G space where there is already a much wider ecosystem of players and higher availability of optics. This is a result of the much more diverse market but also a recognition that there is a potential untapped opportunity that has yet to have an 800lb gorilla in the room – so to speak.
Looking to the future, it is clear that 100G based 25G serial lane technology will continue to have a very long tail. The low cost and maturity in the market are hard to displace quickly. In my view, 400G will have a modest adoption over the next three to five years but 800G will start to eat its lunch at this point due to sharing the same underlying technology foundations.
So lastly, let’s look at the evolution of testing. It is clear that the generation of technologies based on 100G serial lanes will be much more diverse than that 100G/25G lane situation where at a silicon level, a single dominant player has a 70%+ market share. Competition and diversity are great for customers and innovation but this means that customers are not just evaluating one core silicon vendor: now it’s potentially three or four different vendors working together in a mixed ecosystem. This means that there will be inconsistency in performance and potentially interoperability issues and things can go wrong.
The other major issue is backwards compatibility. Eventually, all the silicon coming to power 800G systems will need to support standards all the way down to 10G. The mix and match nature of modern networks mean that performance can vary dramatically. Add in the newly mandated features like FEC and the constant flow of firmware updates, and the need for validation testing is dramatically increased.
The last element is that with a bigger ecosystem, the need for independent accountability within testing is vital. In a world where “white box” solutions are growing and Network Operating Systems are the norm, there is a danger in the 400/800 space for more finger-pointing between vendors where there is an issue – only third-party testing and validation can uncover the true root cause which may reside at the underlying silicon layer.
In summation, the networking space is incredibly exciting and the wealth of new vendors entering the space will offer many benefits for customers – and for ongoing innovation across the industry. The network is the lifeblood for society and looking at the bigger picture, it is clear that more continuous testing will become the new normal and that shift is being accelerated by this next leap to 400G and 800G.