Video Formats (in Particular, Multicast and UDP) and Network Architecture
Some readers will consider this section the hub of all the discussions. Others will disregard it as esoteric and irrelevant in practice. It's up there with Marmite!
First a tech briefing. The Transmission Control Protocol (TCP) is a particular way to reliably send data across IP networks. If you are sending data that needs to arrive intact, TCP is a very reliable way to send data. It ensures that a dialogue between sender and recipient is carried out that confirms sending, receipt, and checks the delivery as intact.
TCP is a well-known network layer application that sorts out that dialogue.
UDP - the User Datagram Protocol - allows users to define every step of that dialogue themselves. In some circumstances reliability may not be the imperative in the network link. When sending a video picture, a few missed pixels won't have much impact on the viewer. There seems little point in stopping proceedings, rewinding the transmission to the missing pixels and getting going again.
UDP allows the developer to define how this type of discarding may occur, and when too much data has gone missing to discard, and so on.
This is a very simplistic example, but it helps highlight that while the dominant use of IP/Internet is perceived to be accurately copying data from one end of a network link to another, actually the way this is done is not deterministic. In certain circumstances network operators may choose to use non-TCP-based transmission layer protocols.
Critically TCP quite quickly reaches a maximum transmission speed when used over long fat networks (LFNs) such as transatlantic cable or satellite.
In these situations UDP is needed to maximize the transmission throughput over the link.
UDP can also be connectionless, and receivers can listen and receive data without needing to acknowledge each packet. This in turn can allow an operator to optimize distribution of data to many locations. UDP underpins IP multicast, and while we will cover that later in some detail, multicast is without a doubt the best way to scale live or one to many distribution of content.
However, much as in the CoDec wars, albeit with fewer options, TCP and UDP actually emerged as battleground opponents in the late 1990s and early 2000s causing developers to split into two factions.
UDP promised better scaling and finer control for high-bandwidth video delivery. However, TCP, in particular when combined with HTTP services, was very accessible to web developers, and at the time it was web developers who were driving the traffic to video online, and not large scaled up high-bandwidth TV platforms (which had yet to emerge).
Essentially, while inferior from a purists point of view (yes, I would include myself in that group), HTTP/TCP was cheap and easy, and good enough. While Windows Media and Real Networks had worked hard to support both TCP and UDP transmission models, the fact was that Adobe Flash proved a simple way for web designers to add a little video to their webpages, and this quickly attracted audiences. Very soon content delivery networks were seeing a huge demand for this type of short form high-bandwidth video, not least because it worked very well for advertising online, and the relative value of continuing to develop better and better technical solutions for UDP (and some niche TCP-based models) became unjustifiable.
The UDP technologies did, however, find a home with IPTV network operators. In the context of an IPTV network all points on the network are under the operator's management, and this means that the scaling and quality advantages can be maximized, making IPTV a better quality proposition for premium customers.
In the meanwhile for Inter-network delivery HTTP/TCP was simple, and commoditised with all the other web traffic. For CDNs this meant, and means to this day, that they need only support a single type of traffic, making their jobs more profitable.
While the market is still emerging, HTTP, despite its inefficiencies, copes extremely well with the audiences we have today. However, we are still drawing breath when a CDN claims to deliver millions of viewers to a live event. This contrasts with the many broadcast measurement systems that report audiences of tens of millions on broadcast infrastructure.
This gap has to be bridged for the Internet to truly evolve to be considered a broadcast medium for scaled up live/linear streaming.
HTTP/TCP is a brute force way to do that, ultimately requiring that every user has a termination at the edge, and the edge itself has the resource to provide an effective proxy service to minimize the core network requirements. As we scale to the point that one TV broadcast regularly requires “much” of the CDN's resource, the question has to be asked how the model can scale to its best efficiency while using TCP.
Those of us that believe in this scaling issue, believe in IP multicast and other UDP- based dark arts that have been left to one side while there is an HTTP gold rush. However, those capabilities are increasingly going to show real value. Aspera, Motama, Zixi (my own company's “GRIT”), and others are already finding seams of new opportunity in this space, and arguably this is true for the channel-bonding live contribution platforms (CellMuxes) and other similar production broadcast telecoms technologies that are still evolving in the enterprise but will eventually reach into the wider consumer market for sure.