HD streaming experiments

This website uses many stream supplied by the server itself. Many of these streams are in HD at high bitrate. See this page for a complete (and hopefully updated) list

To stream HD is required HD material. The first HD material I used are 282 seconds of the movie Home realised by Yann Arthus Bertrand in 2009, copyright Europa Corp, publicly viewable on many web sites. I've added a specific page with all the disclaimer, additional informations and all the related links.

Sizes and bitrate stream:

The first HD stream I erogated were these ones:
1080p h264 VBR with a peak of 3.2 Mbps(about 111 MB). Stream http://95.110.164.61/home_1080p
720p h264 VBR with a peak of 2.7 Mbps (about 99 MB). Stream http://95.110.164.61/home_720p
1080p mpeg2 VBR with a peak of 4.3 Mbps (about 143 MB). Stream http://95.110.164.61/mpg2_4.3
1080p mpeg2 VBR with a peak of 6.4 Mbps (about 211 MB). Stream http://95.110.164.61/mpg2_6.4

other stream The streams (as all the streams used here) use different versions pre-encoded files of the same sample 282 seconds long. The first couple (one 1080p and the other 720p) have been compressed in h264 with mpeg1 layer3 (mp3) audio (in italian) through x264 (open source h264) in two steps., the second couple, added on September 9th, are both mpeg2 HD 1080p streams with mp3 audio encoded at two different bitrates.
The scope of these last files is to make some test of different behaviour of mpeg2 and h264 TS when stream.

Troubles

Http is all but the ideal protocol to stream, but it passes all firewall and it is based on tcp/ip.

At infinite time the stream should be delivered (almost) perfect, potentially as it is. The problem arise from the uneven flow of information, not correctly delivered to the player in time to be decoded and presented at the right time. This can be metered by Media Delivery Index, introduced in 2006 with RFC 4445 by Welch (Ineoquest) and Clark (Cisco). It is explained in the Wikipedia's lemma but perfectly shown by the following graphic taken from Ineoquest's document Media Delivery Index Application Note

.
The image refers to a UDP stream, but it can be extended to any ordinary live stream, including mine.
It clearly represents why a player may be unable to show correctly a flow of 3,75 Mbps despite it get all the required bits: all but not at the right time

But in this case the problem was harder: recalling with VLC the stream http://95.110.164.61:64087/mpg2_6.4, the original one generated by VLC, it flows quite fluently, while recall the stream proxed by Apache (on port 80) or through the webpage handled by Apache it used to be more than simply affected by stops and go: it was almost unviewable and this affected also the dump!.
This is unconceiveable, being the transmission based on TCP/IP

Using VLC you can capture the stream creating a file with the following command line

"[vlc path]\vlc" http://95.110.164.61/mpg2_6.4 :http-caching=1200 :demux=dump :demuxdump-file="c:\stream.ts"

The default VLC path is c:\program files\videolan\vlc (in any case you can find the path from the icon properties).
Beware: the stream is live: it never starts and never ends, so -- even in case of narrowband stream as in the image below -- if you forget it can become a big file.


The evidence: it cannot be a problem of server's computational power or network

Despite the available bandwidth and power of the server is almost untouched, the reproduction of these HD live streams was continously affected by stop and go in mpeg2 as well in h264: the image quality was worst in h264 because the higher compression rate.
In order to understand, I've considered the server load as well the network connection: both them cannot be charged!


Server computational load (single core 4GB RAM in cloud) while draining a remote 6.4 Mbps stream with VLC.

Server network load (on 1Gbps nominal) while draining a remote 6.4 Mbps stream with VLC.

So what remains is the overall quality of network connection: usually from private standard home adsl connections only the 1.5 Mbps PAL stream flow regularly.
I've had the opportunity to test these pages in sites connected with broadband-warranted connections. Well, there even the 6.4 Mbps flow perfectly: it means that the network path of private citizens has many bottlenecks

This trouble let me understood why the live http streaming is worldwide carried out slicing the stream in small portions (seconds) and transferred as small files to be reproduced as a playlist.
This is done by Adobe Http Live Streaming, Microsoft Silverlight (both supporting as many other the MPEG DASH streaming architecture) and Apple Live Streaming.
It is a winning strategy because the TCP-IP (small) file transfer suffers much less than true live streams. It is simpler to transfer small files instead one streams.
These tests confirmed me -- again -- that increasing the connection bandwidth and reducing the round trip time the video would be reproduced correctly.
Only with true broadband connections (sucha as 64 Mbps with warranted bandwidth and a mean RTT to the server of 10 mSec) the HD streams were reproduced perfectly.
With a commercial ADSL with a download of 8 Mbps and a RTT to the server of 70 mSec, only the streams below 2 Mbps were reproduced accettably.
But why the VLC erogated streams were received and play correctly while the Apache proxed were not? It cannot be a problem of network (that would affect the VLC streams as well the Apache ones): it could only be a wrong or incomplete configuration of Apache, the only block between VLC and the network.

The solution!

I'm a newby of Apache but I was quite sure that there were a problem of Apache configuration, because it is the only block between the network and VLC.
I searched the web and also on the rich Apache documentation.

First of all is required to know what are the Workers of Apache. Workers are the Multi-Processing Modules (MPMs) manage the network connections and dispatching the requests.
There are two common MPMs available for Apache2: the mpm_prefork_module uses the traditional (unix) model of one process per request and mpm_worker_module (under windows) that uses a threaded model under multiple processes (only two under windows), each with multiple threads to get better performance with lower overhead. These two MPMs, if present, are usually compiled into the Apache binary and not loaded as a module.
By default the Windows binary includes only the mpm_worker_module.
In other words, Apache for Windows is multithreaded, so it do not use a separate process for each further request (as it happens under Unix).
Under Windows there are only two Apache processes running: a parent process, and a child which handles the requests: within the child process each request is handled by a separate thread.



Only when you stop Apache or kill the processes appears a third httpd process, the killer. On the Apache documentation I've found that for any Operating System it is required a specific section (only one) to be copied from extra\httpd-mpm.conf and pasted into httpd.conf.
The standard httpd-mpm.conf for windows suggest only two lines, but in this case I had the suspect that it was not a problem of efficience (so of threading), but of Apache network buffering.
On Apache documentation (I confess: i do not read it all) I've found informations about SendBufferSize parameter
It increases the TCP send buffer size useful to compensate high latency connection with RTT more than 100ms (as in our case).
By default or when it is set to 0, the server will use the OS default.
I decided to put it at 1000000 (1 MB) because on other parameters I've seen the usage of these high values.

Well: it seems to work! It streams correctly even with 6.4 Mbps video, at least as the native VLC generated streams.



A big step for me, nothing for mankind

But it is still not enough

No, it is not finished, because while in presence of a sufficient downlink connection, VLC client is now able to reproduce fluently the high bitrate streams, the same streams are still not reproduced correctly when framed in a web page using the VLC plug-in, that lacks of documentation.
Using the same approach I've adopted to solve what I called the Apache bandwidth bottleneck, I've changed the cache parameters of the plug-in, and really it seems go better.
Because the high bandwidth stream are reproduced correctly by the VLC client, it means the streams are transmitted correctly (if the bandwidth of your downlink connection is sufficient (30% more than the stream bandwidth).
On the contrary the same streams are not reproduced fluently when recalled by the active-x (under Internet Explorer) or by the VLC plug-in for Netscape derivated browsers (installed with VLC). Please test the possible enhancement opening (in sequence) the different pages of the following table where you can recall the same stream with 300 (normal) mSec network cache, with 1000 (high), or 3000 (higher) and last 5000 mSec of cache.



source stream cache in milliseconds
300 mSec 1000 mSec 3000 mSec 5000 mSec
h264 720p 2.7 Mbps home_720p_300.html home_720p_1000.html home_720p_3000.html home_720p_5000.html
h264 1080p 3.2 Mbps home_1080p_300.html home_1080p_1000.html home_1080p_3000.html home_1080p_5000.html
mpeg2 1080p 4.3 Mbps mpg2_4.3_300.html mpg2_4.3_1000.html mpg2_4.3_3000.html mpg2_4.3_5000.html
mpeg2 1080p 6.4 Mbps mpg2_6.4_300.html mpg2_6.4_1000.html mpg2_6.4_3000.html mpg2_6.4_5000.html

Try to understand what cache is better for your connection. Consider that VLC client by defaut uses 1000 milliseconds (1 Second).
It is also evident that the greater the network cache longer will be the time before the beginning of the reproduction.
If you do not find appreciable changes, consider also that the downlink bandwidth must be at least 30% more than the nominal stream bandwidth (8 Mbps should be enough for all).

Due to the difference of the call between Explorer and Netscape derivated browser it is mandatory to duplicate the parameters in the object section as well in the embed section.
The VLC active-x plug-in lacks of documentation, but it seems that the change of the network-caching parameter affect (in better) the reproduction.
On the web I found an initiative of a FireBreath VLC plug-in client (Firebreath is a plug-in architecture for Internet Explorer active-x as well NPAPI - Netscape Plugin Application Programming Interfacefor for Netscape derivated browsers) where there is written something more.
But be careful: I'm using the original VLC active-x and the embed call, which parameters could not be the same.

back