Video streaming works by using Traditional Streaming or HTTP-based Adaptive Streaming technologies. HTTP-based streaming utilizes the HTTP protocol allowing for easier caching, and this results in faster video streams.
In Traditional Streaming, media servers are used to serve content. It uses special media protocols like RTSP, or Real Time Streaming Protocol, RTMP, or Real Time Messaging Protocol, and RDT, or Real Data protocol. In RTSP, the video will be sent as small packets, in sequence, in a format called RTP packet. The RTP packets are around 1 MB for 1 Mbps quality video.
If there are only three seconds of content in your buffer, the streaming server only serves two seconds, no matter how long you wait resulting into almost no bandwidth loss.
In Adaptive streaming, videos get encoded in various formats and qualities then transmitted via HTTP protocol onto regular webservers. If your video is in three different qualities, 5 Mbps, 10 Mbps and 15 Mbps, and both the network and CPU are in good condition, then the server feeds your desktop app with 10 Mbps, which means you have a lower chance to wait for the video to load.
In Traditional Streaming, once the client-side starts streaming video the quality remains the same, and if the network weakens it still needs to load that same video quality.