Sponsored Article: How to transfer big files faster and safer

Signiant Media Shuttle

This 'Sponsored Article' from Signiant explains how their Media Shuttle technology can transfer big and small files faster through the same bandwidth

Advances in image resolution technology are dramatically increasing the size of content files. At the same time, those files need to be transferred more often throughout the production, post-production and distribution process, as networks of producers, studios and broadcasters continue to globalize.

With 4K only recently commonplace, 8K is already turning up at major sports events. At the Women's World Cup in Canada last week, FIFA worked with Japanese broadcaster NHK "to produce ten matches in ultra-high definition 8K visuals, including the Final Match on July 5th." And Japan also intends to have 8K broadcasting in time for the 2020 Olympics in Tokyo.

Such high image quality is the best way to watch professional sports, and 8K isn't just twice as good as 4K. Image resolution going from 2K to 4K to 8K squares incrementally, with 4K = 8 megapixels and 8K = 32 megapixels. But this also equals exponentially growing file sizes. Raw footage files that were already huge at over 6000 GB on average for 90 minutes in 4K are nearly three times that in 8K.

However, compared to every other type of data that's transferred over the Internet, media files have always been large. Even a 1GB SD file can cause problems for standard Internet transfer protocols like FTP, especially if there's substantial distance between endpoints.

So, ever since the dawn of professional digital filmmaking, media enterprises have relied on large file transfer technology like Signiant's. A decade ago, it was mostly the big companies that used it, while everyone else relied on a mix of FTP and courier services. But in the last few years at Signiant, we've incorporated our core large file transfer technology into Media Shuttle, a SaaS solution that is quickly being adopted by every size of media company.

Currently, Media Shuttle is the only true SaaS (software-as-a-service) solution on the market that can transfer any size file quickly, no matter how big it is or how far it needs to go. But, because it has the easy feel of other cloud file sharing solutions often used for smaller files, it can seem a bit magical on the surface. Recently, we've had several people ask us to explain how it goes so fast. However, it's not just acceleration technology that works the magic behind Media Shuttle, security and reliability play equally important roles.

Acceleration

The kind of technology we've developed to speed file transfers is often called "UDP (user datagram protocol), acceleration." But UDP acceleration doesn't really convey the depth of innovation involved. In order to explain it, we have to go back to the way data is typically transferred over the Internet and how we've improved on TCP and UDP.

TCP (transfer control protocol) is what provides a reliable stream of data from one point to another on the Internet, and forms the basis for both HTTP and FTP. For the large majority of data being sent over the Internet, TCP works very well. But, because it uses a relatively unsophisticated mechanism, it suffers from latency. Essentially, it slows down file transfers because it waits for the other end to acknowledge safe receipt before it will send more data.

With large files, this creates a lot of back and forth, with plenty of room for data loss, data retransmission and failure. Also, without additional functionality added, when TCP fails, it doesn't keep track of data that's already been sent. So, if a network fails or a computer crashes during a transfer, often the entire file has to be resent.  

Another big drawback to TCP is that, due to retries and acknowledgements, it can only effectively utilize a fraction of allocated bandwidth. Many companies purchase more bandwidth in order to send large files with FTP, only to realize they've squandered budget: additional bandwidth doesn't help TCP/FTP.

Reliability

Signiant's acceleration protocol replaces TCP with a more advanced transmission control protocol used on top of UDP, as well as an advanced file transfer protocol to replace FTP. Standard UDP allows a chunk of data to be sent from one place to another on a best effort basis, but UDP doesn't guarantee reliability. Signiant improved on UDP by implementing several reliability mechanisms — such as flow and congestion control — to address any data loss. Additionally, Signiant implements checkpoint/restart to ensure that, in the event of an interruption, the file will automatically pick up transmission where it left off.

Signiant's acceleration technology and reliability work together to make Media Shuttle so fast. The graph below illustrates three transfers using Signiant vs. TCP/FTP. Note that transfers with TCP do not improve with greater bandwidth speeds, and suffer greatly with long distance transfers due to latency. Signiant practically eliminates the effects of latency while capitalizing on available bandwidth, which is why transfers can be up to 200 times faster.

Security

All the speed in the world can't move fast enough to evade cyber criminals or prevent user error when implementing archaic security features. At Signiant, security has always been a top priority in our software development and is an ongoing focus throughout design from core technology to usability. There are numerous articles written about the implementation of secure design principles, so we won't go into too much depth here. However, it's important to know that our software design includes multiple layers of security, including data encryption both in transit and at rest. Additionally, we actively hire people to try and penetrate our network, and take their recommendations into consideration as we further develop our software.

What Harbor Picture Company said about Media Shuttle

In a recent case study, one of our customers spoke about the reasons they chose Media Shuttle:

"We're not just moving dailies, but all of the post production work done outside of our studios, in various locations around the world," said Harbor Picture Company Co-founder Zak Tucker. "The data transfer requirements are always increasing as we go from HD to 2K to 4K…. We needed a better solution for transferring files … and that offered the security protocols we could stand behind. That's a ‘must have' in this business.

"Many organizations in our network were using Signiant's Media Shuttle, so we began vetting that as a replacement for FTP. Using Media Shuttle allows us to support this next generation of collaborative workflows, and do so at a much more reasonable cost."

If you'd like to try sending a file with Media Shuttle for free, visit us at Signiant.com.