SMPTE Publishes a New Archiving Standard

SMPTE Publish a New Archiving Standard

SMPTE, the Society of Motion Picture and Television Engineers, are the creators and guardians of standards in the media industry and they have now turned their attentions to archiving

 SMPTE was founded in 1916 and since then it has done an excellent job of standardising the many different formats of film, audio and other media that are in use today. If it wasn't for them it would be chaos, with each manufacturer creating their own formats whenever they felt like it.

Thanks to SMPTE we have Colour Bars, Time Code, SDI, HD-SDI and many other standards that allow us to work with media from different sources. Now it has turned its attention to the field of digital archiving and published a set of standards that should enable archives to be available on any equipment in the future.

Currently there are many different formats used for media archiving such as LTO, DLT and Sony's Optical Disc Archive but the problem is that they all use different systems for tracking the media and it's associated metadata. How are we to guarantee that our archived material will still be accessible in 100 years time?

That is why SMPTE has published its Archive eXchange Format (AXF) to ensure that any media that is archived using these standards will still be accessible to all in the years to come, as well as bringing interoperability to different systems in use today.

At the most basic level, AXF is an IT–centric file container that can encapsulate any number and any type of files in a fully self–contained and self–describing package. The encapsulated package actually contains its own file system, which abstracts the underlying operating system, storage technology, and the original file system from the AXF object and its valuable payload.

It's like a file system within a file that can be store any type of data on any type of storage media. Archive eXchange Format (AXF) is an open format that supports interoperability among disparate content storage systems and ensures the content's long–term availability no matter how storage or file system technology evolves. AXF inherently supports interoperability among existing, discrete storage systems irrespective of the operating and file systems used and also future–proofs digital storage by abstracting the underlying technology so that content remains available no matter how these technologies evolve.

Read the press release over the page...

 WHITE PLAINS, N.Y. — Sept. 11, 2014 — The Society of Motion Picture and Television Engineers®(SMPTE®), the worldwide leader in motion-imaging standards and education for the communications, media, entertainment, and technology industries, today published a standard that codifies the ArchiveeXchange Format (AXF). An IT-centric file container that can encapsulate any number and type of files in a fully self-contained and self-describing package, AXF supports interoperability among disparate content storage systems and ensures content's long-term availability, no matter how storage or file system technology evolves.

“When faced with the need either to consolidate archives or to migrate them to new generations of storage technology, media companies traditionally have been forced to perform the costly and time-consuming integration of archive systems and other systems,” said S. Merrill Weiss, chair of the SMPTE Working Group on AXF. “Now, by abstracting the underlying technology of digital storage, AXF not only supports interoperability among today’s discrete storage systems regardless of the operating and file systems used, but also future-proofs digital storage so that content remains available despite changing formats and storage technologies.”

Designed for operational storage, transport, and long-term preservation, AXF was formulated as a wrapper, or container, capable of holding virtually unlimited collections of files and metadata related to one another in any combination. Known as “AXF Objects,” such containers can package, in different ways, all the specific information different kinds of systems would need in order to restore the content data. The format relies on the Extensible Markup Language (XML) to define the information in a way that can be read and recovered by any modern computer system to which the data is downloaded.

AXF Objects are essentially immune to changes in technology and formats. Thus, they can be transferred from one archive system into remote storage — geographically remote or in the cloud, for instance — and later retrieved and read by different archive systems without the loss of any essence or metadata.

AXF Objects hold files of any kind and any size. By automatically segmenting, storing on multiple media, and reassembling AXF Objects when necessary, “spanned sets” enable storage of AXF Objects on more than one medium. Consequently, AXF Objects may be considerably larger than the individual media on which they are stored. This exceptional scalability helps to ensure that AXF Objects may be stored on any type or generation of media. The use of “collected sets” permits archive operators to make changes to AXF Objects or files within them, while preserving all earlier versions, even when write-once storage is used.

The nature of AXF makes it possible for equipment manufacturers and content owners to move content from their current archive systems into the AXF domain in a strategic way that does not require content owners to abandon existing hardware unless or until they are ready to do so. In enabling the recovery of archived content in the absence of the systems that created the archives, AXF also offers a valuable means of protecting users’ investment in content. By maintaining preservation information such as fixity and provenance as specified by the OAIS model, AXF further enables effective long-term archiving of content. Resilience of data is ensured through use of redundant AXF structures and cryptographic hash algorithms.

AXF already has been employed around the world to help businesses store, protect, preserve, and transport many petabytes of file-based content, and the format is proving fundamental to many of the cloud-based storage, preservation, and IP-based transport services available today.

Many individuals and organizations have contributed to the development of AXF. The project was led by S. Merrill Weiss, president of the Merrill Weiss Group. Principal contributors included Brian Campanotti of Front Porch Digital, Karl Paulsen of Diversified Systems, James Snyder of the Library of Congress, Steve Posick of ESPN, William Hayes of Iowa Public Television, David Hostetter of Oracle Corporation, Merrick Ackermans of Turner Entertainment Networks, and John Eremic and Robert Wolfe of HBO.

The AXF standards document is now available to subscribers and for purchase in the SMPTE Digital Library:http://standards.smpte.org. Further information about SMPTE and its work is available at www.smpte.org.