By the middle of the last decade, the U.S. military was recording full-motion video (FMV) from airborne platforms–especially UAVs–at an unprecedented rate. An estimated 20 million hours had been generated. Massive amounts of footage had been reviewed in real time by operators and analysts in ground stations and had prompted immediate action. Some of the footage surely had archive value, and if it could be properly stored and recalled, it might provide valuable context to subsequent video coverage. Unfortunately, the ability to unlock that value was stymied. The footage was being collected in multiple formats; it was hardly searchable, and then only by basic time and location.
A senior official at the U.S. National Geospatial-Intelligence Agency (NGA) got to wondering how was it that the television broadcasters of U.S. football games were able to so quickly retrieve and show video clips of moves in previous games, that were relevant to the current action on the field. He traveled to the next convention of the National Association of Broadcasters and realized that industry was 20 years ahead of the military when it came to storing, searching, analyzing and displaying FMV.
One company, Harris Corp. had developed a system for exploiting video metadata that offered much of the functionality required by analysts at the NGA and at deployed locations. It relied on time-stamping of the video to provide a universal reference for annotation, metadata correlation and searching, rather than the spatially based video management system the military was using.
Four years later, the company’s FMV asset management engine (FAME) is turning terabytes of video footage obtained over Afghanistan into real-time tactical knowledge. Harris teamed with Lockheed Martin and NetApp to deliver the Valiant Angel FMV exploitation system to the U.S. military. Lockheed Martin provides a video user interface called Audacity that includes mosaic creation and the use of Google Earth maps to provide analysts with situational awareness of a UAV’s orientation and track. NetApp provides the high-performance commercial servers that store and distribute the enhanced video. But FAME is the heart of the system, responsible for 90 percent of the workflow, according to John Delay, lead architect for FAME at Harris Broadcast Communications.
“Some defense companies had produced video storage and retrieval systems, but none of them was conceived as an enterprise architecture,” Delay told AIN. “FAME is a set of back-end services...that allows users to create markers in the database. We build all the ingest capability, which can include online text chat and voice commentary, as well as annotations and algorithms,” he continued.
When analysts tag video clips with key words, icons, graphics and expressions, the results are automatically captured and saved as metadata in a networked master catalog. The catalog can then be searched for data that can inform front-line decision-making on the latest threats being observed on video. For instance, FAME might provide footage and comments about a white truck that had previously been tagged as suspicious and that was now being followed on the current video feed (see illustration). The information that FAME stores is compliant with the standards devised by the motion picture industry and by NATO (STANAG 4609). “The system can process noncompliant imagery so it can be shared across the community,” Delay noted.
Any PC in the system can act as a browser-based client that relies on a FAME server for the processing power. Moreover, the system allows for situations in the field, where an analyst may be able to use only low-bandwidth connections. “Part of the core FAME product is a real-time transcoder that modifies outbound signals into lower bit rates for disadvantaged networks,” said Delay.
Harris has also developed what Delay claims to be the first scalable airborne video encoder. The Acuity H264 does scaling and cropping of images “on the fly,” he said.
