F-35 test pilots will begin flying this year with a third-generation helmet-mounted display system (HMDS) that incorporates modifications to the earlier-generation display system, which pilots deemed insufficient for missions the Joint Strike Fighter will perform. Last October, after testing the fixes over the course of two years, the Pentagon’s F-35 Joint Program Office (JPO) gained enough confidence in the new “Gen 3” system to stop the development of an alternate helmet-mounted display.

“I definitely have confidence that we are on the right track, that we have the right plan for these fixes in place and that it’s going to be a great system for the fleet,” said U.S. Marine Lt. Col. Matthew Kelly, government flight test director at the F-35 integrated training center at Naval Air Station Patuxent River, Maryland.

In 2011, the Department of Defense (DOD) identified the HMDS as one of several F-35 program risks. According to a “quick-look review” of the jet’s flight-test progress, test pilots found that the “Gen 2” helmet system being developed by the joint venture of Rockwell Collins (Stand Q79) and Elbit Systems (Stand N65) of America–then called Vision System International (VSI)–had inadequate night-vision acuity and experienced display jitter during aircraft buffeting. It was also not timely enough at importing imagery from the F-35’s Northrop Grumman AN/AAQ-37 distributed aperture system (DAS), a set of six infrared sensors, flush-mounted around the aircraft to provide the F-35 pilot with 360-degree, spherical coverage for situational awareness, missile warning and target detection functions.

Getting It Right

Getting the HMDS right is a serious issue because the F-35, the DOD’s costliest weapons program, was designed without a pilot’s heads-up display, a feature that is common to fourth-generation fighters. In September 2011, F-35 prime contractor Lockheed Martin (Stand CS02) awarded a contract to BAE Systems (Stand U67) to develop an alternate HMDS with detachable night-vision goggles (NVGs) as a fallback system in the event VSI failed to resolve issues with the chosen helmet-mounted display.

As recently as last September, Air Force Lt. Gen. Christopher Bogdan, the outspoken, reformist F-35 program executive officer, said the development of an alternate HMDS continued. However, in October, the JPO announced that it had stopped the development of the BAE Systems helmet in order to focus solely on bringing the Gen 2 HMDS to a “fully compliant” Gen 3 standard. “During the past two years, the JPO and Lockheed Martin used a disciplined systems engineering approach and conducted dedicated helmet flight tests to develop solutions to address the helmet’s technical challenges,” the program office said.

The Gen 3 HMDS “will include an improved night-vision camera, new liquid-crystal displays, automated alignment and software improvements,” according to the JPO. Further, a “cost guarantee” that Lockheed Martin, Rockwell Collins and Elbit Systems offered the government resulted in a 12-percent reduction from the previous cost of the HMDS–while the program will also recoup $45 million in funds it had originally allocated for the development of the BAE Systems alternatehelmet, the program office said.

The Test Regime

In a recent interview with AIN, Kelly, an F-35 test pilot, described some of the testing that took place during the intervening period between the start and termination of the alternate HMDS development. Flight tests in a surrogate aircraft using a Gen 2 “shell” helmet with a new night-vision camera, as well as tests involving prototype Gen 2 helmets with newly integrated inertial measurement units (IMUs) and software algorithms, gave test pilots enough knowledge to inform Bogdan’s decision to stop the alternate HMDS effort, he said.

The night-vision acuity of the Gen 2 HMDS, which contains an ISIE 10 sensor for low-light-level detection, was the system’s major deficiency, according to Kelly (ISIE stands for Intevac silicon imaging engine). An ISIE 11 sensor based on Intevac Photonics’ patented electron-bombarded activated pixel sensor (EBAPS) technology brings the system’s night-vision acuity closer to the 20/20 vision NVGs can provide. The alternate helmet with NVGs would not have been a perfect replacement, though. In order to switch between night vision and DAS displays, Kelly said, pilots had to remove the entire NVG assembly and then attach another visor, which was “very clumsy and not very pilot friendly,” he added.

Last summer, the F-35 program tested a production-representative night-vision camera with ISIE 11 sensor in a modified Gen 2 helmet, using a twin-engine King Air surrogate aircraft. Flying from St. Mary’s County Regional Airport in Maryland, close to NAS Patuxent River, pilots tested the system in high- and low-light conditions and compared it to using ANVIS 9 NVGs. Testers also used a ground-based laser designator to determine how far away pilots could spot a laser pointer.

“There were some limitations to the test,” Kelly acknowledged. “It wasn’t in an F-35, but it was close enough that we could make a confident decision about the usability and the effectiveness of the new ISIE 11 night-vision camera in the Gen 3 helmet. [We could say] we know enough now to make the decision to start saving money by not funding the alternate helmet. That fed [Bogdan’s] decision matrix in canceling the alternate helmet.”

Integrated Night Imagery

F-35s will have two night-vision cameras. A canopy bow in the jet obstructs the helmet camera, which is positioned above the pilot’s eye level on the HMDS. A second, dashboard-mounted camera is used in combination with the first; the imagery from both cameras is fused for display to the pilot.

In a separate interview with AIN, Intevac Photonics’ general manager Drew Brugal said, “the plan had always been” to eventually deliver the ISIE 11 sensor, which was not mature when the company was contracted to provide integrated night imagery for the F-35 HMDS. Last fall, Intevac started delivering ISIE 11 sensors to Elbit Systems of America, which builds the sensor into the night-vision camera. “The feedback we received was that [the night-vision camera] met the pilots’ expectations and they are comfortable going forward with the ISIE 11,” Brugal said.

The helmet system’s latency, or response time at importing DAS imagery–measured in milliseconds–was not the problem testers thought it would be, Kelly said. Pilots just hadn’t had the opportunity to use the DAS sensor array during flight testing. “Initially there was concern about the latency of the DAS and what that might look like,” he said. “But we were able to do some testing in the spring and summer of 2013 where we looked at a bunch of different tasks [and] some formation flying and, across-the-board, we found there was really no issue with the latency.”

Test pilots experienced helmet-mounted display jitter in areas of the F-35 flight envelope that haven’t been approved for training, Kelly said. The program addressed the problem by integrating micro IMUs and filtering algorithms in the HMDS to cancel out jitter effects. Pilots flew the fixes using a modified Gen 2 helmet. “It’s still not perfect, but it’s the 95-percent solution and the major issue there is resolved,” he said.

Asked if the jitter effect will be further improved, JPO spokesperson Kyra Hawn, answered: “In the bigger scope of the program, we have a lot of sophisticated technology. The constant challenge is, we’ve gotten to the 95-percent solution, which is viable and usable based on the mission requirements. What does it cost us to get the 5 [percent] in terms of investment and time, and what do we get as a result of getting that 100-percent solution? [What] do we derive from the additional investment and is it worthwhile given all other things?”

According to the JPO, the improved Gen 3 HMDS will be introduced to the F-35 fleet in low-rate initial production Lot 7 in 2016, and complete test and development the following year. The Marine Corps, which plans to declare initial operational capability (IOC) of its F-35Bs in July 2015, will start operations with Gen 2 helmet-mounted displays. “The Marine Corps understands that really for IOC we’re not counting on that [Gen 3] capability,” Kelly said.