Rapid advances in training, including gaming, are cutting costs while honing a wide range of soldiering skills.
Synthetic training seeks to bring together live, virtual, and constructive training in a common environment. Also referred to as Synthetic Training Environments (STE), it can provide individual, small unit and crew training, as well as tactical unit interaction at various levels including brigade and higher. STE elements can be applied to mounted and dismounted ground forces, to aerial platforms, and to command exercises. By combining virtual reality, augmented reality, and artificial intelligence, it can project various scenarios placing participants in seemingly realistic situations and circumstances.
Various operating terrains and tactical events can be set, confronting participants with simple to complex exercises. Training in a synthetic environment provides both the opportunity to review and critique the exercise and to repeat it.
Former US Secretary of Defense and retired Marine Corps General James Mattis in support of the synthetic training concept stated that it “could allow a soldier to fight twenty-five bloodless battles.” The objective is allowing for repetition in a task or situation, while allowing mistakes facilitates in this ‘bloodless’ environment and learning from them. Further, this learning process allows new challenges to be introduced into subsequent iterations as appropriate encouraging adaptability.
Technical Origin
Synthetic training provides an immersed experience to those participating. In this, the military is drawing from existing and rapidly evolving commercial computer and gaming technologies. It parallels and, in some cases, expands on its combination of processing, an array of sensors, Bluetooth links, and display technologies. An example is Microsoft’s application of its HoloLens head mounted gaming display adapted as the US Army Integrated Visual Augmentation System (IVAS).
Doug Bonderud, writing in Northrop-Grumman’s NOW on-line magazine observed that “commercial gaming VR [virtual reality] in mimicking real-world scenarios such as combat situations or high-speed racing, replicates military’s requirements… It is also making its way into the world of quick decision-making. Current games provide the foundation for improving cognitive ability, communications, and teamwork. VR gaming introduces a kind of controlled chaos where gamers are immersed through visual, auditory and even tactile stimuli and must then respond to chaotic situations.” Parallels to the battlefield, combat and military challenges are evident. Taking the process from the living room to the field is the focus of a number of industry and government efforts.
Synthetic Training Environments (STE)
Northrop-Grumman Training and Simulation Solutions’ understanding of STE is as follows: ‘Synthetic Training actually consists of a number of elements including Virtual Reality (VR), Augmented Reality (AR), Live, Virtual and Constructive (LVC), Augmented Virtuality, and computer-based modelling and simulation programmes. The differences are at times subtle but important. VR replaces your world with a simulated one in which you are fully immersed and everything you see, hear, and interact with is virtually generated. Augmented Reality overlays outside data, 3D objects and/or video real or simulated on top of one’s current view of the world, while continuing to let you see the world around you. Live, Virtual and Constructive (LVC) provides a resilient and secure open architecture and modular scenarios so warfighters can adapt to any mission and combat exercise. Augmented Virtuality (AV) is like Virtual Reality, with one exception — not everything one sees, hears, and interacts with is virtually generated. In AV, real-world objects are merged into and interact with the virtual world, such as sounds, smells, sensations, movement, and information. Computer-based modelling and simulation programs provide computer-based modelling and simulation capabilities to enhance mission readiness using models to create and display simulations and analysis on a computer. The computer-based modelling and simulation can be used for training, data analysis, experimentation, and gaming.’
Benefits of Synthetic Training
ST offers a range of benefits such as training on systems and tasks that would be otherwise too expensive or dangerous, such as bomb disposal. It also allows the safe simulation of events such as equipment failures or collisions in order to teach the proper responses. ST can also offer step-by-step guidance in conducting maintenance, repair and assuring the proper execution of procedures. Equally important is its ability to not only introduce various situations and conditions into a simulated event, but to then re-enact that event again and again. Participants confront problems in a ‘real’ scenarios and can then repeat the exercise addressing errors and refining their responses. Through this repetition, procedures and teamwork can be perfected while the insertion of unanticipated contingencies to expand the learning.
Another advantage of training in the synthetic environment is its portability and ready accessibility. As Mike Knowles, president at Cubic Mission and Performance Solutions, shared: “Virtual and augmented reality training mediums can be provided that can be used almost anywhere. They do not necessarily require dedicated on-site infrastructure or even prescribed training times. It allows for training to be conducted at more convenient times and at more locations, even sites across the globe.” This is particularly useful in providing deployed training where maintaining and improving proficiency can be a challenge.
Augmented Reality (AR)
Virtual Reality, according to data gathered by Northrop-Grumman, has demonstrated 80 percent increased effectiveness over traditional learning in many military applications such as individual skill, vehicle, equipment, and operational training. It provides an improved learning experience that fully engages the user. It can reduce training time and costs. Yet, it is augmented and mixed-reality capabilities that are being viewed by the military as offering the most significant advances. As Marlo Brooke, CEO of AVATAR Partners, a company offering AR software, pointed out: “Augmented reality transposes data or other digitally created images on top of a real-world field of view. Mixed reality also merges real and virtual worlds to produce new environments where physical and digital entities co-exist.” Plus, it can be equally applied to training and actual service and combat.
The military’s embrace of AR is well represented by the US Army’s pursuit and recent $21.88 billion ten-year contract to Microsoft for 120,000 augmented-reality headsets. The HoloLens based Integrated Visual Augmentation System (IVAS) combines a computer, a sensor array, a wide lens with internal display and communication and data links to produce, share, and enhance information presented to the soldier wearing it. These allow IVAS to also be used as a training device that can replicate various scenarios and situations reflecting individual to unit tasks. Coupled with outside video and event data collection it would contribute to an interactive and reviewable tactical training system combining physical and virtual features. In fact, Microsoft’s HoloLens 2, its current civilian version, is now being pitched more for surgeons, maintenance engineers and work applications than for gamers.
Embracing Military Synthetic Training
The capabilities and benefits of synthetic training has been recognised by militaries worldwide. The People’s Liberation Army (PLA) of China has taken initiatives to use these technologies to improve the training given to its military forces. The Intelligent Commando VR training system developed by the state-owned China Electronics Technology Group, allows trainees to carry out combat drills on virtual battlefields, alongside either real teammates or AI manufactured avatars. Trainees wear VR headsets in which battlefield simulations are presented. Interactions can be recorded allowing post-event review and critique. Said Wu Zhen Feng, vice director of the 28th Research Institute of the CETC: “we have integrated a number of key technologies including large space positioning technology, optical and inertial navigation sensor interaction space positioning technology, the technology to rapidly construct virtual scenes based on geographical information, as well as the technology to assess combat drills based on machine learning.”
Derek Williams, director of communications at OTR, a United Kingdom IT integrator working in VR/AR since 2007 commented that “the Chinese military have consistently proclaimed that virtual reality technology will provide the PLA with a distinctive edge in combat readiness, vehicle and weapons lifecycle, personnel management, and even information warfare.” China has made significant investment toward realising this goal.
The Republic of Korea (ROK) Army has committed to being ‘slimmer but smarter’ by strengthening individual soldiers’ combat capability. It introduced its Warrior Platform, an advanced combat system for individual soldiers allowing them to become more independent combat capability. A critical part in achieving this is enhancing combat skills. For that the ROK is turning to Synthetic Training. In 2018 it invested in VR comprehensive training systems for firing and tactical training, later adding AR simulation for exercising command and control. In March 2021 Korean Aerospace Industries (KAI) committed to entering the LCV integrated training market. This may be in response to the Ministry of Defence 2020 White Paper plans for establishing LCV with VR and AR as part of its ‘small defence’ innovation.
Singapore’s limited land space, a common constraint in the Indo-Pacific, makes ST especially attractive. In fact, it is viewed as a pioneer in simulation training. According to the Ministry of Defence, it realised a 60 percent reduction in time to train its truck and small vehicle drivers in road and cross-country using VR developed by its Defence Science and Technology Agency (DTSA). It is also adapting commercial games for military applications. As a conscript force levering gaming ST has shown significant benefits.
Swedish defence company Saab has been active in military training systems for over 35 years. Its live combat training solutions are in service with over 25 nations including the US, the UK, a majority of the European nations as well as Thailand, Japan, Australia, New Zealand and Singapore. Asap Thegström, head of Training & Simulation, Saab’s Dynamics business explained: “Every training scenario, environment, or system, needs to reflect authentic battlefield experiences in order to hone combat instincts. The foundation of this is allowing warfighters to train with weapons and equipment that accurately simulate the fit, form and function of their battlefield kit.” This includes reflecting as closely as possible its actual performance so as not to demonstrate false capabilities. “From this standpoint,” he continued, “range, ballistics and time of flight are hugely important characteristics of any weapon system. These characteristics need be in the warfighter’s mind both when conducting training and on operations. So, training systems must also replicate these same characteristics, in order that weapons can be correctly employed, and the necessary and correct lessons learned.”
The Australian Armed Forces have been exploring ST for some years and have introduced VR marksmanship, flight, and other individual skill trainers since 2015. A collaboration by the Defence Science and Technology Group (DST) and Saab Australia recently demonstrated the HoloLens. It and other ST technologies are being investigated toward exploiting existing tools to facilitate the transition to a fifth generation fighting force. GPCAPT Pete Mitchell, for the director of Plan Jericho, shared: “We are exploring the possibilities…As ARV applications become more commercial and mainstream, we expect this potential to grow exponentially.” In fact, the Army’s is using VR as its primary training medium for its new Protected Mobility Vehicle (PMV) and for close combat visualisation. Australian industries like InVeris, Saab and Cubic Australia are offering state-of-the-art ST products. InVeris displayed a range of ST products at the June 2021 Land Forces show in Brisbane. These include the FATS 180MIL high-fidelity simulation trainer that had been endorsed and fielded by the US Army and Marines.
The Future
Craig Beddis, CEO and co-founder, Hadean said: “Synthetic training environments leverage gaming engines to simulate the battlefield and its many different facets. Military training today often involves multi-domain operations, where forces from land, sea, air and now even space and cyberspace come together. These are complex scenarios, with a number of potential outcomes, making them difficult to simulate for training purposes.”
The challenge for ST is incorporating the differing aspects of land, sea, air and cyber into a representative yet common picture and accommodate their interaction. Still, given the projected advances in technology Beddis suggests this is possible explaining that “the technologies (such as Hadean’s) provide robust and secure scalability, enabling vast simulations, including more detail, realism and complexity. Plus, users can connect from anywhere in the world and interact through a common operating picture.”
by Stephen W. Miller
