June 2018 – Unmanned and Autonomous Vehicle Testing
The theme for this issue is “Unmanned and Autonomous Vehicle Testing,” and the issue includes two Guest Editorials, an Inside the Beltway feature, a Cultivating the T&E Workforce feature, the President’s Corner, and six technical articles.
The first of our two Guest Editorials is written by Major General Matthew Molloy. His topic is “T&E in a Time of Risk and Change – Answering the Call: Prototyping, Experimentation, and Acquisition at the Pace of Need.” He states that expediting the fielding of critical warfighting capabilities is essential. We need to find a way to unbridle our processes to allow for serendipity, experimenting, falling forward, pivoting with technology, prototyping, and delivering a center-mass product with clear growth potential.
Our second Guest Editorial, written by Wilson Felder, Ph.D., is “Systems Engineering and Test and Evaluation: a Professional Society Collaboration.” This feature provides new information concerning an initiative to create a roadmap for the growth and development of systems engineering. Several professional societies, including ITEA, have been invited to collaborate in this initiative with the International Council on Systems Engineering (INCOSE). Readers are asked to provide feedback on four possible mechanisms to achieve the desired goals.
Our Inside the Beltway article is titled, “A Conversation with The Honorable Bob Behler, Director, Operational Test and Evaluation (DOT&E).” Director Behler was interviewed by Eileen Bjorkman, Ph.D., a member of the Publications Committee of The ITEA Journal of Test and Evaluation. Director Behler discussed his background and path to becoming DOT&E, and he discussed his desire to bring attention to the growth in the dependence on software and the need to secure it. Director Behler also explained his focus areas and priorities.
The President’s Corner, written by William Keegan, President of the Board of Directors for ITEA, discusses upcoming events, news about ITEA, and key themes, features, and articles in this issue of The ITEA Journal. As a Cultivating the T&E Workforce feature, we’ve included a short “2018 Update on Simulation: Helping Develop the Future Workforce” as an initiative to motivate high school students toward the sciences and engineering. The new national program for certification in Modeling and Simulation (M&S) has increased the number of certifications in Florida and now has been adopted by Virginia. To date, 280 high school students and teachers have earned the M&S Certification, and many of these students are employed in the M&S industry.
Our first of six technical articles, “The Future of Autonomous Ground and Surface Systems Testing,” written by Camille Robbins and Melissa Steffen, Ph.D., discusses the new challenges that require fundamental changes in testing for autonomous systems. Testing sensors and decision-making logic in all possible operational environments are likely not possible. Range and instrumentation improvements are the beginning to improve the ability to more fully test autonomous systems.
The second technical article in this issue, “Overcoming the Challenges in Test and Evaluation of Autonomous Robotic Systems” written by Darryl Ahner, Ph.D., et al., describe eight autonomous system test challenges faced by the test and evaluation community. These challenges span from test adequacy to post-acceptance testing. The authors conclude with five challenges that require test research funding.
For our next article, “A Bioinformatics Approach to Finding Candidate Organism Strains for True Positive Detection Testing of Biological Warfare Agent Detectors,” by Felicia Ellis, et al., proposes a method to identify and use candidate biological agent strains that are less likely to be detected, thus providing a more rigorous test of the detectors. The authors point out that this goal is meant to add realism to the testing of the detectors and to make the controlled lab testing environment more realistic compared to what field detection would encounter if it were feasible. The search for candidates continues.
In our fourth technical article, “Applying the Gaps Model of the Service Industry: Five Ways that Prevent Test & Evaluation from Meeting Customer Expectations,” written by Allan Alfafara, CTEP, Test and Evaluation (T&E) is compared to being a service organization. The author uses the Gaps Model of Service quality to illustrate how T&E as a service might fail to satisfy customers. He presents solutions that might be needed to help recover the good standing of T&E and prevent future unwanted gaps in the services provided by T&E.
In the fifth technical article, “Estimating Operational Resilience in a Cyber Contested Environment,” Paul Johnson discusses a methodology to highlight the complicated problem of estimating operational resilience based on the portfolio of cybersecurity. The author presents a method for estimating operational resilience by using data from a variety of tests and by using a variety of inputs.
In the sixth and last technical article, Angus McLean, Ph.D., et al., authored “LVC-Enabled Testbed for Autonomous System Testing” to highlight how the test requirements for autonomous systems differ from piloted aircraft and to propose a Live, Virtual, Constructive (LVC) environment solution. The authors explain how the LVC environment can be improved using real data, and they discuss the advantages of the LVC approach.
I hope you enjoy this second issue of 2018 for The ITEA Journal of Test and Evaluation. By the time you receive issue 39-2 in June, the September 2018 issue 39-3 is being finalized. That theme in September will be “Test and Evaluation of Hypersonic Systems.” For the next issue (the fourth issue of 2018), 39-4, the deadline for submissions is just after September 1, 2018, and the theme will be “Advanced Instrumentation and Information Systems Technology for T&E.” We have posted all themes and descriptions for the remainder of 2018 – 2021 on the ITEA website. Please provide feedback on the choice of themes, and please write early and often.
March 2018 – Testing Using Facilities Around the World
The theme for this issue is “Testing Using Facilities Around the World,” and the issue includes two Guest Editorials, an Inside the Beltway feature, the President’s Corner, and five technical articles.
Our first Guest Editorial, written by Keith Joiner, PhD, is “International Test and Evaluation (T&E): Looking Towards the United States Not Outward from the United States.” This feature provides a unique description of international T&E looking from a perspective external to the United States. The author also provides information on the areas where Australia has parts to play in international T&E. He foresees the need for allies of the United States to find ways to resynchronize T&E capabilities with the United States.
Our second Guest Editorial, written by Fred Moorefield, is “Modernizing DoD’s Spectrum Technology Test and Evaluation Capabilities to Meet Growing Military Requirements.” In this feature, he states that achieving electromagnetic spectrum (EMS) superiority provides important tactical, operational, and strategic advantages. This goal of EMS superiority will be paramount in future military operations. He concludes that new EMS technology development will require new T&E capabilities to understand impacts and interoperability capabilities in the electromagnetic environment.
Our Inside the Beltway article is titled “Reciprocally Using International Test Facilities Under the Director, Operational Test and Evaluation (DOT&E) Test and Evaluation Program (TEP).” Gloria Deane and Mitchell Dossett state the two fundamental considerations of the TEP: that adequate testing is necessary to ensure that systems work as expected for warfighters and that T&E cooperation with allies improves interoperability in coalition operations. The authors explain the unique TEP Reciprocal Use of Test Facilities project agreement, and they state that TEP agreements have contributed to improved interoperability in the coalition warfare.
President’s Corner, written by William Keegan, President of the Board of Directors for ITEA, discusses upcoming events, news about ITEA, and key themes, features, and articles in The Journal.
Our first of five technical articles, “National Advanced Spectrum and Communications Test Network (NASCTN): A Framework for Leveraging T&E Facilities,” written by Keith Hartley, et al., discusses the need for a collaborative and flexible spectrum-sharing network. This network is needed because the growth of broadband wireless services continues worldwide, and the crowded spectrum environment drives regulators to look increasingly at spectrum sharing. The authors explain that NASCTN was designed and chartered as a cooperative agreement to quickly unite expertise, equipment, facilities, and resources to support spectrum-sharing test measurements. Expanding wireless services across government and commercial users will require increased spectrum access and technologies, and NASCTN is a model for expansion and replication to meet future needs.
The second technical article in this issue, “Next Generation Intelligent Test Network Environments” written by J. Michael Barton, Ph.D., and Venkat Dasari, Ph.D., describes many developments that are leading to a seemingly overwhelming flood of data to be evaluated for tests. For instance, the growth of networked systems of systems, an array of advanced instrumentation providing data at enormous aggregate rates, and testers needing near-real-time feedback about the state of tests as they occur, in combination, may require intelligent test networks to manage the bandwidth and sequence the computations to smart instrumentation. The authors state that current technologies are available to field intelligent instrumentation and programmable networks to process the growing flow of data from tests.
For our next article, “Estimating the Effect of Avionics Complexity on Safety Assurance” by Sarah Sheard, et al., the authors review the research to date from the Federal Aviation Administration’s Complexity and Safety Project. They state that, even with the problems associated with complexity, we must find ways to regain the ability to predict and control systems. The team created and tested a formula for complexity that can be applied early in system development, and they applied the formula to three varied test cases.
Our fourth technical article, “Services-Based Testing of Autonomy (SBTA)” is a peer-reviewed article, and Christopher Eaton, et al. suggest a method to test newly incorporated services, enabling a consistent test and evaluation (T&E) verification and validation (V&V) capability, designated TEVV. The authors state that the Services-Based Testing of Autonomy will provide a cost effective, scalable, and efficient means to test autonomy services, and TEVV will be a key asset as autonomous systems have significant mission growth.
In the fifth and last technical article, “Measuring the Influences of Mission Command,” Robert Pokorny, et al., discuss a means of developing metrics for mission command so that the United States Army can measure the effects of new systems or processes on mission command. This research leveraged a scoring system called Performance Assessment in Complex Environments (PACE) that has been successfully used on other projects in varied domains. Since the effectiveness of mission command enables mission execution, the developed metrics may be used to evaluate the effectiveness of new technology and may also reduce the operational risk of introduction of new, disruptive processes or technologies. Software that can unobtrusively measure mission command effectiveness could provide a key missing metric in operational mission testing.
I hope you enjoy this first issue of 2018 for The ITEA Journal of Test and Evaluation. By the time you receive issue 39-1 in March, the June 2018 issue 39-2 is being finalized. That theme will be “Unmanned and Autonomous Vehicle Testing.” For the next issue (the third issue of 2018), 39-3, the deadline for submissions is just after June 1, 2018, and the theme will be “Test and Evaluation of Hypersonic Systems.” We have posted all themes and descriptions for the remainder of 2018 – 2021 on the ITEA website. Please provide feedback on the choice of themes, and please write early and often.