2018 ITEA Journal Issue Abstracts

2018 ITEA Journal Issue Abstracts

September 2018 – Test and Evaluation of Hypersonic Systems

The theme for this issue is “Test and Evaluation of Hypersonic Systems,” and the issue includes the President’s Corner, the feature “Testing Hypersonic Systems 101,” “Walt Hollis Remembered,” and seven technical articles.

President’s Corner, written by William Keegan, President of the Board of Directors for ITEA, discusses upcoming events, news about ITEA, and key themes, features, and articles in this issue of the The ITEA Journal.

The editorial “Testing Hypersonic Systems 101” is provided as an overview of the motivations to develop hypersonic systems and the requirements for test and evaluation research and development necessary to field safe and effective hypersonic systems for travel, cargo, and military purposes.

Our first of seven technical articles, “Navigating the Testing of Hypersonics in the 21st Century,” written by Elizabeth Hill and Greg DeVogel, discusses the gaps in developmental and operational testing capabilities required for hypersonic systems. The authors propose areas where testing hypersonic systems may require changes to processes or additional test assets.

The second technical article in this issue, “Interface Management in Systems Engineering” written by George Gardner, Ph.D., describes the activity of interface management—particularly in the practice of systems engineering. He compares the naming conventions and the interface management tools used across some federal agencies to ensure smooth integration, improve interoperability, and provide easier and cheaper technical enhancement. This is a reprint to correct errors in a previous printing of this article.

Our next article, “Defining Resiliency” by Sarah Pramanik, Ph.D., proposes a framework for defining resiliency in a way that facilitates how a system is designed, implemented, and tested. She suggests that the definition of resiliency is a necessary first step, and she proposes a resilient system is able to recover to an acceptable level of performance once it has been under duress. The author also discusses examples, such as the story of the A-10 aircraft’s resiliency to combat damage.

In our fourth technical article, “Modernizing Test and Evaluation with High Performance Computing and Advanced Networks,” is written by J. Michael Barton, Ph.D., et al., and the authors summarize progress to date for the High Performance Computing Modernization Program. The authors look at the future of computing, high performance computing, and advanced networking.

In the fifth technical article, “One War Training System,” Matt Morrisset discusses the emerging need for centralized data storage at the tactical training ranges. He discusses the types of information being recorded during a mission and how a centralized data store makes sense.

For the sixth article in this issue, “Independent Verification and Validation (IV&V),” David Thrasher and Jessica Pippard present tools that can be created to assist in performing IV&V. They suggest tools and methods that assist in examining the source code and that help run the executables. These tools assist the development team to improve the final product.

In the seventh and last technical article, Jeff Kalibjian authored “Overcoming Data Security Challenges in Testing Environments” to highlight the fact that data stored (at rest) is often more at risk than data in motion (in transmission). The author explains that once the data are post processed they may be more at risk. He offers some alternative data security solutions.

I hope you enjoy this third issue of 2018 for The ITEA Journal of Test and Evaluation. By the time you receive issue 39-3 in September, the December 2018 issue 39-4 is being finalized. That theme in December will be “Advanced Instrumentation and Information Systems Technology for T&E.” For the next issue (the first issue of 2019), 40-1, the deadline for submissions is just after December 1, 2018, and the theme will be “Statistical Methods in T&E.” We have posted all themes and descriptions for the remainder of 2019 – 2021 on the ITEA website. Please provide feedback on the choice of themes, and please write early and often.


June 2018 – Unmanned and Autonomous Vehicle Testing

The theme for this issue is “Unmanned and Autonomous Vehicle Testing,” and the issue includes two Guest Editorials, an Inside the Beltway feature, a Cultivating the T&E Workforce feature, the President’s Corner, and six technical articles.

The first of our two Guest Editorials is written by Major General Matthew Molloy. His topic is “T&E in a Time of Risk and Change – Answering the Call: Prototyping, Experimentation, and Acquisition at the Pace of Need.” He states that expediting the fielding of critical warfighting capabilities is essential. We need to find a way to unbridle our processes to allow for serendipity, experimenting, falling forward, pivoting with technology, prototyping, and delivering a center-mass product with clear growth potential.

Our second Guest Editorial, written by Wilson Felder, Ph.D., is “Systems Engineering and Test and Evaluation: a Professional Society Collaboration.” This feature provides new information concerning an initiative to create a roadmap for the growth and development of systems engineering. Several professional societies, including ITEA, have been invited to collaborate in this initiative with the International Council on Systems Engineering (INCOSE). Readers are asked to provide feedback on four possible mechanisms to achieve the desired goals.

Our Inside the Beltway article is titled, “A Conversation with The Honorable Bob Behler, Director, Operational Test and Evaluation (DOT&E).” Director Behler was interviewed by Eileen Bjorkman, Ph.D., a member of the Publications Committee of The ITEA Journal of Test and Evaluation. Director Behler discussed his background and path to becoming DOT&E, and he discussed his desire to bring attention to the growth in the dependence on software and the need to secure it. Director Behler also explained his focus areas and priorities.

The President’s Corner, written by William Keegan, President of the Board of Directors for ITEA, discusses upcoming events, news about ITEA, and key themes, features, and articles in this issue of The ITEA Journal. As a Cultivating the T&E Workforce feature, we’ve included a short “2018 Update on Simulation: Helping Develop the Future Workforce” as an initiative to motivate high school students toward the sciences and engineering. The new national program for certification in Modeling and Simulation (M&S) has increased the number of certifications in Florida and now has been adopted by Virginia. To date, 280 high school students and teachers have earned the M&S Certification, and many of these students are employed in the M&S industry.

Our first of six technical articles, “The Future of Autonomous Ground and Surface Systems Testing,” written by Camille Robbins and Melissa Steffen, Ph.D., discusses the new challenges that require fundamental changes in testing for autonomous systems. Testing sensors and decision-making logic in all possible operational environments are likely not possible. Range and instrumentation improvements are the beginning to improve the ability to more fully test autonomous systems.

The second technical article in this issue, “Overcoming the Challenges in Test and Evaluation of Autonomous Robotic Systems” written by Darryl Ahner, Ph.D., et al., describe eight autonomous system test challenges faced by the test and evaluation community. These challenges span from test adequacy to post-acceptance testing. The authors conclude with five challenges that require test research funding.

For our next article, “A Bioinformatics Approach to Finding Candidate Organism Strains for True Positive Detection Testing of Biological Warfare Agent Detectors,” by Felicia Ellis, et al., proposes a method to identify and use candidate biological agent strains that are less likely to be detected, thus providing a more rigorous test of the detectors. The authors point out that this goal is meant to add realism to the testing of the detectors and to make the controlled lab testing environment more realistic compared to what field detection would encounter if it were feasible. The search for candidates continues.

In our fourth technical article, “Applying the Gaps Model of the Service Industry: Five Ways that Prevent Test & Evaluation from Meeting Customer Expectations,” written by Allan Alfafara, CTEP, Test and Evaluation (T&E) is compared to being a service organization. The author uses the Gaps Model of Service quality to illustrate how T&E as a service might fail to satisfy customers. He presents solutions that might be needed to help recover the good standing of T&E and prevent future unwanted gaps in the services provided by T&E.

In the fifth technical article, “Estimating Operational Resilience in a Cyber Contested Environment,” Paul Johnson discusses a methodology to highlight the complicated problem of estimating operational resilience based on the portfolio of cybersecurity. The author presents a method for estimating operational resilience by using data from a variety of tests and by using a variety of inputs.

In the sixth and last technical article, Angus McLean, Ph.D., et al., authored “LVC-Enabled Testbed for Autonomous System Testing” to highlight how the test requirements for autonomous systems differ from piloted aircraft and to propose a Live, Virtual, Constructive (LVC) environment solution. The authors explain how the LVC environment can be improved using real data, and they discuss the advantages of the LVC approach.

I hope you enjoy this second issue of 2018 for The ITEA Journal of Test and Evaluation. By the time you receive issue 39-2 in June, the September 2018 issue 39-3 is being finalized. That theme in September will be “Test and Evaluation of Hypersonic Systems.” For the next issue (the fourth issue of 2018), 39-4, the deadline for submissions is just after September 1, 2018, and the theme will be “Advanced Instrumentation and Information Systems Technology for T&E.” We have posted all themes and descriptions for the remainder of 2018 – 2021 on the ITEA website. Please provide feedback on the choice of themes, and please write early and often.


March 2018 – Testing Using Facilities Around the World

Journal Mar18 Cov webThe theme for this issue is “Testing Using Facilities Around the World,” and the issue includes two Guest Editorials, an Inside the Beltway feature, the President’s Corner, and five technical articles.

Our first Guest Editorial, written by Keith Joiner, PhD, is “International Test and Evaluation (T&E): Looking Towards the United States Not Outward from the United States.” This feature provides a unique description of international T&E looking from a perspective external to the United States. The author also  provides information on the areas where Australia has parts to play in international T&E. He foresees the need for allies of the United States to find ways to resynchronize T&E capabilities with the United States.

Our second Guest Editorial, written by Fred Moorefield, is “Modernizing DoD’s Spectrum Technology Test and Evaluation Capabilities to Meet Growing Military Requirements.” In this feature, he states that achieving electromagnetic spectrum (EMS) superiority provides important tactical, operational, and strategic advantages. This goal of EMS superiority will be paramount  in future military operations. He concludes that new EMS technology development will require new T&E capabilities to understand impacts and interoperability capabilities in the electromagnetic environment.

Our Inside the Beltway article is titled “Reciprocally Using International Test Facilities Under the Director, Operational Test and Evaluation (DOT&E) Test and Evaluation Program (TEP).” Gloria Deane and Mitchell Dossett state the two fundamental considerations of the TEP: that adequate testing is necessary to ensure that systems work as expected for warfighters and that  T&E cooperation with allies improves interoperability in coalition operations. The authors explain the unique TEP Reciprocal Use of Test Facilities project agreement, and they state that TEP agreements have contributed to improved interoperability in the coalition warfare.

President’s Corner, written by William Keegan, President of the Board of Directors for ITEA, discusses upcoming events, news  about ITEA, and key themes, features, and articles in The Journal.

Our first of five technical articles, “National Advanced Spectrum and Communications Test Network (NASCTN): A Framework for Leveraging T&E Facilities,” written by Keith Hartley, et al., discusses the need for a collaborative and flexible spectrum-sharing network. This network is needed because the growth of broadband wireless services continues worldwide, and the crowded  spectrum environment drives regulators to look increasingly at spectrum sharing. The authors explain that NASCTN was designed  and chartered as a cooperative agreement to quickly unite expertise, equipment, facilities, and resources to support spectrum-sharing test measurements. Expanding wireless services across government and commercial users will require increased  spectrum access and technologies, and NASCTN is a model for expansion and replication to meet future needs.

The second technical article in this issue, “Next Generation Intelligent Test Network Environments” written by J. Michael Barton, Ph.D., and Venkat Dasari, Ph.D., describes many developments that are leading to a seemingly overwhelming flood of data to be evaluated for tests. For instance, the growth of networked systems of systems, an array of advanced instrumentation  providing data at enormous aggregate rates, and testers needing near-real-time feedback about the state of tests as they occur, in combination, may require intelligent test networks to manage the bandwidth and sequence the computations to smart instrumentation. The authors state that current technologies are available to field intelligent instrumentation and programmable networks to process the growing flow of data from tests.

For our next article, “Estimating the Effect of Avionics Complexity on Safety Assurance” by Sarah Sheard, et al., the authors  review the research to date from the Federal Aviation Administration’s Complexity and Safety Project. They state that, even with the problems associated with complexity, we must find ways to regain the ability to predict and control systems. The team  created and tested a formula for complexity that can be applied early in system development, and they applied the formula to  three varied test cases.

Our fourth technical article, “Services-Based Testing of Autonomy (SBTA)” is a peer-reviewed article, and Christopher Eaton, et al. suggest a method to test newly incorporated services, enabling a consistent test and evaluation (T&E) verification and validation (V&V) capability, designated TEVV. The authors state that the Services-Based Testing of Autonomy will provide a cost effective, scalable, and efficient means to test autonomy services, and TEVV will be a key asset as autonomous systems have significant mission growth.

In the fifth and last technical article, “Measuring the Influences of Mission Command,” Robert Pokorny, et al., discuss a means of developing metrics for mission command so that the United States Army can measure the effects of new systems or processes on mission command. This research leveraged a scoring system called Performance Assessment in Complex Environments (PACE) that has been successfully used on other projects in varied domains. Since the effectiveness of mission command enables  mission execution, the developed metrics may be used to evaluate the effectiveness of new technology and may also reduce the operational risk of introduction of new, disruptive processes or technologies. Software that can unobtrusively measure mission  command effectiveness could provide a key missing metric in operational mission testing.

I hope you enjoy this first issue of 2018 for The ITEA Journal of Test and Evaluation. By the time you receive issue 39-1 in  March, the June 2018 issue 39-2 is being finalized. That theme will be “Unmanned and Autonomous Vehicle Testing.” For the next issue (thethird issue of 2018), 39-3, the deadline for submissions is just after June 1, 2018, and the theme will be “Test and Evaluation of Hypersonic Systems.” We have posted all themes and descriptions for the remainder of 2018 – 2021 on the ITEA  website. Please  provide feedback on the choice of themes, and please write early and often.