2022 ITEA Journal Issues Abstracts
Building Trust through Testing by Flournoy, Haines, and Chefitz
ITEA is going online! As part of our shift to an online Journal we will be making select articles available to the general public. The December 2020 featured column is by Michèle Flournoy, Avril Haines, and Gabrielle Chetitz. The column provides an executive summary of a recent report the three authored entitled, “Building Trust through Testing.” The summary highlights how new technologies such as machine learning (ML) and artificial intelligence (AI) require new testing approaches. They make 11 recommendations providing a useful framework to pave the way for future Test and Evaluation methods for ML/AI.
Interested in more great articles like this, join ITEA for only $95 or contribute articles to the Journal – all authors get a full copy of the issue in which their article appears.
This month’s issue starts off with another great Conversations with Experts column in which J. Michael Barton, Ph.D., interviews Malcolm Tutty, Ph.D. Dr. Tutty has served in the Royal Australian Air Force, Public Service and Industry in a multitude of test, operations, engineering, staff, project management and command roles. Dr. Tutty shares his experiences in and out of T&E. He also calls back to numerous quotes that I am sure will keep ITEA readers engaged. A quick line of advice that he slips in, “Enjoy your service, make friends everywhere, and I hope you have successful careers in and out of the T&E space along with having as much fun as I have had with many of y’all.”
In our Workforce of the Future column this month we have contributions from undergraduate and graduate student-led research at the University of Memphis. The first article, “Testing the Control Software in Autonomous Cyber-Physical Systems,” by Bhavana Garimilla and Mark L. Gillenson, Ph.D., examines how to test software in autonomous cyber-physical systems. The authors highlight the need for ensuring safety and the critical role of testing software extensively when it incorporates neural networks. In the next article, “Managing the Security of Data in the Cloud,” Venkata Akshara Motati and Mark L. Gillenson, Ph.D., discuss the security of cloud-based data storage. They provide recommendations that merit consideration before moving data to the cloud. In our third article, “Simulation and Testing of AI/Autonomous Systems,” Faizan Zafar and Eddie Jacobs, Ph.D., report on state-of-the-art practices and conduct simulations using the Mississippi State University’s Autonomous Vehicle Simulation (MAVS) library.
Our first technical article comes from Nick Jones and Wayne Adams at the Scientific Test and Analysis Techniques Center of Excellence (STAT COE), Air Force Institute of Technology. Their article entitled, “Model Selection and Use of Empiricism in Digital Engineering,” addresses the important role that modeling plays in test and evaluation. The article outlines a process based in STAT by which a digital engineering test program should leverage the digital engineering (DE) environment by first utilizing existing resources on the digital thread and then first-principles and empirical data to build new models when needed.
In our next article Donald H. Costello III, Ph.D., from the United States Naval Academy and Huan Xu, Ph.D., from the University of Maryland discuss “Using a Run Time Assurance Approach for Certifying Autonomy Within Naval Aviation.” The article illustrates how to use run time assurance as the framework for a safety of flight certification of autonomous behavior. They leverage an aerial refueling task as a use case for the framework for certification.
In our final article a research team from Virginia Tech including Laura Freeman, Ph.D.; Justin Kauffman, Ph.D.; Erin Lanus, Ph.D.; Daniel Sobien; and Tyler Cody, Ph.D., summarizes some best practices they have adopted in testing Artificial Intelligence Enabled Systems (AIES). As the lead author on the article entitled, “Best Practices for Addressing New Challenges in Testing and Evaluating Artificial Intelligence Enabled Systems,” I would like to highlight that these practices might prove useful to others as they embark on developing test programs for AIES, but they are surely not complete and will need to be updated as our understanding of testing AIES evolves.
Enjoy the Issue!
In this month’s Conversations with Experts column J. Michael Barton, Ph.D., interviews Eileen Bjorkman, Ph.D., Executive Director at the Air Force Test Center. In the article, “Lofty Goals, A Celebration of Flying,” Dr. Bjorkman discusses her career in the US Air Force, the decision to pursue a Ph.D., and her career in Test and Evaluation (T&E). In her advice to people interested in T&E careers, Dr. Bjorkman recommends that individuals “try out different types of jobs early on and don’t be afraid to try something outside your area of expertise.” She highlights how technologies change and to think of your academic degree as just a starting point.
In our Workforce of the Future column with three articles written by students that participated in the High Performance Computing Modernization Program Internship Program:
In our first technical article Alexander Hillman, a major in the US Air Force and a Ph.D. candidate in aeronautical and astronautical engineering at the Massachusetts Institute of Technology, discusses how testers are uniquely equipped to lead new data strategy efforts. Hillman’s article, “Leveraging a Tester’s Experience to Build a Successful Enterprise Data Strategy,” describes the years of experience, knowledge, and the challenges that testers bring to the data strategy discussion. He outlines the critical elements of data strategies and provides astute observations on how Department of Defense processes and protections can inhibit progress in achieving strategic objectives.
In our next technical article Everett H. Oliver, Ph.D., and Shahram Sarkani, Ph.D., P.E., and Thomas A. Mazzuchi, D.Sc., discuss the value of systems thinking and model-based systems engineering in assessing the resilience of critical infrastructure sectors. In their article, “Assessing the Resilience of Critical Infrastructure Systems and Supply Chains Against Cascading Disruptions,” they provide a case study to demonstrate the approach leveraging the historical example of Superstorm Sandy and the food supply chain.
In our final technical article Frank Cruz from the US Air Force, Applied Spectrum Technology Research Office (ASTRO), at Edwards AFB discusses, “Optimizing Antenna Placement Using Modeling and Simulation.” Mr. Cruz’s paper showcases how modeling and simulation can be used to enable trade-space exploration in order to optimize hardware design. He highlights the potential cost savings when compared to the cost of chamber testing.
Enjoy the issue!