Upcoming Issue Themes for The ITEA Journal
The ITEA Journal of Test and Evaluation Upcoming Themes
Authors can submit articles for standard review and editing or for the lengthier ITEA peer review process. Please consider writing an article, share this document with coworkers, and provide feedback. Articles of general interest to ITEA members and The ITEA Journal of Test and Evaluation readers are always welcome, and authoring these articles is a great way to contribute to our industry. Authors do not need to be ITEA members.
The ITEA Journal offers a forum for sharing knowledge and ideas crucial to our changing T&E workforce. Please submit your contributions today to firstname.lastname@example.org, attention: Publications Chair. Steve Gordon, PhD (email@example.com)
Articles for The ITEA Journal include specialty features, each 2-3 pages long, and key technical articles:
Test and Evaluation of Hypersonic Systems (Issue 39-3, September 2018). Two technological challenges of testing hypersonic systems are to (1) advance hypersonic technology itself, staying ahead of rivals, and (2) develop countermeasures for use in combat situations when an adversary employs such technology against our systems. By the very nature of these hypersonic systems, testing and evaluation will have very significant challenges – some yet to be adequately described. These challenges include having the range space, instrumentation, and data capture systems to conduct the testing live or having the trusted simulations to conduct the testing in a live-virtual-constructive environment. Secondly, testing the countermeasures on both sides of engagements may be a challenge. It should be noted that hypersonic speed provides potential improvements in operational flexibility for accomplishing the mission (such as short notice urgent space launch or engaging time-critical targets), diminishing the effectiveness of current detection and countermeasures systems, and potential reductions in operational cost to accomplish the mission. Because of hypersonics, we will need testing improvements in aerothermodynamics, materials science, hypersonic navigation, guidance and control systems, endo-atmospheric and exo-atmospheric flight dynamics, instrumentation including telemetry, and extremely large data flows.
(Manuscript deadline: June 1, 2018)
Advanced Instrumentation and Information Systems Technology for T&E (Issue 39-4, December 2018) Testing requires data and analysis of the data from instrumentation and information systems. In order to improve these systems, science and technology (S&T) advancements are sought in these areas: Time-Space Position Information (TSPI), Advanced Sensors, Advanced Energy and Power Systems, Non-Intrusive Instrumentation, Range Environmental Encroachment, and Human Performance Measurement and Assessment. For this theme, articles related to these S&T areas are requested, and articles describing advancements in real-time data analytics, automated analysis, and automated test reporting are also welcomed. Advanced instrumentation and information systems could produce an overwhelming amount of data from tests, causing a sense of drowning in data — absent advances in data analytics.
(Manuscript deadline: September 2018)
Statistical Methods in T&E (Issue 40-1, March 2019). Mathematical and statistical methods have traditionally been used in testing. Some new approaches in using statistical methods provide new tools to allow testers to estimate how much testing is enough. Too little testing and too much testing waste money. Application of statistical methods coupled with disciplined up-front analyses, may help increase the scientific underpinning of tests. Up-front analyses include following accepted processes to determine outputs, determining how to measure the outputs accurately, identifying what super-set of inputs may affect the outputs, determining the critical few inputs and how to measure the inputs accurately, setting statistical confidence, evaluating the design for statistical power, and making trade-offs. How are designation of standard operating procedures and conducting measurement system analyses related to reducing unnecessary noise and maintaining statistical power? Would upfront analyses also include verification and validation of requirements and verification, validation, and accreditation of modeling and simulation to support the test? What are the consequences of unnecessary noise in the systems that support testing? How can we increase statistical power of tests without increasing testing?
(Manuscript deadline: December 2018)
Accelerating Test and Evaluation with LVC and Agile (Issue 40-2, June 2019). New! – Draft Theme: Fielding effective, secure systems to warfighters at the speed of need is essential, but this goal is difficult to achieve given that industrial-age acquisition and systems engineering processes, including T&E, do not mesh well with use of modern software-intensive systems. Agile software processes that combine acquisition events with developmental and operational testing show promise in decreasing historic timelines. Combining built-in information technology security (Sec) with software development (Dev) and information technology operations (Ops) throughout the SecDevOps software build is also streamlining the delivery of secure software-intensive systems. Finally, increasing the focus on what the warfighter needs now and what is necessary for potential conflicts will provide more usable and effective systems. Other key ideas for improving effectiveness and accelerating this process include early prototyping via modeling, simulation, and gaming; evaluating hardware prototypes; combining test events; the use of Artificial Intelligence to improve data gathering and reporting; and evolutionary program development. Simulation and gaming environments can be used to allow warfighters to evaluate the advantages of system variants and alternative tactics before the hardware and software are finalized. Using early synthetic and live prototyping, early and repeated testing, continuous system refinement based on warfighter and test feedback, SecDevOps, and other innovative ideas will enable more rapid acquisition of systems that are also more fully tested by warfighters and testers, evolutionary delivery of capability, and swift modification of secure systems. The ultimate goal is to more quickly place new, fully capable weapon systems in the hands of warfighters.
(Manuscript deadline: March 2019)
Aligning Modernization of DoD Test Ranges with National Defense Strategy (Issue 40-3, September 2019). DoD test ranges need to be modernized in order to adequately test advanced systems in development and current, updated systems designed to perform in future warfighting environments. Our systems must be tested sufficiently in the anticipated environments and scenarios to make sure they are resilient. Unless properly handled or accommodated, environmental encroachment can cause limitations to testing and training; these limitations can include restrictions, degradations, interference, pressures to reduce maneuver space, and increased cost. The science and technology (S&T) research in this area is extensive. Are there lessons learned or success stories in preventing incompatible development near test and training ranges? Papers related to these topics, to developing improved and expanded ranges, and to resolving encroachment issues are requested. Also, papers that discuss ways to complement range testing with gaming or simulation are welcomed. Range challenges will continue to impact test and training ranges, but the limitations can be moderated. Best practices in this area would be of interest to the readers.
(Manuscript deadline: June 2019)
Drowning in Data: How to Gain Timely Information and Knowledge from Data (Issue 40-4, December 2019). Digital technology and accelerating improvements to digital technology provide us with the ability to acquire, create, and store data at unprecedented rates and volumes. Literature searches that would have taken days or weeks in the age of library card catalogs now can be accomplished in seconds over the internet. Data, or more generally, information, has become big business, in addition to being the business of T&E. Yet, technological solutions come with their own problems: the immense flow of data from tests has not been met by a commensurate growth in the ability to exploit the data to gain information and knowledge. This theme examines the issues and the potential solutions for the need to extract meaning and value from the mountains of data. Articles are invited on such topics as data acquisition, storage, archiving, access, validation, exploitation, and visualization; data as a service; cloud computing; service-oriented architecture; metadata syntax and semantics; instrumentation; accelerating the process of acquiring data to making a decision; data and sensor fusion; data preservation; distributed and nonrelational databases; and related topics. The theme seeks insights, lessons learned, and success stories of gaining information and knowledge in a timely manner from test data.
(Manuscript deadline: September 2019)
The Right Mix of T&E Infrastructure (Issue 41-1, March 2020). Our T&E infrastructure is regularly evaluated for downsizing, improvement, or changes in ownership. The right footprint of T&E infrastructure depends on the tests in the pipeline and future systems in design. How do we know what and how much is necessary? How do we find the right sources of information (who knows?), and how can we search for facilities available nationally and internationally? Can we share government, industry, and university facilities within and across country boundaries? Would this type of sharing cause conflict of interest issues? Will overlap of contractor testing, developmental testing, and operational testing reduce or increase the demand on test infrastructure? Can a shift to earlier developmental testing in representative operational environments and a push for integrated testing reduce the load on test infrastructure?
(Manuscript Deadline: December 2019)
Systems Engineering and T&E Synchronization (Issue 41-2, June 2020). These activities are part of an integrated, solid T&E process. Systems engineering provides the process and tools to build the right effective products in the best way. Reliability strives to develop a system that is available and suitable for the intended use and resilient to disruption. Lifecycle support looks at maintainability and supportability with long-term ownership costs in mind. Testing makes sure these requirements and others are satisfied by the designed and produced system. Blending these initiatives into an integrated T&E program could help us field the right system for the user. Invited papers could include discussions of success stories, lessons learned, drawbacks, benefits, good intentions gone awry, and alternative views.
(Manuscript Deadline: March 2020)
Innovations in Modeling and Simulation (M&S) Use for T&E (Issue 41-3, September 2020). M&S allows depiction of places and times where the users of the M&S cannot otherwise go. These battlespaces and times could be in the past, future, or into areas where enemy defenses or other physical barriers prevent immediate travel. In light of this powerful benefit of M&S, there are at least two situations where M&S gives the most benefit in T&E. The first is the reduction of testing cost. Rather than running flight tests or destroying weapons systems, we simulate the activity in the laboratory to determine if the system satisfies the developmental documentation and performance and if the system meets operational effectiveness and suitability. We can then verify performance with much fewer flight tests — saving money by running fewer high-cost live tests. The second is the ability to use M&S to generate operational edge conditions that we cannot create in the live operational environment. We can simulate operational conditions that would be expensive or impossible to create in a physical operational test. Examples could be testing aircraft in lethal environments or after the departure from safe flight or load testing an air traffic control system with more tracks than would be possible in the live operational environment to prove performance resilience requirements.
(Manuscript Deadline: June 2020)
T&E for Cyber Security and Readiness (Issue 41-4, December 2020). Key information passed through network connections improves the speed and lethality of combat operations; yet, use of networks opens doors to vulnerabilities. Network connections for home computers, smart phones, social media, and entertainment add enjoyment; yet, ease of use often equates to increased ease of misuse and scamming. Systems that support the military, our finances, our health records, and our other personal information must pass risk management, information assurance/security, net readiness, and cyber readiness tests. Yet, these tests, when passed, do not provide 100% assurance of protection. Systems and the networks that connect them are subject to attacks from many sources; however, the goal of the attack is nearly always to take something valuable. Money, personal information, trust, freedom, military information and plans, or intellectual property are often taken with very minimal effort and cost. How much testing is required to provide an acceptable level of risk and enduring protection to expected attacks?
(Manuscript deadline: September 2020)
New Initiatives in Developmental and Integrated T&E (Issue 42-1, March 2021). These initiatives include, but are not limited to, mission engineering and improved interoperability testing across mission support levels, implementing the Developmental Evaluation Framework (DEF), designation of Chief Developmental Testers, improving reliability T&E, and improving cybersecurity testing. Systems need to be tested in relevant mission areas and relevant levels of warfare so that the systems perform as intended in the Service, Joint, and Coalition warfare missions in which the system may be employed. Part of that requirement includes the interoperability to effectively and safely operate in those types of operations, levels of warfare, and battlespaces. What are the benefits and lessons learned of implementing the DEF to improve DT&E planning and streamline some T&E documentation? In designating Chief Developmental Testers, how has that initiative improved DT and OT results? Has the emphasis on designing for reliability planning and reliability growth and improved estimation of reliability resulted in realized improved reliability and risk reduction? How have tabletop exercises and consistent cybersecurity T&E assistance helped improve cybersecurity DT&E.
(Manuscript deadline: December 2020)
Training the Future T&E Workforce (Issue 42-2, June 2021). Test and evaluation over the next decade will need a workforce of professionals from many academic disciplines. The academic majors will certainly include science, technology, engineering, and math (STEM); yet, management, communications, psychology, and other types of majors also may be needed for the T&E profession. We will need a steady supply of the right academic majors from our technical schools, colleges, and universities, and we will need initial training for the incoming workforce to be ready to become T&E professionals. The need for an inflow of new talent suffers from a constrained supply and competes with many demands for the same disciplines from industry, academia, and other parts of the government. Increasing the throughput of the right new talent would help considerably. And, innovative ways to attract the new workforce, provide recurring training to the existing workforce, and fund career enhancement will help T&E retain the workforce needed.
(Manuscript deadline March 1, 2021)
Testing Artificial Intelligence and Collaborative Autonomous Systems (Issue 42-3, September 2021). For these systems, we must test and train as we fight. Unmanned and autonomous vehicles fly reconnaissance, target location, and combat sorties. They work for us. They fly in combat and public airspaces. They travel on the ground, in the air and space, on the water, and under water on a variety of missions. Unmanned and autonomous vehicles may be fully controlled by humans, semi-automated, or fully autonomous individually or in swarms or dissimilar teams. How can we test the collaborative software that allows these systems to complete many missions, including operating in autonomous swarms. How have we conducted T&E of these systems in the past? Are there best practices or lessons learned? Are there any guidelines for how to test the semi-automated and autonomous behaviors in representative operational environments? How can we test these systems and their “brains” in degraded environments in order to determine operational robustness to combat conditions and to cyber degradations?
(Manuscript deadline: June 1, 2021)
Success Stories in T&E (Issue 42-4, December 2021). Are there improved testing regimens under which systems have been evaluated, resulting in lives saved, costs reduced, and/or battles won? For this issue, we will mostly focus on the high-profile T&E experiences where the testing helped influence product changes that made the product better in terms of cost, safety, or effectiveness. Feedback from tests at or before the design phase can be implemented with minimal cost to the program and can help improve the cost, suitability, and effectiveness of the system. Product or system modifications as a result of feedback from tests after the development phase of acquisition may cost more, but the test results later in a program are often essential to building the right product. This issue is also seeking examples of tests where, for some reason, systems were fielded even though they were lacking in effectiveness, suitability, or resiliency. So, we would also welcome “Less than Successful Stories in T&E” in this issue as well. Examples of how things have gone wrong help us understand the importance of making sure things go right.
(Manuscript deadline: September 1, 2021)
Accelerating T&E to he Speed of Need (Issue 43-1, March 2022). The speed of need is the speed required to transition the system tested from definition of user needs to the initial necessary operational capability. The demands of war have shortened the timeline on requirements for military systems. Rapid acquisition and rapid fielding initiatives arising from urgent operational needs have created an entire industry in the defense community. The pace at which transportation security and border protection measures need to be deployed is similarly increased. We see information technology (IT), especially software and cyber threats and defenses, change with a frequency of months, not years. Test and evaluation (T&E) must be responsive to the acquisition timelines. The Federal Aviation Administration, Border Patrol, law enforcement, and many other organizations are adapting to the changing speed of need. This issue takes a candid look at agile software development processes, cyberspace T&E, defense IT acquisition reform, rapid acquisition and fielding, reconfigurable test capability, testing on demand, reuse, and other ideas for streamlining the T&E process in support of accelerating deployment of new products, services, and capabilities.
(Manuscript Deadline: December 1, 2021)