JOINREGISTERLOGINCONTACT
OWNER LOG IN

JOINREGISTERCONTACT

Upcoming Events

  • This event has passed.
5th Cybersecurity Workshop

Challenges Facing Test and Evaluation

Register Now

PROGRAM GUIDE

ABSTRACTS AND BIOGRAPHIES

REGISTER ONLINE NOW

DOWNLOAD REGISTRATION FORM

PROGRAM OVERVIEW

Cybersecurity continues to be at the forefront of the DoD acquisition community. As this contested environment brings new challenges at as accelerated rate, the T&E community must be prepared to meet new requirements.

This Workshop provides an opportunity to share ideas among experienced T&E professionals regarding threat ad requirements, test capabilities, autonomous systems, and evaluation methodologies. Our goal is to share ideas on how to better characterize cybersecurity threats, evaluate system performance when attacked by a cybersecurity threat, and assess risk of using the system in the presence of a cybersecurity threat.

Please join us in Belcamp, Maryland, as members of the T&E community from academia, industry, and government discuss the evolving discipline of Cybersecurity T&E. Come share your thoughts, connect with others, and learn from some of the leading experts at this Workshop. The cyber threat will only increase with time. See you there!

PRE-WORKSHOP TUTORIALS

Pre-Workshop Tutorials require a separate fee from the Workshop. Single Tutorial – $205, Two Tutorials – $385.

 

Integrated Systems Engineering, Agile DevSecOps, and Test and Evaluation

Instructor: Dave Brown, PhD, CTEP – Consulting Engineer, Chesapeake Systems Engineering

With recent emphasis on Agile and DevSecOps development methodologies, many practitioners now believe that these new methodologies completely negate or replace the elements of program management, systems engineering, and independent test and evaluation. Nothing could be farther from the truth. In fact, Agile and DevSecOps incorporate many elements, often with only different names. For many programs, especially tightly coupled hardware and software programs, like almost all modern military systems, a hybrid systems engineering and Agile development approach is required. This approach must then be top level managed with program management techniques, and verified and validated with independent test and evaluation, especially for developmental test of integrated hardware-software, operational test, and cyber test and assessment.

This tutorial consists of a quick overview of systems engineering and test and evaluation. This is followed by a brief introduction to Agile and DevSecOps. Finally we will discuss techniques to effectively integrate the above when and where required.

 

Blockchain 101: Blockchain De-Mystified

Instructor: Mr. Duane Wilson, SURVICE Engineering

Why is the Blockchain such a technology phenomena in the today’s tech lingo? What are the current use cases of Blockchain? Can the Blockchain solve all of our problems? Is there any application to Test & Evaluation? Will it be here in the future? The aim of this tutorial is to attempt to answer all of these questions and provide a baseline understanding of what Blockchain technology is and what it is not – which is often even more important. We have broken our tutorial down into six distinct – yet related sections to attempt to appease a very diverse audience: Blockchain 101, Blockchain Components, Blockchain Applications, Blockchain Demo, Blockchain Development, and Blockchain Test & Evaluation.

  • In Blockchain 101 we will discuss Foundational Concepts of the Blockchain and demystify this term that so widely used today.
  • In Blockchain Components we break down the Blockchain into its logical components to show you how simple it is at its core.
  • In Blockchain Applications we discuss the myriad of use cases for Blockchain Technology and the different domains in which it is being used in practice.
  • In Blockchain Demo we allow you as the attendee to participate by presenting an interactive demo of the Blockchain.
  • In Blockchain Development we show the code behind the demo and all Blockchains in circulation today to appeal to the developers in attendance.
  • Lastly, in Blockchain Test & Evaluation we demonstrate how Blockchain applications can be tested and evaluated and where the T&E community at large would find some relevant uses for this innovative technology.
 

Building Better Models Using Robust Machine Learning Methods

Instructor: Thomas A. Donnelly, PhD – SAS Institute
 
Through case studies, you will learn to build better and more robust models with machine learning and predictive modeling techniques. Featured methods will include many types of regression (linear, logistic, penalized), neural networks (single layer, dual layer, boosted), and decision trees (simple, bagged, boosted). To make these methods robust you’ll learn to split your data into training, validation (tuning) and test subsets to prevent over fitting. And, when there are not enough data to support splitting, learn how to use penalization criteria to prevent over fitting. You will also see how to use graphical and statistical comparison techniques to help choose the best predictive model.
 
Featured case studies include building surrogate models of a computer simulation of a helicopter flying surveillance and identifying the best predicting model of the various logistic, decision tree, neural, spline, and regression models. A derivative data set of the 1998 KDD Cup Cyber Attack Data set with over 40 possible causes of 20 types of attack will be used to show the benefit of building a robust ensemble predictor model. It will also be shown how to use penalized regression methods for highly correlated data to create in many cases, models that are almost as good as complex neural networks, but much more interpretable – even offering confidence intervals about predictions. This tutorial is for analysts, scientists, engineers and researchers interested in learning how machine learning can help them use the data they have today to better predict tomorrow.
 
 

Introduction to Cybersecurity Test & Evaluation

Instructor: Mr. Pete Christensen – Director, Cyber Support to OSD Programs, The MITRE Corporation
 
Now more than ever, Program Managers (PM) must ensure that cybersecurity be given careful consideration throughout the system lifecycle. Specifically, this includes identifying cybersecurity requirements early in the acquisition and systems engineering lifecycle. Initiating a focus on cybersecurity earlier will provide PMs the opportunity to give careful consideration, upfront, to related cybersecurity testing activities that can be integrated into the engineering planning and design phases.
 
Results of informal cybersecurity testing can then be applied to influence design and development efforts and to posture programs for success in Developmental Test (DT) and Operational Test (OT). The Deputy Assistant Secretary of Defense (DASD) Developmental Test and Engineering (DT&E) has collaborated with key systems engineering stakeholders to develop disciplined processes that will assist Program Managers (PM) in implementing an incremental and iterative phased approach to develop cyber secure systems.
 

Fundamentals of Distributed Testing (1 hour), and Identifying Requirements and Vulnerabilities for Cybersecurity (3 hours)

Instructors:
Fundamentals of Distributed Testing
Mr. Scott “Gunner” Thompson – Electronic Warfare Associates – GSI

Identifying Requirements and Vulnerabilities for Cybersecurity
Mike Lilienthal, PhD, TRMC, and Mr. Patrick “Preacher” Lardieri, Lockheed Martin

There are two tutorials presented during the 4 hour block of time allocated. The first hour “Fundamentals” was developed to provide information and an approach for the DoD T&E community on how to use distributed methodologies to plan for, prepare, and execute distributed test events. The tutorial is intended to present executive level material on fundamental concepts of Distributed Testing, as well as generate a discussion on considerations and requirements that can be used for the design of Integrated Cyber Security T&E in a Joint mission environment. Desired outcome is to have attendees incorporate Distributed Testing methodologies into their own processes and guidelines.

The last three hours of the tutorial was developed for the many Service acquisition, System Engineering (SE), and Test and Evaluation (T&E) teams that are starting to move their programs from “checklist information assurance or compliance” cyber security approach to a proactive, iterative risk management process with the goal of ensuring personnel can still carry out their duties in a cyber contested environment. Many people are struggling to formulate a practical and effective approach to develop requirements and a plan to incorporate cyber security into their SE and T&E activities using the recent spate of cybersecurity policies and guidelines released by the Office of the Secretary of Defense.

The tutorial will:

  • Explain the DOT&E 6 Step Cyber T&E process
  • Explain the OSD Cyber Table Top (CTT) process
  • Explain the National Cyber Range’s cyber T&E methodology
  • Describe how both tools improve the engineering and testing of cyber resilient systems and how they support the DOT&E 6 Step Cyber T&E Process
  • Present lesson learned using these tools over the past 5 years

The CTT (which has been adopted by the Navy and DT&E) is a rigorous, intellectually intensive and interactive data collection and analysis process that introduces and explores the potential effects of cyber offensive operations on the capability of a system to carry out its designed functions. It produces a prioritized list of actionable recommendations to support more informed decisions and tradeoffs in a fiscally constrained environment.

The National Cyber Range is an OSD TRMC capability that provides the ability to conduct cybersecurity test and evaluation of DoD systems in support of cyber risk assessments. It is capable of instantiating systems in classified close test range and enabling red and penetration test teams to conduct hands-on evaluation of cyber attacks on the systems under evaluation.

The tutorial is based on the lessons learned from using the CTT and NCR processes to support acquisition programs across the services.

Intended Audience: It is intended for attendance by Acquisition Program Management Offices, Systems Engineers, Chief Developmental Testers, and Lead Developmental Test and Evaluation (DT&E) Organizations.

How to Successfully Plan Test Strategy for Agile Development in a Gov Framework 

Mr. Hans Miller and Ms. Colleen Murphy – The MITRE Corporation

This course provides a framework and guidance for programs transitioning to an agile construct or new programs established with an agile construct. The intended audience includes requirements managers, program managers and test managers executing DoD programs; however, the overall principles could apply to multiple agencies. This course is not a singular solution for agile testing; it acknowledges the different approaches needed for different programs and is intended to provide students with an understanding of concepts that can be tailored to their specific program. 

This course will walk through characteristics of agile process and where it does and does not apply to help inform expectations. It will cover US code, OSD and service policy as it applies to agile testing and planned policy updates designed to allow greater flexibility. The core of the course covers upfront planning and strategy considerations for successful testing; requirements, contracting, infrastructure investments, automation and test execution. It concludes with how to translate that strategy into concise, timely, and relevant documentation from the TEMP, test plan, and test reporting.

 
 

Cyber Test and Training Solutions with TENA and JMETC

Instructor: Mr. Gene Hudgins, KBRwyle
 
Together, TENA and JMETC enable interoperability among ranges, facilities, and simulations in a timely and cost-efficient manner. TENA provides for real-time system interoperability, as well as interfacing existing range assets, C4ISR systems, and simulations; fostering reuse of range assets and future software systems. JMETC is a distributed, LVC capability which uses a hybrid network architecture; the JMETC Secret Network (JSN), based on the SDREN, is used for secret testing and the JMETC Multiple Independent Levels of Security (MILS) Network (JMN) is the T&E enterprise network solution for all classifications and for cyber testing. JMETC provides readily available connectivity to the Services’ distributed test and training capabilities and simulations, as well as industry resources.
 
This tutorial will address the current impact of TENA and JMETC on distributed systems engineering as well as their significance to the cyber Test and Training community.
 
 

Planning and Executing Cyber Tables Tops, Facilitator Training

Instructor: Mr. Vinny Lamolinara – Defense Acquisition University (DAU)
 
The workshop introduces and applies the Cyber Table Top (CTT) mission-based cyber risk assessment (MBCRA) method to help discover cyber vulnerabilities, gauge their risk, propose mitigations and inform other competencies, documents and events across the DoD acquisition lifecycle. The workshop will establish an understanding of the threat and “thinking like a Hacker”; provide a “wheel of access” methodology to identify and diagram surface-attack characteristics; include cross-competency personnel, including users, to identify and prioritize cyber-attacks / vulnerabilities in a Red / Blue / White Team “wargame” mission scenario; and provide a construct to characterizes and report risk and mitigations in order to design and maintain cyber resilient systems and personnel in the acquisition and operational phases of an Information or Platform weapons system.
 
Participants will conduct exercises in each phase to reinforce and apply the concepts and methodology will learn how cybersecurity principles apply to their career fields. Students will create a surface attack taxonomy, role play different competencies including engineering, test, cybersecurity, logistics, safety, intelligence, contracts and the adversary.
 
The case studies and scenarios will build up in complexity culminating in a mini-CTT execution and Cyber Risk outbrief (to a simulated PM) for an exemplar weapons systems at the UNCLAS level. Students will also apply CTT results to inform Test, AoA ICD/CDD/CPD, RFP/SOW, Specification, Architecture and upgrade / patch / ECP requirements as well as acquisition and risk management strategy. This workshop will allow enable students to participate in CTT efforts in their respective programs. Tailorable to the specific customer needs.
 
Objectives: Given a cybersecurity scenario, use Surface-attack characterization and Cyber Table Top Methodology to discover cyber vulnerabilities, gauge their risk, propose mitigations and inform other competencies, documents and events across the DoD acquisition lifecycle.
 
  1. Understand and apply the “think like a Hacker” adversarial threat concept to cybersecurity.
  2. Understand, apply and create the “wheel of access” surface-attack methodology to create a taxonomy useable to discover cyber vulnerabilities for DoD systems.
  3. Understand and apply CTT methodology for various acquisition scenarios. 
  4. Create program manager level outbrief delineating risks, mitigations and implications for test, requirements, design, logistics and safety. 
 
Target Attendees: The acquisition workforce, including industry partners, who design, build, procure, maintain, and provision cybersecurity capabilities.
 
 

Software Assurance

Instructor: Mr. Robert Martin – Senior Secure Software & Technology Principal Engineer, The MITRE Corporation
 
This tutorial will explore how the the directed activities in the DoDI 5200.44, DoDI 8510.01–2014, and DoDI 8500.01–2014, and their Program Protection Plans, Developmental test and evaluation, Systems Engineering design & architecture reviews can be used to gain assurance about DOD Software and its resilience to attack.
 
Improving our assurance that the mission will not be circumvented, undermined, or unnecessarily put at risk through attacks on the software that provides critical mission capabilities requires a shift in focus and integration of many types of assessment activities across the acquisition life cycle.
 
This tutorial will also cover how the public vulnerability information, along with an understanding of the weaknesses in commercial and open source software puts the mission at risk. Publicly available about these weaknesses and the patterns of attacks they are susceptible to can be used to test GOTS and custom software so we have insight into how attackable DOD Software is and what can be done to address those risks.

PLENARY SPEAKERS

Mr. Mike Zwiebel – Director, Test Management, U.S. Army Test and Evaluation Command (ATEC)

Mitch Crosswait, PhD – Director, Operational Test & Evaluation (DOT&E)

Mr. Chip Ferguson – Deputy Executive Agent, DOD Cyber Test Ranges, Test Resource Management Center (TRMC)

Mr. Joe Bradley – Director, Cyber Resiliency Office for Weapons Systems, AFMC

Mr. Patrick Kastner – Deputy Director (Acting), National Preparedness Programs, DHS Office of Test and Evaluation

Ms. Kim Ploskonka – C5ISR Center

Mr. Bill Hughes – Institute for Defense Analysis (IDA)

Mr. Paul Johnson – Scientific Advisor, MCOTEA

SPECIAL PANELS

 

Cyber Test Range Panel
Moderator Mr. Chip Ferguson Deputy Executive Agent, DOD Cyber Test Ranges Test Resource Management Center (TRMC)
Panelists Jon Davis, PhD Principle Investigator RAND Corporation
Rob Tamburello, PhD Deputy Director National Climate-Computing Research Center (NCRC)
Ms. Sophia Paros   Defense Information Systems Agency (DISA)
LtCol Ben George, USAF   AFSP 346 TS/CC
 

OTA Panel
Moderator COL Jason Woodford, USA Chief, Survivability Evaluation Directorate U.S. Army Evaluation Center (AEC)
Panelists Ms. Ellena Millar Director, Cybersecurity OT&E (01D) U.S. Navy Operational Test and Evaluation Force (OPTEVFOR)
Mr. Maurice Sanders Chief, Cyber Division Marine Corps Operational Test and Evaluation Activity  (MCOTEA)
Mr. Matt Frandsen Chief, Cyber Test Operations Division Air Force Operational Test and Evaluation Center (AFOTEC) 
Mr. Rob Aguilera Senior Vice President, Garud Technology Services DHS, FEMA, OTA
 

Industry Cyber Range/Capabilities Panel
Moderator Mr. Pete Christensen   The MITRE Corporation
Panelists Mr. Randy Smith   BOEING Test and Evaluation
Mr. Patrick Ladieri   Lockheed Martin Corporation (LMCO)
Mr. Mark Bradbury Chief Engineer, Intelligence, Information and Services, Cyber Warfare Campaign Raytheon
 

Workforce Development Panel
Moderator Mr. Terry Murphy   Department of Homeland Security (DHS)
Panelists Tom Meservy, PhD Associate Professor Brigham Young University
Diana Burley, PhD   Institute for Information Infrastructure Protection (I3P)
Portia Crowe, PhD Chief, Cyber & Systems Engineering U.S. Army Program Executive Office Command, Control and Communications-Tactical (PEO C3T)
Mr. Isidore Venetos   FAA

 

TECHNICAL TRACK SESSIONS

  • Analysis — Innovative Techniques
  • Cybersecurity Implications for Autonomous Systems
  • Testing and Test Methodologies – Current Capabilities and Future Needs
  • Threat and Requirements Definition – Viewpoints from T&E Customers

CONTINUING EDUCATION UNITS (CEUs)

Each of the 4-hour Pre-Workshop Tutorials provide 4 contact hours of instruction (4 CEUs) that are directly applicable to your professional development program, including the Certified Test and Evaluation Professional Credential (CTEP).

In addition to the Pre-Workshop Tutorials, the Workshop provides 4 contact hours of instruction (4 CEUs) for each half-day, 8 contact hours of instruction (8 CEUs) for each full-day, or 20 contact hours of instruction (20 CEUs) for attending the full Workshop, that are directly applicable to your professional development program, including the Certified Test and Evaluation Professional Credential (CTEP).

MEETING LOCATION

Water’s Edge Event Center
4687 Millennium Drive
Belcamp, MD 21017
410-297-9467

SPONSORSHIP INFORMATION

ITEA is a 501(c)(3) professional education association dedicated to the education and advancement of the test and evaluation profession. Registration fees, membership dues, and sponsorships are tax deductible.

Your sponsorship dollars help defer the cost of the Symposium and support the ITEA scholarship fund, which assists deserving students in their pursuit of academic disciplines related to the test and evaluation profession. Sponsorship and related benefits will become effective on receipt of payment.

NOTE:  This ITEA event is a non-competitive environment meant for a free exchange of ideas and information. 

REGISTRATION

Early Bird Registration until February 25th
$645 – Regular Registration*
$495 – ITEA Member / Government Employee / Active Duty Military

Regular Registration February 26th to March 12th
$745 – Regular Registration*
$595 – ITEA Member / Government Employee / Active Duty Military

Late Registration after March 12th
$845 – Regular Registration*
$695 – ITEA Member / Government Employee / Active Duty Military

*Regular Registration rate includes one-year membership to ITEA.

New T&E Professional (less than 5 years of T&E experience) VERIFICATION REQUIRED – Includes two Lunches, the Networking Reception, and a one-year ITEA membership for Non-ITEA Member.

  • $95 – Early Bird Registration prior to February 26, 2019
  • $195 ($120 ITEA Member) – Regular Registration February 26 – March 12, 2019
  • $295 ($220 ITEA Member) – Late Registration after March 12, 2019
  • Download verification form here

SUBSTITUTION AND CANCELLATION POLICY: Substitutions are permitted. Refunds are not available within ten (10) days prior to the start of the event. Requests for cancellation submitted between ten (10) to 45 days prior to start date of the event will be subject to a $250 cancellation fee. Requests for cancellation greater than 45 days prior to the start date of the event will be subject to a $100 cancellation fee.

WORKSHOP COMMITTEE - Cyber@itea.org

Workshop Chair – Ms. Chris Susman, SURVICE Engineering Co

Technical Program Co-Chairs
– Dave Brown, PhD, CTEP, Chesapeake Systems Engineering
– Mr. Pete Christensen, CTEP, The MITRE Corporation
– Paul Dailey, PhD, CTEP, Johns Hopkins University Applied Physics Lab
– Mr. Bruce Einfalt, Applied Research Laboratory, The Penn State University

Sponsorship Co-Chairs
– Ms. Cathy Pritts
– Mr. Jim Myers

Registration – Mr. James Gaidry, CAE –  jgaidry@itea.org

Event Details

Date

25 - 28 Mar 2019

Venue

Water’s Edge Event Center

EVENT QUICK LINKS

SPONSORS

Gold

dfgdfg dfgdfg

Silver

dfgdfg

Location Map

Share: