Process Improvement at the Federal Aviation Administration for the Acquisition of Software Intensive Systems

 

Linda Ibrahim, FAA, 800 Independence Ave. SW, Washington DC

Larry LaBruyere, TRW/SETA, One Federal Systems Park Drive, Fairfax, VA

 

Linda Ibrahim is the process improvement lead at the Federal Aviation Administration.  She chairs the Corporate Software Engineering Process Group and is project leader, architect, and lead author on FAA’s integrated Capability Maturity Model (CMM) Project.  Linda has over 30 years of software engineering.  She holds a BA in Mathematics (Duke University), MS in Information Science, and Ph.D. in Electrical Engineering (University of Hawaii). 

 

Larry LaBruyere is the TRW task lead for process improvement support to the FAA Chief Scientist for Software Engineering.  Larry has over 25 years experience in software and systems engineering and over 15 years with TRW.  He has a BA in Mathematics (Anderson University) and an MS in Technology of Management (American University). He is authorized by the SEI as a lead assessor for CMM Based Appraisal for Internal Process Improvement, and is authorized to teach the Practical Software Measurement course.

__________________________________________________________________

Introduction

The Federal Aviation Administration (FAA) is taking steps to improve the processes it uses for the acquisition of software intensive systems.  The FAA integrated Capability Maturity Modelsm (FAA-iCMMâ) was developed as the framework for this improvement, along with the FAA-iCMM appraisal method that is being used to assess the maturity of the FAA's systems acquisition processes.  Process action teams have been formed across the FAA to focus on improving their processes.  A major training effort is ongoing to educate FAA personnel and support contractors in both the applicable disciplines and in how to do process improvement.   Measurement is a major aspect of FAA process improvement at several levels.  This paper will briefly address these topics to provide an overview of process improvement at the FAA.

FAA’s Process Improvement Infrastructure

The FAA Corporate Software Engineering Process Group (SEPG) is responsible for planning, motivating, facilitating, and implementing FAA process improvement.  The SEPG is led from the Office of the Chief Scientist for Software Engineering, and receives overall guidance from a cross-FAA Software Engineering Executive Committee.  SEPG members come from 10 directorates across several FAA lines of business.  Typically each corporate representative chairs a local, directorate level SEPG.  Corporate SEPG working groups focus on metrics, training, process action team coordination, communication, and the process improvement framework that guides improvement: the FAA-iCMM.

Overview of the FAA CMM Integration Project

The FAA developed the FAA-iCMM to guide improvement of the engineering, management, and acquisition processes it uses in acquiring software intensive systems.  Three CMMs had been being used separately in different FAA directorates that work on different aspects of acquisition, the CMM for: Software, Systems Engineering, and Software Acquisition.  These CMMs have different architectures, goals, terminology, and appraisal methods and none alone covers all FAA system acquisition activities.  While some improvements were being made using one model, the goal of FAA-wide, full lifecycle, process improvement remained elusive.  In addition, the FAA had moved to using integrated product teams as the implementation arm for its new Acquisition Management System and these teams needed processes that interrelated their disciplines.

The FAA-iCMM initiative began in the fall of 1996 with an analysis and preliminary merger of these 3 CMMs at the process area level, and one sample process area at the base practice level.  These efforts demonstrated that it was possible to integrate CMMs of different architectures and that the resultant model contained a significant reduction in the number of process areas and practices while still covering the individual CMM disciplines.

In March 1997, the FAA formed a team of FAA and external CMM and discipline experts and began work on the integrated model.  The project purpose was to derive a reference model that would: describe key elements of an effective system acquisition process, describe an evolutionary improvement path, have an associated appraisal method, and faithfully and robustly capture all features of its 3 source CMMs.  Meanwhile, the Software Engineering Institute (SEI) started development of a Common CMM Framework to provide guidance to multiple CMM users and to assist CMM developers and integrators.  The FAA-iCMM project followed those draft guidelines.

A complete draft of the FAA-iCMM was completed by June 1997 and submitted to the SEI for review. FAA management adopted an FAA-iCMM-related performance goal that same month.  In late September, a joint SEI-FAA session was held to ensure consensus that the FAA’s work captured its source CMMs and followed CMM principles, construction guidelines, and requirements.  Version 1.0 of the FAA-iCMM was released in November 1997 with endorsement by the SEI as a new product type - an iCMM.

Overview of the Model

The FAA-iCMM is structured to answer 3 process improvement questions: what activities should be performed (the domain aspect), how can performance be improved (the capability aspect), and what processes should be focused on next (maturity levels).  The FAA-iCMM Appraisal Method (FAM) supports application of the model.

·   The Domain Aspect.  The domain is the acquisition of software intensive systems.  There are 23 process areas derived from integrating the 52 process areas/key process areas of the 3 source CMMs.  These process areas are grouped into 4 categories: life cycle or engineering, management or project, supporting, and organizational process areas.  Table 1 lists the 23 process areas of the FAA-iCMM. Each process area description includes a purpose, goals, and from 2 to 10 fully elaborated base practices.

 

Life Cycle or Engineering Processes

Management or Project Processes

PA01 Needs

PA02 Requirements

PA03 Architecture

PA04 Alternatives

PA05 Outsourcing

PA06 Software Development/maintenance

PA07 Integration

PA08 System Test and Evaluation

PA09 Transition

PA10 Product Evolution

 

PA11 Project  Management

PA12 Contract  Management

PA13 Risk Management

PA14 Coordination

Supporting Processes

PA15 Quality Assurance & Management

PA16 Configuration Management

PA17 Peer Review

PA18 Measurement

 Organizational  Processes

PA19 Prevention

PA20 Organization Process Definition

PA21 Organization Process Improvement

PA22 Training

PA23 Innovation

 

               Table 1. The Integrated Process Areas of the FAA-iCMM

 

·   The Capability Aspect.  There are 5 capability levels in the FAA-iCMM, and generic practices at each level provide guidance for improving any process.  Generic practices are additive as process capability increases through the 5 levels.  Capability level 2 is summarized in Table 2.

 

Capability Level 2:  Repeatable: Planned and Tracked

Generic Practices:

  2.1 Establish policy

  2.2 Allocate adequate resources 

  2.3 Assign responsibility

  2.4 Ensure training

  2.5 Document the process

  2.6 Plan the process

  2.7 Use a repeatable process

2.8 Manage configurations

2.9 Assess process compliance

2.10 Verify work products

2.11 Measure process

2.12 Review status

2.13 Take corrective action

2.14 Coordinate within the project

 

Table 2.  Capability Level 2 - Summary

 

·   Maturity Levels.  Maturity levels in the FAA-iCMM are groupings of process areas and generic practices.  They “stage” the process areas to provide guidance for improving organizational maturity.  Maturity levels are conceptually the same as capability levels, i.e. the 5 levels, but they provide guidance on what processes together contribute to each step of organizational maturity. Maturity level 2 is summarized in Table 3.

 

Maturity Level 2 Process Areas.  

  Lifecycle/Engineering Processes

PA 01 Needs, PA 02 Requirements, PA 05 Outsourcing, PA 08 System Evaluation, PA 09 Transition

Management/Project Processes

PA 11 Project Management, PA 12 Contract Management

Supporting Processes

PA 15 Quality Assurance & Management, PA 16 Configuration Management

For an organization to have level 2 maturity, the above process areas should be at level 2 (or higher) capability according to an FAA-iCMM appraisal. This would indicate a “level 2” organizational maturity.

 

Table 3. Maturity Level 2 - Summary

·   Appraisal Method. FAA developed the FAM, including several variations.  The full internal appraisal is similar to the CMM-Based Appraisal for Internal Process Improvement method, except it has been adapted to a continuous model with both process area goals and capability level goals.  Other appraisal types include facilitated discussion/training-based, document-intensive, questionnaire-based, interview-intensive, and external.  These draw on and adapt from several existing appraisal methods. FAA’s concept is to integrate various appraisal methods, just as it integrated its source CMMs.  All FAM variations are tailorable and cover needs for initial, interim or full appraisal. 

 

Using the Model

FAA management adopted the FAA-iCMM by setting an aggressive improvement goal for FAA’s major software-intensive programs to achieve maturity level 2 by December 1999, and level 3 by December 2001. In the first 9 months of FAA-iCMM usage, over 900 managers and practitioners were trained in FAA-iCMM-related topics, and about 20 programs (including the targeted “major” programs, plus programs voluntarily signing up) are using the model to guide their process improvement.  FAA-iCMM process improvement workshops and appraisals are finding that the model raises and promotes resolution of process integration issues across the disciplines and across the acquisition lifecycle.  Working to improve the Requirements and the Transition process areas for example (both staged at maturity level 2) has required extensive cross-directorate, cross-discipline, cross-lifecycle participation.

 

A major appraisal is currently underway to determine interim status, to facilitate process improvement plan adjustment, and to promote even broader discussions and learning about process improvement.  Meanwhile, the FAA process improvement goal is being strengthened to include new programs as they are initiated.

 

Other models may be included in future versions of the FAA-iCMM, (such as models generated from the Government-Industry-SEI Capability Maturity Model Integration project or the Systems Security Engineering CMM) and other disciplines (including Human Factors) are being studied for inclusion now. Other government organizations, including Warner Robins Air Force Base and the Internal Revenue Service have received FAA-iCMM training and are looking towards adopting an integrated approach to process improvement.  Several companies, including Lockheed Martin, have also expressed interest.  The model is available in the public domain for organizations seeking to improve their acquisition processes.

Measurement

It is important to measure process improvement to ensure that the effort is on track with the process improvement plan.  At the FAA, this measurement is accomplished in a number of different ways, including: process appraisals (described above); project and executive measurements; and staff surveys.

 

At the project level the Practical Software Measurement (PSM) approach has been adopted. The FAA is a transition partner with PSM and as a result is authorized to provide training and to participate in future development of their products such as the expansion of their philosophy to include system and process measurements.  The PSM approach is aimed at the project level and starts with the identification and prioritization of project level issues.  This process helps to focus measurements on those parts of the project that will provide the greatest return on investment.  These issues are then categorized and measurements are identified and the process continues throughout the full life cycle, reviewed periodically so that the current issues are being addressed.  The FAA is keeping the focus not only on project management issues, but includes product, process, and their interrelationships.  It is important to determine if improving processes also improves the product and management.

 

The FAA is also focusing on executive management’s measurement needs.  Based on a periodic review of executive issues, a list of the following six measurements are reported monthly to executive management: Contractor Costs/Earned Value (within budget?); Schedule Milestones (on schedule?); Stability in Requirements (requirements stable?); Product Quality (defects too high? fixing problems timely?); SW Size Growth (size estimates on track?); Project Risk Performance Parameters (performance risks status?).  Each report presents 6 months of history, planned, and projected data.

 

The other aspect of process improvement measurement is staff views: the perceptions of those involved in the process either as managers, performers, or receivers of the products.  This data is obtained through periodic surveys eliciting staff impressions of the process improvement effort.

 

-> For more information,, please visit the SEPG website:  http://www.faa.gov/ait/sepg


sm Capability Maturity Model is a service mark of Carnegie Mellon University.  CMMâ is registered in the U.S. Patent and Trademark Office.