A Guide to the Systems Engineering Body of Knowledge (SEBoK)
IV. SE Process Capability Assessment
Whilst the previous sections of the SEBoK have been oriented to the definition of the processes and practices of systems engineering, this section deals with assessing and improving the capability of implementation of Systems Engineering within working organizations. First is the focus is on what establishes an organization’s capability to accomplish effective and efficient systems engineering. Second is a discussion of the models used for assessing systems engineering capability. Third is a discussion of what an organization can do to improve its capability.
The capability of an organization to do systems engineering is a resultant of three capability vectors: 1) the processes used by the organization; 2) the defined context in which the processes are applied; and 3) the workforce that applies the processes within the defined context.
Processes: The processes are necessary in that they define what has to be done to convert customer requirements into system products that satisfy the customer and the other interested parties. These processes are selected as applicable to the organization’s business. By themselves processes are not sufficient, however. Processes are applied within a given organizational context to include the concepts of systems engineering adopted by the organization and the common methods and tools adopted to perform process activities and tasks.
Context: A uniform understanding of basic systems engineering concepts facilitate enhanced organization-wide communications though use of a set of common terminology. One important concept that needs to be defined is how the organization looks at the system to be engineered. One system definition has a focus on the products that will perform the operational functions. This system concept is called the system-of-interest in ISO/IEC 15288. Another system definition defines the system to consist of not only the products that perform the operational functions but also the products that enable the operational products to perform within the various stages of the system life cycle. This system concept is defined in ANSI/EIA 632 calls the products that perform operational functions end products and those that perform life cycle service functions as enabling products. ISO/IEC 15288 includes consideration of the life cycle service functions and calls the products as enabling systems. ISO/IEC 15288 has only a loose connection between systems-of-interest and enabling systems whereas ANSI/EIA 632 has a strong system project link between end products and enabling products. The organization’s concept of the system will influence the scope of a projects concern with respect to engineering a solution to a customer’s problem that provides life cycle satisfaction to the customer and also considers the interest of other parties involved in that life cycle.
This leads to the second important concept that needs to be defined by an organization. This concept is how the organization looks at the role of systems engineering within the life cycle model used by the organization to guide a system through its service life. Such a model has early exploratory phases such a pre-concept definition and/or concept definition to study technologies and feasible concepts and later an execution phase in which the system is designed; pre-production models created and proofed; products produced, utilized and supported; and then retired. Typically organizations have gates within a life cycle model that are used to determine whether a system is continued to the next service life phase, terminated or retired, or redirected (through appropriate improvements). To provide management with the information to make decisions at each gate the system has to be appropriately engineered to create the needed information to meet decision gate criteria.
Thus systems engineering context capability includes not only engineering the right system but also creating the information required to make appropriate life cycle management decisions with respect to the system.
[A similar discussion of methods and tools would be needed]
Workforce: Two attributes of the workforce help determine the systems engineering capability of an organization-Knowledge and skills. The workforce requires the domain knowledge essential to the system being engineered and also knowledge of the processes to be used for systems engineering, understanding of the concepts adopted by the organization, and knowledge of project management. The individuals within the workforce also need the special skills required to perform systems engineering processes; use the common methods and tools applicable to their assigned work; and work in an integrated, multi-disciplinary team environment.
Major models dealing with capability of Systems Engineering and organizations are:
· SE-CMM (Software Engineering Institute)
· SECAM (International Council on Systems Engineering)
· EIA-731 (Electronic Industries Alliance)
· CMMI (National Defense Industries Association)
· ISO/IEC 15504 (International Organization of Standards)
Capability Assessment deals with the determination of the capability of a particular organization to do Systems Engineering. The Assessment process transforms a set of requirements into a quotation, which aims to indicate the level of Capability of an organization. Since the Assessment provides a clear snapshot of the activities of an organization, it is a valuable tool to understand organizational behavior and correct it. As these models are essential measuring tools when evaluating the capability of a given organization, it usually does not fit the needs of procurement agencies or services, since the result of a few past projects is typically difficult to extrapolate to new ones.
Unlike audits where an auditor seeks what is missing and tracks what is incorrect, the assessment is a constructive approach that aims to understand why the organization is at a given level and how it can improve its Capability. Assessments are mostly based on cooperative interviews and work product shared examination. Starting with very open questions, the assessment team gathers from the answers and from the discussions that the engineers may start together, the evidences that activities are performed. The different existing processes to assess organizations can be tailored to get a proper process that will be used to create the organization’s assessment process.
Although most of the processes are closely related to a particular model, some defined assessment processes may be used with different models.
· CBI-IPI
· EIA 731-2
· ISO 15504 – Part 3 & 4
The sponsor of the assessment is given a profile of the assessed organization, showing its major strengths and weaknesses. From this material, the organization can start thinking about addressing the flaws that prevent it from reaching some of their objectives. Some models embed an improvement path that can be directly used to determine what actions are to be performed to gain a better capability. Those models are said to be “Staged”, since they define a number of classes of organizations, that the improving organization must cross one after another.
Some models do not provide this improvement path, which then must be built by the user of the model, based on the different requirements expressed by the sponsor. These models are called “continuous”, since there is a path to follow step by step. As those two approaches are complementary and often used together, some models have been designed to fit both. Those models provide two views of the same information: one as a profile for each elementary process of the organization, and one as a measurement on a defined improvement path. The architecture of the improvement path, regardless of the model to be used, usually follows simple rules: technical matters must be done first, then management and support at a project level, then management and support at an institution level.
The greatest challenge in Capability improvement is usually not in the techniques of capability evaluations and improvement definition. The most problematic achievement is to conduct organizational changes where all the people are modifying their uses. This goal implies often widely spread training efforts, acquisition of new tools, introduction of new procedures, collection of a series of indicators, etc. Success in such a project is expensive and runs, by definition, over several years. The price for improvement is high, but the returns on investment have been found, across several enterprises and institutions, highly attractive. Many resulting organizations have been able to drastically reduce their inefficient use of money and their sensitivity to major risks, while becoming more cost effective.
The major pitfall to overcome is the strong feeling that improvement is a non-regressive act. The Capability of an organization can easily sink if no attention is paid to sustain the effort and the willingness for improvement.