This is just an Excerpt from a larger document, click here to view the entire document.
SECTION TWO GENERAL CONSIDERATIONS FOR ASSESSING RELIABILITY PROGRESS

This section addresses assessment issues that should be considered if a continuous reliability program that updates the status of the design is needed.

2.1 The Goals of Reliability Assessment

Reliability is traditionally considered to be a performance attribute that is concerned with the probability of success and frequency of failures, and is defined as:

       The probability that an item will perform its intended function understated conditions, for either a specified interval or over its useful life.

Reliability assessments are performed to assess design progress towards meeting customer needs. In addition, assessments of product design alternatives, options and changes can be performed to evaluate their impact on customer needs, schedule and costs. The assessment process should be considered an iterative one to review reliability progress throughout the product design and development phases. Each assessment should be thought of as one step in the design decision process.

Section Three of this Blueprint describes each assessment task, indicates the proper time for implementing the task and may include an example to aid in understanding its application. Table 1 identifies those tasks (historically classified as design, analysis and test) that have been proven to effectively assess the product design throughout the entire design cycle. Continuous evaluation of product reliability will add value for the customer by reducing the number of design problems and component defects.

Table 1. Reliability Tasks for Assessing Reliability Progress
Type of Activity Tasks and Description Section
D
E
S
I
G
N		 Critical Item Control. Monitoring in-house and suppliers'activities to reduce the risk to product reliability from items identified as critical. Can include hardware and software. 3.2
Design Reviews. Formal or informal independent evaluation and critique of a design to identify and correct hardware or software deficiencies. 3.3
Supplier Control. Monitoring suppliers' activities to assure that purchased hardware and software will have adequate reliability. 3.4
A
N
A
L
Y
S
I
S		 Design of Experiments (DOE). Systematically determining the impact of process and environmental factors on a desired product parameter, in order to reduce product variability by controlling the factors. 3.5
Dormancy Analysis. Determination of the effects of expected periods of storage or other non-operating conditions on the reliability of the product. 3.6
Durability Analysis. Determination of whether or not the mechanical strength of a product will remain adequate for its expected life. 3.7
Failure Modes, Effects&Criticality Analysis (FMECA). Systematically determining the effects of part or software failures on the product's ability to perform its function. This task includes FMEA. 3.8
Failure Reporting Analysis & Corrective Action System (FRACAS). A closed- loop system of data collection, analysis and dissemination to identify and correct failures of a product or process. 3.9
Fault Tree Analysis (FTA). Using inductive logic to determine the possible causes of a defined undesired operational result. 3.10
Finite Element Analysis (FEA). Determining the mechanical stresses present in products through simulation by decomposing the product into simple elements. 3.11
Life Cycle Planning. Determining reliability (and other) requirements by considering the impact over the expected useful life of the product. 3.1
Parts Obsolescence. Analysis of the likelihood that changes in technology will make the use of a currently available part undesirable. 3.12
Predictions. Estimation of reliability from available design, analysis or test data, or data from similar products. 3.13
Sneak Circuit Analysis (SCA). Investigation to discover the existence of unintended signal paths in a product. 3.14
Thermal Analysis. Analysis of the heat dissipations, transfer paths and cooling sources to determine if part/product temperatures are consistent with reliability needs. 3.15
Worst Case Circuit Analysis (WCCA). Analysis of the effects of variability in the components of a product on the product's performance. 3.16
T
E
S
T		 Accelerated Life Testing. Testing at high stress levels over compressed time periods to draw conclusions about the reliability of a product under expected operating conditions, based on formulated correlation factors. 3.18
Reliability Growth Test (RGT)/Test Analyze and Fix (TAAF). Testing a product to identify reliability deficiencies in order to eliminate their causes. 3.19
Test Strategy. Determination of the most cost effective mix of tests for a product. 3.17

2.2 Product Program Phases

Each product, from the simplest to the most complex, passes through a sequence of phases during its life cycle. The definitions of the phases vary among commercial companies, and within the military. Table 2 describes the sequence of general phases that will be used in this document to describe a product's life.

Table 2. Product Life Cycle Phases
Concept/ Planning Design/ Development Production/ Manufacturing Operation/ Repair Wearout/ Disposal
  • Formulate ideas, estimate resources and financial needs
  • Identify risks & requirements
  • Program objective
  • Identify and allocate needs and requirements
  • Propose alternate approaches
  • Design and test the product
  • Develop manufacturing, operating, and repair/ maintenance tasks
  • Refine and implement manufacturing procedures
  • Finalize production equipment
  • Establish quality processes
  • Build&distribute the product
  • Implement operating, installation and training procedures
  • Provide repair and maintenance service
  • Repair warranty items
  • Provide for performance feedback
  • Implement refurbish- ment and disposal tasks
  • Resolve potential wearout issues


What sometimes distinguishes one phase from the next is a decision milestone, sometimes referred to as a "gate." It represents a point in time where the program can go forward or stop. For many products, the phases may be abbreviated or combined. For example, the Concept/Planning and Design/Development phases may be combined under a compressed schedule for a new product that is simply an update or slightly modified version of an older, proven product. Reliability tasks for this type of program would concentrate only on the differences between the old and the modified product. As a result, the number of engineering tasks would be reduced. It is important to understand that tasks performed in one phase are often the result of the analysis, trade- offs and planning performed in an earlier phase. For example, trade-offs addressing approaches to manufacturing printed circuit boards would be performed during Design/Development, with the implementation of the process decision to follow during the Production/Manufacturing phase.

2.3 Task Selection Guide

The performance of any of the reliability tasks described in this Blueprint requires a financial and schedule commitment by the product manufacturer. Therefore, selection of the tasks should be on a value-added basis. Figure 2 shows some of the failure causes that a product might experience and, for each cause, appropriate reliability analysis techniques are indicated. For example, if a product is expected to be used by a variety of operators and may be subjected to possible operator error, tasks such as fault tree or sneak analysis should be considered to find and eliminate potential problems. Using this figure, a manufacturer could establish an appropriate list of reliability assessment tasks that will potentially enhance their product. Figure 3 was adapted from an article in the ITEA Journal of Test and Evaluation to show which reliability tasks result in the most design changes. As can be seen, thermal analysis is by far the most effective task and should be considered if the operational environment is more severe than a typical office environment.

Figure 2. Product Failure Causes and Assessment Techniques
Figure 2. Product Failure Causes and Assessment Techniques (Click to Zoom)


Figure 3. Design Changes As A Result Of Analysis Type*
Figure 3. Design Changes As A Result Of Analysis Type* (Click to Zoom)


2.4 Tailoring Instructions

For most products, the customer's reliability needs are satisfied through sound design practices, proper application of parts and components, and good manufacturing processes. However, for complex products that involve many vendors and designers, interim assessment of the progress may be needed as indicated in Table 3. This table lists a number of techniques that are useful in assessing reliability progress and includes guidance for their use. Most of these techniques provide valuable means of understanding a product's design strengths and weaknesses so that appropriate changes can be implemented.

Table 3. Application Guidance for Assessing Reliability Progress
Tasks Application Guidance
Accelerated Life Testing Effective on parts, components or assemblies to identify failure mechanisms and life limiting critical components.
Critical Item Control
Apply when safety margins, process procedures and new technology present risk to the production of the product.
Design of Experiments (DOE) Use when process physical properties are known and parameter interactions are understood. Usually done in early design phases, it can assess the progress made in improving product or process reliability.
Design Reviews Continuing evaluation process to ensure details are not overlooked. Should include hardware and software.
Dormancy Analysis Use for products that have "extended" periods of non-operating time, unusual non-operating environmental conditions, or high cycle on-and-off periods.
Durability Analysis Use to determine cycles to failure or determine wearout characteristics. Especially important for mechanical products.
Failure Modes, Effects and Criticality Analysis (FMECA) Applicable to equipment performing critical functions (e.g., control systems) when the need to know consequences of lower level failures is important.
Failure Reporting Analysis and Corrective Action System (FRACAS) Use when iterative tests or demonstrations are conducted on breadboard, or prototype products to identify mechanisms and trends for corrective action.
Fault Tree Analysis (FTA) Use for complex systems evaluation of safety and system reliability. Apply when the need to know what caused a hypothesized catastrophic event is important.
Finite Element Analysis (FEA) Use for designs that are unproven with little prior experience/test data, that use advanced/unique packaging/design concepts, or will encounter severe environmental loads.
Life Cycle Planning Use to strategize value-added mix of reliability analysis/test assessment techniques.
Parts Obsolescence Use to determine need and risk of application of parts and lifetime buys.
Predictions Use as a general means to develop goals, choose design approaches, select components, and evaluate stresses.
Reliability Growth Test (RGT)/Test Analyze and Fix (TAAF) Use when technology or risk of failure is critical to the success of the product. These tests are costly in comparison to alternative analytical assessment techniques.
Sneak Circuit Analysis (SCA) Apply to operating and safety critical functions. Important for space systems and others of extreme complexity. May be costly to apply.
Supplier Control Apply when high volume or new technologies for parts, materials or components are expected.
Test Strategy Use when critical technologies result in high risk of failure.
Thermal Analysis Use for products with high power dissipation, or thermally sensitive aspects of design. Typical for modern electronics, particularly densely packaged products.
Worst Case Circuit Analysis (WCCA) Use when the need exists to determine critical component parameter variation and environmental effects on circuit performance.

The assessment methods chosen should be appropriate to the product under development and the operating environment expected. For example, a thermal analysis may not be needed for a product operated in an air conditioned office, but should be considered for a product operated in an outside unprotected environment. The methods chosen should represent a reasonable level of investment when compared to the value of the results. For nondevelopmental items, only methods that confirm suitability of the product to the intended environment and application should be considered. Table 4 contains a list of recommended tasks as a function of several product classifications as a starting point. Tasks can be added or deleted depending on the consequence of failure of the product and the customers' expectations.

Table 4. Assessment Tasks Tailored by Product Classification
Assessment Tasks Consumer Industrial Military
Product Durable Equip- ment System Structure Equip- ment System Strategic
Accelerated Test     X     X    
Critical Items             X X
Design of Experiment         X     X
Design Review       X     X  
Dormancy           X    
Durability         X     X
Failure Modes   X X X   X X X
Failure Reporting   X X X   X X X
Fault Tree Analysis       X X   X X
Finite Element Analysis         X     X
Life Cycle Planning         X   X X
Part Obsolescence               X
Predictions X X X X   X X X
Reliability Growth Test           X    
Sneak Circuit       X     X  
Supplier Control       X   X X X
Test Strategy       X     X  
Thermal Analysis   X X X   X X X
Worst Case Analysis               X


START 99-2 Add to Read Later list 
Performance-Based Requirements (PBRs)
Journal Article V10, N4 Add to Read Later list 
Program Managers Handbook - Common Practices to Mitigate the Risk of Obsolescence
Journal Article V12, N3 Add to Read Later list 
An Introduction to Task Analysis
Journal Article V14, N1 Add to Read Later list 
Information Management for Systems Design for RMQSI