For many years, the de facto standard on relia- bility used in the United States was MIL-STD- 785. With the demise of that standard under MIL-SPEC Reform, much more attention has been given to commercial standards. Un- fortunately, what many reliability managers and practitioners have found is a confusing and endless list of standards on reliability. In many cases, standards not labeled "reliability" are, nonetheless, reliability standards. This article explores how national and international stan- dards are developed, the proliferation (real and perceived) of reliability standards brought about by defense acquisition reform, and ways for improving the development of reliability standards. Standardization in the United States Within the United States, standards are devel- oped by either the US Government or through a voluntary process within the private sector. Within the Government, the Department of Defense published most reliability standards. As already stated, these standards have either been canceled (e.g., MIL-STD-785) or been converted to handbooks. For all essential pur- poses, the development of reliability standards in the US is now done in the private sector. The American National Standards Institute (ANSI) has been the administrator and coordi- nator of the United States private sector volun- tary standardization system for 81 years. ANSI does not develop standards but facilitates devel- Introduction Telecommunications and computer & infor- mation systems are part of our daily routine for personal and business activities. Telephone, finance, Federal Reserve, energy, Federal elec- tions, stock exchanges, air traffic control, as well as critical governmental responsibilities for public safety and law enforcement all depend heavily on networked information systems which are potentially vulnerable to network- based attacks. The massive interconnection of computerized communications and informa- tion networks has provided infinite opportuni- ties to our adversaries, placing our nation at serious risk. Private industry manages risk by protecting their information systems assets in accordance with their perceptions of the commercial value of the asset, its vulnerability, and the threats to it. However, private industry has the alternative of writing off losses as a cost of doing business or purchasing some form of protection through insurance. This alternative is not possible at the national level. The situation will likely worsen. The factors that contribute to increased risk, in particular the huge growth in inter-networking and the tremendous expansion in data handling capaci- ties, show no signs of let-up. We continue to put massive amounts of information in electronic RAC is a DoD Information Analysis Center Sponsored by the Defense Technical Information Center and Operated by IIT Research Institute INSIDE T h e J o u r n a l o f t h e Standardization in Reliability Engineering Information Assurance into Information Systems 5 FAQ's About IEC Dependability Standards 6 Call for Papers 14 Industry News 15 Got Data? Let's Make a Deal 16 From the Editor 18 New from RAC 18 Panelists Sought to Advise 19 In Memoriam 20 Calendar 22 Help from RAC By Ned H. Criscimagna, Reliability Analysis Center By Deborah A. Cerino, Air Force Research Laboratory Reliability Analysis Center Fourth Quarter--1999 continued on page 2 continued on page 8 T h e J o u r n a l o f t h e R e l i a b i l i t y A n a l y s i s C e n t e r F o u r t h Q u a r t e r ­ 1 9 9 9 2 opment by establishing consensus among qualified groups. Currently, 175 distinct entities1 are accredited to develop stan- dards under one of the three ANSI federation's methods of accreditation: organization, committee, or canvass. · Organization Method. Associations and societies that have an interest in developing standards on specific topics most often use this method. Although participation on the consensus bodies is open to all interested parties, members of the consensus body are most often members of the association or society. SAE and IEEE are examples of societies that develop reliability standards. · Committee Method. Accredited standards committees are standing committees created for the purpose of developing con- sensus in support of a document. This method is most often used when a standard affects a broad range of diverse interests or where multiple associations or societies with similar interests exit. The Z1 Dependability Subcommittee is an example of an accredited standards committee (ASC) that develops reliability standards. A discussion of Z1 follows later in this paper. · Canvass Method. Small trade associations or societies that have documented current industry practices and wish to have these standards recognized nationally use this method. Most canvass developers are responsible for fewer than five standards. Underwriters Laboratories uses the canvass method for develop- ing standards. Regardless of the method used to develop standards, the ANSI's guiding principles of standardization must be followed. These principles are consensus, due process, and openness. ANSI expects its accredited standards developers to be commit- ted to supporting the development of national and, in many cases, international standards that address the critical trends of technological innovation, marketplace globalization, and regu- latory reform. As mentioned earlier, the Z1 ASC Dependability Subcommittee develops reliability standards. The Z1 ASC is part of the US Standards Group on Quality, Environment, Dependability (which includes reliability and maintainability), and Statistics (QEDS). The group consists of the members and leadership of organizations concerned with the development and effective use of generic and sector-specific standards on quality control, assurance and management; environmental management systems and auditing; dependability; and the application of statistical methods. Z1 is responsible for coordi- nating all US national efforts related to standards in QEDS and related technologies. Within each technical discipline, there is a Technical Advisory Group (see next section) focused on representing US interests in the development of international standards, and a corresponding ANSI ASC Z1 subcommittee focused on nation- al standards in that area as well as adoptions of international standards. All of the technical work on standards within the ANSI ASC Z1 is done within the four QEDS subcommittees. The US and International Reliability Standards ANSI promotes the use of US standards internationally, advo- cates US policy and technical positions in international (e.g., ISO, the international organization for standardization, and IEC, the International Electrotechnical Commission) and regional standards organizations. It encourages the adoption of international standards when they meet the needs of the user community in the US. The IEC is one of the two non-treaty international standards organizations for which ANSI is the sole US representative and dues-paying member. Headquartered in Geneva, Switzerland (as is ISO), the IEC has 49 member countries. Some 300 tech- nical committees develop standards on a variety of subjects. The IEC is the major international body that publishes internation- al standards on reliability.2 The technical committee (TC) with- in IEC that address reliability is TC56, the Dependability Technical Committee. Since reliability is not in its title, someone who is trying to find reliability standards, but is unfamiliar with international standards, might overlook TC56. TC56 develops standards in maintainability and availability, as well as reliabili- ty. Hence the name, dependability. For the same reason, the Z1's subcommittee responsible for reliability is called the Dependability Subcommittee. Each of the member countries in the IEC has one vote. Thus, the US has, through the US National Committee (USNC), only one vote on any particular reliability standard developed by TC56. The US National Committee develops its position large- ly on the advice of the US Technical Advisory Group (TAG) to TC56. The US TAG to TC56 is one of several accredited by ANSI to develop and transmit to the USNC, via ANSI, US positions on activities and ballots of the TC56. The US TAG to TC56 is administered by the American Society for Quality, which also administers the Z1 Accredited Standards Committee. Proliferation of Standards It may now be obvious why so many standards on reliability are to be found. Within the US alone, the following organizations develop standardization documents in reliability: · National Aeronautics and Space Administration (NASA) · Department of Defense (DoD) · Electronics Industries Association (EIA) · Institute of Electrical and Electronic Engineers (IEEE) · Society of Automotive Engineers (SAE) · Z1 Accredited Standards Committee When US standards on reliability are added to those devel- oped by the IEC, other national bodies (such as the British Standards Institute (BSI)), foreign government agencies (such as the British Ministry of Defence (MOD)), and multi-regional organizations (such as the North Atlantic Treaty Organization (NATO)), it is not surprising that we have many reliability stan- dards from which to choose. At one time, the DoD standards on reliability, such as MIL- STD-785, were considered by many to be the standards of choice. The standards did suffer from misuse, primarily lack of tailoring, and were often allowed to become badly out-of-date. However, they provided a common framework for the defense industry and much of commercial industry for developing reli- able systems and equipment. In 1994, the DoD acquisition community was given new direction by then Secretary of Defense William Perry. On June 29, 1994, he issued a five-page memorandum, "Specifications & Standards - A New Way of Doing Business." The intent of the memorandum can be summarized as three "overarching" objectives: · Establish a performance-oriented solicitation process · Implement a document improvement process · Create irreversible cultural change in the way DoD does business Even prior to the Perry memorandum, a Process Action Team (PAT), chartered by Colleen Preston, Deputy Under Secretary for Acquisition Reform, and chaired by Darold Griffin, was tasked to review the system of military standardization docu- ments and develop recommendations to (emphasis added): · Eliminate unnecessary and obsolete specifications and standards · Use performance specifications and standards · Use commercial standards and specifications to the greatest extent practicable · Encourage industry to propose alternative solutions to military specifications and standards · Reduce paperwork Since 1994, all of the DoD reliability standards have been can- celed or revised and converted to handbooks. As a result, the interest in developing commercial standards in reliability has dra- matically increased. On occasion, two or more of the SDOs are simultaneously developing a standard on the same subject! Such redundant efforts come at a time when companies are less likely to support their employees in participating in SDOs. So, in addi- tion to producing standards that are similar, if not identical, in content, these efforts strain the capabilities of many SDOs. Having a choice is usually good. However, it can be confus- ing, especially when it is difficult to compare the choices. Such is the case with reliability standards. Even when one is aware of the choices, it is often difficult and expensive to obtain all of the possible reliability standards on a particular aspect of relia- bility so that an intelligent comparison can be made. More importantly, one has to wonder if cooperative efforts among SDOs and international bodies wouldn't result in more useful and technically sound standards. Improving the Reliability Standardization Picture Through SDOs and the US TAG to TC56, US companies and individuals can work to improve the current state of reliability standards. One SDO, the Z1 Dependability Subcommittee, has decided not to focus on developing more reliabili- ty standards. Instead, this subcommittee is devel- oping a strategy to work together with other US SDOs to: · Develop joint US standards · Adopt international standards that meet US needs · Work with the US TAG to TC56 to lead the development of international standards The US TAG to TC56 is the best and most appropriate means of working in the arena of international reliability stan- dards. Participation to date has been limited to a handful of dedicated individuals who diligently work to ensure US inter- ests are reflected in international reliability standards. A wider base of members would improve the TAG's effectiveness and help the US change its traditional role from follower to leader in the development of international standards on reliability. In addition to the work being done by the Z1 and the US TAG to TC56, other organizations are working to improve the often confusing picture of reliability standards. One such organ- ization is the Reliability Analysis Center (RAC), operated by IIT Research Institute (IITRI). The RAC, based on work fund- ed by Rome Laboratory (now the Air Force Research Laboratory Information Directorate) publishes the "Worldwide Reliability & Maintainability Standards: A Primer of US and Non-US Commercial and Government Documents." Unlike other attempts that simply list a "snapshot" of avail- able standards, the Primer provides an abstract and table of con- tents for 88 national and international standards on reliability and maintainability (R&M). This information makes it much easier to find a standard on a particular topic or to compare two or more documents that ostensibly cover the same topic. In addition to the information on specific standards, the Primer includes information on standards issuing bodies, including contact information; information on changes to US military R&M documents; standards under development by national and international groups; and background information on stan- dards development. Another way in which the RAC works to improve reliability standards is to support the work done by national and interna- tional SDOs. Technical members of the RAC's staff participate and lead a number of activities related to standards development (see Table 1). Only through the support of organizations such as the RAC and companies like IITRI, can our private sector voluntary T h e J o u r n a l o f t h e R e l i a b i l i t y A n a l y s i s C e n t e r F o u r t h Q u a r t e r ­ 1 9 9 9 3 Table 1: RAC Support of Reliability Standardization Activities NAME ACTIVITY Ned H. Criscimagna · Chair, Z1 Dependability Subcommittee · Member, US TAG to TC56 · Member, Working Groups 6 and 8, TC56 · Chair of Operations, SAE G-11 Division · Member, American Society for Quality · Member, International Society of Logistics William Denson · Chair, SAE Reliability Standards Committee, Motor Vehicle Council, Electrical Systems Group David Nicholls · Member, IEST (currently participating in WG-PR781 to update MIL-HDBK-781A) T h e J o u r n a l o f t h e R e l i a b i l i t y A n a l y s i s C e n t e r F o u r t h Q u a r t e r ­ 1 9 9 9 4 standardization system be successful. In recent years, this support has been eroded by the innumerable downsizings and rightsizings within industry. Yet, this support is more important and can benefit industry more than ever before. So, the RAC encourages companies that do not now support standardization activities to do so vigorously. The essential element of this support is to encourage and support the par- ticipation of individual staff members in standards develop- ment activities.3 Closing Remarks The commercial marketplace is global, no longer bounded by national borders. Companies develop, produce, and mar- ket their products worldwide. Multi-national conglomerates blur the distinction between domestic-made and foreign- made products. In such a world, standards become more important to ensure that companies, regardless of where they produce or sell their products, can use common and proven procedures and methods. Reliability standardization is an area especially affected by defense acquisition reform. The effect has not been confined to the United States because many companies and govern- ment agencies worldwide used US military reliability stan- dards. Among the many changes wrought by defense acqui- sition reform has been a proliferation of reliability standards from many sources. These standards include those devel- oped by various SDOs in the United States and internation- al standards developed by the IEC. With the problem of proliferation, however, comes the opportunity to improve the standards used by reliability managers and engineers. By working together and pooling their collective knowledge and experience, and by bringing together different perspectives of reliability, US SDOs can develop standards that improve American industry's ability to develop and produce the most reliable products and serv- ices in the world. About the Author: Ned H. Criscimagna is a Senior Engineer with IIT Research Institute (IITRI). At IITRI, he has been involved in projects related to defense acquisition reform. These have included a project for the Department of Defense in which he led an effort to benchmark commercial reliability practices. He led the development of a handbook on maintainability to replace MIL-HDBK-470 and MIL-HDBK-471, and the update to MIL-HDBK-338, Electronic Reliability Design Handbook. Before joining IITRI, he spent 7 years with ARINC Research Corporation and, prior to that, 20 years in the United States Air Force. He has over 32 years experience in project management, acquisition, logistics, reliability and maintainability (R&M), and availability. Mr. Criscimagna holds a Bachelor's degree in Mechanical Engineering from the University of Nebraska-Lincoln, a Master's degree in Systems Engineering from the Air Force Institute of Technology, and did post-graduate work in Systems Engineering and Human Factors at the University of Southern California. He completed the US Air Force Squadron Officer School in residence, the US Air Force Air Command and Staff College by seminar, and the Industrial College of the Armed Forces correspondence program in National Security Management. He is also a graduate of the Air Force Instructors Course and completed the ISO 9000 Assessor/Lead Assessor Training Course. Mr. Criscimagna is a member of the American Society for Quality (ASQ) and a Senior Member of the Society of Logistics Engineers. He is a certified Professional Logistician, chairs the ASQ/ANSI Z- 1 Dependability Subcommittee, is a member of the US TAG to IEC TC56, and Secretary for the G-11 Division of the Society of Automotive Engineers. Footnotes: 1 Organizations that develop standards are referred to collectively as standards development organizations (SDO)s. 2 Professional groups, such as IEEE, maintain that their stan- dards are international, because their membership and organization are international. The author uses the term "international standards" to mean standards that ANSI considers to be international. This use limits the term to standards developed by IEC, ISO, and regional bodies. 3 Readers interested in any of the activities shown in Table 1 may contact the respective RAC staff member by sending an E-mail. The address would be the person's initial of the first name, then the last name, followed by "@iitri.org." For example, the E-mail address for Ned Criscimagna is . The appearance of advertising in this publication does not constitute endorsement by the Department of Defense or RAC of the products or services advertised.