Army-Materiel-System-Analysis-Activity Maturity Projection Model Based on a Subsystem Stein Estimator

LI ZhongshengFAN Jinwei(范晉偉)PAN Ri

Beijing Key Laboratory of Advanced Manufacturing Technology，Beijing University of Technology，Beijing 100124，China

Abstract：A reliability-growth test is often used to assess complex systems under development. Reliability-growth models are usually used to quantify the achievable reliability indices and predict the expected reliability values. The Crow army-materiel-system-analysis-activity (Crow-AMSAA) projection model and the AMSAA maturity projection (AMPM)-Stein model are suitable for modelling delayed corrective strategies. The AMPM-Stein model，which involves more failure data and requires limited assumptions，is more robust than the Crow-AMSAA projection model. However，the rationality of the Stein factor introduced in the AMPM-Stein model has always been controversial. An AMPM-Stein extended projection model，derived from data regrouping based on similar failure mechanisms，is presented to alleviate the problem. The study demonstrated that the proposed model performed well，the prediction results were credible，and the robustness of the proposed model was examined. Furthermore，the Stein-shrinkage factors，which are derived from components with similar inherent failure mechanisms，are easier to understand and accept in the field of engineering. An example shows that the proposed model is more suitable and accurate than the Crow-AMSAA model and the AMPM-Stein model，by comparing the projection values based on the failure data of the previous phases with the actual values of the current phases. This study provides a technical basis for extensive applications of the proposed model.

Key words:reliability projection；army-materiel-system-analysis-activity maturity projection model (AMPM)；Stein estimator；subsystem

Introduction

The requirement of high reliability for complex systems is obvious，and it is usually reflected in a reliability-growth process.This includes finding failure modes，researching the modes，and incorporating fixes to the occurring modes.The definition of reliability-growth is the process of improving reliability indicators by changing design，manufacturing methods，and testing processes.The ratio of reliability improvement is based on the on-going ratio at which new failure modes occur and the number and effectiveness of the fixes that are incorporated into the system.Thus，the way in which the reliability changes are depicted throughout the growth process is important，because it indicates whether the requirements of the system can be met.

Reliability-growth models are usually used to quantify the achievable reliability indices after the initial test stage and predict the expected later reliability-growth values，based on assumed fix-effectiveness factors (FEFs) of corrective actions.Some scholars have already done significant research on reliability-growth models in recent decades.

Weiss[1]attempted to determine the level of reliability，whether the reliability is increasing and，if so，how rapidly，and the desired reliability after the development testing of complex systems.Weiss developed methods to fit special reliability-growth curves for failure data to monitor and extrapolate system-reliability growth.Furthermore，mathematical models for an idealized system development program were developed to present a base for selecting a suitable growth curve.

Duane[2]，from the General Electric Company，observed and measured the accumulative failure rate versus accumulative running hours.This study showed that they approached a direct line when plotted on double logarithmic-coordinate paper.It was concluded that several different and complex systems had apparently similar ratios of reliability improvement under development.As a result，a learning-curve method was presented to control development programs，forecast growth routes，and plan schemes for reliability improvement.

Crow[3]extended this research to a non-homogeneous Poisson process based on Duane’s hypothesis，with respect to failures that were repaired instantaneously in a repairable system.Then，the army-materiel-system-analysis-activity (AMSAA) model for reliability-growth analyses，also named the tracking model，was presented，irrespective of whether all failures received corrective actions.In 1982，Crow[4]assumed that，considering the effectiveness factor for type-B modes，fixing the occurred failures during the development testing would be implemented as deferred fixes after each test phase.An improved projection model was proposed to project the impact of these fixes.This is called projection model.

To assess the management strategy of the corrective measures in and after the testing，Crow[5]presented an extended model in which the type-B modes were divided into two different types：type-BC modes and type-BD modes，by combining the tracking model and the projection model.To continuously evaluate and manage system-reliability growth，which include multiple test phases，Crow[6]presented a model that was focused not only on design issues，but also on human factor issues in 2010.This model was named extended continuous evaluation model.

On account of the Crow projection model，Ellner and Wald[7]derived a rate function for new failure in type-B mode occurrence and presented the AMSAA maturity projection model (AMPM)，which utilized this rate function whenever corrective actions were performed.Then，Ellner and Hall[8]observed that type-A failure modes might be switched to type-B modes for a variety of reasons.Thus，a projection model that could regard different failure modes as a uniform style in a situation was developed，where all corrective actions were deferred after the test phase.This was called the AMPM-Stein model.

As stated above，various reliability-growth models based on different testing strategies have already been provided.The testing strategies are normally driven by the cost and the test duration；however，they are decided by the practical management actions for improving failure modes that have surfaced.A widely recognized method of determining which mode should be fixed and when it is difficult to finalize.Experience shows that it is sufficient to replace，or repair the components when they have failed during a development test and then continue，the implementation of only corrective actions after the test.Design changes or element reselection is reasonable and acceptable.This can be called a test-find-test strategy[9].The deferred corrective actions are normally merged into a group，and the reliability indices of the system generally show an obvious jump.In this sense，both the AMPM-Stein model and Crow-AMSAA projection model are suitable for modelling these situations.The AMPM-Stein model，which involves more failure data and requires limited assumptions，is statistically more robust against the effects of FEFs variability than the Crow-AMSAA model[10].Moreover，the Stein-based approach presents more-exact reliability-assessment results than the latter，given that minimizing the prospective squared error between the estimated and the original failure rates[11].

Since Stein’s pioneering work[12]，the Stein estimator has received increasing attention in the academic and engineering fields[13-14].However，the rationality of the Stein factor introduced in the model has always been controversial[15]，the focus of controversy is the rationality of the algorithm of Stein factor.The algorithm does not consider the difference of the failure mechanisms of different type-B failures when solving the Stein factor，so when the failure rates of different type-B failures are significantly different，the calculated shrinkage factor will also have a large deviation.Many scholars have done some effective research based on AMSAA model in recent years[16-19].However the issue in the use of the AMPM model have not been mentioned.

To alleviate this problem，an algorithm of Stein factor based on similar inherent failure mechanisms and maintenance strategies is proposed.Then an extended AMPM-Stein model，derived from partitioned subsystem failure data，is presented.The main objective of the model proposed is to assess the system-reliability growth continuously in multiple test phases，through dividing the failure data into several sections in which the failure mechanisms are similar.Then，the robustness of the proposed model is examined.Finally，the performances of the three models discussed in this study are compared.

1 Reliability Projection Techniques

Reliability projection is a program of evaluating the system reliability indices that can be expected because of the implementation of corrective actions to the observed failure modes.Reliability prediction technology relies on test information and effectiveness assessments of the implemented corrective actions.Other existing projection approaches make a distinction between the failure modes that will be solved if they surface (type-B modes) and those that will not be solved (type-A modes).This mode classification affects the projected results directly.Before discussing the capability of the current and proposed projection models，their backgrounds and assumptions will be provided below.

1.1 Crow-AMSAA projection model

The Crow projection model assumes that individual failure numbers in a test occur in terms of a non-homogeneous Poisson process[20-21].The system failure intensity is expressed by

ρ(t)=abtb-1，

(1)

The expected fault-intensity function regarding type-B failure modes is given by

(2)

The Crow projection model aims to assess the system reliability before the subsequent test stage by considering the reliability improvement of delayed fixes and the portion of unsurfaced type-B failure modes.However，if the failure modes are reclassified，the projection accuracy degrades.

1.2 AMPM-Stein projection model

The AMPM-Stein projection model assigns a value of zero or a positive value to FEFs for surfaced modes，rather than classifying failures into type-A or type-B modes like the Crow projection model.The basis of this model is that the incidences of the following independent failure modes are subject to a gamma random distribution[8].

(3)

To compare it intuitively with the Crow projection model，the expanding model of Eq.(3)，which covers the type-A and type-B modes，is obtained as

(4)

(5)

(6)

In this model，only one average failure rate of all type-B failures and one average failure rate of all type-B failure modes are solved，and then a Stein-shrinkage factor is obtained without considering the difference of failure rates between type-B failures.

1.3 AMPM-Stein extended projection model

There are two differences between the AMPM-Stein projection model and the Crow-AMASS projection model.First，all the data，both for first and repeat occurrence failures，are used in the model to develop the model projection.Second，to improve the projection accuracy，the shrinkage estimation builds a direct connection to Bayesian modeling using squared error loss functions[22].

Here，the same assumptions are made as in the AMPM-Stein model.The continuous systems have massive potential fault modes with an original rate of occurrenceλ1，λ2，…，λm0.The failure modes are corrected at the end of the test if they surface during the test，irrespective of which mode results in individual failure and the system fails whenever a mode failure happens，and new failure modes are not created by corrective actions.

Moreover，it is supposed that there arej(j=1，2，…，q) subsystems of the complex system，in terms of the similar failure mechanisms and corrective strategies.Hence，the Stein factors based on the divided subsystems are easier to understand and accept in the field of engineering.As a result，the Stein projection of the system failure rate developed by divided subsystems is obtained.

(7)

(8)

Similarly，

(9)

Meanwhile，

(10)

The first term in Eq.(7) is the constant failure intensity resulting from type-A modes.The second term is the prospective failure intensity caused by type-B modes that have surfaced，and finished corrective actions after the testing，according to the distinguished subsystems.The last term is the unconditional prospective failure intensity caused by type-B modes that have not surfaced by timeT.Comparing with Eq.(4)，all occurred type-B modes are divided intojdifferent groups in the second term.Considering the difference of failure rates between type-B failures，totallyjdifferent Stein-shrinkage factors need to be solved in the model，that is，three arejfailure rates of type-B failures andjfailure rates of type-B failure models need to be solved.

2 Illustrative Examples

The data used in the study consist of type-A and type-B modes.Type-A modes in the Crow model can be considered as type-B modes in the Stein model by considering the FEFs to be zero.The number of occurrences and time to failure for each failure mode which surfaces are from ReliaSoft in 1999.

Table 1 Failure data

As shown in Table 1，the experiment lasted 400 h.The total number of failures was 42，and all corrective actions were implemented after the 400 h test.There were 10 type-A failures and 32 type-B failures，which were distinguished into 16 unique modes during the test.The FEFdiis the portion decrease inλi，because a corrective action has been performed for theith type-B mode.Thus，the final failure rate of theith type-B failure mode after corrective actions is(1-di)λi.

As shown in Table 2，the 16 unique type-B modes are randomly divided into three sections by MATLAB，which can be deemed as the 16 type-B modes derived from three subsystems.

Table 2 Type-B modes divided into three subsystems

(Table 2 continued)

Table 3 shows the computed results of Stein-shrinkage factors and failure intensities for three projection models：Crow，AMPM-Stein，and AMPM-Stein with three subsystems.

Table 3 System failure intensity results of three subsystems

The results show that the three models perform well under special test conditions that follow the hypothetical conditions of the models.The failure intensity bias is very small between the AMPM-Stein model and the AMPM-Stein model with three subsystems，even when the Stein-shrinkage factors are different.

To examine the robustness of the proposed projection model，that is，how the numbers of subsystemsjand Stein-shrinkage factorθS，jvariation impact the failure intensity，two new random data groups were studied additionally.One group was divided as shown in Table 4.

Table 4 Type-B modes divided into four subsystems

As shown in Table 4，the 16 unique type-B modes are randomly divided by MATLAB into four groups，which can be considered as the 16 type-B modes from four different subsystems.

The other group is divided into five groups，as shown in Table 5.

Table 5 Type-B modes divided into five subsystems

(Table 5 continued)

Table 6 System failure intensity comparison

The AMPM-Stein model，which involves more failure data and requires limited assumptions，is statistically more robust against the effects of fix effectiveness variability than the Crow-AMSAA projection model[10].Thus，the AMPM-Stein model is elected as the baseline for performance comparison.

The present study indicates that the performances of the three forms of the proposed models are acceptable.With respect to the AMPM-Stein model with three subsystems，the deviation rate is just 0.349% when compared with the baseline.As a result，for the AMPM-Stein model with four subsystems，the deviation rate is 0.871%.Finally，for the AMPM-Stein model with five subsystems，the deviation rate is 1.050%.The results show that the deviation rate is also increasing with the increase of the number of subsystems，but the prediction results of three data reorganization methods are acceptable.This proves the robustness of the proposed AMPM-Stein extended model.

3 Case Study

A development test with three testing phases was conducted on a CNC grinder system in sequence in a factory.All corrective actions of the type-B modes that occurred were incorporated after the first test phase，before starting a follow-up test phase.This means that the first test phase was considered to be phase I，and the follow-up test phase was considered to be phase II，then the subsequent test phase was considered as phase III.

The failure data are shown in Tables 7-9.The CNC grinder system was divided into five subsystems based on different failure mechanisms and management strategies：spindle system，head-stock/feed system，electronic control system，base/body/protection system，and hydraulic/pneumatic/lubrication system.Consequently，the failures that occurred were classified into these five subsystems.

Table 7 Failure data of CNC grinder in phase I

The modes that surfaced were still distinguished into type-A and type-B modes to accommodate the Crow projection model.Type-B failure modes shall increase the reliability of the CNC grinder，as they were fixed after the testing.In contrast，type-A failure modes shall not increase the reliability of the CNC grinder，as they were not be solved because of limitations.The AMPM-Stein models do not require distinction，but assigne a positive or zero FEF individually.

As shown in Table 7，the experiment of phase I continued for 200 h.The total number of failures was 18，and all corrective actions were implemented after the testing.Three type-A failures and 15 type-B failures were found in the experiment.As shown in Table 7,13 different type-B modes were identified which required 13 different corrective actions to be implemented after testing.

Table 8 Failure data of CNC grinder in phase II

As shown in Table 8，the experiment in phase II continued for 400 h.The total number of failures was 23，and all corrective actions were implemented after the testing.Four type-A failures and 19 type-B failures were found in the experiment.As shown in Table 8,10 different type-B modes were identified which required 10 different corrective actions to be implemented after testing.

Table 9 Failure data of CNC grinder in phase III

(Table 9 continued)

As shown in Table 9，the experiment of phase III continued for 800 h.The total number of failures was 25，and all corrective actions were implemented after the testing.Three type-A failures and 22 type-B failures were found in the experiment.As show in Table 9,11 different type-B modes were identified which required 11 different corrective actions to be implemented after testing.

The calculation results of the three phases are shown in Table 10.

Table 10 Calculation results of three phase data

The intent of the AMPM-Stein extended model is to present an assessment for system-reliability because of the implementation of deferred fixes.The model also directly evaluates the achievable system failure intensity，which is alleviated by the failure mode.As shown in Table 10，the system-failure intensities show an obvious downward trend between phases.With respect to the AMPM-Stein extended model，the system failure intensity was 0.063 729 after implementing all the corrective actions associated with phase I according to Table 7.In phase II，the system failure intensity decreased to 0.034 200 after implementing all the corrective actions according to Table 8.Finally，the system failure intensity reduced，again to 0.018 353，after implementing all the corrective actions associated with phase III according to Table 9.It meaned that the reliability of this complex system was significantly increased through the development test.

Fig.1 Projection performance of phase I

Fig.2 Projection performance of phase II

Fig.3 Projection performance of phase III

By comparing the predicted failure rates with the actual value of phase II and，similarly，the predicted failure rates with the actual value of phase III，the prediction results of the proposed AMPM-Stein extended model are found to be more accurate than those of the Crow-AMSAA projection model and AMPM-Stein model.For all three models，the actual intensity rates at a certain stage are less than the predicted intensity rate at the same stage.This is probably because conservative FEFs are used instead of realized FEFs；for instance，in Table 7，the FEF of the eighth type-B mode (B8) is only 0.59.

Additionally，the Stein-shrinkage factors of the proposed model，which derive from five partitioned subsystems，have similar failure mechanisms.This makes the proposed model easier to understand and more acceptable to engineers than the AMPM-Stein model.

4 Conclusions

To alleviate the problem of Stein factor’s algorithm and to effectively evaluate the reliability of the under-development system with a delayed corrective strategy，an AMPM-Stein extended model was proposed based on failure data reconstruction.Then，the Stein-shrinkage factors were calculated based on the divided subsystem data.Finally，the system failure intensities were derived based on the proposed model.Based on the study，the following conclusions can be drawn.

(1) The Stein-shrinkage factors of subsystems derived from similar inherent failure mechanisms and management strategies are easier to be understood and accepted in the engineering field.This provides a technical basis for the extensive application of the proposed model.

(2) The presented model is more reliable than the Crow-AMSAA projection model，because more data are involved.Furthermore，the model is more plausible and accurate than the AMPM-Stein model for complex systems，as the Stein factors are solved separately.

(3) The research indicates that the proposed projection model has optimal performance，regardless of the complex system being randomly divided into three，four，or five subsystems.Furthermore，the performance is still good，even if the Stein-shrinkage factor of partitioned subsystem equals zero.This fully proves the robustness of the proposed model.

All of the above indicate that the proposed projection model using failure data regrouping according to the similar failure mechanisms seems more reasonable and understandable for the complex systems.The model can provide an assessment for estimated reliability because of the implementation of deferred fixes.It can also directly evaluate the achievable system failure intensity，which is alleviated by the failure mode.

This model does not consider the difference between the actual FEFs and the theoretical FEFs，which may have a non-negligible influence on the reliability-prediction results.In addition，the goodness-of-fit procedure and confidence interval evaluation of the model are not mentioned.Furthermore，how to control the deviation rate of prediction results if there are too many subsystems are divided.All these need to be studied further.