Author’s Accepted Manuscript

Ensemble of Data-Driven Prognostic Algorithms for Robust Prediction of Remaining Useful Life Chao Hu, Byeng D. Youn, Pingfeng Wang, Joung Taek Yoon PII: DOI: Reference:

S0951-8320(12)00042-7 doi:10.1016/j.ress.2012.03.008 RESS4607

To appear in:

Reliability Engineering and System Safety

Received date: Revised date: Accepted date:

29 March 2011 13 February 2012 8 March 2012

www.elsevier.com/locate/ress

Cite this article as: Chao Hu, Byeng D. Youn, Pingfeng Wang and Joung Taek Yoon, Ensemble of Data-Driven Prognostic Algorithms for Robust Prediction of Remaining Useful Life, Reliability Engineering and System Safety, doi:10.1016/j.ress.2012.03.008 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Ensemble of Data-Driven Prognostic Algorithms for Robust Prediction of Remaining Useful Life Chao Hu 1, Byeng D. Youn 2, Pingfeng Wang 3, and Joung Taek Yoon 4 1

Department of Mechanical Engineering, the University of Maryland at College Park, College Park, MD 20742, USA (Currently at Medtronic Inc.)

2,4

SchoolofMechanicalandAerospaceEngineering,theSeoulNationalUniversity, Seoul151742,Korea

3

DepartmentofIndustrialandManufacturingEngineering,theWichitaStateUniversity, Wichita,KS67260,USA

Abstract — Prognostics aims at determining whether a failure of an engineered system (e.g., a nuclear power plant) is impending and estimating the remaining useful life (RUL) before the failure occurs. The traditional data-driven prognostic approach is to construct multiple candidate algorithms using a training data set, evaluate their respective performance using a testing data set, and select the one with the best performance while discarding all the others. This approach has three shortcomings: (i) the selected standalone algorithm may not be robust; (ii) it wastes the resources for constructing the algorithms that are discarded; (iii) it requires the testing data in addition to the training data. To overcome these drawbacks, this paper proposes an ensemble data-driven prognostic approach which combines multiple member algorithms with a weighted-sum formulation. Three weighting schemes, namely the accuracy-based weighting, diversity-based weighting and optimization-based weighting, are proposed to determine the weights of member algorithms. The k-fold cross validation (CV) is employed to estimate the prediction error required by the weighting schemes. The results obtained from three case studies suggest that the ensemble approach with any weighting scheme gives more accurate RUL predictions compared to any sole algorithm when member

 1 Ph.D.,[email protected]. 2

AssistantProfessor,[email protected],CorrespondingAuthor.

3

AssistantProfessor,[email protected].

4GraduateStudent,[email protected].

1

algorithms producing diverse RUL predictions have comparable prediction accuracy and that the optimization-based weighting scheme gives the best overall performance among the three weighting schemes.

Keywords:Ensemble;Kfoldcrossvalidation;Weightingschemes;Datadrivenprognostics;RUL prediction.

I. Introduction To support critical decision-making processes such as maintenance replacement and system design, activities of health monitoring and life prediction are of great importance to high-risk engineered systems composed of multiple components, complex joints, and various materials, such as aerospace systems, nuclear power plants, chemical plants, advanced military systems and so on. Stressful conditions (e.g., high pressure, high temperature and high irradiation field) imposed on these systems are the direct causes of damage in their integrity and functionality, which necessitates the continuous monitoring of these systems due to the health and safety implications [1,2,3]. Research on real-time diagnosis and prognosis which interprets data acquired by distributed sensor networks, and utilizes these data streams in making critical decisions provides significant advancements across a wide range of applications. Maintenance and life-cycle management of these high-risk engineered systems for minimizing the cost [4,5,6], maximizing the availability [7] and extending the service life [8] is one of the beneficiary application areas because of the pervasive nature of monitoring and maintenance activities throughout the manufacturing and service sectors and, especially, the extremely high failure costs. For instance, in nuclear power plants, unexpected breakdowns can be prohibitively expensive and disastrous since they immediately result in lost power production, correct maintenance cost, reduced public confidence, and, possibly, human injuries and deaths. In order to reduce and possibly eliminate such problems, it is necessary to accurately assess current system health condition and precisely predict the remaining useful lives (RULs) of operating components, subsystems, and systems in the highrisk engineered systems.

Ingeneral,prognosticapproachescanbecategorizedintomodelbasedapproaches[9,10,11,12,13], datadrivenapproaches[14,15,16,17,18]andhybridapproaches[19,20,21].Theapplicationofgeneral modelbasedprognosticapproachesreliesontheunderstandingofsystemphysicsoffailureand underlyingsystemdegradationmodels.Myötyrietal.[9]proposedtheuseofastochasticfiltering 2

techniqueforrealtimeRULpredictioninthecaseoffatiguecrackgrowthwhileconsideringthe uncertaintiesinbothdegradationprocessesandconditionmonitoringmeasures.Asimilarparticle filteringapproachwaslaterappliedtoconditionbasedcomponentreplacementinthecontextof fatiguecrackgrowth[10].Luoetal[11]developedamodelbasedprognostictechniquethatreliesonan accuratesimulationmodelforsystemdegradationpredictionandappliedthistechniquetoavehicle suspensionsystem.GebraeelpresentedadegradationmodelingframeworkforRULpredictionsof rollingelementbearingsundertimevaryingoperationalconditions[12]orintheabsenceofprior degradationinformation[13].Ashighriskengineeredsystemsgenerallyconsistofmultiplecomponents withmultiplefailuremodes,understandingallpotentialphysicsoffailuresandtheirinteractionsfora complexsystemisalmostimpossible.Withtheadvanceofmodernsensorsystemsaswellasdata storageandprocessingtechnologies,thedatadrivenapproachesforsystemhealthprognostics,which aremainlybasedonthemassivesensorydatawithlessrequirementofknowinginherentsystemfailure mechanisms,havebeenwidelyusedandbecomepopular.Agoodreviewofdatadrivenprognostic approacheswasgivenin[14].Datadrivenprognosticapproachesgenerallyrequirethesensorydata fusionandfeatureextraction,statisticalpatternrecognition,andforthelifeprediction,the interpolation[15,16],extrapolation[17],ormachinelearning[18]andsoon.Hybridapproachesattempt totakeadvantageofthestrengthfromdatadrivenapproachesaswellasmodelbasedapproachesby fusingtheinformationfrombothapproaches.Gargaetal.[19]describedadatafusionapproachwhere domainknowledgeandpredictorperformanceareusedtodetermineweightsfordifferentstateof chargepredictors.Goebeletal.[20]employedaDempsterShaferregressiontofuseaphysicsbased modelandanexperiencebasedmodelforprognostics.Sahaetal.[21]combinedtheofflinerelevance vectormachine(RVM)withtheonlineparticlefilterforbatteryprognostics.Similartomodelbased approaches,theapplicationofhybridapproachesislimitedtothecaseswheresufficientknowledgeon systemphysicsoffailuresisavailable.

3

ImplicitrelationshipbetweentheRULandthesensorysignalsmakesitdifficulttoknowwhich prognosticalgorithmperformsbestinaspecificapplication.Furthermore,therearemanyfactorsthat affectthepredictionaccuracyandrobustness,suchas(i)dependencyofthealgorithm’saccuracyonthe numberofunitsinatrainingdataset,(ii)significantvariabilityinmanufacturingconditionsandlarge uncertaintiesinenvironmentalandoperationalconditions,(iii)theamountofeffectivesensorysignals forRULpredictions,and(iv)theformofdegradationtrend(e.g.,linear,nonlinear,noisy,smooth). Therefore,nosingleprognosticalgorithmworkswellforallpossiblesituations.Insteadofusingan individualprognosticalgorithm,itwouldbebeneficialtocombinemultiplealgorithmstoformahybrid algorithm.

Combiningapproximatealgorithmsintoanensemble,i.e.ensemblemethods,wasmotivatedto improvetherobustnessandaccuracyofalgorithminthemachinelearningcommunity.Themethodscan beclassifiedbythecombiningstrategy;byconsensusorbylearning.Examplesofnotedensemble methodsandbriefdescriptionsarearrangedbelowinTable1[22].





Table1 Examplesofnotedensemblemethods

Combinin Ensemble gstrategy method

By consensus

By learning

Description

Reference

Bagging

Baggingdeterminesaclasslabelwithmajor votingbymultipleclassifiers.

BreimanL. (1996)[23]

Random Forest

Randomforestimprovestheperformanceof baggingbycombiningwithrandomfeature selectionscheme.

BreimanL. (2001)[24]

Boosting

Boostingtrainsweakclassifiersandcombines themintoastrongclassifier.

Schapire R.E.(1990)

4

[25] Adaboosttrainseachbaseclassifierwitha weighteddatasetofwhichweightingcoefficients Adaboost arecomputedfromclassificationerrorsbythe previousclassifiers,andthenaggregatesthebase classifiersintoone.

FreundY.& Schapire R.E.(1997) [26]

Notonlyuseabasisfunctionasabaseclassifier,a ruleensembleincludesaruleasabaseclassifier. Rule Astherulehasasimpleform,itiseasyto Ensemble understandtheinfluencesofrulesonpredictions andthedegreeofdependencyoneachother.

Friedman J.H.& Popescu B.C.(2008) [27]



Theensemblemethodfoundsitsapplicationsinawidevarietyofresearchfields,suchasthe developmentofcommitteesofneuralnetworks[28,29],themetamodelingforthedesignofmodern engineeredsystems[30,31,32],thediscoveryofregulatorymotifsinbioinformatics[33],thedetection oftrafficincidents[34],thetransientidentificationofnuclearpowerplant[35],andthedevelopmentof ensembleKalmanfilters[36].However,theutilizationoftheensembleapproachforthedatadriven prognosticsisstillininfancy.Theonlyrelevantworkweareawareofcomesfrom[37]whereonlytwo datadrivenalgorithmsareemployedasmemberalgorithmsandtheirweightsareempirically determinedbasedonaonetimetrainingerrorwithoutasystematicschemeforerrorestimationand performancevalidation.Mostdatadrivenprognosticpracticesselectasinglealgorithmwiththebest accuracyfromthealgorithmpoolwhilediscardingtheothers.Thisapproachnotonlywastesthe resourcedevotedtodevelopingdifferentalgorithms,butalsosuffersfromthelackofrobustness.

Estimatingtheaccuracyofaprognosticalgorithmisimportantnotonlyforevaluatingitsprediction accuracybutalsoforchoosingthebestalgorithmfromagivenset(algorithmselection),orcombining algorithms.Manydatadrivenapproaches[14,16]usethesocalledholdoutmethod,whichdividesthe originalruntofailuredatasetintotwomutuallyexclusivesubsetscalledatrainingsetandatestingset,

5

orholdoutset.Theholdoutmethodisstraightforwardandcomputationallyefficient.However,itoften producesalargevarianceoftheresultingestimateandrequiresthetestingdatasetwhichincreasesthe overallexpenses.

Toovercometheaboveshortcomings,thisstudyproposesanensembleapproachthatemploysthe kfoldcrossvalidation(CV)toestimatetheaccuracyofagivenensembleandproposesthreeweighting schemestodeterminetheweightvalues.Assumptionsforthisstudyarelistedbelow: (1) Sensory data from multiple run-to-failure units are available, either from the computer simulation or field testing. (2) A single failure mode is considered, i.e., the RUL prediction is exclusively for this failure mode. (3) The underlying physics of the system fault propagation is not comprehensive or it is too expensive to derive a reliable physical damage model for a complex engineered system. Both cases entail the use of the data-driven prognostics.

Therestofthepaperisorganizedasfollows.SectionIIgivesabriefintroductiontothedatadriven prognosticalgorithmsselectedinthisstudy.SectionIIIpresentstheproposedensembleapproachwith thekfoldCVandthreeweightingschemes.Applicationsoftheproposedmethodologyarepresentedin SectionIVandtheconclusionofthisworkisgiveninSectionV.

II. Description of Prognostic Algorithms II.1. A general description of member algorithms

Thissectionprovidesabriefoverviewofthefiveselecteddatadrivenprognosticalgorithms:Method 1asimilaritybasedinterpolation(SBI)approachwiththerelevancevectormachine(RVM)asthe regressiontechnique(RVMSBI)[15,49],Method2SBIwiththesupportvectormachine(SVM)asthe regressiontechnique(SVMSBI)[15,47],Method3SBIwiththeleastsquareexponentialfitting(Exp SBI)[15],Method4aBayesianlinearregressionwiththeleastsquarequadraticfitting(QuadBLR)[17], andMethod5arecurrentneuralnetwork(RNN)approach[18,50].Adataprocessingschemewitha

6

generichealthindexsystemisusedforthefirstfouralgorithmswhileadataprocessingschemewitha simplenormalizationschemeforthelastalgorithm. II.2. Methods 1-3: Similarity-based interpolation approaches II.2.1.

Data processing with a generic health index system

Successfulimplementationsofprognosticalgorithmsrequiretheextractionofthehealthcondition signaturesandbackgroundhealthknowledgefrommassivetraining/testingsensorysignalsfrom engineeredsystemunits.Todoso,thisstudywilluseagenerichealthindexsystemthatiscomposedof twodistinguishedhealthindices:physicshealthindex(PHI)andvirtualhealthindex(VHI).Ingeneral,the PHIusesadominantphysicalsignalasadirecthealthmetricandisthusapplicableonlyifsensorysignals aredirectlyrelatedtophysicsoffailures.Intheliterature,mostengineeringpracticesofhealth prognosticsarebasedonvariousPHIs,suchasthebatteryimpedance[21],themagnitudeofthe vibrationsignal[45]andtheradiofrequency(RF)impedance[46].Incontrast,thevirtualhealthindex (VHI)isapplicableevenifsensorysignalsarenotdirectlyrelatedtosystemphysicsoffailures.Inthis study,theVHIsystemwillbeemployedwhichtransformsthemultidimensionalsensorysignalstoone dimensionalVHIwithalineardatatransformationmethod[15].TheVHIsystemwillbedetailedinwhat follows.

Supposetherearetwomultidimensionalsensorydatasetsthatrepresentthesystemfailedand healthystates,Q0ofM0uDmatrixandQ1ofM1uDmatrix,respectively,whereM0andM1arethedata sizesforsystemfailedandhealthystates,respectively,andDisthedimensionofeachdataset.With thesetwodatamatrices,atransformationmatrixTcanbeobtainedtotransformthemultidimensional sensorysignalintotheonedimensionalVHIas



T

Q Q T

7

1

QT S off 

(1)

whereQ=[Q0;Q1],Soff=[S0,S1]T,S0isa1uM0zerovectorandS1isa1uM1unityvector.This transformationmatrixTcantransformanysensorysignalfromtheofflinelearningoronlineprediction processtothenormalizedVHIasH=QoffTorH=QonT,whereQoffandQonaretheofflineandonline multidimensionalsensorydatasets,respectively,and,ifweassumethedatasizesforQoffandQonare respectivelyMonandMoff(i.e.,QoffofMoffuDmatrixandQonofMonuDmatrix),Hwillbeacolumnvector ofthesizeMofforMon.TheVHIcanalsobedenotedash(ti)fori=1,…,Moff(fortheofflinecase)orfori= 1,…,Mon(fortheonlinecase),varyingapproximatelybetween0and1.ThisVHIcanbeusedtoconstruct backgroundhealthknowledge(e.g.,predictivehealthdegradationcurves)intheofflinetrainingprocess andtofurtherconducttheonlinepredictionprocess.

 II.2.2.

Training with RVM, SVM, or exponential fitting

Threeregressiontechniques,namely,relevancevectormachine(RVM),supportvectormachine (SVM),andleastsquareexponentialfitting,areemployedtoconstructthebackgroundhealth knowledgeusingtheVHIofofflinesystemunitsobtainedfromSectionII.2.1.Theseregression techniquescangeneratedifferentsetsofpredictivehealthdegradationcurvesforofflinesystemunits. II.2.2.1. Relevance vector machine

Supposewehaveatrainingdataset{ti,hi},i=1,…,Ms,sampledfromascalarvaluedfunctionwith additivezeromeanGaussiannoisewithavariance2.TheRVMisaspecialcaseofasparselinear modeltoapproximatethisdatasetas



h t

Ms

¦Z I  t, t  Z i

i

0



(2)

i 1

where=(0,...,Ms)TisakernelweightvectorandI(t,ti)isakernelfunctioncenteredatthetraining pointti.Assumingtheindependenceof{hi},i=1,…,Ms,wehavethelikelihoodoftheobserveddataas

8

p  h | , V 2

 2SV

2 Ms /2

1 2· § exp ¨  h   ¸ 2 © 2V ¹

(3)

whereh=(h1,...,hMs)T,isanMs×(Ms+1)designmatrixconstructedthetrainingpointstwithij=I(ti, tj). ToevaluatetheunknownparametersinEq.(3)fromaBayesianperspective,asparseweightprior distributioncanbeassigned,insuchawaythatadifferentvarianceparameterisassignedforeach weight,as

p  | 



Ms

– N Z | 0, D  1 i

i

(4)

i 0

where=(0,…,Ms)isavectorconsistingofMs+1hyperparameters,whicharetreatedasindependent randomvariables.TospecifythishierarchicalBayesianinferencemodel,priordistributionsforandthe noisevariance2mustbedefined.Forscaleparametersand2,itiscommontouseGammaprior distributionsas

p 



p E

Ms

– Gamma D i 0

i

| a, b



(5)

Gamma  E | c , d

where=–2,Gamma(|a,b)=(a)–1baa–1e–bwith(a)beingthegammafunction,a,b,canddare thehyperparametersandsettosmallvaluestoformaflatGammaprior.

Withthepriordefined,theposteriordistributionovertheweightsisgivenbytheBayesianinference as



p   | h,  , V 2

 2S 

 M s 1 / 2



1/ 2

T § 1 · exp ¨       1     ¸  © 2 ¹

wheretheposteriormeanvectoroftheweightsis=–2h,andthecovariancematrixis=(–

9

(6)

2

T+A)–1withA=diag(0,…,Ms).Appropriateiterativeoptimizationmethods[49],suchasmarginal

likelihoodoptimization,expectationmaximization(EM)algorithmsorincrementaloptimization algorithms,canbeemployedtofindthehyperparameterposteriormodesormostprobablevaluesm andm2thatmaximizep(,2|h) v p(h|,2)p()p(2).MoredetaileddescriptionsoftheRVMcanbe foundin[49]. II.2.2.2. Support vector machine

SimilartotheRVM,theSVMisalsoaspecialcaseofasparselinearmodelthatapproximatesthe dataset{ti,hi},i=1,…,Ms,as



h t

Ms

¦Z I  t, t  Z i

i

0



(7)

i 1

wherethelinearornonlinearkernelfunctionI(t,ti)arecenteredatthetrainingpointti.Theoptimum regressionfunctioncanbeobtainedbysolvingthefollowingoptimizationproblem

Minimize

Ms 1 2 w  C ¦ [i  [i 2 i 1 Ms



Subject to hi  ¦ ZiI  ti , t j  Z0 d H  [i j 1

Ms

¦Z I t , t  Z i

i

j

0



(8)

 hi d H  [i

j 1  i

[ , [i t 0, i 1,..., M s

wheretheregularizationparameterCspecifiesthetradeoffbetweentheflatnessandtolerance; i–and i+areslackvariablesdefiningtheupperandlowerconstraintsonthepredictionswithaninsensitive lossfunction.WiththeKarushKuhnTucker(KKT)conditions,theoptimizationproblemcanbe reformulatedas

10

Maximize 



Ms Ms 1 Ms Ms

, D  D D  D I t t  D  D ˜ H      Di  Di yi ¦¦ i i j j i j ¦ ¦ i i 2i1 j1 i 1 i 1

Ms

Subject to

¦ Di  Di 0, 0 d Di ,Di d C, i 1,..., M s



(9)

i 1

Theweightsandbiastermcanbecomputedas



wi

Di  Di , i 1,..., M s

w0



 1 Ms Di  Di ª¬I  xi , xr  I  xi , xs º¼  ¦ 2i1

(10)

Then,theregressionfunctioncanbeexpressedas

h t



Ms

¦ D i 1

i

 Di I  t , ti  Z0 

(11)

DetailedinformationregardingtheuseofSVMforregressioncanbefoundin[47].Thisstudyusedthe MATLAB®programdevelopedbyRakotomamonjy[48].



 II.2.2.3. Exponential fitting

ComparedtotheRVMandSVM,anexponentialfittingismuchsimplerandeasiertoimplement.In thisstudy,anexponentialfunctionwasconstructedas



h t

b1 ª¬ exp  b2 t  1º¼ 

(12)

wheretheoptimumcoefficientsb1andb2canbeobtainedthatminimizetheleastsquareerrorofthe exponentialfitting.



11

II.2.3.

RUL prediction using SBI

TheRULpredictionprocessinvolvestwoprocedures:(i)determinationofaninitialhealthcondition and(ii)theRULpredictionusingthesimilaritybasedinterpolation(SBI).Thisprocesscloselyfollowsthe similaritybasedapproachin[15]. II.2.3.1. Determination of initial health condition

Differentonlinetestingunitsoftenexhibitdifferentinitialhealthindicesduetodifferentinitialhealth conditions.Thus,accurateestimationsofinitialhealthconditionsforonlinetestingunitsareofgreat importancetopreciseRULpredictions.Basedonthepredictivehealthdegradationcurve(hp)froman offlinesystemunit,anoptimumfittingisconductedtodetermineatimescaleinitialhealthcondition (T0)thatminimizesthesumofsquareddifferencesSSDbetweentheonlineandofflinehealthindex data.Theoptimumfittingcanbeformulatedas Ms



Minimize SSD

¦  h t  h t r

j

j 1

p

j

 T0



2



(13)

subject to T0  [0, TS  't ]

wherehr(tj)andhp(tj)aretheonlineandpredictivehealthindicesattj,respectively,Msisthelengthof theonlinehealthindexdata;T0isthetimescaleinitialhealthcondition;'tisthetimespan(=tMst1)of theonlinehealthindexdata;TSisthetimespanofapredictivehealthdegradationcurve,i.e.,thelife spanofanofflinesystemunit.Itisnotedthatthepredictivehealthdegradationcurvehpbuiltinthe offlinetrainingprocess(seeSectionII.2.2)isessentiallytheregressionmodelinEq.(2),(7)or(12).This optimizationprocessbasicallymovestheonlinehealthindexdatahralongthetimeaxisofapredictive healthdegradationcurvehptofindtheoptimumtimescaleinitialhealthstate(T0)thatbestmatcheshr withhpwithrespecttotheSSD.AssumingthatthedatasizeoftheofflinesystemunitisMoff,itfollows that,amongMoffofflinedatapointsontheofflinehealthdegradationcurve,weonlyselectMs consecutivedatapointsoverlappingwiththeonlinedataalongthetimeaxistocomputetheSSD.Once

12

T0isdetermined,theprojectedremaininglifeofanonlinesystemunitbasedonagivenpredictive healthdegradationcurvecanbecalculatedas

LP TS  't  T0 



(14)

RepeatingtheoptimumfittingprocessonKpredictivehealthdegradationcurvesfromKdifferentoffline systemunitsgivesKRULestimates(LiPfori=1,…,K).  II.2.3.2. Similarity-based interpolation

Inthesimilaritybasedinterpolation(SBI),thepredictiveRULisalinearinterpolationfunctionin termsofdifferentprojectedRULs(Lifori=1,…,K)ofanonlineunitas



L

1 W

K

¦ W ˜ L i

i

K

where W

i 1

¦W

i



(15)

i 1

whereLiistheprojectedRULontheithpredictivehealthdegradationcurve(forsimplicity,weuseLi insteadofLiPhere),Wiistheithsimilarityweight.AsimilarityweightWicanbedefinedastheinverseof thecorrespondingSSDi,i.e.,Wi=(SSDi)–1.Thisdefinitionensuresthatagreatersimilaritygivesagreater weight.

 II.3. Method 4: Extrapolation-based approach

Unlike the similarity-based interpolation (SBI) approach, the extrapolation-based approach employs the training data set not for the comparison with the testing data set but rather for obtaining prior distributions of the degradation model parameters. The testing data set is then used to update these prior distributions. An RUL estimate can be obtained by extrapolating the updated degradation model to a predefined failure threshold. For the construction and updating of the degradation model, this study employed the Bayesian linear regression method used in [17].

13

II.3.1.

Data processing with a generic health index system

ThissectionfollowsexactlythedescriptionsinSectionII.2.1andwillnotbedetailedhere.

 II.3.2.

Training with quadratic fitting

In this study, the least-square quadratic fitting was employed as the degradation model for both the training and testing units. Aquadraticfunctionwasconstructedas h t



b1t 2  b2 t  b3 

(16)

wheretheoptimumcoefficientsb1,b2andb3canbeobtainedthatminimizetheleastsquareerrorof thequadraticfitting.Theoptimummodelparametersforadataset{ti,hi},i=1,…Ms,areestimatedas



b



T

 1  T  1h  1

(17)

wherethecoefficientvectorb=[b1,b2,b3]T,thedesignmatrixisofthesizeMs×3withij=tij,the diagonalcovariancematrixisofthesizeMs×Mswiththediagonalelementsij=2andtheoffdiagonal elementsbeingzero,theVHIvectorh=[h1,…,hMs]T.Forthetrainingprocesswithoutpriorinformation, thevalueof2doesnotaffecttheleastsquarefittingresultsandwesimplyset2=1.

 II.3.3.

RUL prediction with Bayesian linear regression

TheBayesianlinearregressionemploystheBayesianupdatingtechniquetoupdatetheprior distributionsofthedegradationparametersbandtheRULpredictionisthenaccomplishedby extrapolatingthemodelshowninEq.(16)withtheposteriordistributionsofb.TheBayesianlinear regressionmethodforRULpredictionisbrieflyexplainedhereandamorecompletediscussionabout thismethodcanbefoundinreferences[13,41].Thepriorinformationofbcanbeobtainedfromthe

14

trainingdata,i.e.thepriordistributionofbjisrepresentedbyanormaldistribution,N(Pj,Vj2),withthe mean jandthestandarddeviationj,j=1,2,3.Thevalueofj2canbeestimatedfromthetraining datasetastherootmeansquareerrorbetweenpredictedandtruedegradationcurves.Inorderto includethispriorinformationintotheregressionestimationofmodelparametersbasshowninEq.(17), thedesignmatrix,theVHIvectorh,andthediagonalcovariancematrixneedtobechanged accordingly.Astherearethreeparametersintotal,b1,b2andb3,thedesignmatrixconstructedwitha testingdatasetisappendedwiththreeadditionalrowswith(Ms+j)j=1,j=1,2,3,andallotherelements beingzero.AccordinglytheVHIvectorhisappendedwiththreeadditionalrowswithhMs+j= j,j=1,2,3, andthecovariancematrixisaugmentedwiththreeadditionalrowsandcolumnswith(Ms+j)(Ms+j)=j2,j =1,2,3,andallotherelementsbeingzero.Withtheupdateddesignmatrix,theVHIvectorh,andthe diagonalcovariancematrix,theposteriorestimateofthedegradationparametersbicanbeobtained withEq.(17),andtheupdatedquadraticdegradationmodelshowninEq.(16)canbeextrapolatedto thedegradationthresholdhc=0toobtaintheRULas:

RUL



t  t0 :

^t  >t , f 0

| b1t 2  b2 t  b3

0

`

(18)

where t0 is the time that the system has been operating until the RUL prediction is made. II.4. Method 5: Recurrent neural network approach

II.4.1.

Data processing with a normalization scheme

Asastandarddataprocessingtechnique,thenormalizationprovidesacommonscaleamongallthe dimensionsofadataset.SupposethatamultidimensionalsensorydatasetQintheformofamatrixof thesizeMsuDcomesfromthesameoperationcondition,whereMsisthedatasizeandDisthe dimension.ThenormalizeddatasetQNofthedatasetQcanbeexpressedas



QijN

Qij  P j

Vj

15



(19)

whereQijNisthenormalizedvalueofQij, jandjarethemeanandstandarddeviationofthejth dimensionofthedatasetQ,respectively.

 II.4.2.

Recurrent neural network

TheRNNiscapableoflearningthenonlineardynamictemporalbehaviorduetotheuseofan internalstateandfeedback.AfirstordersimpleRNNisanexampleofmultilayerperceptron(MLP)with feedbackconnections(seeFigure1).Thenetworkiscomposedoffourlayers,namely,theinputlayerI, recurrentlayerR,contextlayerCandoutputlayerO.Unitsoftheinputlayerandtherecurrentlayerare fullyconnectedthroughtheweightsWRIwhileunitsoftherecurrentlayerandoutputlayerarefully connectedthroughtheweightsWOR.ThroughtherecurrentweightsWRC,thetimedelayconnectionslink currentrecurrentunitsR(t)withthecontextunitsC(t)holdingrecurrentunitsR(t–1)intheprevioustime step.LetI(t)=(I1(t),…,Ij(t),…,I|I|(t)),R(t)=(R1(t),…,Rj(t),…,R|R|(t))andO(t)=(O1(t),…,Oj(t),…,O|O|(t))betheinput patterns,recurrentactivitiesandoutputactivitiesatthetimestept,respectively,where|I|,|R|and |O|denotethenumbersoftheinput,recurrentandoutputunits,respectively.Thenetinputoftheith recurrentunitcanbecomputedas



R it

¦W

RI ij

j

I j t  ¦ WijRC R jt 1 

(20)

j

Giventhelogisticsigmoidfunctionastheactivationfunctionf,theoutputactivityoftheithrecurrent unitcanthenbecomputedas



Ri t



f R i t





1

ª1  exp  R i t º  ¬ ¼

Thenetinputandoutputactivityoftheithoutputunitcanbecomputed,respectively,as

16

(21)

t O i



¦W

OR ij

t R j 

(22)

j

and Oi t





f O i t





1

ª1  exp O i t º  ¬ ¼

(23)

Inthisstudy,theinputstotheRNNarethenormalizedsensorydatasetQNandtheoutputsarethe RULsassociatedwiththedataset.WeusedtheMATLAB®programdevelopedbyCernansky[50].Inthe RNNtrainingprocess,thebackpropagationthroughtimeandextendedKalmanfilterwereusedto calculatethegradientsofnetworkweightsandtoupdatenetworkweights,respectively.



Output layer O

O1

…

Ok

…

O|O|

Time delay

Time delay

WOR

WOR R1

Recurrent layer R WRC

…

Rj

…

R|R|

WRC

WRI

WRI C1

…

Cj

…

I1

C|C|

…

Ii

…

I|I|

Input layer I

Context layer R



 (b)

(a) Figure1.

Simplified(a)andmoredetailedrepresentation(b)ofElman’ssimpleRNN[50]



III. Ensemble of Prognostic Algorithms ItisessentialtoproposearobustprognosticsolutionthataccuratelypredictstheRULusingdata featuresextractedfrommultidimensionalsensorydegradationsignals.Forbuildingsuchaunified structuralhealthprognosticframework,thispaperproposes(i)aweightedsumformulationforan ensembleofprognosticalgorithms,(ii)kfoldcrossvalidation(CV)toevaluatetheerrormetric

17

associatedwithacandidateensemblemodel;and(iii)threeweightingschemestodeterminetheweight valuesforthememberalgorithms.Thissectionisorganizedasfollows.SectionIII.1presentsthebasic weightedsumformulationfortheRULprediction.SectionIII.2describesthebackgroundofthekfoldCV andhowitcanbeappliedforestimatingtheaccuracyofaprognosticalgorithm.SectionIII.3describes thethreeproposedweightingschemes.Theoverallprocedureoftheensembleapproachisdescribedin SectionIII.4.

III.1.Weighted-sum formulation

AsimpleaverageofRULpredictionsobtainedusingthememberalgorithmsmeansassigningequal weightstothememberalgorithmsusedforprognostics.Thisisacceptableonlywhenthemember algorithmsprovidethesamelevelofaccuracyforagivenproblem.However,itismorelikelythatan algorithmtendstobemoreaccuratethanothers.Itisidealtoassignagreaterweighttoamember algorithmwithhigherpredictionaccuracyinordertoenhanceitspredictionaccuracyandrobustness. Hence,memberalgorithmswithdifferentpredictionperformanceshouldbemultipliedbydifferent weightfactors.

LetY={y1,y2,…,yN}beadatasetconsistingofmultidimensionalsensorysignals(e.g.,acceleration, strain,pressure)fromNdifferentruntofailureunits.Anensembleofprognosticmemberalgorithmsfor RULpredictioncanbeexpressedinaweightedsumformulationas

Lˆ

M

¦ w Lˆ  y , Y j

j

t

(24)

j 1

where Lˆ denotes the ensemble predicted RUL for the testing data set yt; M denotes the number of algorithm members in the ensemble; wj denotes the weight assigned to the jth prognostic algorithm; Lˆ j (yt, Y) denotes the predicted RUL by the jth prognostic member algorithm trained with the data set Y. Let the weight vector w = [w1,…,

18

ˆ [Lˆ ,..., Lˆ ]T , the weighted-sum formulation in Eq. wM]T and the vector of predicted RULs by member algorithms L M 1





ˆ (24) can be expressed in a vector form as Lˆ w, L

ˆ. wTL

III.2.K-fold cross validation

The k-fold cross validation is used in the offline process to evaluate the accuracy of a given ensemble. It randomly divides the original data set Y into k mutually exclusive subsets (or folds) Y1, Y2,…, Yk having an approximately equal size [38]. Of the k subsets, one is used as the test set and the other k1 subsets are put together as a training set. The CV process is performed k times, with each of the k subsets used exactly once as the test set. Let Im = {i: yi  Ym}, m = 1, 2,…, k denote the index set of the run-to-failure units whose sensory signals construct the subset Ym. Then the CV error is computed as the average error over all k trials and can be expressed as

HCV





1 k ˆ  y , Y \ Y , LT S Lˆ w, L ¦¦ i m i N m 1 iIm

(25)

where S(x) is a predefined evaluation metric that measures the accuracy of the ensemble-predicted RUL; N denotes the number of run-to-failure units for CV; LiT denotes the true RUL of the ith unit. The above formula indicates that all units in the data set are used for both training and testing, and each unit is used for testing exactly once and for training k–1 times. Thus, the variance of the resulting estimate is likely to be reduced compared to the traditional holdout approach, resulting in superior performance when employing a small data set. It is important to note that the disadvantage of the k-fold CV against the holdout method is greater computational expense because the training process has to be executed k times. As a commonly used setting for CV, a 10-fold CV is employed in this study.

III.3.Weighting schemes

This section will introduce three schemes to determine the weights of member algorithms: the accuracy-based weighting, diversity-based weighting and optimization-based weighting.

III.3.1. Accuracybasedweighting ThepredictionaccuracyofthejthmemberalgorithmisquantifiedbyitsCVerror,expressedas

19

H CVj



1 k ¦¦ S Lˆ j  yi , Y \ Ym , LTi N m 1 iIm



(26)

Theweightwjofthejthmemberalgorithmcanthenbedefinedasthenormalizationofthe correspondinginverseCVerror,expressedas

wj

H ¦ H j CV

M

i 1

1

i CV

(27)

1

Thisdefinitionindicatesthatalargerweightisassignedtoamemberalgorithmwithhigherprediction accuracy.Thus,amemberalgorithmwithbetterpredictionaccuracyhasalargerinfluenceonthe ensembleprediction.Thisweightingschemereliesexclusivelyonthepredictionaccuracytodetermine theweightsofmemberalgorithms.



III.3.2. Diversitybasedweighting TheweightformulationinEq.(27)reliesexclusivelyonthepredictionaccuracytodeterminethe weights.However,thepredictionaccuracyofmemberalgorithmsisnottheonlyfactorthataffectsthe ensembleperformance.Thepredictiondiversity,whichmeasurestheextenttowhichthepredictionsby amemberalgorithmaredistinguishablefromthosebytheothers,alsohasasignificanteffectonthe ensembleperformance,especiallyontherobustness.Morespecifically,alargerweightshouldgenerally beassignedtoamemberalgorithmwithhigherpredictiondiversitybecauseofitslargerpotentialto enhancetheensemblerobustness.

WebeginbyformulatinganNdimensionalerrorvectorconsistingofabsoluteRULpredictionerrors bythejthmemberalgorithmas

ej

ª Lˆ j  y1 , Y \ Y1  LT1 ,..., Lˆ j  y N , Y \ Ym  LTN º ¬ ¼

20

T

(28)

RepeatedlycomputingtheerrorvectorsforallMmemberalgorithmsgivesMerrorvectorse1,e2,…,eM. ThepredictiondiversityofthejthmemberalgorithmcanthenbecomputedasthesumofEuclidean distancesbetweentheerrorvectorejandalltheothererrorvectors,givenby M

Dj

¦

e j  ei

(29)

i 1;i z j

Thepredictiondiversitymeasurestheextenttowhichthepredictionsbyamemberalgorithmare distinguishablefromthosebyanyother.Basedonthedefinedpredictiondiversity,thenormalized weightwjofthejthmemberalgorithmcanthenbecalculatedas

wj

Dj

¦

M i 1

Di

(30)

Thisdefinitionsuggeststhatamemberalgorithmwithhigherpredictiondiversitywillbegivenalarger weightandthuscontributesmoretotheensemblepredictedRUL.Forexample,if,amongallthe memberalgorithms,onealgorithmconsistentlygivesearlyRULpredictionswhileanyoftheotherslate RULpredictions,theformerwilllikelybegivenalargerweightthanthelatter.Itisalsonotedthatthe weightformulationinEq.(30)considersthepredictiondiversityastheonlycriterionfortheweight determination.



III.3.3. Optimizationbasedweighting Neithertheaccuracybasednordiversitybasedweightingschemetakesintoaccountboththe predictionaccuracyanddiversityintheweightcalculation.Thus,thetwoschemescannotproducean ensemblealgorithmtoachievebothhighpredictionaccuracyandrobustness.Inwhatfollows,an optimizationbasedweightingschemeisproposedtomaximizetheaccuracyandrobustnessofdata drivenprognosticsbyadaptivelysynthesizingthepredictionaccuracyanddiversityofeachmember

21

algorithm.

Intheoptimizationbasedweightingscheme,theweightsinEq.(24)canbeobtainedbysolvingan optimizationproblemofthefollowingform

 

Subject to

¦



H CV Lˆ w, Lˆ  yi , LTi , i 1,..., N

Minimize H CV M j

w 1 j



(31)

1

AfterthepredictionofRULsusingtheMmemberalgorithmsthroughthe10foldCV,theabove optimizationproblemcanbereadilysolvedwithalmostnegligiblecomputationaleffortsincetheweight optimizationprocessdoesnotrequiretheexecutionofmemberalgorithms.Thus,theoverall computationalcostmainlycomesfromthetrainingandtestingintheCVprocess.Weexpectthat,by solvingtheoptimizationprobleminEq.(31),theresultingensembleofalgorithmswilloutperformanyof theensemble’sindividualmemberalgorithmsintermsofbothaccuracyandrobustness.Thecapability ofthisweightingschemetoadaptivelysynthesizethepredictionaccuracyanddiversityofeachmember algorithmwillbedemonstratedinthecasestudysection.



III.4.Overall procedure

Figure2showstheoverallprocedureoftheproposedensembleapproachwiththekfoldCVand threeweightingschemes.Thisdatadrivenprognosticapproachiscomposedoftheofflineandonline processes.Intheofflineprocess,theofflinetraining/testingprocesswiththekfoldCVisemployedto computetheCVerrorofanensembleformulation;theweightsofmemberalgorithmsaredetermined usingtheaccuracybasedweighting,diversitybasedweightingandoptimizationbasedweighting.The onlinepredictionprocesscombinestheRULpredictionsfromallmemberalgorithmstoforman ensembleRULpredictionusingtheweightsobtainedfromtheofflineprocess.Thisprocessenablesthe continuousupdateofthehealthinformationandprognosticresultsinrealtimewithnewsensory 22

signals.0detailstheproposedensembleprognosticsapproachwiththefivesteps.STEPS24canbe repeatedtoincorporatenewtrainingsensorysignalsandtoupdatetheweightsandRULpredictions. Sincethecomputationallyexpensivetrainingprocesswithmultiplealgorithmsisdoneofflineandthe onlinepredictionprocesswithmultiplealgorithmsrequiresarelativelysmallamountofcomputational effort,theensembleapproachraiseslittleconcernsinthecomputationalcomplexity.Indeed,inmany engineeredsystems,theprognosticaccuracyistreatedasofmuchhigherimportancecomparedtothe computationalcomplexitysincetheoccurrenceofacatastrophicsystemfailurecausesmuchmoreloss thantheincreaseofthecomputationalefforts.Therefore,incaseswheretheensembleapproach achievesconsiderableimprovementinthepredictionaccuracyoveranysolememberalgorithm,we shouldalwaysprefertheuseoftheformer.



 Figure2.

Aflowchartoftheensembleapproach





Table2 Detailedprocedureoftheensembleapproach

23

STEP1

Determinesensorconfigurationsandacquiretrainingsensorysignalsfromoffline systemunits;

STEP2a

Performtheofflinetraining&testingprocesseswiththekfoldCVwiththetraining sensorysignalstocomputetheCVerror;

STEP2b

Determinetheweightsusingtheaccuracybasedweighting,diversitybasedweighting andoptimizationbasedweightingschemes;

STEP3

Acquiretestingsensorysignalsfromonlinesystemunits;

STEP4a

PredictRULsusingthememberalgorithmsthroughtheonlinepredictionprocesswhich employsthebackgroundhealthknowledgeobtainedfromtheofflinetrainingprocess;

STEP4b PredicttheensembleRULswiththeoptimumweightsobtainedfromSTEP2b.

IV. Case Studies Inthissection,theproposedensembleofdatadrivenprognosticalgorithmsisdemonstratedwith threePHMcasestudies:(i)2008IEEEPHMchallengeproblem,(ii)powertransformerproblem,and(iii) electriccoolingfanproblem.Ineachcasestudy,theensembleapproachcombinesRULpredictionsfrom fivepopulardatadrivenprognosticalgorithms,namely,asimilaritybasedinterpolation(SBI)approach withRVMastheregressiontechnique(RVMSBI)[15,49],SBIwithSVM(SVMSBI)[47,15],SBIwiththe leastsquareexponentialfitting(ExpSBI)[15],aBayesianlinearregressionwiththeleastsquare quadraticfitting(QuadBLR)[17],andarecurrentneuralnetwork(RNN)approach[50,18].Details regardingthefiveprognosticalgorithmsaregiveninSectionII. IV.1.

2008 IEEE PHM challenge problem

In an aerospace system (e.g., an airplane, a space shuttle), system safety plays an important role since failures can lead to dramatic consequences. In order to meet stringent safety requirements as well as minimize the maintenance cost, condition-based maintenance must be conducted throughout the system’s lifetime, which can be enabled by system health prognostics. This case study aims at predicting the RULs

24

of aircraft engine systems in an accurate and robust manner with massive and heterogeneous sensory data.  IV.1.1. Description of data set

The data set provided by the 2008 IEEE PHM Challenge problem consists of multivariate time series signals that are collected from an engine dynamic simulation process. Each time series signal comes from a different degradation instance of the dynamic simulation of the same engine system [39]. The data for each cycle of each unit include the unit ID, cycle index, 3 values for an operational setting and 21 values for 21 sensor measurements. The sensor data were contaminated with measurement noise and different engine units start with different initial health conditions and manufacturing variations which are unknown. Three operational settings have a substantial effect on engine degradation behaviors and result in six different operation regimes as shown in Table 3. The 21 sensory signals were obtained from six different operation regimes. The whole data set was divided into training and testing subsets, each of which consists of 218 engine units. In the training data set, the damage growth in a unit was allowed until the occurrence of a system failure when one or more limits for safe operation have been reached. In the testing data set, the time series signals were pruned some time prior to the occurrence of a system failure. The objective of the problem is to predict the number of remaining operational cycles before failure in the testing data set.

Table3 Sixdifferentoperationregimes

Operating Operating Operating RegimeID parameter1 parameter2 parameter3 1

0

0

100

2

20

0.25

20

3

20

0.7

0

4

25

0.62

80

5

35

0.84

60

6

42

0.84

40

25



 IV.1.2. Implementation of ensemble approach

FortheCVprocess,thetrainingdatasetwith218unitsweredividedto10datasubsetswithasimilar size.Eachdatasubsetwasusedforbothtrainingandtestingand,morespecifically,9timesfortraining andoncefortesting.Thetrainingdatasubsetscontaincompletedegradationinformationwhilethe testingdatasubsetscarryonlypartialdegradationinformation.Thelatterweregeneratedbytruncating theoriginaldatasubsetsafterpreassignedRULs.TheRULpreassignedtoeachunitinatestingdata subsetwasrandomlygeneratedfromauniformdistributionbetweenitszeroandhalfremaininglife. Thisrangeintheuniformdistributionwasselectedbasedonthefollowingtwocriteria:(i)thepre assignedRULsshouldbesmallenoughtoallowtheoccurrenceofsubstantialdegradation;and(ii)the variationofthepreassignedRULsshouldbelargeenoughtotesttherobustnessofalgorithms.

Amongthe21sensorysignals,somesignalscontainnoorlittledegradationinformationofanengine unitwhereastheothersdo.ToimprovetheRULpredictionaccuracy,importantsensorysignalsmustbe carefullyselectedtocharacterizethedegradationbehaviorofengineunitsforhealthprognostics. Followingtheworkin[15],thisstudyselected7sensorysignals(2,3,4,7,11,12and15)amongthe21 sensorysignalsfortheuseinthememberalgorithms:RVMSBI,SVMSBI,ExpSBIandQuadBLR.A monotoniclifetimetrendcanbeobservedfromthese7sensorysignalsofwhichthenoiselevelsare relativelylow.FortheVHIconstruction,thesystemfailurematrixQ0wascreatedwiththesensorydata inasystemfailurecondition,0Ld4,whilethesystemhealthymatrixQ1withthoseinasystem healthycondition,L>300.TheRVMemployedalinearsplinekernelfunctionwiththeinitialmost probablehyperparametervectorforkernelweightsm=[1×104,…,1×104]andtheinitialmostprobable noisevariancem2=1×10–4.IntheSVM,aGaussiankernelfunctionisusedwiththeparametersettings

26

as:theregularizationparameterC=10andtheparameteroftheinsensitivelossfunction=0.10.In theRNNtraining,the21normalizedsensorysignalstogetherwiththeregimeIDateachcyclewereused asthemultidimensionalinputsoftheRNNandtheRULatthecorrespondingcyclewasusedasthe output.Theimplementationdetailscanbefoundin[18].IntheRNNarchitecture,thenumbersofthe input,recurrentandoutputunitsare|I|=22,|R|=8and|O|=1.

Theevaluationmetricconsideredforthisexampleemployedanasymmetricscorefunctionaround thetrueRULsuchthatheavierpenaltiesareplacedonlatepredictions[39].Thescoreevaluationmetric Scanbeexpressedas



S Lˆi , LTi



­°exp  di /13  1, di  0 ® °¯exp  di /10  1, di t 0

where di

Lˆi  LTi

(32)

where Lˆi and LiT denote the predicted and true RUL of the ith unit, respectively. This score function was used to compute the CV error CV using Eq. (25) for the accuracy- and optimization-based weighting schemes. In this study the weight optimization problem in Eq. (31) was solved using a sequential quadratic optimization (SQP) method which is a gradient-based optimization technique.

 IV.1.3. Results of ensemble approach

ThefiveselectedmemberalgorithmsareRVMSBI(RS),SVMSBI(SS),ExpSBI(ES),QuadBLR(QB) andRNN(RN).Thethreeweightingschemesaretheaccuracybasedweighting(AW),diversitybased weighting(DW)andoptimizationbasedweighting(OW).Table4summarizestheweightingresultsby thethreeweightingschemesaswellascomparestheCVandvalidationerrorsoftheindividualand ensembleapproaches.Itisobservedthattheensembleapproacheswithallthreeweightingschemes outperformsanyoftheindividualmemberalgorithmintermsoftheCVerrorandthattheonewiththe optimizationbasedweightingachievesthesmallestCVerrorof4.8387onthetrainingdataset,a 38.62%improvementoverthebestindividualmemberalgorithm,ES,whoseCVerroris7.8834.As 27

expected,theaccuracybasedweightingschemeyieldsbetterpredictionaccuracythanthediversity basedweighting.Thiscanbeattributedtothefactthattheformerassignslargerweightstomember algorithmswithbetterpredictionaccuracywhilethelatterdoesnotconsiderthepredictionaccuracyin theweightdetermination.Totesttherobustnessoftheensembleapproaches,thetestingdatasetwith 218unitswereemployedtocomputethevalidationerrors.Notethatthetestingdatasetisdifferent fromthetrainingdatasetthatwasusedtodeterminetheweightsintheensembleapproach.Itis apparentthattheensembleapproachesagainoutperformtheindividualmemberalgorithmsandthat theonewiththediversitybasedweightingperformsbest,witha34.7%improvementoverthebest individualmemberalgorithm,SS.Thissuggeststhatthediversitybasedweighting,comparedtothe accuracybasedweighting,providesamorerobustensembleofthememberalgorithms.Itisnotedthat theoptimizationbasedweightingschemestillachievesacomparablevalidationerrortothatofthe diversitybasedweightingscheme.

Undertheoptimizationbasedweightingscheme,theRULpredictionsbytwoindividualalgorithms, ESandQB,withthelargestweightsandtheensembleapproachareplottedfor218trainingandtesting unitsinFigure3.TheunitsaresortedbytheRULsinanascendingorder.ItisseenthatEStendstogive consistentlyearlyRULpredictionswhileQBtendstoprovideconsistentlylateRULpredictions.In contrast,theensembleapproachgivesRULpredictionsclosertothetruevalueswhileeliminatingmany outliersproducedbythetwoindividualalgorithms.Theoptimizationbasedweightingschemeprovides betterperformancesincetheschemeemploysanoptimumensembleformulation.



28





(a) Figure3.

(b)

RULpredictionsoftrainingunits(a)andtestingunits(b)for2008PHM challengeproblem(optimizationbasedweighting)





Table4 Weightingresults,CVandvalidationerrorsfor2008PHMchallengeproblem



RSSSESQBRN RS

SS

ES

QB

RN



AW

DW

OW

WeightbyAW

 0.3063

0.3029

 0.3137

0.0151 0.0620







WeightbyDW

 0.1478

0.1488

 0.1488

0.3354 0.2191







WeightbyOW

 0.0000

0.0470

 0.7462

0.2068 0.0000

















CVerror

 8.0743

8.1646

 7.8834

163.3376 39.8583

Validation error

10.2393

9.3907

10.4710

247.0079



29



20.1499







 6.9159

 7.0852

 4.8387

8.5544

6.1280

6.1955

 IV.1.4. Comparison of different combinations of member algorithms

Outofthefivememberalgorithms,31differentcombinationscanbechosentoformulatean ensembleapproach.Itwouldbeinterestingtostudyhowachoiceofcombinationaffectsthe performanceofanensembleapproach.Table5summarizestheCVerrorsforensembleapproacheswith allpossiblecombinationsofthememberalgorithmsundertheoptimizationbasedweightingscheme. Threeimportantremarkscanbederivedfromtheresults.Firstofall,itisobservedthattheES,asthe individualmemberalgorithmwiththebestperformance,alwaysservesasamemberalgorithmofthe bestensembleapproach.WealsoobservethattheES,wheninvolvedintheensembleapproach,always hadalargerweightthananyother.Itindicatesthatthebestmemberalgorithmexhibitsgood cooperativeperformancewhichcanbeidentifiedbytheoptimizationbasedweightingscheme. Secondly,theQB,whichgivestheworstindividualperformance,wassurprisinglyselectedasan importantmemberofthebestensembleapproach.Theseresults,thoughcounterintuitive,suggestthat theensembleapproachcanadaptivelysynthesizethepredictionabilityanddiversityofeachindividual algorithmtoenhancetheaccuracyandrobustnessofRULpredictions.Indeed,theQBispronetogive lateRULpredictionsasshowninFigure3andthuspossesseshigherpredictiondiversity.Thirdly,both themeanandstandarddeviationofCVerrorsdecreaseasthenumberofmemberalgorithmsincreases. ThemeanandstandarddeviationofCVerrorsofensembleapproacheswithasinglememberalgorithm are45.4636and67.3188,respectively,andtheymonotonicallydecreaseto5.1896and0.7440, respectively,bytheensembleapproachwithfourmemberalgorithms.Thusitwouldbebeneficialto havemorememberalgorithmstoenhancethepredictionaccuracyandreducetheuncertaintyofthis accuracy.



Table5 ComparisonofCVerrorsofdifferentcombinationsofmemberalgorithmsfor2008PHM

30

challengeproblem(optimizationbasedweighting) Combination

CVerror



Combination

CVerror



Combination

CVerror

RS

8.0743



RSSS

8.0769



RSSSES

7.8834

SS

8.1646



RSES

7.8834



RSSSQB

4.9123

ES

7.8834



RSQB

4.9162



RSSSRN

6.7983

QB

163.3376 

RSRN

6.8002



RSESQB

4.8391

RN

39.8583



SSES

7.8834



RSESRN

6.5194

Mean

45.4636













Stda

67.3188





























RS SS ES QB

4.8387



SSQB

4.9362



RSQBRN

4.9162

RSSSESRN

6.5194



SSRN

6.8376



SS ES QB

4.8387

RSSSQBRN

4.9123



ES QB

4.8391



SSESRN

6.5194

RSESQBRN

4.8391



ESRN

6.5194



SSQBRN

4.9362

SS ES QB RN

4.8387



QBRN

17.5868



ESQBRN

4.8391

Mean

5.1896



Mean

7.6279



Mean

5.7002

Std

0.7440



Std

3.7182



Std

1.1234

















RSSSESQBRN

4.8387













a

Standarddeviation

IV.2.

Power transformer problem

Thepowertransformerisacriticalpowerelementinnuclearpowerplants,sinceanunexpected breakdownofthetransformercausesplantshutdownandsubstantialsocietalexpense.Soitisvery importanttoensurehighreliabilityandsafetyofthetransformerduringitsoperation.Investigationsof

31

thefailurescauseshaverevealedthatmechanicalbreakdownsconstitutealargeportionofunexpected breakdownsoftransformersinnuclearpowerplants[40].Therefore,healthmonitoringandprognostics ofthetransformerwithrespecttomechanicalfailuresisofsignificantimportancetopreventing unexpectedbreakdownsandminimizinginterruptionstoreliablecustomerservice.Thiscasestudy conductstransformerhealthprognosticswithsensorysignalsobtainedfromafiniteelement(FE)model ofapowertransformer. IV.2.1. Model description

TheFEmodelofapowertransformerwascreatedinANSYS10asshowninFigure4,whereone exteriorwallisconcealedtomaketheinteriorstructurevisible.Thetransformerisfixedatthebottom surfaceandavibrationloadwiththefrequencyof120Hzisappliedtothemagneticcore.Thethree windingshaveatotalnumberoftwelvesupportjoints,witheachhavingfoursupportjoints.Therandom parametersconsideredinthisstudyarelistedinTable6,whichincludesthematerialpropertiesof supportjointsandwindingsaswellthegeometriesofthetransformer.Theuncertaintiesinvibration responsespropagatedfromtheseuncertainparameterswillbeaccountedforwhengenerating prognosticdata. Sinceitisverydifficult,ifnotimpossible,toobtaindirectmeasurementsofthehealthconditionof transformers,indirectmeasurementsaremostoftenusedtodiagnosethehealthconditionandpredict theRULsoftransformers[41].Inparticular,thevibrationsofthemagneticcoreandofthewindings couldcharacterizetransitoryoverloadsandpermanentfailuresbeforeanyirreparabledamageoccurs [42,43].Thus,thiscasestudyemploysthevibrationsignalsofthemagneticcoreandofthewindingsofa powertransformertopredicttheRULsoftransformers.

32

Damagedjoint

Figure4.



ApowertransformerFEmodel(withoutthe coveringwall)

Table6 Randomgeometriesandmaterialpropertiesforpowertransformerproblem

Component

Physicalmeaning

Distri.type

x1

WallThickness

Normal

3

0.015

x2

Angularwidthofsupportjoints

Normal

15

0.075

x3

Heightofsupportjoints

Normal

6

0.03

x4

Young’smodulusofsupport joint

Normal

2E+12

1E+10

x5

Young’smodulusofwinding

Normal

1.28E+12

6E+8

x6

Poissonratioofjoints

Normal

0.27

0.0027

x7

Poissonratioofwinding

Normal

0.34

0.0034

x8

Densityofjoints

Normal

7.85

0.000785

x9

Densityofwindings

Normal

8.96

0.0896





33

Mean

Std

IV.2.2. Prognostic data generation

Thefailuremodeconsideredinthisstudyisthelooseningofawindingsupportjoint(seeFigure4) inducedbythemagneticcorevibration.Thejointlooseningwasrealizedbyreducingthestiffnessofthe joint.Thefailurecriterionisdefinedasa99%stiffnessreductionofthejoint.Tomodelthetrajectoryof changeinstiffnessovertime,thisstudyusesadamagepropagationmodelwithanexponentialformas [39] E t

E0  bE 1  exp  aE t

(33)

where E0 is the initial Young’s modulus of the joint; aE and bE are the model parameters; t is the cycle time. The initial Young’s modulus E0 follows the same normal distribution with x4 (see Table 6). The model parameters aE and bE are independent and normally distributed with means 0.002 and 4E+12, each of which has a 10% coefficient of variation.

Sincedatadrivenprognosticapproachesrequirealargeamountofprognosticdata,itis computationallyintolerable,ifnotimpossible,tosimplyrunthesimulationtogenerateeverydatapoint. Toovercomethisdifficulty,thisstudyemployedtheunivariatedecompositionmethodthatonlyusesa certainnumberofunivariatesamplepointstoconstructtheresponsesurfaceforageneralmultivariate responsefunctionwhileachievinggoodaccuracy[44].Thisstudyselected5straingauges(seeFigure5) fromtheoptimallydesignedsensornetworkconsistingof9straingaugesandthusrequiresthe constructionof5responsessurfaces.Thedatagenerationprocessinvolvesfoursequentiallyexecuted procedures:(i)fourunivariatesamplepointswereobtainedfromtheharmonicanalysistoconstruct responsesurfaces,alongthedamagepropagationpath,thatapproximatethestraincomponentsatfive sensorlocationsasfunctionsofrandomvariablesdetailedinTable6;(ii)400randomlygenerated samplesofE0,aEandbEwereusedinconjunctionwithEq.(33)toproduce400damagepropagation paths,ofwhich200pathswereassignedtothetrainingunitsandtheresttothetestingunits;(iii)the constructedresponsesurfaceswereusedtointerpolatethestraincomponentsatfivesensorlocations

34

foragivensetofrandomlygeneratedgeometriesandmaterialpropertiesanddamagepropagation paths,andrepeatedlyexecutingthisprocessfor400timesgavethetrainingdatasetwith200training unitsandthetestingdatasetwith200testingunits;(iv)measurementnoisefollowingazeromean normaldistributionwasaddedtoboththetrainingandtestingdatasetstofinalizethedatageneration. Thecubicsplinewasusedasthenumericalschemefortheresponsesurfaceconstructionand interpolation.Simulatedmeasurementsbysensors1and5areplottedagainsttheadjustedcycleindex, definedasthesubtractionofthecycletofailurefromtheactualoperationalcycle, inFigure6forall200 trainingunitsinthetrainingdataset.

4 3 5

1 2

 Figure5.

5straingaugeslocatedonthesidewallofpower transformer

35





(a) Figure6.

(b)

Simulatedmeasurementsbysensors1(a)and5(b)forpowertransformer problem

 IV.2.3. Implementation of ensemble approach

Thetrainingdatasetwith200unitswereequallyandrandomlydividedto10subsets.Similartothe firstexample,whenusedforthetestinginCV,eachunitinasubsetwasassignedwitharandomly generatedRULfromauniformdistributionbetweenitszeroandhalfremaininglife.Allthefivemember algorithmsusedthesameparametersettingswiththosedetailedinSectionIV.1.2.Thescorefunctionin Eq.(32)wasagainusedtocomputetheCVerrorCVfortheaccuracyandoptimizationbasedweighting schemes.

 IV.2.4. Results of ensemble approach

Table7summarizestheweightingresultsbythethreeweightingschemesaswellascomparestheCV andvalidationerrorsoftheindividualandensembleapproaches.Comparedtothefirstexample,similar resultscanbeobserved:(i)theensembleapproacheswithallthreeweightingschemesyieldsmallerCV errorthananyoftheindividualmemberalgorithmandtheonewiththeoptimizationbasedweighting givesthesmallestCVerrorof2.7258onthetrainingdataset,a66.48%improvementoverthebest 36

individualmemberalgorithm,RN,whoseCVerroris8.1323;(ii)theaccuracybasedweightingscheme yieldsacomparableCVerrortothatofthediversitybasedweighting;(iii)theoptimizationbased weightingschemeachievesavalidationerrorof5.6138,whichiscomparabletothesmallestvalidation errorof5.6119bythediversitybasedweightingscheme.

Undertheoptimizationbasedweightingscheme,theRULpredictionsbytwoindividualalgorithms, ESandQB,withthelargestweightsandtheensembleapproachareplottedfor218trainingandtesting unitsinFigure7.ItcanbeobservedthatESandQBarepronetoproduceearlyandlateRULpredictions, respectively,whiletheensembleapproachgivesRULpredictionsclosertothetruevalueswithamuch smallernumberofoutliers.

Table7 Weightingresults,CVandvalidationerrorsforpowertransformerproblem



RSSSESQBRN RS

SS

ES

QB

RN



AW

DW

OW

WeightbyAW

 0.2128

0.2265

 0.2343

 0.0677

0.2588







WeightbyDW

 0.1488

0.1486

 0.1688

 0.3290

0.2048







WeightbyOW

 0.0000

0.0000

 0.6303

 0.2336

0.1361



















CVerror

 9.8922

9.2945

 8.9849

Validation error

 6.5737

6.8847

 7.8251







 8.1323 31.0891

 3.4874

 3.4124

 2.7258

 20.0356 15.2265

5.7825

5.6119

5.6138



37

 (a) Figure7.

 (b)

RULpredictionsoftrainingunits(a)andtestingunits(b)forpower transformerproblem(optimizationbasedweighting)

 IV.2.5. Comparison of different combinations of member algorithms

Acomparisonstudyofdifferentcombinationsofmemberalgorithmswasagaincarriedoutusingthe optimizationbasedweightingschemeforthepowertransformerproblem.Table8summarizesthe comparisonresultsfromwhichseveralimportantremarkssimilartothoseinthefirstexamplecanbe derived.Firstofall,thememberalgorithmsESandQBcanalwaysbeobservedinthebestensemble approachwithmorethanonememberalgorithms.WealsoobservethatthecombinationESandQB, wheninvolvedintheensembleapproach,alwayshadalargerweightthananyother.Thisresultis differentfromwhatweobserveinthefirstexample,wherethelargestweightwasassignedtothebest individualmemberalgorithm.Thissuggeststhattheoptimizationbasedweightingschememakesless useoforevendiscardedthebestmemberalgorithmthatdoesnotexhibitgoodcooperative performancewithothermembers.Secondly,theQB,whichgivestheworstindividualperformanceand ispronetogivelaterRULpredictions(seeFigure7),wasselectedasanimportantmemberofthebest ensembleapproach.Thisagainsuggeststhatthepredictiondiversityplaysanimportantroleinthe weightdetermination.Thirdly,asisthecaseinthefirstexample,boththemeanandstandarddeviation

38

ofCVerrorsdecreaseasthenumberofmemberalgorithmsincreases.Thustheadditionofmember algorithmstendstoenhancethepredictionaccuracyandreducetheuncertaintyofthisaccuracy.

Table8 ComparisonofCVerrorsofdifferentcombinationsofmemberalgorithmsforpowertransformer

problem(optimizationbasedweighting) Combination

CVerror 

Combination

CVerror



Combination

CVerror

RS

9.8922



RSSS

9.2945



RSSSES

8.9561

SS

9.2945



RSES

8.9849



RSSSQB

3.1688

ES

8.9849



RSQB

3.1764



RSSSRN

3.9651

QB

31.0891 

RSRN

3.9744



RSESQB

2.7894

RN

8.1323



SSES

8.9561



RSESRN

3.4557

Mean

13.4786













Std

9.8650





























RSSSESQB

2.7894



SSQB

3.1815



RSQBRN

3.1470

RSSSESRN

3.4557



SSRN

3.9671



SSESQB

2.7894

RSSSQBRN

3.1433



ES QB

2.7894



SSESRN

3.4557

RS ES QB RN

2.7258



ESRN

3.4557



SSQBRN

3.1559

SS ES QB RN

2.7258



QBRN

6.9724



ES QB RN

2.7258

Mean

2.9680



Mean

5.4752



Mean

3.7609

Std

0.3232



Std

2.7412



Std

1.8640

















RSSSESQBRN

2.7258













 IV.3.

Electric cooling fan problem

Inadditiontothenumericalstudies,wealsoconductedexperimentaltestingtoverifythe

39

effectivenessoftheensembleapproach.Inthiscasestudy,weappliedtheensembleapproachtothe healthprognosticsofelectroniccoolingfanunits.Coolingfansareoneofthemostcriticalpartsin systemthermalsolutionofmostelectronicproducts[51]andincoolingtowersofmanychemicalplants [52].Thisstudyaimstodemonstratetheproposedensembleprognosticswith32electroniccooling fans. IV.3.1. Experimental setup

Inthisexperimentalstudy,thermocouplesandaccelerometerswereusedtomeasuretemperature andvibrationsignals.Tomaketimetofailuretestingaffordable,theacceleratedtestingconditionfor theDCfanunitswassoughtwithinclusionofasmallamountoftinymetalparticlesintoballbearings andanunbalancedweightononeofthefanunits.TheexperimentblockdiagramofDCfanaccelerated degradationtestisshowninFigure8.Asshowninthediagram,theDCfanunitsweretestedwith12V regulatedpowersupplyandthreedifferentsignalsweremeasuredandstoredinaPCthroughadata acquisitionsystem.Figure9(a)showsthetestfixturewith4screwsateachcornerfortheDCfanunits. AsshowninFigure9(b),anunbalancedweightwasusedandmountedononebladeforeachfan. Sensorswereinstalledatdifferentpartsofthefan,asshowninFigure10.Inthisstudy,threedifferent signalsweremeasured:thefanvibrationsignalbytheaccelerometer,thePrintedCircuitBoard(PCB) blockvoltagebythevoltmeter,andthetemperaturemeasuredbythethermocouple.Anaccelerometer wasmountedtothebottomofthefanwithsuperglue,asshowninFigure10(a).Twowireswere connectedtothePCBblockofthefantomeasurethevoltagebetweentwofixedpoints,asshownin Figure10(b).AsshowninFigure10(c),athermocouplewasattachedtothebottomofthefanand measuresthetemperaturesignalofthefan.Vibration,voltage,andtemperaturesignalswereacquired bythedataacquisitionsystemandstoredinPC.ThedataacquisitionsystemfromNationalInstruments Corp.(NIUSB6009)andthesignalconditionerfromPCBGroup,Inc.(PCB482A18)wereusedforthe dataacquisitionsystem.Intotal,32DCfanunitsweretestedatthesameconditionandallfanunitsrun

40

tillfailure.

 Figure8.

DCfandegradationtestblockdiagram







(a) Figure9.

(b)

DCfantestfixture(a)andtheunbalanceweightinstallation (b)



 (a)

(b)

41

(c)

Figure10.

SensorinstallationsforDCfantest:(a)accelerometer,(b)voltmeterand(c) thermocouples

 IV.3.2. Implementation of Ensemble Approach

ThesensorysignalscreeningfoundthatthefanPCBblockvoltageandthefantemperaturedidnot showcleardegradationtrend,whereasthevibrationsignalshowedhealthdegradationbehavior.This studyinvolvedtherootmeansquares(RMS)ofthevibrationspectralresponsesatthefirstfive resonancefrequenciesanddefinedtheRMSofthespectralresponsesasthePHIfortheDCfan prognostics.Figure11showstheRMSsignalsofthreefanunitstodemonstratethehealthdegradation behavior.TheRMSsignalgraduallyincreasedasthebearinginthefandegradedovertime.Itwasfound thatthePHIishighlyrandomandnonmonotonicbecauseofmetalparticles,sensorysignalnoise,and inputvoltagenoise.

Among32fanunits,thefirst20fanunitswereusedtoconstructthetrainingdatasetfortheCV, whiletherestwereusedtobuildthetestingdatasetforthevalidation.Duetothesmallamountof trainingdata,thiscasestudyemployedthe5foldCVwherethetrainingdatasetwith20unitswas equallyandrandomlydividedto5subsets.Similartothepreviousexamples,whenusedforthetesting inCV,eachunitinasubsetwasassignedwitharandomlygeneratedRULfromauniformdistribution betweenitszeroandhalfremaininglife.Toexpandthenumberoftestingunits,eachtestingfanunit wasassignedwithtworandomlygeneratedRULfromauniformdistributionbetweenitszeroandhalf remaininglife,resultingintotally24testingunits.TheparametersettingsdetailedinSectionIV.1.2was againusedforthefivememberalgorithms.Withonecycledefinedaseverytenminutes,thescore functioninEq.(32)wasagainusedtocomputetheCVerrorCVfortheaccuracyandoptimization basedweightingschemes.

42

 Figure11.

SampledegradationsignalsfromDCfantesting

 IV.3.3. Results of Ensemble Approach

TheweightingresultsbythethreeweightingschemesandtheCVandvalidationerrorsofthe individualandensembleapproachesaresummarizedinTable9.Comparedtothepreviousexamples, weobservedquitedifferentresultsfromwhichthreeimportantremarkscanbederived.Firstofall,the ensembleapproachwiththediversitybasedweightingschemegivesconsiderablylargerCVand validationerrorsthanthebestindividualmemberalgorithms,RSandES.Thisresultisduetothefact thatthediversitybasedweighting,whichreliesexclusivelyonthepredictiondiversityfortheweight determination,assignedlargerweightstothememberalgorithms,QBandRN,whichproducedverylow predictionaccuracyduetotherandomandnonmonotonicnatureofthePHI(seeFigure11).Secondly, comparedtothebestindividualmemberalgorithms,RSandSS,theensembleapproachwiththe optimizationbasedweightinggavesmallerCVandvalidationerrors.However,theimprovementis insignificant.Sincenonzeroweightsareonlyassignedtothetwomemberalgorithms,RSandES,with superbpredictioncapability,theperformanceoftheresultingensembleistotallydeterminedbythese twoalgorithms.However,RSandESgavesimilarRULpredictionsandtheresultingensemble,whichis 43

indeedacombinationoftwoalgorithmswithsimilarpredictionbehavior,cannotachievesignificant improvementinthepredictionperformance.Therefore,weexpectthattheensembleapproach achievessignificantimprovementinthepredictionperformanceonlyincaseswherememberalgorithms withcomparablepredictionaccuracyproducediverseRULpredictions.Thirdly,althoughthemember algorithms,QBandRN,havelargerpredictiondiversity,theirpredictionaccuracyisnotcomparablewith thatofthebestmemberalgorithms,RSandSS.Asaresult,thesetwoalgorithmswerediscardedfrom thealgorithmpoolbytheoptimizationbasedweighting.Undertheoptimizationbasedweighting scheme,theRULpredictionsbytheensembleapproachareplottedforthetrainingandtestingunitsin Figure12whereweobservedveryaccurateRULpredictionsbytheensembleapproach.

Table9 Weightingresults,CVandvalidationerrorsforelectriccoolingfanproblem



RSSSESQBRN RS

SS

ES

QB

RN



AW

DW

OW

WeightbyAW

 0.3646

0.3767

 0.2552

0.0008 0.0027







WeightbyDW

 0.1423

0.1427

 0.1496

0.3285 0.2369







WeightbyOW

 0.1155

0.8845

 0.0000

0.0000 0.0000

















CVerror

 1.4770

1.4298

 2.1100

717.8430 199.0067

Validation error

 0.7027

0.9223

 0.7037

461.5064



44



84.3975







 1.5188

11.8520

 1.4292

0.7185

11.0177

0.6984

(a) Figure12.

(b)

RULpredictionsoftrainingunits(a)andtestingunits(b)forelectriccooling fanproblem(optimizationbasedweighting)

 V. Conclusion Thispaperproposedanovelensembleapproachforthedatadrivenprognosticsofhighrisk engineeredsystems.Bycombiningthepredictionsofallmemberalgorithms,theensembleapproach achievesbetteraccuracyinRULpredictionscomparedtoanysolememberalgorithm.Furthermore,the ensembleapproachhasaninherentflexibilitytoincorporateanyadvancedprognosticalgorithmthat willbenewlydeveloped.Tothebestofourknowledge,thisisthefirststudyofanensembleapproach withthreeweightingschemeforthedatadrivenprognostics.Sincethecomputationallyexpensive trainingprocessisdoneofflineandtheonlinepredictionprocessrequiresasmallamountof computationaleffort,theensembleapproachraiseslittleconcernsinthecomputationalfeasibility. Threeengineeringcasestudies(2008PHMchallengeproblem,powertransformerproblemandelectric coolingfanproblem)demonstratedthesuperbperformanceoftheproposedensembleapproachfor thedatadrivenprognostics.Amongthethreeweightingscheme,theoptimizationbasedweighting schemeshowedthecapabilityofadaptivelysynthesizingthepredictionaccuracyanddiversityofeach memberalgorithmtoenhancetheaccuracyofRULpredictions.Consideringtheenhancedaccuracyand

45

robustnessinRULpredictions,theproposedensembleapproachleadstothepossibilityofeffective conditionbasedmaintenancepracticeandriskinformedlifetimemanagementofhighriskengineered systems.

Acknowledgement ThisworkwaspartiallysupportedbyagrantfromtheEnergyTechnologyDevelopmentProgram (2010101010027B)andInternationalCollaborativeR&DProgram(042020110161)ofKoreaInstituteof EnergyTechnologyEvaluationandPlanning(KETEP),fundedbytheKoreangovernment’sMinistryof KnowledgeEconomy,theNationalResearchFoundationofKorea(NRF)grant(No.20110022051) fundedbytheKoreagovernment,theBasicResearchProjectofKoreaInstituteofMachineryand Materials(ProjectCode:SC0830)supportedbyagrantfromKoreaResearchCouncilforIndustrial Science&Technology,andtheInstituteofAdvancedMachineryandDesignatSeoulNationalUniversity (SNUIAMD).



References 1

Dekker R., 1996, “Applications of maintenance optimization models: a review and analysis,”

ReliabilityEngineeringandSystemSafety,v51,n3,p229–240. 2

MarseguerraM.,ZioE.,andPodofilliniL.,2002,“Conditionbasedmaintenanceoptimizationby

means of genetic algorithms and Monte Carlo simulation,” Reliability Engineering and System Safety, v77,n2,p151–165. 3

Zio E., 2009, “Review reliability engineering: Old problems and new challenges,” Reliability

EngineeringandSystemSafety,v94,n2,p125–141. 4

Barata J., Guedes Soares C., Marseguerra M., and Zio E., 2002, “Simulation modelling of

46

repairable multicomponent dteriorating systems for ‘on condition’ maintenance optimization,” ReliabilityEngineeringandSystemSafety,v76,n3,p255–264. 5

Grall A., Berenguer C., and Dieulle L., 2002, “A conditionbased maintenance policy for

stochasticallydeterioratingsystems,”ReliabilityEngineeringandSystemSafety,v76,n3,p167–180 6

WeideJ.A.M.vander,PandeyM.D.,andNoortwijkJ.M.van,2010,“Discountedcostmodelfor

conditionbased maintenance optimization,” Reliability Engineering and System Safety, v95, n3, p236– 246. 7

SmidtDestombesK.S.de,HeijdenM.C.vander,andHartenA.van,2004,“Ontheavailabilityof

a koutofN system given limited spares and repair capacity under a condition based maintenance strategy,”ReliabilityEngineeringandSystemSafety,v83,n3,p287–300. 8

Tinga T., 2010, “Application of physical failure models to enable usage and load based

maintenance”,ReliabilityEngineeringandSystemSafety,v95,n10,p1061–1075. 9

Myotyri E., Pulkkinen U., and Simola K., 2006, “Application of stochastic filtering for lifetime

prediction,”ReliabilityEngineeringandSystemSafety,v91,n2,p200–208. 10

CadiniF.,ZioE.,andAvramD.,2009,“ModelbasedMonteCarlostateestimationforcondition

basedcomponentreplacement,”ReliabilityEngineeringandSystemSafety,v94,v3,p752–758. 11

LuoJ.,PattipatiK.R.,QiaoL.,andChigusaS.,2008,“Modelbasedprognostictechniquesapplied

toasuspensionsystem,”IEEETransactionsonSystems,ManandCybernetics,PartA,v38,n5,p1156– 1168. 12

Gebraeel N., and Pan J., 2008, “Prognostic degradation models for computing and updating

residuallifedistributionsinatimevaryingenvironment,”IEEETransactionsonReliability,v57,n4,p539– 550.

47

13

Gebraeel N., Elwany A., and Pan J., 2009, “Residual life predictions in the absence of prior

degradationknowledge,”IEEETransactionsonReliability,v58,n1,p106–117. 14

Schwabacher M., 2005, “A survey of datadriven prognostics”, Proceedings of AIAA

Infotech@AerospaceConference,Arlington,VA. 15

Wang T., Yu J., Siegel D., and Lee J., 2008, “A similaritybased prognostics approach for

remaining useful life estimation of engineered systems,” International Conference on Prognostics and HealthManagement,Denver,CO,Oct69. 16

ZioE.,andDiMaioF.,2010,“Adatadrivenfuzzyapproachforpredictingtheremaininguseful

life in dynamic failure scenarios of a nuclear power plant,” Reliability Engineering and System Safety, v95,n1,p49–57. 17

Coble J.B., and Hines J.W., 2008, “Prognostic algorithm categorization with PHM challenge

application,”IEEE,InternationalConferenceonPrognosticsandHealthManagement,Denver,CO,Oct6 9. 18

Heimes F.O., 2008, “Recurrent neural networks for remaining useful life estimation,” IEEE,

InternationalConferenceonPrognosticsandHealthManagement,Denver,CO,Oct69. 19

KozlowskiJ.D.,WatsonM.J.,ByingtonC.S.,GargaA.K.,andHayT.A.,2001,“Electrochemicalcell

diagnostics using online impedance measurement, state estimation and data fusion techniques,” Proceedingsof36thIntersocietyEnergyConversionEngineeringConference,Savannah,Georgia. 20

Goebel K., Eklund N., and Bonanni P., 2006, “Fusing Competing Prediction Algorithms for

Prognostics,”Proceedingsof2006IEEEAerospaceConference,NewYork. 21

SahaB.,GoebelK.,PollS.,andChristophersenJ.,2009,“Prognosticsmethodsforbatteryhealth

monitoringusingaBayesianframework,”IEEETransactiononInstrumentationandMeasurement,v58,

48

n2,p291–296. 22

GaoJ.,FanW.andHanJ.,“Onthepowerofensemble:Supervisedandunsupervisedmethods

reconciled,”TutorialonSIAMDataMiningConference(SDM),Columbus,OH,2010 23

BreimanL.,1996,“BaggingPredictors,”MachineLearning,v24,p123140

24

BreimanL.,2001,“RandomForests,”MachineLearning,v45,p532

25

SchapireR.E.,1990,“TheStrengthofWeakLearnability,”MachineLearning,v5,p197227

26

FreundY.,andSchapireR.E.,1997,“Adecisiontheoreticgeneralizationofonlinelearningand

anapplicationtoboosting”,Journalofcomputerandsystemsciences,v55,p119139 27

Friedman J.H. and Popescu B.C., 2008, “Predictive learning via rule ensembles,” The Annals of

AppliedStatistics,v2,n3,p916954 28

PerroneM.P.,andCooperL.N.,“Whennetworksdisagree:ensemblemethodsforhybridneural

networks,”NeuralNetworksforSpeechandImageProcessing.R.J.Mammone,ed.,ChapmanHall,1993. 29

BishopC.M.,2005,Neuralnetworksforpatternrecognition.OxfordUniversityPress.

30

ZerpaL.E.,QueipoN.V.,PintosS.,andSalagerJ.L.Zerpa,2005,“Anoptimizationmethodologyof

alkaline–surfactant–polymer flooding processes using field scale numerical simulation and multiple surrogates,”JournalofPetroleumScienceandEngineering,v47,n3–4,p197–208. 31

Goel T., et al., 2007, “Ensemble of surrogates,” Structural and Multidisciplinary Optimization,

v33,n3,p199–216. 32

Acar E., and RaisRohani M., 2009, “Ensemble of metamodels with optimized weight factors,”

StructuralandMultidisciplinaryOptimization,v37,n3,p279–294. 33

HuJ.,YangY.D.,andKiharaD.,2006,“EMD:anensemblealgorithmfordiscoveringregulatory

49

motifsinDNAsequences,”BMCbioinformatics,v7,n342. 34

ChenS.,WangW.,andZuylenH,2009,“Constructsupportvectormachineensembletodetect

trafficincident,”ExpertSystemswithApplications,v36,n8,p10976–10986. 35

BaraldiP.,RazaviFarR.,andZioE.,2011,“Classifierensembleincrementallearningprocedure

for nuclear transient identification at different operational conditions,” Reliability Engineering and SystemSafety,v98,n4,p480–488. 36

Evensen G., 2003, “The ensemble Kalman filter: Theoretical formulation and practical

implementation,”OceanDynamics,v53,n4,p343–367. 37

Niu G., Yang B., and Pecht M., 2010, “Development of an optimized conditionbased

maintenance system by data fusion and reliabilitycentered maintenance,” Reliability Engineering and SystemSafety,v95,n7,p786–796. 38

KohaviR.,1995,“Astudyofcrossvalidationandbootstrapforaccuracyestimationandmodel

selection,”ProceedingsoftheInternationalJointConferenceonArtificialIntelligenceIJCAI'95'. 39

Saxena A., and Goebel K., 2008, “Damage propagation modeling for aircraft engine runto

failuresimulation,”IEEE,InternationalConferenceonPrognosticsandHealthManagement,Denver,CO, Oct69. 40

LeeW.R.,JungS.W.,YangK.H.andLeeJ.S.,2005,“Astudyonthedeterminationofsubjective

vibration velocity ratings of main transformers under operation in nuclear power plants,” 12th InternationalCongressonSoundandVibration,Paper1017,Portugal,July1114. 41

Wang M., Vandermaar A. J., and Srivastava K. D., 2002, “Review of condition assessment of

powertransformersinservice,”IEEEElectricalInsulationMagazine,v18,n6,p12–25. 42

Ji S., Luo Y., and Li Y., 2006, “Research on Extraction Technique of Transformer Core

50

Fundamental Frequency Vibration Based on OLCM”, IEEE Transactions on Power Delivery, v21, n4, p1981–1988. 43

GarcíaB.,Member,BurgosJ.C.,andAlonsoÁ.M.,2006,“TransformerTankVibrationModeling

asaMethodofDetectingWindingDeformations—PartII:ExperimentalVerification,”IEEETransactions onPowerDelivery,v21,n1,p164–169. 44

Xu H., and Rahman S., 2005, “Decomposition methods for structural reliability analysis,”

ProbabilisticEngineeringMechanics,v20,n3,p239–250. 45

Gebraeel N.Z., Lawley M.A., Li R., and Ryan J.K., 2005, “Residuallife distributions from

component degradation signals: A Bayesian approach,” IIE Transactions on Reliability, v37, n6, p543– 557. 46

Kwon D., Azarian M., and Pecht M., 2008, “Detection of solder joint degradation using RF

impedanceanalysis,”IEEEElectronicComponentsandTechnologyConference,LakeBuenaVista,FL,27 30May,p606–610. 47

Smola A.J., and Schölkopf B., 2004, “A tutorial on support vector regression,” Statistics and

Computing,v14,n3,p199–222. 48

CanuS.,MaryX.,andRakotomamonjyA.,2003,“Functionallearningthroughkernel,”Advances

inLearningTheory:Methods,ModelsandApplications,v190,p89–110. 49

Tipping M.E., 2001, “Sparse Bayesian learning and the relevance vector machine,” Journal of

MachineLearningResearch,v1,p211–244. 50

CernanskyM.,MakulaM.,andCernanskyL.,2007,“Organizationofthestatespaceofasimple

recurrentnetworkbeforeandaftertrainingonrecursivelinguisticstructures,”NeuralNetworks,v20,n2, p236–244.

51

51

Tian X., 2006, “Cooling Fan Reliability, Failure Criteria, Accelerated Life Testing, Modeling, and

Quantification,”IEEE,AnnualReliabilityandMaintainabilitySymposium,NewportBeach,CA,Jan2326. 52

Burger R. Cooling tower technology—maintenance, updating and rebuilding. Fairmont Press;

1995. 53

LindelyD.V.,andSmithA.F.,“BayesEstimatesforLinearModels,”JournaloftheRoyalStatistical

Society(B),v34,n1,p1–41,1972. 

52