7RZDUGV$1RYHO,QWHJUDWHG1HXUR&RJQLWLYH$UFKLWHFWXUH,1&$  5LFKDUG-2HQWDU\R6WXGHQW0HPEHU,(((DQG0LFKHO3DVTXLHU 

$EVWUDFW²$UWLILFLDO LQWHOOLJHQFH UHVHDUFK LV QRZ IORXULVKLQJ ZKLFK DLPV DW DFKLHYLQJ JHQHUDO KXPDQOHYHO LQWHOOLJHQFH $FFRUGLQJO\FRJQLWLYHDUFKLWHFWXUHVDUHLQFUHDVLQJO\HPSOR\HG DVEOXHSULQWVIRUEXLOGLQJLQWHOOLJHQWDJHQWVWREHHQGRZHGZLWK YDULRXVSHUFHSWLYHDQGFRJQLWLYHDELOLWLHV7KLVSDSHUSUHVHQWVD QRYHO ,QWHJUDWHG 1HXUR&RJQLWLYH $UFKLWHFWXUH ,1&$  ZKLFK HPXODWHWKHSXWDWLYHIXQFWLRQDODVSHFWVRIYDULRXVVDOLHQWEUDLQ VXEV\VWHPV YLD D OHDUQLQJ PHPRU\ PRGHOLQJ DSSURDFK 7KH VWUHQJWK RI,1&$OLHVLQVHOIRUJDQL]LQJ FRQQHFWLRQLVWOHDUQLQJ WR LQGXFH KLJKOHYHO V\PEROLF NQRZOHGJH DXWRQRPRXVO\ DQG VXSSRUWIRUPHWDFRJQLWLYHIXQFWLRQV,WVRYHUDOORSHUDWLRQVDUH JRYHUQHGE\LWVFRQVROLGDWLRQDQGLQIHUHQFHF\FOHVZKLFKSRVLW DKXPDQSODXVLEOHZD\IRUIRUPLQJDQGH[SORLWLQJNQRZOHGJH

, ,1752'8&7,21

7

+( XOWLPDWH JRDO RI $UWLILFLDO ,QWHOOLJHQFH $,  LV WR FUHDWHDQDXWRQRPRXVV\VWHPWKDWFDQRSHUDWHLQYDULHG GRPDLQVDQG VROYH FRPSOH[ SUREOHPV FRPSHWHQWO\ LQD KXPDQOLNH PDQQHU >@ $V ERWK FRJQLWLYH VFLHQFH DQG $, H[SDQGHGKRZHYHU LWDSSHDUV WKDW WKLV REMHFWLYH KDV IDGHG DPRQJWKHP\ULDGRIVXEILHOGVIRFXVLQJRQYHU\VSHFLDOL]HG EXW KDUGO\ H[WHQVLEOH PHWKRGV @ 5HFHQW DGYDQFHV IURPQHXURVFLHQFHWR$,PDNHLWYLDEOHDJDLQWRFRQWHPSODWH EXLOGLQJ DQ LQWHJUDWHG IUDPHZRUN IRU JHQHUDO LQWHOOLJHQFH SRVVLEO\ E\ DPDOJDPDWLQJ YDULRXV H[LVWLQJ $, DSSURDFKHV ZKLOH LQFRUSRUDWLQJ QRYHO LGHDV VXFK DV IURP PRUH UHFHQW ELRORJLFDOO\LQVSLUHGPRGHOV 7KH QHHG IRU DQ LQWHJUDWHG IUDPHZRUN ZLWK HYHULQFUHD VLQJFDSDELOLWLHVKDVOHGRYHUWKH\HDUVWRWKHGHYHORSPHQWRI QXPHURXVFRJQLWLYHDUFKLWHFWXUHV%HLQJE\GHVLJQDJHQHULF EOXHSULQW IRU EXLOGLQJ LQWHOOLJHQW DJHQWV DQG V\VWHPV WKHVH DUFKLWHFWXUHV SOD\ DQ HVVHQWLDO UROH LQ WKH GHYHORSPHQW DQG YDOLGDWLRQRIFRPSXWDWLRQDOPRGHOVH[KLELWLQJWKHVDPHNLQG RIJHQHUDOLQWHOOLJHQFHDVWKDWRIKXPDQV&RJQLWLYHDUFKLWHF WXUHV RIIHU DQ LQIUDVWUXFWXUH IRU FUHDWLQJ LQWHOOLJHQW DELOLWLHV DW D KROLVWLF OHYHO WKDW LV IL[HG DFURVV D ZLGH YDULHW\ RI GRPDLQVDQGDSSOLFDWLRQVLQVKDUSFRQWUDVWZLWKFODVVLFDO$, PHWKRGV ZKLFK IRFXV RQ VSHFLILF DGKRF VROXWLRQV HJ H[SHUW V\VWHPV >@  $V VXFKWKH\ IRUP DQ HVVHQWLDO SDUW LQ WKHHQGHDYRUWRDFKLHYHJHQHUDOKXPDQOHYHOLQWHOOLJHQFH 7ZREDVLFSURSHUWLHVWKDWXQGHUOLHWKHGHVLJQRIFRJQLWLYH DUFKLWHFWXUHVDUHPHPRU\DQGOHDUQLQJDVZLGHO\HVWDEOLVKHG LQSV\FKRORJLFDODQGQHXURVFLHQWLILFVWXGLHV>@0HPRU\ UHIHUVWRWKHUHSRVLWRU\IRUEDFNJURXQGNQRZOHGJHDERXWWKH ZRUOGDQGRQHVHOIZKLOHOHDUQLQJLVWKHSURFHVVWKDWVKDSHV WKLV NQRZOHGJH >@ 7RJHWKHU WKH\ IRUP WKH LQGLVSHQVDEOH VXEVWUDWHVRIFRJQLWLRQRQZKLFKKLJKHURUGHUIXQFWLRQVDQG  7KH DXWKRUV DUH ZLWK WKH &HQWUH IRU &RPSXWDWLRQDO ,QWHOOLJHQFH &L  6FKRRO RI &RPSXWHU (QJLQHHULQJ 1DQ\DQJ 7HFKQRORJLFDO 8QLYHUVLW\ 1DQ\DQJ$YHQXH6LQJDSRUHHPDLODVPESDVTXLHU#QWXHGXVJ 

LQWHOOLJHQWFDSDELOLWLHV²VXFKDVSODQQLQJH[HFXWLYHFRQWURO DQGFUHDWLYLW\²DUHEXLOW$IXQGDPHQWDOWD[RQRP\PD\DOVR EHGHYLVHGEDVHGXSRQWKHVHWZRSURSHUWLHVGLYLGLQJYDULRXV GHVLJQDSSURDFKHVWRFRJQLWLYHDUFKLWHFWXUHVLQWRWKUHHPDMRU JURXSVV\PEROLFHPHUJHQWDQGK\EULGPHWKRGV>@ 6\PEROLF DUFKLWHFWXUHV VXFK DV 6WDWH 2SHUDWRU $QG 5HVXOW 62$5  > @ DQG ,&$586 > @ DUH W\SLFDOO\ GHULYHG IURP D WRSGRZQ DQDO\WLFDO $, DSSURDFK RSHUDWLQJ RQDEVWUDFWV\PEROV RUGHFODUDWLYHNQRZOHGJH7KLVPHWKRG HPSKDVL]HV WKH H[LVWHQFH RI D FHQWUDOL]HG RU KRPXQFXOXV OLNHFRQWURORYHUWKHLQIRUPDWLRQIORZIURPVHQVRU\LQSXWVWR PRWRU RXWSXWV DQG DFFHVV WR VHPDQWLF PHPRU\ IRU NQRZ OHGJHUHWULHYDO6\PEROLFDUFKLWHFWXUHVSRVVHVVWKHDELOLW\WR SURFHVV LQIRUPDWLRQ DQG WR FDSWXUH KLJKOHYHO FRJQLWLYH IXQFWLRQV LQ D ZD\ DQDORJRXV WR KXPDQV +RZHYHU WKH\ W\SLFDOO\ODFNWKHPHDQVWROHDUQV\PEROLFHQWLWLHVIURPORZ OHYHO LQIRUPDWLRQ WR KDQGOH ODUJH DPRXQWV RI LQIRUPDWLRQ DQGFRSHZHOOZLWKXQFHUWDLQW\2WKHUFKDOOHQJHVLQWKLVOLQH FRQFHUQV\VWHPVFDODELOLW\DQGSDUDOOHOSURFHVVLQJDELOLWLHV %\FRQWUDVWHPHUJHQWDUFKLWHFWXUHVVXFKDV&RQIDEXODWLRQ > @ DQG WKH ,QWHJUDWHG %LRORJLFDOO\EDVHG &RJQLWLYH $UFKLWHFWXUH,%&$ >@DUHLQVSLUHGE\WKHERWWRPXS FRQQHFWLRQLVW PRGHO RI ORZOHYHO DFWLYDWLRQ VLJQDOV IORZLQJ WKURXJK D QHWZRUN RI SURFHVVLQJ QRGHV 7KHVH QRGHV FDQ VHOIRUJDQL]H DQG LQWHUDFW ZLWK HDFK RWKHU LQ VSHFLILF ZD\V PRGLI\WKHLULQWHUQDOVWDWHVDQGFDSWXUHSURSHUWLHVRILQWHUHVW (PHUJHQWDUFKLWHFWXUHVDUHFDSDEOHRIPRGHOLQJWKHFRQWH[W VSHFLILFLW\ RI KXPDQ SHUIRUPDQFH DQG KDQGOLQJ ORZOHYHO LQIRUPDWLRQ FRQFXUUHQWO\ @ (DFK FDQ UHPHG\ WKH OLPLWDWLRQV RI WKH RWKHU HQDEOLQJ D SULRULWKHFUHDWLRQRIDFRPSUHKHQVLYHDUFKLWHFWXUHFRYHULQJ DOOOHYHOVRISURFHVVLQJIURPVWLPXOLWRKLJKOHYHOFRJQLWLRQ 1XPHURXV K\EULG DUFKLWHFWXUHV KDYH WKXV EHHQ SURSRVHG VXFK DV $GDSWLYH &RPSRQHQWV RI 7KRXJKW5DWLRQDO $&7 5 >@/HDUQLQJ,QWHOOLJHQW'LVWULEXWLRQ$JHQW/,'$  > @ DQG &RQQHFWLRQLVW /HDUQLQJ $GDSWLYH 5XOH ,QGX FWLRQ21OLQH&/$5,21 >@+RZHYHUPRVWRIWKHVH DUFKLWHFWXUHV DUH VWLOO ODFNLQJ VFDODELOLW\ IRU HPXODWLQJ WKH IXQFWLRQV RI WKH EUDLQ DQG DXWRQRPRXV VHOIRUJDQL]DWLRQ FDSDELOLW\ SDUWLFXODUO\ IRU DXWRPDWLFDOO\ FRQVWUXFWLQJ WKH UHOHYDQWPHPRU\VWUXFWXUHVZLWKRXWDSULRULNQRZOHGJH $QRYHO,QWHJUDWHG1HXUR&RJQLWLYH$UFKLWHFWXUH,1&$  LVSURSRVHGWRDGGUHVVVRPHRIWKHDERYHLVVXHVDQGSRVVLEO\ FRPSOHPHQWRUVXEVXPHYDULRXVH[LVWLQJFRJQLWLYHDUFKLWHF WXUHV,1&$LVGHVLJQHGWRHPXODWHWKHSXWDWLYHPDFURVFRSLF

1902 c 978-1-4244-1821-3/08/$25.002008 IEEE

  IXQFWLRQV RI WKH SULQFLSDO EUDLQ VXEV\VWHPV XVLQJ K\EULG V\PEROLFQHXUDO PRGHOV HDFK IHDWXULQJ IXOO\ DXWRQRPRXV VWUXFWXUDO DQG SDUDPHWHU VHOIRUJDQL]LQJ PHFKDQLVPV WR LQGXFHH[SOLFLWNQRZOHGJHIURPVHQVRU\GDWD7KHGHVLJQRI ,1&$ LV DOVR H[WHQVLEOH WR PHWDFRJQLWLYH FDSDELOLWLHV VXFK DV DWWHQWLRQ VKLIWLQJ DIIHFW PRGXODWLRQ DQG H[HFXWLYH UHJXODWLRQ ZKLFK DLG WR RUFKHVWUDWH DQG HQKDQFH RQJRLQJ FRJQLWLYH SURFHVVHV 7KH KROLVWLF RSHUDWLRQV RI WKH ,1&$ PRGHOVWHPIURPWKHLQWHUDFWLRQVRILWVFRQVWLWXHQWPRGXOHV LQWKHIRUPVRIFRQVROLGDWLRQDQGLQIHUHQFHF\FOHVWRJHWKHU RIIHULQJ D SODXVLEOH FRPSUHKHQVLYH SLFWXUH RI WKH KXPDQ NQRZOHGJHDFTXLVLWLRQDQGUHWULHYDOSURFHVVHVUHVSHFWLYHO\ 7KLV SDSHULV RUJDQL]HG DV IROORZV 6HFWLRQ ,, GHWDLOV WKH PRGXOHV DQG VXSSRUWLQJ SURWRFROV LQFRUSRUDWHG LQ ,1&$ IROORZHGE\LWVWZRPDLQRSHUDWLRQF\FOHVLQVHFWLRQ,,,7KH FRJQLWLYHSODXVLELOLW\RI,1&$DQGDFRPSDULVRQZLWKRWKHU HVWDEOLVKHGDUFKLWHFWXUHVDUHGLVFXVVHGLQVHFWLRQV,9DQG9 UHVSHFWLYHO\6HFWLRQ9,ILQDOO\FRQFOXGHVWKHSDSHU ,, 287/,1(2)7+(,1&$)5$0(:25. $VLOOXVWUDWHGLQ)LJ,1&$FRQVLVWVRIVHYHUDOVSHFLDOL]HG VXEV\VWHPV WKH VHQVRU\ DQG PRWRU UHJLVWHUV D WUDQVLHQW HSLVRGLF  PHPRU\ GLVWLQFW GHFODUDWLYH DQG SURFHGXUDO PHPRULHV DQ H[HFXWLYH PDQDJHU DQG DQ DIIHFW PRGXODWRU $OWKRXJKHDFKPRGXOHFRUUHVSRQGVWRDVDOLHQWUHJLRQRIWKH EUDLQ DV VKRZQ  QRQH LV GHVLJQHG WR PRGHO LWV ELRORJLFDO GHWDLOVZKLFKDUHWRRFRPSOH[DQGLPSUDFWLFDO,QVWHDGWKH PRGXOHV DLP WR UHDOL]H IXQFWLRQDO DVSHFWV RI WKH UHVSHFWLYH EUDLQDUHDVXVLQJK\EULGQHXUDOV\PEROLF OHDUQLQJPHPRU\ PRGHOV 6HYHUDO EDVLF SURWRFROV DUH DOVR LQFOXGHG LQ ,1&$ IRULQWHUPRGXOHFRPPXQLFDWLRQDFRQVROLGDWLRQDOJRULWKP DQ DWWHQWLRQ IRFXV SURFHGXUH DQG D GHFLVLRQ FRPELQHU PH WKRG7KHVHPRGXOHVDQGSURWRFROVDUHGHWDLOHGKHUHDIWHU $ $UFKLWHFWXUH0RGXOHV 7KHVHQVRU\DQGPRWRUUHJLVWHUVLQ,1&$FRQVWLWXWHWKHWZR HOHPHQWDU\LQWHUIDFHVIRUUHFHLYLQJVHQVRU\LQSXWVIURPDQG FDUU\LQJRXWUHOHYDQWPRWRUFRPPDQGVLQWKHH[WHUQDOZRUOG UHVSHFWLYHO\,QKHUHQWO\WKHVHUHJLVWHUVLPSOHPHQWEDVLFSUH RU SRVWSURFHVVLQJ SURFHGXUHV IRU VLPXODWLQJ WKH IXQFWLRQDO QRWELRORJLFDO DVSHFWVRIWKHUHDFWLYHSULPLWLYHSHUFHSWXDO DQGDFWLRQV\VWHPVLQWKHEUDLQUHVSHFWLYHO\>@2XUSUHVHQW DVVXPSWLRQ LVWKDWWKH VHQVRU\ DQGPRWRUUHJLVWHUVKDYH WKH LQQDWHDELOLW\WRH[WUDFWUHOHYDQWVHQVRU\LQSXWVDQGUHOLDEO\ KROGDQGH[HFXWHPRWRUFRPPDQGVJHQHUDWHGE\,1&$ 0HDQZKLOH WKH WUDQVLHQW PHPRU\ LQKHUHQWO\ VHUYHV DV D NH\PHGLDWRUIRUWKHIRUPDWLRQRIORQJWHUPGHFODUDWLYHDQG SURFHGXUDOPHPRULHV,WHSLWRPL]HVERWKUROHVRIWKHKLSSR FDPSXVLQWKHEUDLQLQHQFRGLQJSHUFHSWXDOLQIRUPDWLRQWKDW FRQWULEXWHVWRRXUH[SHULHQFHVRUHSLVRGLFNQRZOHGJHDQGLQ SURJUHVVLYHO\OLQNLQJWKHPEDVHGRQWKHLUFRPPRQHOHPHQWV WR IRUP IOH[LEOH LQIHUHQWLDO PHPRULHV >@ 7R UHDOL]H WKLV PRGXOH D VHOIRUJDQL]LQJ QHXURIX]]\ OHDUQLQJ PHPRU\ PRGHOKDVEHHQGHYLVHG>@$NH\IHDWXUHRIWKLVV\VWHPLV WKDW OLNH WKH KLSSRFDPSXV LW FDQ UDSLGO\ OHDUQ IURP QRYHO VHQVRU\ LQSXWV DQG LQFUHPHQWDOO\ IRUPXODWH IURP ORZOHYHO GDWDKLJKOHYHOLQWXLWLYHOLQJXLVWLFUXOHV([LVWLQJORQJWHUP PHPRU\FRQWH[WPD\FRQYHUVHO\EHXVHGDVSULRUNQRZOHGJH WRGLUHFWRUH[SHGLWHOHDUQLQJRIWKHWUDQVLHQWPHPRU\

$IIHFW0RGXODWRU $P\JGDOD 

7UDQVLHQW0HPRU\ +LSSRFDPSXV 

$WWHQWLRQ)RFXV

([HFXWLYH0DQDJHU )URQWDO&RUWH[ 

&RQVROLGDWLRQ

&RPELQHU

6HQVRU\5HJLVWHU 7KDODPXV 

'HFODUDWLYH0HPRU\ 3RVWHULRU&RUWH[ 

3URFHGXUDO0HPRU\ &HUHEHOOXP%DVDO*DQJOLD 

0RWRU5HJLVWHU 0RWRU&RUWH[ 

(;7(51$/:25/' )LJ6WUXFWXUDORYHUYLHZRIWKHSURSRVHG,1&$IUDPHZRUN

7KH GHFODUDWLYH PHPRU\ LQ ,1&$ FDSWXUHV WKH JHQHUDO IDFWV DERXW WKH ZRUOG RU RQHVHOI WKH ³ZKDW´  WKDW DUH UHODWLYHO\ VWDWLF DOEHLW PRGLILDEOH  6LPLODU WR WKH WUDQVLHQW PHPRU\WKLVPRGXOHFDQLQGXFHOLQJXLVWLFUXOHVIURPVHQVR U\GDWD+RZHYHULQVWHDGRIDUDSLGOHDUQLQJLWFDUULHVRXWD VORZ ORFDOL]HG OHDUQLQJ DWRS VRPH UXOH QHLJKERUKRRG WKDW SRUWUD\V WKH UHODWLYHO\ VWDEOH PHPRULHV LQ WKH SRVWHULRU DVVRFLDWLRQ  FRUWH[ >@ KHOSLQJ FDSWXUH NH\ SURSHUWLHV RI WKHZRUOG$ORFDOL]HGQHXURIX]]\OHDUQLQJPHPRU\PRGHO KDVEHHQGHYHORSHGXSRQWKLVSULQFLSOHZKLFKH[KLELWVJRRG ORFDOJHQHUDOL]DWLRQDQGIDVWFRPSXWDWLRQ>@$QH[WHQVLRQ ZDV DOVR GHYLVHG WR HQKDQFH WKH PRGHO¶V VFDODELOLW\ DQG LQWHUSUHWDELOLW\FRPELQLQJDFRUUHODWLRQEDVHGILOWHULQJDOJR ULWKPDQGDUXOHUHGXFWLRQWHFKQLTXH>@ 7KH SURFHGXUDO PHPRU\ LV GHVLJQHG WR OHDUQ DQG UHWDLQ NQRZOHGJH DERXW ZRUNLQJ VNLOOV RU SURFHGXUHV WKH ³KRZ´  ,QRXUFRQWH[WSURFHGXUDOPHPRU\LVXWLOL]HGVSHFLILFDOO\WR UHJXODWHWKHJHQHUDWLRQRIPRWRUDFWLRQFRPPDQGV&RPSOH[ WUDMHFWRULHVRIPRYHPHQWDUHOHDUQWYLDDORFDOHUURUFRUUHF WLRQ PHWKRG RQ HPEHGGHG V\QDSWLF ZHLJKWV WKDW URXJKO\ FRUUHVSRQGVWRWKHIXQFWLRQDOUROHRIWKHFHUHEHOOXPRUEDVDO JDQJOLDLQWKHEUDLQ>@2XUDSSURDFKWRVLPXODWLQJWKLVUROH LVYLDDORFDOL]HGOHDUQLQJPHPRU\PRGHOVLPLODUWRWKDWRI WKH GHFODUDWLYH PHPRU\ > @ +RZHYHU WKH FXUUHQW PRGHO IRFXVHV RQ WKH IRUPDWLRQ DQG UHWULHYDO RI ORQJWHUP SURFHGXUDONQRZOHGJHLQVWHDGRIGHFODUDWLYHNQRZOHGJH 7KHPHWDFRJQLWLYHIXQFWLRQVRI,1&$DUHUHDOL]HGLQWKH DIIHFWPRGXODWRUDQGH[HFXWLYHPDQDJHUV\VWHPV7KH DIIHFW PRGXODWRU LV FRQFHUQHG ZLWK PRWLYDWLRQV DQG WKHLU LQWHUDF WLRQVWKH³ZK\´ DQGLWVIXQFWLRQLVWDLORUHGZLWKWKHH[HFX WLYH PDQDJHU 7KH UHOHYDQFH RI WKH DIIHFW PRGXODWRU WR RSHUDWLRQVLQ,1&$OLHVLQWKHIDFWWKDWLWSURYLGHVDFRQWH[W LQ ZKLFK JRDO VHWWLQJ DQGUHLQIRUFHPHQW DUH GHWHUPLQHG IRU WKHRWKHUVXEV\VWHPV&RJQLWLYHO\WKLVPRGXOHFDQEH YLH ZHG DV D IXQFWLRQDO DQDORJXH RI WKH OLPELF FLUFXLW LQ WKH EUDLQ HVSHFLDOO\ WKH DP\JGDOD ZKLFK SURYLGHV WKH JRDOV WR GLUHFWV\VWHPRSHUDWLRQVDQGPRGXODWHWKHPHPRU\FRQVROL GDWLRQSURFHVVHV>@$SODXVLEOHLPSOHPHQWDWLRQRIWKLV

2008 International Joint Conference on Neural Networks (IJCNN 2008)

1903

  PRGXOHLVYLDDJOREDOL]HGOHDUQLQJPHPRU\PRGHO>@ 7KHH[HFXWLYHPDQDJHURQWKHRWKHUKDQGLVUHVSRQVLEOH IRUDFWLYHPRQLWRULQJFRQWURODQGRUFKHVWUDWLRQRIFRJQLWLYH SURFHVVHVWRHQKDQFHRYHUDOOV\VWHPSHUIRUPDQFH7KLVUROH LVIXQFWLRQDOO\FRUUHODWHGZLWKWKHIURQWDOFRUWLFDOFLUFXLWVLQ ,QSXWV WKH EUDLQ > @ ,W PD\ FRPSULVH VHWWLQJ JRDOV IRU WKH HSLVRGLF SURFHGXUDODQG GHFODUDWLYH PHPRULHV FKDQJLQJ RU LQWHUUXSWLQJRQJRLQJSURFHVVHVLQWKHVHPRGXOHVWXQLQJWKH PRGXOH SDUDPHWHUV HWF 5HJXODWLRQ LV DOVR DFKLHYHG YLD VHWWLQJ UHLQIRUFHPHQW IXQFWLRQV RQ WKH EDVLV RI PRWLYDWLRQ VLJQDOVIURPWKHDIIHFWLYHPRGXODWRU)RUWKHVHSXUSRVHVWKH H[HFXWLYH PDQDJHU LQ ,1&$ LQFOXGHV D QXPEHU RI VXE PRGXOHV D JRDO VHWWLQJ PDQDJHUDSDUDPHWHUWXQLQJPDQD JHUDQGDUHLQIRUFHPHQWPHWULFVPDQDJHU(DFKFDQEHLP SOHPHQWHGDOJRULWKPLFDOO\>@RUYLDWKHORFDOL]HGOHDUQLQJ PHPRU\PRGHOGLVFXVVHGEHIRUH>@ % 6XSSRUWLQJ3URWRFROV 7KH DWWHQWLRQ IRFXVLQJ SURWRFRO VKRZQ LQ )LJ  HVVHQWLDOO\ LPSOHPHQWV WKH QRWLRQ RI WKH VHOHFWLYH DWWHQWLRQ LQ KXPDQ FRJQLWLRQ7KHJRDORIWKHSURWRFROLVWRILOWHURXWLUUHOHYDQW RUUHGXQGDQWVHQVRU\IHDWXUHVIRFXVLQJLQWXUQDOOFRJQLWLYH SURFHVVLQJRQWKHPRVWLQIRUPDWLYHRUXVHIXODWWULEXWHV)LO WHULQJLV W\SLFDOO\ DSSOLHG SULRUWR SURSDJDWLQJ VHQVRU\ GDWD LQWR WKH UHPDLQLQJ PRGXOHV WR UHGXFH WKH FRPSXWDWLRQDO FRPSOH[LW\ RI VXEVHTXHQW RSHUDWLRQV 7KLV FDQ EH PRGHOHG LQ,1&$XVLQJDUHOHYDQFHEDVHGHJFRUUHODWLRQLQIRUPD WLRQWKHRUHWLF  IHDWXUH ILOWHU >@ DQGRU DKLHUDUFKLFDO FOXV WHULQJDOJRULWKPHJGHQGURJUDPEDVHG >@ 7KH FRPELQHU SURWRFRO LQ ,1&$ DLPV WR LPSURYH RYHUDOO NQRZOHGJH H[SORLWDWLRQ DQG UHFDOO SHUIRUPDQFH E\ PHUJLQJ WKH GHFLVLRQ RXWSXWV IURP WKH WUDQVLHQW PHPRU\ DQG GHFODUDWLYH PHPRU\ $ JDWLQJYRWLQJ RU D OLQHDU ZHLJKWHG FRQMXQFWLYHRSHUDWRUFDQEHDSSOLHGWRHYDOXDWHWKHVDOLHQF\ HJ DYHUDJH ILULQJ IUHTXHQF\  RI HDFK PHPRU\ W\SH DQG FRPSXWHDFFRUGLQJO\WKHSURSHUFRPELQHGGHFLVLRQRXWSXWV 7KHVH RXWSXWV SURYLGH LQ WXUQ D GHOLEHUDWLYH FRQWH[W WKDW LV WUDQVPLWWHG WR WKH SURFHGXUDO PHPRU\ LQ RUGHU WR JXLGH LWV PRWRUFRRUGLQDWLRQRSHUDWLRQV 7KHGHVLJQRIWKHFRQVROLGDWLRQSURWRFROLQ,1&$DLPVDW VLPXODWLQJWKHJUDGXDOWUDQVIHURIVDOLHQWUHJXODUO\DFFHVVHG NQRZOHGJHLWHPVLQWKHWUDQVLHQWPHPRU\LQWRWKHORQJWHUP VWRUDJH2XUPRGHOLQJDSSURDFKLVVSHFLILFDOO\EDVHGRQWKH QRWLRQ RI FRPSOHPHQWDU\ LQWHUOHDYHG OHDUQLQJ EHWZHHQ WKH WUDQVLHQW KLSSRFDPSDO PHPRU\ DQG WKH ORQJWHUP FRUWLFDO PHPRU\V\VWHPVLQWKHEUDLQ>@$NH\EHQHILWRIVXFK OHDUQLQJLVWKDWLWHQDEOHVWKHGHFODUDWLYHH[HFXWLYHPHPRU\ LHFRUWH[DQDORJXH WREHRSWLPL]HGIRUJUDGXDOGLVFRYHU\ RIWKHVKDUHGVWUXFWXUHVRIHYHQWVDQGH[SHULHQFHVZKLOHDOVR PDNLQJ WKH WUDQVLHQW PHPRU\ LH KLSSRFDPSXV DQDORJXH  DOZD\V DYDLODEOH IRU DFTXLULQJ QRYHO LQIRUPDWLRQ ZLWKRXW LQWHUIHULQJ SUHYLRXVO\ OHDUQW NQRZOHGJH ,Q WKLV ZD\ WKH LVVXHV UHODWHG WR PHPRU\ GLVUXSWLRQV DQGRU FDWDVWURSKLF IRUJHWWLQJFDQEHDGGUHVVHGHOHJDQWO\>@ 7KHFRQVROLGDWLRQSURWRFROLV FHQWUDOWRWKHFRQVROLGDWLRQ F\FOHLQ,1&$GHVFULEHGLQVHFWLRQ,,,7KLVSURWRFROLVEXLOW RQWRSRIDFRQVROLGDWLRQQHWZRUNGHSLFWHGLQ)LJZKLFK FRPSULVHV WKH ORQJWHUP GHFODUDWLYHH[HFXWLYH  PHPRU\ UHDOL]HG LQ ,1&$ XVLQJ D VORZ ORFDOL]HG OHDUQLQJ PHPRU\

1904

'HFODUDWLYH([HFXWLYH  0HPRU\   /5       /5

/5

$/

&/

2XWSXWV

/5

'ULYHVLJQDOV )URPDIIHFW PRGXODWRU 

$/

7HDFK VLJQDOV

 $/

&/

65

65

65

65

7UDQVLHQW 0HPRU\

)LJ6WUXFWXUH RI WKH FRQVROLGDWLRQ QHWZRUNLQ,1&$/5  ORQJWHUP UXOH65 VKRUWWHUPUXOH$/ DQWHFHGHQWODEHO&/ FRQVHTXHQWODEHO 

PRGHO > @ DQG WKH WUDQVLHQW PHPRU\ UHDOL]HG XVLQJ D UDSLG JOREDOL]HG PRGHO >@ %RWK PRGHOV FRQWDLQ DVVRFLDWLYH UXOHV WKDW PDS D VHW RI DQWHFHGHQW OLQJXLVWLF ODEHOV WR VRPH FRQVHTXHQW ODEHOV ,Q WKLV QHWZRUN WKH WUDQVLHQW PHPRU\ LV UHVSRQVLEOH IRU UDSLGO\ FDSWXULQJ WKH UHSUHVHQWDWLRQVRIQHZLWHPVDQGSURJUHVVLYHO\WHDFKLQJWKH ORQJWHUPPHPRU\DWWKHVDPHWLPH7KLVVXEVHTXHQWO\OHDGV WRDQLQWHUOHDYHGOHDUQLQJUHJLPHQWKDWEOHQGVUHSHWLWLRQVRI WKH ROG DQG QHZ UHSUHVHQWDWLRQV LQ WKH ORQJWHUP PHPRU\ HQDEOLQJWKHFRQVROLGDWLRQQHWZRUNWRJDLQQHZLQIRUPDWLRQ ZKLOHPLQLPL]LQJFDWDVWURSKLFLQWHUIHUHQFH ,,, 23(5$7,21&<&/(6 7KLVVHFWLRQGHVFULEHVWKHFRQVROLGDWLRQDQGLQIHUHQFHF\FOHV WKDWIRUPWKHPDFUROHYHORSHUDWLRQVDQGKROLVWLFEHKDYLRURI WKH ,QWHJUDWHG 1HXUR&RJQLWLYH $UFKLWHFWXUH ,1&$  )RU LOOXVWUDWLRQ SXUSRVH DQ RSHQHQGHG VFHQDULR RI DXWRPDWHG GULYLQJVNLOODFTXLVLWLRQE\DQLQWHOOLJHQWYHKLFOHLVSURYLGHG WRJHWKHU ZLWKWKH GHVFULSWLRQV RI WKHWZR F\FOHV 3UHYLRXVO\ SXEOLVKHGZRUNIRFXVHGRQWKHSURFHGXUDOPHPRU\RI,1&$ HJWROHDUQEDVLFSDUNLQJDQGGULYLQJPDQHXYHUV >@ 7KLV ZRUN LV FXUUHQWO\ EHLQJ H[WHQGHG WR PRUH FRPSOH[ RSHUDWLRQVDQGHYHQWDFWLFDOGHFLVLRQPDNLQJ,QYHVWLJDWLRQV DUHDOVRXQGHUZD\WRHPSLULFDOO\PRGHODWWHQWLRQIRFXVDQG LWV LPSDFW RQ WKH FRQVROLGDWLRQSURFHVV WKDW \LHOGV OLIHORQJ GULYLQJVNLOOH[SHUWLVH0HDQZKLOHWKHVFHQDULRLQWKLVSDSHU SUHVHQWV VHYHUDO SODXVLEOH WHVW FDVHV WKDW UHTXLUH WKH LQWHJUDWLRQ RI KLJKHURUGHU FRJQLWLYH IXQFWLRQV WR UHDOL]H PRUHVRSKLVWLFDWHGGULYLQJVNLOOVVLPLODUWRKXPDQV¶ ,QWKLVH[DPSOHWKHHPERGLHG,1&$EDVHGGULYLQJDJHQW RSHUDWHVLQDYLUWXDOWUDIILFZRUOGDQGKDVLQQDWHVHQVHVDQG

2008 International Joint Conference on Neural Networks (IJCNN 2008)

  HIIHFWRUVWKDWFRUUHVSRQGWRLWVVHQVRU\DQGPRWRUUHJLVWHUV UHVSHFWLYHO\ 7KHVSDFHWLPHRIWKHZRUOGLVFRQWLQXRXV,Q SDUWLFXODUWKHVHQVRU\UHJLVWHULPSOHPHQWVVHYHUDOSULPLWLYH VHQVLQJIXQFWLRQVWRH[WUDFWIHDWXUHVRILQWHUHVWIURPWKHHQ YLURQPHQW HJ HGJH GHWHFWLRQ DOJRULWKP IRU YLVXDO LQSXWV ORZSDVV QRLVH ILOWHU IRU DXGLWRU\ LQSXWV  7KH HIIHFWRUV RQ WKHRWKHUKDQGDUHLPSOHPHQWHGLQWKHPRWRUUHJLVWHU,QWKLV FDVH WKH PRGXOH LV UHVSRQVLEOH IRU VHQGLQJ RXW PRWLRQ FRQWURO VLJQDOV WR WKH H[WHUQDO ZRUOG HJ VWHHULQJ ZKHHO DQJOHEUDNHDFFHOHUDWLRQLQWHQVLW\JHDUVKLIWLQJ  $ &RQVROLGDWLRQ&\FOH 7KHNQRZOHGJHFRQVROLGDWLRQF\FOHRI,1&$FRQVWLWXWHVWKH SULPDU\LQWHUQDOUHKHDUVDOSURFHVVLQWKHV\VWHPIRUIRUPLQJ DQGFU\VWDOOL]LQJLWVNQRZOHGJH EDVH7KLVF\FOH RFFXUVDWD UHFXUULQJWLPHDOORZLQJDFFHVVRINQRZOHGJHWKDWLVLQLWLDOO\ GHSHQGHQWRQWKHWUDQVLHQWPHPRU\WKDWOHDUQVIDVWEXWKDVD OLPLWHGFDSDFLW\ WREHFRPHJUDGXDOO\UHOLDQWXSRQWKHORQJ WHUPGHFODUDWLYHH[HFXWLYH PHPRU\WKDWOHDUQVVORZO\ \HW KDVDODUJHFDSDFLW\ $VFKHPDWLFGLDJUDPVXPPDUL]LQJWKH VWHSVLQWKHFRQVROLGDWLRQF\FOHLVSUHVHQWHGLQ)LJZKHUH WKH,1&$PRGXOHVLQYROYHGDWHDFKVWHSDUHOLVWHGLQEXOOHWV 7KHGHWDLOVRIHDFKVWHSDUHJLYHQLQWXUQDVIROORZV   ,QLWLDOL]DWLRQ 7KH EDVLF DUFKLWHFWXUH RI WKH ORQJWHUP PHPRU\ LQ ,1&$ LV ILUVW IRUPXODWHG XVLQJ D FRPSHWLWLYH OHDUQLQJ >@ DQG D FRUUHODWLRQ OHDUQLQJ PHFKDQLVP >@ ZKLFK EXLOG WKH URXJK LQLWLDO VWUXFWXUHV RI WKH DQWHFHGHQW  ODEHOVDQGUXOHFHOOVUHVSHFWLYHO\7KH IRUPHUEHJLQVZLWKD ODUJHQXPEHURIODEHOQHXURQVDQGWKHQUHPRYHVWKRVH ZLWK ORZVLJQLILFDQFHGHJUHHVWKHVHFRQGDOORFDWHVPHPRU\VSDFH IRU WKH LQLWLDO UXOH FHOOV EDVHG XSRQ WKH VXUYLYLQJ ODEHO QHXURQV 7KLV SURFHVV UHVHPEOHV WKH LQLWLDO GHYHORSPHQW VWDJHRIRXUFRUWLFDOV\VWHPVZKHUHFRDUVHPHPRU\VWUXFWXUH LV ILUVW ODLG RXW E\ DFWLYLW\LQGHSHQGHQW SURFHVV >@ $FWLYH QHXURQVWKHQ VWDELOL]H WKURXJK WKH XSWDNH RI WURSKLF IDFWRUV ZKLOH WKH XQVXFFHVVIXO QHXURQV GLH ,Q RXU GULYLQJ VNLOO OHDUQLQJH[DPSOHWKLVSURFHVVFRQFHUQVWKHLQLWLDOIRUPDWLRQ RILQSXWODEHOVVXFKDV0LOG6WHHS IRUWKHURDGFXUYDWXUHRU )DU 1HDU  IRU WKH GLVWDQFH WR REVWDFOH DQG PHPRU\ DOORFDWLRQIRUWKHFRUUHVSRQGLQJUXOHFHOOV   ,QWHUOHDYHG OHDUQLQJ  )RU HDFK GDWD VDPSOH WKH WUDQ VLHQWPHPRU\LQ,1&$SHUIRUPVDUDSLGRQHVKRWVWUXFWXUDO DQG SDUDPHWHU OHDUQLQJ WR FUHDWH DQGRU WXQH LWV NQRZOHGJH EDVH LH D VHW RI HSLVRGLF UXOHV DQG WKHLU VXSSRUWLQJ DQWHFHGHQW DQG FRQVHTXHQW ODEHOV  $ IRUZDUG LQIHUHQFH LV WKHQUXQDWRS WKLV XSGDWHG UXOH EDVH 7KH UHVXOWDQW RXWSXWV PRGXODWHG E\ GULYH VLJQDOV IURP WKH DIIHFW PRGXODWRU DUH WKHQVHQWWRDQGDFWDVDWHDFKHUIRUWKHORQJWHUPPHPRU\ 7KH ODWWHU WKHQ XQGHUJRHV D VORZ VWUXFWXUDO DQG SDUDPHWHU OHDUQLQJ WR FUDIW WKHUHOHYDQW OLQJXLVWLFUXOHV DQG XSGDWH WKH DQWHFHGHQW DQG FRQVHTXHQW ODEHOV ,QKHUHQWO\ WKLV SURFHVV H[HPSOLILHV WKH QH[W FRUWH[ GHYHORSPHQW SKDVH ZKHUH WKH LQLWLDOO\ FRDUVH PHPRU\ VWUXFWXUH LV UHILQHG E\ DFWLYLW\ GHSHQGHQW PHFKDQLVPV >@ ,Q RXU GULYLQJ H[DPSOH WKLV FRUUHVSRQGVWRWKHDFTXLVLWLRQRIORQJWHUPUXOHVVXFKDV³,) URDG FXUYDWXUH LV 6WHHS DQG GLVWDQFH WR REVWDFOH LV 1HDU 7+(1GHFLVLRQLVWR6ORZGRZQ´   5HRUJDQL]DWLRQ 6LPSOLILFDWLRQUHGXFWLRQ RI WKH OHDUQW NQRZOHGJH EDVH LQ WKH ORQJWHUP PHPRU\ LV DFKLHYHG E\

6WDUW

,QLWLDOL]DWLRQ • 'HFODUDWLYH([HFXWLYH0HPRU\

,QWHUOHDYHG/HDUQLQJ • 'HFODUDWLYH([HFXWLYH0HPRU\ • 7UDQVLHQW0HPRU\ • $IIHFW0RGXODWRU

5HRUJDQL]DWLRQ • 'HFODUDWLYH([HFXWLYH0HPRU\

&OHDUDQFH • 7UDQVLHQW0HPRU\

(QG )LJ&RQVROLGDWLRQF\FOHLQWKH,1&$IUDPHZRUN

GLVFDUGLQJ DQWHFHGHQW OLQNV RI WKH UXOH FHOOV ZKLFK GR QRW FRQWULEXWHVLJQLILFDQWO\WRWKHGHULYDWLRQRIWKHILQDORXWSXWV DQGSUXQLQJVXEVHTXHQWO\WKHGXSOLFDWHGREVROHWHUXOHVHJ WKDW ZLWK WKH VDPH VHW RI DQWHFHGHQW OLQNV  6RPH GHFLVLRQ FRQVLVWHQF\ PHDVXUH FDQ EH XVHG WR DFFRPSOLVK WKH WDVN HJFXUUHQWO\URXJKVHWWKHRU\>@ ,QWKLVZD\DVHW RI KLJKO\ LQWXLWLYH FRQFLVH UXOHV ZLWK ORZ UHGXQGDQF\ FDQ EH GHULYHG)RULQVWDQFHWKHVDPSOHGULYLQJUXOHVKRZQLQVWHS   VLPSOLILHV LQWR ³,) GLVWDQFH WR REVWDFOH LV 1HDU 7+(1 GHFLVLRQ LV WR 6ORZ GRZQ´ DV WKH URDG FXUYDWXUH EHFRPHV LQVLJQLILFDQWZKHQWKHGLVWDQFHWRREVWDFOHLVFULWLFDO)XUWKHU UHRUJDQL]DWLRQ RI WKH ORQJWHUP PHPRU\ PD\ LQYROYH D VHTXHQFLQJ RI WKH UXOH DFWLRQ VWHSV WR GHYLVH H[SOLFLW SODQV LQDFFRUGDQFHZLWK$,SODQQLQJPHWKRG DQGVHJPHQWDWLRQ RIDFWLRQVHTXHQFHVWREXLOWDKLHUDUFK\RIVXEVHTXHQFHVWKDW FDSWXUHV WKH KLHUDUFKLFDO VSDWLRWHPSRUDO VWUXFWXUH RI WKH ZRUOG >@ 7KHVH H[WHQVLRQV IRUP WKH NH\ PHFKDQLVPV IRU SUHGLFWLRQ DQG LPDJLQDWLRQ RQ ZKLFK QRYHO PHWDFRJQLWLYH FDSDELOLWLHV DUH EXLOW ,Q RXU GULYLQJ VFHQDULR WKHVH FDQ EH XVHIXO WR SURYLGH WKH LQWHOOLJHQWDJHQW ZLWK DQWLFLSDWLRQDQG QDYLJDWLRQFDSDELOLWLHV>@  &OHDUDQFH8SRQFRQVROLGDWLRQRIWKHNQRZOHGJHLQWKH ORQJWHUP PHPRU\ RI ,1&$ WKH FRQWHQW RI WKH WUDQVLHQW PHPRU\ LV GHOHWHG SDUWLDOO\ HJ EDVHG RQ FLUFXODU EXIIHU  PDNLQJ LW DYDLODEOH WR OHDUQQHZ LQIRUPDWLRQ 7KLV UHFWLILHV VXEVHTXHQWO\ WKH EDVLFUROH RI WKHKLSSRFDPSDO V\VWHP DV D WHPSRUDU\ PHGLDWRU ZLWKOLPLWHG VWRUDJHDQGKLSSRFDPSDO LQGHSHQGHQWSURFHVVLQJLQWKHFRUWH[DIWHUFRQVROLGDWLRQ

2008 International Joint Conference on Neural Networks (IJCNN 2008)

1905

  % ,QIHUHQFH&\FOH 7KHLQIHUHQFHF\FOHLQ,1&$EDVLFDOO\UHDOL]HVDVHULDOIORZ RIUHDVRQLQJSURFHVVHVLQYROYLQJDFRPSRVLWHLQWHUDFWLRQRI WKHFRQVWLWXHQWV\VWHPPRGXOHVVWDUWLQJIURPSHUFHSWLRQDQG HQGLQJZLWKDFWLRQ$VLPLODUF\FOHKDVEHHQQHXURORJLFDOO\ YDOLGDWHGLQKXPDQVZKLFKKDSSHQVDERXWILYHWRWHQWLPHVD VHFRQG FDVFDGLQJ VR WKDW VRPH VWHSV LQWKH DGMDFHQW F\FOHV DUHUXQLQSDUDOOHO>@7KLVSDUDGLJPURXJKO\FRUUHVSRQGV WR WKH 3HUFHLYH,QWHUSUHW$FW 3,$  F\FOH > @ ZKLFK GHVFULEHVWKHRYHUDOOSURFHGXUHVIRUNQRZOHGJHH[SORLWDWLRQ RU UHDVRQLQJ LQ KXPDQ FRJQLWLRQ DQG ZKLFK VKDUHV VRPH VLPLODULWLHVZLWKWKHQDWXUHRIV\VWHPHQJLQHHULQJOLIHF\FOHV $QRYHUYLHZRIWKHLQIHUHQFHF\FOHLQ,1&$LVJLYHQLQ)LJ ZKLFKFRQVLVWVRIWKUHHPDLQVWDJHVDVIROORZV   3HUFHSWLRQ 7KLV SKDVH LV FRQFHUQHG ZLWK D SULPLWLYH SUHFRQVFLRXVLQIRUPDWLRQSURFHVVLQJEDVHGRQVHQVHV0RUH VSHFLILFDOO\ LQSXW VWLPXOL IURP WKH H[WHUQDO ZRUOG DUH ILUVW NHSW DQG HQFRGHG LQ WKH VHQVRU\ UHJLVWHU RI ,1&$ $ EDVLF FRPSXWDWLRQDO DQDO\VLV LV SHUIRUPHG RQ VWLPXOL FRQWDLQLQJ YLVXDO DXGLWRU\ DQG RWKHU IRUPV RI SHUFHSWXDO LQIRUPDWLRQ LQRUGHUWRSUHSURFHVVDQGH[WUDFWDVHWRIDWWULEXWHVIHDWXUHV RI LQWHUHVW WR WKH FXUUHQW WDVN )RU LQVWDQFH WKH VHQVRU\ UHJLVWHU LQ RXU GULYLQJ H[DPSOH SHUIRUPV WDVNVSHFLILF HGJH GHWHFWLRQRQLWVYLVXDOLQSXWVWRH[WUDFWWKHURDGFXUYDWXUHDV ZHOODVDFWLYHYLVLRQEDVHGGHSWKHVWLPDWLRQWRFRPSXWHWKH GLVWDQFHWRWKHQHDUHVWREMHFWDKHDGDQGVRRQ  ,QWHUSUHWDWLRQ7KLVVWDJHHQFDSVXODWHVWKHLQWHUPHGLDWH SURFHVVHVEHWZHHQSHUFHSWLRQDQGDFWLRQZKLFKFRQVWLWXWHLQ WXUQWKHFRUHLQIRUPDWLRQSURFHVVLQJVWHSVLQKXPDQVDQGLQ ,1&$ 7KH LQWHUSUHWDWLRQ SKDVH LV IXUWKHU GLYLGHG LQWR D QXPEHURIVWHSVHODERUDWHGKHUHDIWHU D  $WWHQWLRQ IRFXV )LOWHULQJ LV SHUIRUPHG RQ WKH IHDWXUHV H[WUDFWHG LQ WKH SHUFHSWLRQ SKDVH EDVHG XSRQ WKH DWWHQWLRQ IRFXVSURWRFROJLYHQLQVHFWLRQ,,%DQGXQGHUWKHJXLGDQFH RI WKH H[HFXWLYH PDQDJHU 7KLV SURFHVV FDQ EH YLHZHG DV D IRUPRIFRPSHWLWLRQIRUDWWHQWLRQEDVHGRQKRZLQIRUPDWLYH HDFKIHDWXUHRUVXEVHWRIIHDWXUHV LVDVFRPSDUHGWRRWKHUV 7KH VHOHFWHG LQSXW IHDWXUHV DUH WKHQ EURDGFDVW DV FXHV IRU NQRZOHGJHUHWULHYDODQGIXUWKHUSURFHVVLQJLQWKHUHPDLQLQJ ,1&$PRGXOHV2QHLQVWDQFHRIDWWHQWLRQIRFXVSURFHGXUHLQ RXU GULYLQJ VFHQDULR FRQFHUQV WKH GHFLVLRQ RI ZKHWKHU WR DWWHQGWRWKHYLVXDOLQSXWVIURPWKHOHIWULJKWRUUHDUPLUURU ZKHQLQLWLDWLQJDODQHFKDQJLQJPDQHXYHU E 0HWDFRJQLWLYHUHJXODWLRQ,QWKLVSKDVHWKHH[HFXWLYH PDQDJHUDQGDIIHFWPRGXODWRULQ,1&$ZRUNMRLQWO\LQRUGHU WR FRQILJXUH DQG RUFKHVWUDWH WKH RSHUDWLRQV RI WKH WUDQVLHQW GHFODUDWLYHDQGSURFHGXUDOPHPRU\PRGXOHV7KLVPD\EHLQ WKHIRUPRIVHWWLQJXSDJRDOFRQWH[WIRUWKHWKUHHPRGXOHV UHURXWLQJ WKHLU FXUUHQW WDVNDOWHULQJWKHLUOHDUQLQJLQIHUHQFH PRGH RU WXQLQJ WKHLU V\VWHP SDUDPHWHUV 7KHVH SURYLGH LQ WXUQ D PHFKDQLVP HQDEOLQJ WKH DJHQW WR EHFRPH VHOIDZDUH RI DQG LPSURYH DFFRUGLQJO\ LWV FXUUHQW SHUIRUPDQFH ,Q RXU GULYLQJVFHQDULRWKHVHFRUUHVSRQGWRYDULRXVPHFKDQLVPVDW WKH WDFWLFDO OHYHO HJ VZLWFKLQJ PDQHXYHUV PRGLI\LQJ WKH UHZDUGIXQFWLRQVIRUJXLGLQJRWKHUPRGXOHVHWF  F $VVRFLDWLRQUHWULHYDO7KHDVVRFLDWLYHNQRZOHGJHOHDUQW E\ ,1&$ WKURXJK WKH FRQVROLGDWLRQ F\FOH  LQ WKH WUDQVLHQW DQGGHFODUDWLYHPHPRULHVLVUHWULHYHGEDVHGRQWKHVWLPXOXV FXHV ILOWHUHG E\ WKH DWWHQWLRQ IRFXV SURWRFRO DQG VLJQDOV

1906

6WDUW

3HUFHSWLRQ • 6HQVRU\5HJLVWHU

,QWHUSUHWDWLRQ

 D$WWHQWLRQIRFXV • 6HQVRU\5HJLVWHU • ([HFXWLYH0DQDJHU

E0HWDFRJQLWLYHUHJXODWLRQ • ([HFXWLYH0DQDJHU • $IIHFW0RGXODWRU

F$VVRFLDWLRQUHWULHYDO • 7UDQVLHQW0HPRU\ • 'HFODUDWLYH0HPRU\

$FWLRQ

 D$FWLRQVHOHFWLRQ • 3URFHGXUDO0HPRU\

E0RWRUH[HFXWLRQ • 0RWRU5HJLVWHU

(QG )LJ,QIHUHQFHF\FOHLQWKH,1&$IUDPHZRUN

IURPWKHH[HFXWLYHPDQDJHUDQGDIIHFWPRGXODWRU7KLVPD\ EH DFKLHYHG DW GLIIHUHQW FRPSOH[LW\ OHYHOV XVLQJ HLWKHU D EDVLF UHDFWLYH LQIHUHQFH SURFHGXUH WKDW JHQHUDWHV GHFLVLRQ RXWSXWV E\ GLUHFWO\ LQYRNLQJ DVVRFLDWLRQ UXOHV  RU D GHOLEHUDWLYH LQIHUHQFH SURFHVV WKDW SODQV GHFLVLRQDFWLRQ VHTXHQFHVOHDGLQJWRWKHJRDO $QH[DPSOHRIWKHIRUPHULV WKHILULQJRIGHFLVLRQUXOHVIRUVZLWFKLQJODQHZKHQWKHUHLVD VORZYHKLFOHDKHDG>@ZKLOHDQLQVWDQFHRIWKHODWWHULVRQ URDGQDYLJDWLRQEDVHGRQDFRPSRVLWHVHTXHQFHRIGHFLVLRQV VXFKDVZKHQQHJRWLDWLQJDKLJKZD\H[LW   $FWLRQ 7KLV SKDVH FRPSULVHV WKH JHQHUDWLRQ RI WKH H[WHUQDODFWLRQVWKDWDIIHFWWKHDJHQW¶VVLWXDWLRQLQWKHZRUOG DQGWKHQH[WLQWHUQDORSHUDWLRQVRI,1&$UHVSHFWLYHO\7KH SKDVHFRPSULVHVWZRVWHSVDVIROORZV D  $FWLRQ VHOHFWLRQ 7KH GHFLVLRQ RXWSXWV IURP WUDQVLHQW

2008 International Joint Conference on Neural Networks (IJCNN 2008)

  DQGGHFODUDWLYHPHPRULHVDUHLQWHJUDWHGXVLQJWKHFRPELQHU SURWRFRO DV GLVFXVVHG LQ VHFWLRQ ,,% 7KH FRPELQHG GHFL VLRQ SRVVLEO\ WRJHWKHU ZLWK WKH ILOWHUHG VWLPXOL IURP WKH DWWHQWLRQIRFXVSURWRFRODQGJXLGLQJVLJQDOVIURPWKHH[HFX WLYH PDQDJHUDUH VXEVHTXHQWO\ IRUZDUGHG WR WKH SURFHGXUDO PHPRU\7KHODWWHUWKHQLGHQWLILHVLQLWVNQRZOHGJHEDVHSUR FHGXUDO UXOHV PDWFKLQJ WKHVH LQSXWV DQG JHQHUDWHV WKH FRUUHVSRQGLQJPRWRURXWSXWV)RUH[DPSOHLIWKHFRPELQHG GHFLVLRQLQVWUXFWVWKHYHKLFOHWRVZLWFKWRWKHULJKWODQHDQG FXUUHQWVWLPXOLLQGLFDWHWKDWWKHODQHLVYDFDQWWKHUXOHVIURP SURFHGXUDOPHPRU\ZRXOGLQGLFDWHVWHHULQJWKHZKHHOWRWKH ULJKWDQGSUHVVLQJEUDNHDFFHOHUDWRUSHGDOVDSSURSULDWHO\ E  0RWRU H[HFXWLRQ7KH PRWLRQ FRQWURO VLJQDOV IURP WKH SURFHGXUDOPHPRU\DUHWKHQSURSDJDWHGWRWKHPRWRUUHJLVWHU IRU H[HFXWLRQ 7KLV UHVXOWV LQ D VSHFLILF EHKDYLRU DIIHFWLQJ ERWKWKH H[WHUQDO ZRUOG HJ SRVLWLRQ RI WKH YHKLFOH RQ WKH URDG YHORFLW\ HWF  DQGRU WKH LQWHUQDO V\VWHP VWDWHV HJ GULYHU¶VSURSULRFHSWLRQ LQWKHVXEVHTXHQWLQIHUHQFHF\FOH ,9 &2*1,7,9(3/$86,%,/,7< 7KHGHVLJQRIWKH,1&$IUDPHZRUNGHVFULEHGLQVHFWLRQV,, DQG,,,FDSWXUHVLQKHUHQWO\VHYHUDOHVWDEOLVKHGWHUPLQRORJLHV DQGRU NH\ SULQFLSOHV LQ FRJQLWLYH VFLHQFHV DQG $, ZKLFK MXVWLI\LWVWKHRUHWLFDOSODXVLELOLW\DVIROORZV   ([SOLFLW YV LPSOLFLW FRJQLWLRQ 7KH ,1&$ IUDPHZRUN XWLOL]HVDK\EULGV\PEROLFQHXUDOPRGHOLQJDSSURDFKWKDWLV FORVHO\ UHODWHGWR WKH HVWDEOLVKHG GLFKRWRP\ RI LPSOLFLWDQG H[SOLFLWFRJQLWLRQ>@7KHIRUPHULVW\SLFDOO\GHVFULEHGDV EHLQJLQDFFHVVLEOHDQGVXEFRQVFLRXVZKLOHWKHODWWHULVPRUH DFFHVVLEOHDQGGHOLEHUDWHZKHUHDFFHVVLELOLW\GHQRWHVGLUHFW LPPHGLDWHDYDLODELOLW\RIFRQWHQWLQPHQWDORSHUDWLRQ 7KHUH H[LVWV D JHQHUDO FRQVHQVXVWKDWLPSOLFLW NQRZOHGJH LV EHWWHU FDSWXUHGXVLQJDGLVWULEXWHGPHPRU\HJQHXUDOQHWZRUN  ZKHUHDVH[SOLFLWNQRZOHGJHE\V\PEROLFRUORFDOLVWPHPRU\ HJUXOHRUJUDSKEDVHGPHPRU\ >@,QWKLVUHVSHFW KLGGHQOD\HUXQLWVLQQHXUDOQHWZRUNVDUHFDSDEOHRISDUDOOHO FRPSXWDWLRQVEXWKDYHQRVHPDQWLFLQWHUSUHWDWLRQLHWKHUH LVQRVHPDQWLFODEHODVVRFLDWHGZLWKWKHQXPHULFDOGDWD %\ FRQWUDVW HDFK XQLW LQ D V\PEROLF RU ORFDOLVW PHPRU\ LV DVVRFLDWHGZLWKDVHPDQWLFODEHOWKDWLVFOHDUO\LQWHUSUHWDEOH ,Q ,1&$K\EULGL]DWLRQ RI ORFDOLVW DQG GLVWULEXWHGUHSUHVHQ WDWLRQV LV DFKLHYHG XVLQJ D QHXURIX]]\ V\VWHP DSSURDFK ZKLFKV\QHUJL]HVWKHLPSOLFLWOHDUQLQJDGDSWDWLRQDELOLWLHVRI QHXUDOQHWZRUNVDQGH[SOLFLWKXPDQUHDVRQLQJVW\OHRIIX]]\ UXOHEDVHGV\VWHPV>@   'HFODUDWLYH YV SURFHGXUDO FRJQLWLRQ 7KH GHVLJQ RI ,1&$LVDOVREXLOWXSRQWKHHVWDEOLVKHGGLVWLQFWLRQEHWZHHQ GHFODUDWLYHDQGSURFHGXUDONQRZOHGJH>@7KHIRUPHULV FRQFHUQHGZLWKVWDWLFJHQHUDOIDFWVDERXWWKHZRUOGDQGVHOI WKDWFDQEHDUWLFXODWHGZKHUHDVWKHODWHULQYROYHVVNLOOVDQG SURFHGXUHVXVXDOO\ H[SUHVVHGWKURXJKEHKDYLRUV7KH,1&$ IUDPHZRUNDFFRUGLQJO\LQFRUSRUDWHVVHSDUDWHGHFODUDWLYHDQG SURFHGXUDO PRGXOHV WR FDSWXUH WKHVH WZR DVSHFWV H[SOLFLWO\ DVGHWDLOHGLQVHFWLRQ,,$,QVXPWKHIRUPHUVSHFLDOL]HVLQ QRQDFWLRQRULHQWHG FRJQLWLYH SURFHVVLQJ ZKLOH WKH ODWWHU FRQFHQWUDWHVPDLQO\RQDFWLRQRULHQWHGSURFHVVLQJ   5HDFWLYH YV GHOLEHUDWLYH FRQWURO 6ORPDQ >@ DUJXHG WKDWWKHKXPDQPLQGFDQEH FRQVLGHUHGDVDFRQWUROV\VWHP ZKLFK FRPSULVHV WKUHH OHYHOV RI DEVWUDFWLRQ UHDFWLYH

GHOLEHUDWLYHDV ZHOODVUHIOH[LYHPHWDFRJQLWLRQ 7KHILUVW OHYHO LV FKDUDFWHUL]HG E\ WULJJHULQJ FRQGLWLRQV \LHOGLQJ LQVWDQWDQHRXVDXWRPDWLFUHVSRQVHVDQGRSHUDWHVVLPLODUO\WR D ORRNXS WDEOH $ SXUHO\ UHDFWLYH V\VWHP KRZHYHU LV QRW FDSDEOH RI H[SOLFLWO\ GHYLVLQJ H[DPLQLQJ RU VHOHFWLQJ EHWZHHQ PXOWLSOH DFWLRQV RU SRVVLEOH VROXWLRQV $W WKH GHOLEHUDWLYH OHYHO RQ WKH RWKHU KDQG RSWLRQV DQG SODQV DUH ODLG RXW DQG HYDOXDWHG YLD K\SRWKHVHV LPDJLQDWLRQ  FRQGXFWHGDVLQWHUQDOVLPXODWLRQRIWKHLQWHUDFWLRQVZLWKWKH H[WHUQDO ZRUOG 6XFK D V\VWHP ZRUNLQJ DWRS UHDFWLYH SURFHVVHV FDQ SODQ DKHDG DQG PDNH LQIRUPDWLYH GHFLVLRQV +RZHYHU LQ D FKDQJLQJ ZRUOG WKH UHVXOW RI GHOLEHUDWLYH SURFHVVHV VWRUHG HDUOLHU DQG LQWHQGHG IRU IXWXUH XVH PD\ EHFRPH REVROHWH VR WKDW NQRZOHGJH FUDIWHG LQ HDUO\ GHYH ORSPHQWVWDJHVPD\QRWORQJHUEHUHOHYDQWLQODWHUVWDJHV$ UHIOH[LYH PHFKDQLVP LV WKXV UHTXLUHG WR PDLQWDLQ PRQLWRU DQG HYDOXDWH WKH SURFHVVHV LQYRNHG DQG GHFLVLRQV PDGH DW WKHGHOLEHUDWLYHOHYHOUHODWLYHWRWKHFXUUHQWJRDOFRQWH[W,Q ,1&$ UHDFWLYH FRQWURO LV HPEHGGHG ZLWKLQ DOO PRGXOHV ZKLOHGHOLEHUDWLYHPHFKDQLVPVDUHLQWHJUDWHGLQWKHGHFODUD WLYH DQG SURFHGXUDO PHPRULHV /DVWO\ WKH UHIOH[LYH V\VWHP OLHVZLWKLQWKHH[HFXWLYHPDQDJHUDQGDIIHFWPRGXODWRU   6HQVRU\ YV VKRUWWHUP YV ORQJWHUP UHSRVLWRULHV ,1&$DOVRLQFRUSRUDWHVLQLWVGHVLJQWKHGLVWLQFWLRQVDPRQJ WKH VHQVRU\ VKRUWWHUP DQG ORQJWHUP PHPRU\ VWRUDJHV SUHVHQWLQWKHKXPDQEUDLQ>@7KHVHQVRU\PHPRU\HJ LFRQLF DQG HFKRLF UHJLVWHUV  KDV D OLPLWHG DELOLW\ WR VWRUH LQIRUPDWLRQDERXWWKHZRUOGIRUDYHU\EULHISHULRGDQGLQD IDLUO\ XQSURFHVVHG IDVKLRQ 7KLV LQIRUPDWLRQ LV QRW \HW UHFRJQL]HG RU LQWHUSUHWHG DQG W\SLFDOO\ LV YHU\ EULWWOH 7KH FRQWHQWRIWKHVHQVRU\PHPRU\LVVHOHFWHGWKURXJKDWWHQWLRQ ILOWHUVDQGSDVVHGRQWRWKHVKRUWWHUPVWRUHZKLFKFDQW\SL FDOO\ PDLQWDLQ LQIRUPDWLRQ XS WR WKLUW\ VHFRQGV DQG KDV D FDSDFLW\ RI ILYH WR QLQH LWHPV ,QIRUPDWLRQ IURP WKH VKRUW WHUP VWRUH LV WKHQ WUDQVIHUUHG YLD UHKHDUVDO SURFHVVHV WR WKH ORQJWHUP VWRUH ZKLFK VSRUWV D WKHRUHWLFDOO\ XQOLPLWHG FDSDFLW\ RWKHUZLVH LW LV IRUJRWWHQWKURXJK WKHSURFHVVHV RI GLVSODFHPHQW GHFD\  ,Q ,1&$WKHKXPDQ VHQVRU\ VWRUH LV UHDOL]HG LQ LWV VHQVRU\ UHJLVWHU ZKLOH WKHKXPDQ VKRUWWHUP DQGORQJWHUPUHSRVLWRULHVPDSURXJKO\WRWKHWUDQVLHQWDQG GHFODUDWLYHH[HFXWLYH PRGXOHV UHVSHFWLYHO\ 7KH DWWHQWLRQ DQG UHKHDUVDO SURFHVVHV RQ WKH RWKHU KDQG DUH FRUUHODWHG ZLWK WKH DWWHQWLRQ IRFXV DQG FRQVROLGDWLRQ SURWRFROV LQ ,1&$UHVSHFWLYHO\   3DUDOOHO YV VHULDO IORZ RI LQIRUPDWLRQ 7KH KROLVWLF EHKDYLRURI,1&$FRQVLVWVRIUHSHWLWLYHMRXUQH\VWKURXJKWKH LQIHUHQFH DQG FRQVROLGDWLRQ F\FOHV ZKLFK LQKHUHQWO\ EOHQG WKH VHULDO DQG SDUDOOHO IRUPV RI FRJQLWLYH SURFHVVLQJ LQ D V\VWHPDWLFIDVKLRQ7KLVLGHDLVFORVHO\UHODWHGWRWKH JOREDO ZRUNVSDFHIUDPHZRUNRILQIRUPDWLRQIORZLQZKLFKDVHULDO SURFHVVLRQ RI VWDWHV HPHUJHV IURP WKH LQWHUDFWLRQV RI PDQ\ GLVSDUDWHVSHFLDOL]HGSDUDOOHOSURFHVVHV>@7KLVVHULDO SURFHVVLRQRI VWDWHVZKLFKLQYROYHVXQIROGLQJRIFRQVFLRXV FRQWHQWVLQWKHKXPDQZRUNLQJPHPRU\IDFLOLWDWHVWKHUHIRUH KLJKHUOHYHOFRJQLWLYHIXQFWLRQVHJDQWLFLSDWLRQSODQQLQJ HWF DQGLVLQWXUQDQLPSURYHGIRUPRIDFWLRQVHOHFWLRQ
2008 International Joint Conference on Neural Networks (IJCNN 2008)

1907

 

$UFKLWHFWXUDO)HDWXUH 'HVLJQSURSHUW\ 0HWKRGRORJ\ 0HPRU\RUJDQL]DWLRQ /HDUQLQJPHFKDQLVP $EVWUDFWLRQOHYHO ,QWHUDFWLRQSURWRFRO 6XSSRUWHGFDSDELOLW\ 3HUFHSWLRQDQGDFWLRQ 3ODQQLQJSUREOHPVROYLQJ 5HDFWLYLW\ ([HFXWLYHFRQWURO (PRWLRQ 3DUDOOHOH[HFXWLRQ +LHUDUFKLFDOSURFHVVLQJ (PERGLPHQW $XWRQRPRXVJURZWK

7$%/(, &203$5,6216%(7:((1,1&$$1'2 7+(5(67$%/,6+('&2*1,7,9( $5&+,7(&785(6 62$5 ,&$586 ,%&$ &RQIDEXODWLRQ $&75 &/$5,21

/,'$

,1&$

 6\PEROLF 'H3U([ 6W3D 5D'E5I 

 6\PEROLF 'H3U([ 6W3D 5D'E 

 (PHUJHQW 'H3U,P 3D 5D'E5I 

 (PHUJHQW 'H3U([,P 3D 5D'E 

 +\EULG 'H3U([,P 6W3D 5D'E5I 

 +\EULG 'H3U([,P 3D 5D'E5I 

 +\EULG 'H3U([,P 6W3D 5D'E5I &J

 +\EULG 'H3U([,P 6W3D 5D'E5I &V,Q









'H GHFODUDWLYH3U SURFHGXUDO([ H[SOLFLW,P LPSOLFLW6W VWUXFWXUDO3U SDUDPHWHU5D UHDFWLYH'E GHOLEHUDWLYH5I UHIOH[LYH&J  FRJQLWLYHF\FOH&V FRQVROLGDWLRQF\FOH,Q LQIHUHQFHF\FOH

XSFRQWULEXWLQJWRWKHRQJRLQJVHULDOWKUHDG 9 &203$5,6216:,7+27+(5$5&+,7(&785(6 7DEOH , SUHVHQWV D EULHI VXPPDU\ RI FRPSDULVRQV EHWZHHQ WKH SURSRVHG ,1&$ IUDPHZRUN DQG D QXPEHU RI RWKHU UHQRZQHGFRJQLWLYHDUFKLWHFWXUHV62$5>@,&$586> @ &RQIDEXODWLRQ > @ ,%&$ > @ $&75 > @ &/$5,21 > @ DQG /,'$ > @ &RPSDULVRQV DUH ILUVWPDGHLQWHUPVRIVDOLHQWGHVLJQSURSHUWLHVDVGHILQHGLQ WKH SUHYLRXV VHFWLRQV LQFOXGLQJ GHVLJQ PHWKRGRORJ\ LH V\PEROLF HPHUJHQW RU K\EULG  PHPRU\ RUJDQL]DWLRQ LH GHFODUDWLYH RU SURFHGXUDO H[SOLFLW RU LPSOLFLW  OHDUQLQJ SURFHGXUHLHVWUXFWXUDORUSDUDPHWHU OHYHORIDEVWUDFWLRQ LHUHDFWLYH GHOLEHUDWLYH RU UHIOH[LYH  DV ZHOO DV IRUPDO  LQWHUDFWLRQSURWRFROIRULQWHUPRGXOHFRPPXQLFDWLRQ:KLOH WKLVSDSHUFDQQRWHODERUDWHIXUWKHUGXHWRVSDFHFRQVWUDLQWV WKH DXWKRUV DUH FXUUHQWO\ ZRUNLQJ RQ D FRPSUHKHQVLYH VXUYH\ RI FRJQLWLYH DUFKLWHFWXUHV DQ DEULGJHG YHUVLRQ RI ZKLFKLVDYDLODEOH>@ 7DEOH,DOVROLVWVVHYHUDONH\ FDSDELOLWLHVRIWKHH[HPSODU FRJQLWLYHDUFKLWHFWXUHVWKDWFRUUHVSRQGWRVHYHUDOHVWDEOLVKHG WHUPLQRORJLHVLQ$,OLWHUDWXUH7KH\LQFOXGHSHUFHSWLRQLH DELOLW\ WR UHFRJQL]H REMHFWV LQ WKH ZRUOG  SUREOHP VROYLQJ DQGSODQQLQJLHDELOLW\WRGHYLVHDVHTXHQFHRIDFWLRQVWR DFKLHYHDJRDO UHDFWLYLW\LHDELOLW\WRKDQGOHXQIRUHVHHQ HYHQWVLQDERXQGHGWLPH H[HFXWLYHFRQWUROLHDELOLW\WR PRQLWRU DQG UHJXODWH RQH¶V RZQ FRJQLWLYH SURFHVVHV  HPR WLRQLHDELOLW\WRSURYLGHWKHPRWLYDWLRQWRGLUHFWFRJQLWLYH SURFHVVHV  SDUDOOHO H[HFXWLRQ LHDELOLW\ WR SURFHVV D ODUJH DPRXQW RI  LQIRUPDWLRQ VLPXOWDQHRXVO\ RU DV\QFKURQRXVO\  KLHUDUFKLFDO SURFHVVLQJ LH DELOLW\ WR GHFRPSRVHDEVWUDFW WDVNVWRIURPVPDOOHUVXEWDVNV HPERGLPHQWLHJURXQGLQJ RI FRJQLWLYH SURFHVVHV LQWR SK\VLFDO VHQVRULPRWRU UHSUHVHQ WDWLRQV DQGODVWEXWQRWOHDVWDXWRQRPRXVJURZWKLHJUD GXDO DFTXLVLWLRQ RI GHFLVLRQPDNLQJ DQG SUHGLFWLYH DELOLWLHV RYHUDOLIHWLPHYLDH[SHULHQWLDOOHDUQLQJ  2YHUDOOWKHSURSRVHG,1&$IUDPHZRUNSRVLWVDULFKHUVHW RIGHVLJQSURSHUWLHVDQGFDSDELOLWLHVWKDQWKHRWKHUFRJQLWLYH

1908

DUFKLWHFWXUHVOLVWHG+RZHYHULWPXVWEHQRWHGKHUHWKDWRXU DSSURDFKLVQRWFRPSHWLQJZLWKWKHVWDWHRIWKHDUWLQ$,DQG FRJQLWLYH VFLHQFHV ,QVWHDG LW DVSLUHV WR SURYLGH D FRPSOH PHQWDU\ PHWKRGRORJ\ WR DGGUHVV VRPH RI WKH NH\ LVVXHV LQ WKH FXUUHQW DSSURDFKHV²HVSHFLDOO\ ZLWK UHJDUG WR EULGJLQJ WKH JDS IURP ELRORJLFDOO\LQVSLUHG DSSURDFKHV WR V\PEROLF UHSUHVHQWDWLRQV²DQGWREXLOGWRZDUGVWKHXSSHUOHYHORQWKH VWDWHRIWKHDUWDQGLWVXQGHUO\LQJGHYHORSPHQWV 9, &21&/86,21 7KLV SDSHU HODERUDWHV WKH GHVLJQ RI D QRYHO EUDLQLQVSLUHG ,QWHJUDWHG1HXUR&RJQLWLYH$UFKLWHFWXUH,1&$ 7KHPDLQ REMHFWLYH RI ,1&$ LV WR V\QWKHVL]H WKH SXWDWLYH IXQFWLRQDO OHDUQLQJDQGPHPRU\DVSHFWVDQGLQWHUDFWLRQVRIWKHYDULRXV NH\VXEV\VWHPVLQWKHEUDLQZRUNLQJLQFUHPHQWDOO\WRZDUGV D FRPSUHKHQVLYH PRGHO RI KXPDQ FRJQLWLYH PHFKDQLVPV 7KHQRYHOW\FODLPVRI,1&$OLHLQLWVIXOO\DXWRQRPRXVVHOI RUJDQL]LQJFDSDELOLWLHVDQGLWVH[WHQVLELOLW\WRPHWDFRJQLWLYH IXQFWLRQVDVZHOODVWKHSURYLVLRQRIIRUPDORSHUDWLRQF\FOHV JRYHUQLQJ LWV PRGXOH LQWHUDFWLRQV V\VWHPDWLFDOO\ ,W LV H[SHFWHGWKDWZKLOHDGYDQFLQJIXUWKHULQWKHUHDOL]DWLRQDQG H[SHULPHQWDOYDOLGDWLRQRIWKHSURSRVHG,1&$IUDPHZRUND QHZFODVVRIFRJQLWLYHDUFKLWHFWXUHVDQGPRGHOVZLOOHPHUJH ZLWKWKHDELOLW\WRDFTXLUHVNLOOVFRQFHSWVDXWRQRPRXVO\DQG DFFRUGLQJO\OHDUQDQGUHJXODWHLWVHOIDWHYHQKLJKHUOHYHOVRI DEVWUDFWLRQ ,W ZLOO WKHQ EH SRVVLEOH WR HPSOR\ WKHVH QHZ DUFKLWHFWXUHV RU PRGHOV LQ D YDULHW\ RI FRJQLWLYH SDUDGLJPV UHTXLULQJ UREXVW OHDUQLQJ DQG VRFLDO FRJQLWLRQ HYHQWXDOO\ UHDOL]LQJKXPDQOLNHJHQHUDOLQWHOOLJHQFH 7KH ,1&$ SURMHFW LV RQJRLQJ ZLWK WKH FXUUHQW SKDVH RI GHYHORSPHQW IRFXVLQJ RQWKH FRQVROLGDWLRQQHWZRUN DV SHU VHFWLRQ,,% 7KHQH[WVWDJHZLOOLQYROYHGHYLVLQJDVHTXHQ FLQJ DOJRULWKP WR FKDLQ WKH H[SOLFLW UXOHV GHYLVHG LQ WKH FRQVROLGDWLRQQHWZRUNIRUPLQJLQWXUQEDVLFPHFKDQLVPVIRU DQWLFLSDWLRQDQGSODQQLQJFDSDELOLWLHVDVZHOODVRWKHUPHWD FRJQLWLYHIXQFWLRQV7RSURYLGHDQLQLWLDOHYDOXDWLRQRIWKHVH PHFKDQLVPV RQ WKH RWKHU KDQG VHYHUDO EDVLF SDWWHUQ FRPSOHWLRQDQGSODQQLQJWDVNVDUHEHLQJLQYHVWLJDWHGZKLFK

2008 International Joint Conference on Neural Networks (IJCNN 2008)

  LQYROYHLQWHUDFWLRQVZLWKDVLPSOHSK\VLFDOZRUOGDQGZKLFK UHTXLUHLQIDQWOLNHOHDUQLQJDELOLWLHV,QDGGLWLRQDQXPEHURI WHVW FDVHV WKDW H[WHQG SUHYLRXV ZRUN RQ WKH LQWHOOLJHQW FDU V\VWHP>@DUHDOVREHLQJGHYHORSHGWRVKRZFDVHWKH RSHUDWLRQDO SULQFLSOHV LH FRQVROLGDWLRQ DQG LQIHUHQFH F\FOHV LQ,1&$DVGHVFULEHGLQWKLVSDSHU 5()(5(1&(6 >@ $ 1HZHOO 8QLILHG 7KHRULHV RI &RJQLWLRQ +DUYDUG 8QLYHUVLW\ 3UHVV  >@ 00LQVN\6FLHQWLVWRQWKH6HW$Q,QWHUYLHZZLWK0DUYLQ0LQVN\ LQ +$/ V /HJDF\  V &RPSXWHU DV 'UHDP DQG 5HDOLW\ ' * 6WRUN(G0,73UHVVSS >@ ( 5 .DQGHO - + 6FKZDUW] DQG 7 0 -HVVHO 3ULQFLSOHV RI 1HXUDO 6FLHQFHWKHG1HZ@ - 5 $QGHUVRQ /HDUQLQJ DQG 0HPRU\ QG HG 1HZ @ : 'XFK 5 - 2HQWDU\R DQG 0 3DVTXLHU &RJQLWLYH DUFKLWHFWXUHV :KHUHGRZHJRIURPWKHUH"LQ$UWLILFLDO*HQHUDO,QWHOOLJHQFH 3 :DQJ % *RHUW]HO DQG 6 )UDQNOLQ (GV $PVWHUGDP ,26 3UHVV SS >@ -(/DLUG365RVHQEORRPDQG $1HZHOO6RDU$QDUFKLWHFWXUH IRU JHQHUDO LQWHOOLJHQFH $UWLILFLDO ,QWHOOLJHQFH YRO  SS   >@ 3 /DQJOH\ $Q DGDSWLYH DUFKLWHFWXUH IRU SK\VLFDO DJHQWV LQ 3URFHHGLQJV RI WKH ,(((:,&$&0 ,QWHUQDWLRQDO &RQIHUHQFH RQ ,QWHOOLJHQW$JHQW7HFKQRORJ\&RPSLHJQH)UDQFHSS >@ 3 /DQJOH\ DQG ' &KRL /HDUQLQJ UHFXUVLYH FRQWURO SURJUDPV IURP SUREOHP VROYLQJ-RXUQDORI0DFKLQH/HDUQLQJ5HVHDUFKYROSS  >@ 5 +HFKW1LHOVHQ $ FRUWURQLF QHXUDO QHWZRUN PRGHOV RI FRUWLFDO IXQFWLRQ LQ 3URFHHGLQJV RI WKH ,QWHUQDWLRQDO &RQIHUHQFHRQ 1HXUDO ,QIRUPDWLRQ 3URFHVVLQJ 6 8VXL DQG 7 2PRUL (GV .LWD.\XVKX -DSDQ,2$3UHVV >@ 5+HFKW1LHOVHQ&RQIDEXODWLRQ7KHRU\7KH0HFKDQLVPRI7KRXJKW 6SULQJHU >@ 5 & 2 5HLOO\ 7 6 %UDYHU DQG - ' &RKHQ $ ELRORJLFDOO\EDVHG FRPSXWDWLRQDO PRGHO RI ZRUNLQJ PHPRU\ LQ 0RGHOV RI :RUNLQJ 0HPRU\ $ 0L\DNH DQG 3 6KDK (GV &DPEULGJH 8QLYHUVLW\ 3UHVV SS >@ 5 & 2 5HLOO\ DQG < 0XQDNDWD &RPSXWDWLRQDO ([SORUDWLRQV LQ &RJQLWLYH1HXURVFLHQFH8QGHUVWDQGLQJRIWKH0LQGE\6LPXODWLQJWKH %UDLQ&DPEULGJH0$0,73UHVV >@ 5 6XQ DQG ) $OH[DQGUH &RQQHFWLRQLVW 6\PEROLF ,QWHJUDWLRQ +LOOVGDOH1-(UOEDXP >@ - 5 $QGHUVRQ DQG & /HELHUH 7KH 1HZHOO WHVW IRU D WKHRU\ RI FRJQLWLRQ%HKDYLRUDODQG%UDLQ6FLHQFHYROSS >@ -5$QGHUVRQ'%RWKHOO0'%\UQH6'RXJODVV&/HELHUHDQG < 4LQ $Q LQWHJUDWHG WKHRU\ RI WKH PLQG 3V\FKRORJLFDO 5HYLHZ YROSS >@ 6 )UDQNOLQ 7KH /,'$ DUFKLWHFWXUH $GGLQJ QHZ PRGHV RI OHDUQLQJ WR DQ LQWHOOLJHQW DXWRQRPRXV VRIWZDUH DJHQW LQ 3URFHHGLQJVRI WKH ,QWHUQDWLRQDO &RQIHUHQFH RQ ,QWHJUDWHG 'HVLJQ DQG 3URFHVV 7HFKQRORJ\6DQ'LHJR6RFLHW\IRU'HVLJQ 3URFHVV6FLHQFH >@ 6)UDQNOLQ $ .HOHPHQDQG/0F&DXOH\,'$ $ FRJQLWLYHDJHQW DUFKLWHFWXUH LQ 3URFHHGLQJV RI WKH ,((( &RQIHUHQFH RQ 6\VWHPV 0DQDQG&\EHUQHWLFVYRO6DQ'LHJR&$SS >@ 5 6XQ DQG 7 3HWHUVRQ $XWRQRPRXV OHDUQLQJ RI VHTXHQWLDO WDVNV ([SHULPHQWV DQG DQDO\VHV ,((( 7UDQVDFWLRQV RQ 1HXUDO 1HWZRUNV YROSS >@ + (LFKHQEDXP 7KH &RJQLWLYH 1HXURVFLHQFH RI 0HPRU\ $Q ,QWURGXFWLRQ1HZ@ 5 - 2HQWDU\R 0 3DVTXLHU DQG & 4XHN *HQ6R)11@ - 6LP : / 7XQJ DQG & 4XHN )&0$&
>@ 5 - 2HQWDU\R 0HWDFRJQLWLYH LQWHOOLJHQW V\VWHP LQ FRPSOH[ HQYLURQPHQWV 1DQ\DQJ 7HFKQRORJLFDO 8QLYHUVLW\ 6LQJDSRUH )LUVW @ - /H'RX[ %UDLQ PHFKDQLVPV RI HPRWLRQ DQG HPRWLRQDO OHDUQLQJ &XUUHQW2SLQLRQLQ1HXURELRORJ\YROSS >@ (/XPDQSDXZ03DVTXLHUDQG5-2HQWDU\R*HQHULF*$EDVHG PHWDOHYHOSDUDPHWHURSWLPL]DWLRQIRUSDWWHUQUHFRJQLWLRQV\VWHPVLQ 3URFHHGLQJV RI WKH ,((( &RQJUHVV RQ (YROXWLRQDU\ &RPSXWDWLRQ 6LQJDSRUHSS >@ : 'XFK )LOWHU PHWKRGV LQ )HDWXUH ([WUDFWLRQ )RXQGDWLRQV DQG $SSOLFDWLRQV 6HULHV 6WXGLHV LQ )X]]LQHVV DQG 6RIW &RPSXWLQJ , *X\RQ6*XQQ01LNUDYHVKDQG/=DGHK(GV 3K\VLFD9HUODJ 6SULQJHUSS >@ :'XFK5$GDPF]DNDQG1-DQNRZVNL,QLWLDOL]DWLRQRIDGDSWLYH SDUDPHWHUVLQGHQVLW\QHWZRUNVLQ 3URFHHGLQJVRIWKH&RQIHUHQFHRQ 1HXUDO1HWZRUNVDQG7KHLU$SSOLFDWLRQV.XOHSS >@ -/0F&OHOODQG%/0F1DXJKWRQDQG5&2 5HLOO\:K\WKHUH DUH FRPSOHPHQWDU\ OHDUQLQJ V\VWHPV LQ WKH KLSSRFDPSXV DQG QHRFRUWH[ ,QVLJKWV IURP WKH VXFFHVVHV DQG IDLOXUHV RI FRQQHFWLRQLVW PRGHOVRIOHDUQLQJDQGPHPRU\ 3V\FKRORJLFDO5HYLHZYROSS  >@ 03DVTXLHU&4XHNDQG07RK)X]]\ORW$QRYHOVHOIRUJDQLVLQJ IX]]\QHXUDO UXOHEDVHG SLORW V\VWHP IRU DXWRPDWHG YHKLFOHV 1HXUDO 1HWZRUNVYROSS >@ 03DVTXLHUDQG5-2HQWDU\R/HDUQLQJWRGULYHWKHKXPDQZD\$ VWHS WRZDUGV LQWHOOLJHQW YHKLFOHV ,QWHUQDWLRQDO -RXUQDO RI 9HKLFOH $XWRQRPRXV 6\VWHPV 6SHFLDO ,VVXH RQ $GYDQFHV LQ $XWRQRPRXV 9HKLFOH7HFKQRORJLHVIRU8UEDQ(QYLURQPHQWYROSS >@ 7 . .RKRQHQ 6HOIRUJDQL]HG IRUPDWLRQ RI WRSRORJLFDOO\ FRUUHFW IHDWXUHPDSV%LRORJLFDO&\EHUQHWLFVYROSS >@ = 3DZODN 5RXJK VHWV ,QWHUQDWLRQDO -RXUQDO RI &RPSXWHU DQG ,QIRUPDWLRQ6FLHQFHVYROSS >@ 56XQ 6XSSOHPHQWLQJQHXUDOUHLQIRUFHPHQWOHDUQLQJ ZLWK V\PEROLF PHWKRGV LQ +\EULG 1HXUDO 6\VWHPV 6 :HUPWHU DQG 5 6XQ (GV %HUOLQ6SULQJHU9HUODJSS >@ 0 &KDQ 0 3DVTXLHU DQG ' 3DUWRXFKH $Q LQWHOOLJHQW GULYLQJ V\VWHPIRUDXWRPDWLFDOO\DQWLFLSDWLQJDQGQHJRWLDWLQJURDGFXUYHVLQ 3URFHHGLQJVRIWKH,(((56-,QWHUQDWLRQDO&RQIHUHQFHRQ,QWHOOLJHQW 5RERWVDQG6\VWHPV6DQ'LHJR86$SS >@ %-%DDUVDQG6)UDQNOLQ+RZFRQVFLRXV H[SHULHQFHDQG ZRUNLQJ PHPRU\ LQWHUDFW 7UHQGV LQ &RJQLWLYH6FLHQFHV YRO  SS   >@ 5 6XQ DQG / %RRNPDQ $UFKLWHFWXUHV ,QFRUSRUDWLQJ 1HXUDO DQG 6\PEROLF3URFHVVHV%RVWRQ.OXZHU >@ & 7 /LQ DQG & 6 * /HH 1HXUDO )X]]\ 6\VWHPV $ 1HXUR)X]]\ 6\QHUJLVP WR ,QWHOOLJHQW 6\VWHPV 8SSHU 6DGGOH 5LYHU 1- 3UHQWLFH +DOO >@ $ 6ORPDQ :KDW VRUW RI DUFKLWHFWXUH LV UHTXLUHG IRU D KXPDQOLNH DJHQW LQ )RXQGDWLRQV RI 5DWLRQDO $JHQF\ 0 :RROGULGJH DQG $ 5DR'RUGUHFKW(GV.OXZHU$FDGHPLF3XEOLVKHUV >@ 5 & $WNLQVRQ DQG 5 0 6KLIIULQ +XPDQ PHPRU\ $ SURSRVHG V\VWHPDQGLWVFRQWUROSURFHVVHVLQ7KH3V\FKRORJ\RI/HDUQLQJDQG 0HPRU\$GYDQFHVLQ5HVHDUFKDQG7KHRU\.: 6SHQFHDQG-7 6SHQFH(GV1HZ@ % - %DDUV $ &RJQLWLYH 7KHRU\ RI &RQVFLRXVQHVV &DPEULGJH 0$ &DPEULGJH8QLYHUVLW\3UHVV 



2008 International Joint Conference on Neural Networks (IJCNN 2008)

1909