Investigating the Local-Meta-Model CMA-ES for Large Population Sizes Zyed Bouzarkouna1,2 1 IFP 2 TAO
Anne Auger2
Didier Yu Ding1
(Institut Fran¸cais du P´ etrole)
Team, INRIA Saclay-Ile-de-France, LRI
April 07, 2010
Statement of the Problem
Objective To solve a real-world optimization problem formulated in a black-box scenario with an objective function f : Rn 7→ R. multimodal noisy f may be: non-separable ...
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
non-smooth non-convex computationally expensive
page 2 of 15
A Real-World Problem in Petroleum Engineering History Matching The act of adjusting a reservoir model until it closely reproduces the past behavior of a production history.
A fluid flow simulation takes several minutes to several hours !!
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 3 of 15
Statement of the Problem (Cont’d) Difficulties Evolutionary Algorithms (EAs) are usually able to cope with noise, multiple optima . . . Computational cost build a model of f , based on true evaluations ; use this model during the optimization to save evaluations.
⇒
How to decide whether: the quality of the model is good enough to continue exploiting this model ?
or new evaluations on the “true” objective function should be performed ? Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 4 of 15
Table of Contents
1
CMA-ES with Local-Meta-Models Covariance Matrix Adaptation-ES Locally Weighted Regression Approximate Ranking Procedure
2
A New Variant of lmm-CMA A New Meta-Model Acceptance Criterion nlmm-CMA Performance
3
Conclusions
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 5 of 15
Covariance Matrix Adaptation-ES CMA-ES (Hansen & Ostermeier 2001) Initialize distribution parameters m, σ and C, set population size λ ∈ N. while not terminate Sample xi = m + σNi (0, C), for i = 1 . . . λ according to a multivariate normal distribution Evaluate x1 , . . . , xλ on f Update distribution parameters (m, σ, C) ← (m, σ, C, x1 , . . . , xλ , f (x1 ), . . . , f (xλ )) where m ∈ Rn : the mean of the multivariate normal distribution σ ∈ R+ : the step-size C ∈ Rn×n : the covariance matrix. Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 6 of 15
Covariance Matrix Adaptation-ES (Cont’d) Moving the mean µ(= λ2 )
m=
P
ωi xi:λ .
i=1 where xi:λ is the i th ranked individual: f (x1:λ ) ≤ . . . f (xµ:λ ) ≤ . . . f (xλ:λ ) , µ P ω1 ≥ . . . ≥ ωµ > 0, ωi = 1. i=1
Other updates Adapting the Covariance Matrix Step-Size Control
⇒
Updates rely on the ranking of individuals according to f and not on their exact values on f .
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 7 of 15
Locally Weighted Regression
q ∈ Rn : A point to evaluate
⇒
fˆ(q) : a full quadratic meta-model on q.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 8 of 15
Locally Weighted Regression
A training set containing m points with their objective function values (xj , yj = f (xj )) , j = 1..m
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 8 of 15
Locally Weighted Regression
We select the k nearest neighbor data points to q according to Mahalanobis distance with respect to the current covariance matrix C.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 8 of 15
Locally Weighted Regression
h is the bandwidth defined by the distance of the k th nearest neighbor data point to q.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 8 of 15
Locally Weighted Regression
Building the meta-model fˆ on q 2 k n(n+3) P ˆ min f (xj , β) − yj ωj , w.r.t β ∈ R 2 +1 . j=1
T fˆ (q) = β T q12 , · · · , qn2 , · · · , q1 q2 , · · · , qn−1 qn , q1 , · · · , qn , 1 . Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 8 of 15
Approximate Ranking Procedure Every generation g , CMA-ES has λ points to evaluate.
⇒
Which are the points that must be evaluated with: the true objective function f ? the meta-model fˆ ?
Approximate ranking procedure (Kern et al. 2006) 1 2 3 4 5 6 7 8 9 10 11
approximate fˆ and rank the µ best individuals evaluate f on the ninit best individuals “ λ−n ” init for nic := 1 to do n b
approximate fˆ and rank the µ best individuals if (the exact ranking of the µ best individuals changes) then evaluate f on the nb best unevaluated individuals else break fi od adapt ninit depending on nic
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 9 of 15
A New Meta-Model Acceptance Criterion
Requiring the preservation of the exact ranking of the µ best individuals is a too conservative criterion to measure the quality of the meta-model. New acceptance criteria (nlmm-CMA) The meta-model is accepted if it succeeds in keeping: the best individual and the ensemble of the µ best individuals unchanged or the best individual unchanged, if more than one fourth of the population is evaluated.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 10 of 15
nlmm-CMA Performance Success Performance (SP1): SP1 =
mean (number of function evaluations for successful runs) . ratio of successful runs
Speedup (algo) =
SP1(algo) SP1(CMA−ES) .
8
Speedup
6
4
2
0
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
(Dimension, Population Size)
page 11 of 15
nlmm-CMA Performance 4 nlmm-CMA fSchwefel1/4 8
6
6
6
4
Speedup
8
2
4
2
0 (2, 6) (4, 8) (8, 10) (Dimension, Population Size)
(16, 12)
0 (2, 6)
fRosenbrock
4
2
(4, 8) (5, 8) (Dimension, Population Size)
(8, 10)
0 (2, 6) (4, 8) (8, 10) (Dimension, Population Size)
fAckley 8
6
6
6
2
Speedup
8
4
4
2
0 (2, 6)
(4, 8) (5, 8) (Dimension, Population Size)
⇒
(8, 10)
0 (2, 5)
(16, 12)
fRastrigin
8
Speedup
Speedup
fNoisySphere
8
Speedup
Speedup
fSchwefel
lmm-CMA
4
2
(5, 7) (Dimension, Population Size)
(10, 10)
0 (2, 50)
(5, 140) (Dimension, Population Size)
nlmm-CMA outperforms lmm-CMA, on the test functions investigated with a speedup between 1.5 and 7.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 11 of 15
nlmm-CMA Performance for Increasing Population Sizes 4 nlmm-CMA
lmm-CMA Dimension n = 5 fRosenbrock
fRastrigin 5
4
4
4
3 2 1 0 8
Speedup
5
Speedup
Speedup
fSchwefel1/4 5
3 2 1
16 24 32
⇒
48 Population Size
96
0 8
3 2 1
16 24 32
48 Population Size
96
0 70
140 Population Size
280
nlmm-CMA maintains a significant speedup,between 2.5 and 4, when increasing λ while the speedup of lmm-CMA drops to one.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 12 of 15
Impact of the Recombination Type
nlmm-CMA a default weighted recombination type ωi =
ln(µ+1)−ln(i) µ ln(µ+1)−ln(µ!) ,
for i = 1 . . . µ.
nlmm-CMAI an intermediate recombination type ωi = µ1 , for i = 1 . . . µ.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 13 of 15
Impact of the Recombination Type (Cont’d) nlmm-CMAI (with equal RT) fSchwefel1/4 fNoisySphere 8
8
6
6
6
4
2
4
2
0 (2, 6) (4, 8) (8, 10) (Dimension, Population Size)
(16, 12)
0 (2, 6)
fRosenbrock
4
2
(4, 8) (Dimension, Population Size)
(8, 10)
0 (2, 6) (4, 8) (8, 10) (Dimension, Population Size)
fAckley 8
6
6
6
2
0 (2, 6)
⇒ ⇒
Speedup
8
4
4
2
(4, 8) (Dimension, Population Size)
(8, 10)
0 (2, 5)
(16, 12)
fRastrigin
8
Speedup
Speedup
Speedup
8
Speedup
Speedup
4 nlmm-CMA fSchwefel
4
2
(5, 7) (Dimension, Population Size)
(10, 10)
0 (2, 50)
(5, 140) (Dimension, Population Size)
nlmm-CMA outperforms nlmm-CMAI . The ranking obtained with the new acceptance criterion still has an amount of information to guide CMA-ES.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 14 of 15
Summary CMA-ES with meta-models The speedup of lmm-CMA with respect to CMA-ES drops to one when the population size λ increases.
⇒
The meta-model acceptance criterion is too conservative.
New variant of CMA-ES with meta-models A new meta-model acceptance criterion: It must keep: the best individual and the ensemble of the µ best individuals unchanged the best individual unchanged, if more than one fourth of the population is evaluated. nlmm-CMA outperforms lmm-CMA on the test functions investigated with a speedup in between 1.5 and 7. nlmm-CMA maintains a significant speedup, between 2.5 and 4, when increasing the population size on tested functions.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 15 of 15
Thank You For Your Attention
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding
page 16 of 15
Investigating the Local-Meta-Model CMA-ES for Large Population Sizes Zyed Bouzarkouna1,2 1 IFP 2 TAO
Anne Auger2
Didier Yu Ding1
(Institut Fran¸cais du P´ etrole)
Team, INRIA Saclay-Ile-de-France, LRI
April 07, 2010
