Music-Inspired Optimization Algorithm

Harmony Search Zong Woo Geem

Mountain Climbing

What is Optimization?

What is Optimization? Procedure to make a system or design as effective, especially the mathematical techniques involved. (F (F Meta Meta--Heuristics) n Finding Best Solution u Minimal Cost (Design) u Minimal Error (Parameter Calibration) u Maximal Profit (Management) u Maximal Utility (Economics) n

Types of Optimization Algorithms n

n

n

Mathematical Algorithms u Simplex (LP), BFGS (NLP), B&B (DP) Drawbacks of Mathematical Algorithms u LP: Too Ideal (All Linear Functions) u NLP: Not for Discrete Var. or Complex Fn., Feasible Initial Vector, Local Optima u DP: Exhaustive Enumeration, Wrong Direction Meta--Heuristic Algorithms Meta u GA, SA, TS, ACO, PSO, …

Existing Nature-Inspired Algorithms

Existing Meta-Heuristic Algorithms

n

Definition & Synonym u Evolutionary, Soft computing, Stochastic Evolutionary Algorithm (Evolution) Simulated Annealing (Metal Annealing) Tabu Search (Animal’s Brain) Ant Algorithm (Ant’s Behavior) Particle Swarm (Flock Migration)

n

Mimicking Natural or Behavioral

n n n n n

Phenomena → Music Performance

Algorithm from Music Phenomenon

Algorithm from Jazz Improvisation

Analogy Do, Re, Mi

Mi, Fa, Sol

Sol, La, Si

= Do

= Mi

= Sol

x1

x2

x3

100mm 200mm 300mm

300mm 400mm 500mm

500mm 600mm 700mm

= 100mm

= 300mm

= 500mm

f (100, 300, 500)

Comparison Factors n

Musical Inst. → Decision Var.

n

Pitch Range → Value Range

n

Harmony → Solution Vector

n

Aesthetics → Objective Function

n

Practice → Iteration

n

Experience → Memory Matrix

Good Harmony & Bad Harmony

#

An Algorithm which Keeps Better Harmonies!

Procedures of Harmony Search n

Step 0. Prepare a Harmony Memory.

n

Step 1. Improvise a new Harmony with Experience (HM) or Randomness (rather than Gradient).

n

Step 2. If the new Harmony is better, include it in Harmony Memory.

n

Step 3. Repeat Step 1 and Step 2.

HS Operators 1. 2. 3. 4. 5.

Random Playing Memory Considering Pitch Adjusting Ensemble Considering Dissonance Considering

Random Playing

x ∈ Playable Range = {E3, F3, G3, A3, B3, C4, D4, E4, F4, G4, A4, B4, C5, D6, E6, F6, G6, A6, B6, C7}

Memory Considering

x ∈ Preferred Note = {C4, E4, C4, G4, C4}

Pitch Adjusting

x+ or x-, x ∈ Preferred Note

Ensemble Considering

( )

{

(

x i ¬ f x j , Max Corr x i , x j j

)}

Rule Violation (Parallel 5th)

b

Penalty = a (Violation Amount) + c

Example of Harmony Search

Initial Harmony Memory

Next Harmony Memory

With Three Operators

{1, 2, 3, 4, 5}

+1

f

1

4

2

=6

HS Applications for Benchmark Problems

Six-Hump Camel Back Function 1 4 2 Min f(x) = (4 - 2.1x + x1 ) x1 + x1 x2 + (-4 + 4x22 ) x22 3 2 1

*

Minimum Point f (0.08983,-0.7126) *

or f (-0.08983,0.7126)=-1.0316285

2.5

100

2.0

40.0 30.0

20.0

1.5

4.99

1.0

1.02

0.089 0.486

0.5

0.089

-0.882

x

2

2.38 0.0

2.38

0.486

-0.5

2.38

-1.0 -1.5

0.486 0.089

-0.882

10.0 40.0

15.0

30.0

-2.0

100 -2.5 -2.5 -2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

x1

f*(-0.08983, 0.7126) = -1.0316285 (Exact) f (-0.08975, 0.7127) = -1.0316285 (HS)

Multi-Modal Function

Artificial Neural Network - XOR

Bias

Sum of Errors in BP = 0.010 Sum of Errors in HS = 0.003

T

T

F

T

F

T

F

T

T

F

F

F

HS Applications for Real-World Problems

Sudoku Puzzle 2

9

4

5

3

8

7

6

1

5

6

1

2

7

9

3

4

8

8

3

7

1

6

4

2

5

9

7

4

9

8

1

3

6

2

5

6

2

3

9

4

5

1

8

7

1

8

5

7

2

6

9

3

4

4

5

2

6

9

1

8

7

3

3

1

6

4

8

7

5

9

2

9

7

8

3

5

2

4

1

6

Music Composition – Medieval Organum

Interval

Rank

Interval

Rank

Fourth

1

Fifth

2

Unison

3

Octave

3

Third

4

Sixth

4

Second

5

Seventh

5

Project Scheduling (TCTP)

University Timetabling

Internet Routing

Web-Based Parameter Calibration

RMSE: 1.305 (Powell), 0.969 (GA), 0.948 (HS)

Truss Structure Design Z

75 in.

(1)

2

n

W ( A ) = å g Ai Li

75 in.

i =1

(6)

22

14 20

(10)

1

3 (3) 7

5 10 13 15 (5)

12

75 in.

6

11

(4)

23

19

18 (7)

21

100 in.

(2)

4 8

9

17

100 in. Y

24

25 16

200 in. (9)

GA = 546.01, HS = 484.85

(8)

200 in.

X

School Bus Routing Problem 15 5

8

4 7

Depot

5

3

1 5

School 4

10 6

6 4

10

5

20

15 5

5

10 7

5

10 4

4

9

8

20

10

7

15 2

5

5 8

3

Min C1 (# of Buses) + C2 (Travel Time) s.t. Time Window & Bus Capacity GA = $409,597, HS = $399,870

Generalized Orienteering Problem Max. Multi-Objectives 1. Natural Beauty 2. Historical Significance 3. Cultural Attraction 4. Business Opportunity

ANN HS

Case1 12.38 12.38

Case2 13.05 13.08

Case3 12.51 12.56

Case4 12.78 12.78

Case5 12.36 12.40

Water Distribution Network Design 1 1

§

15

2

15

2

§

14

3

§

14

3 4

13

4

13

5

19

12

5 6

12

18 17

18

6 7 7 11 8

9 10 16 17

§

19

10 9

20 21

20 16

§ §

11

8

§

§

MP: $78.09M GA: $38.64M (800,000) SA: $38.80M (Unknown) TS: $37.13M (Unknown) Ant: $38.64M (7,014) SFLA: $38.80M (21,569) CE: $38.64M (70,000) HS: $38.64M (3,373) § 5 times out of 20 runs

Large-Scale Water Network Design

Huge Variables u (454 Pipes) n GA = 2.3M Euro n HS = 1.9M Euro n

Multiple Dam Operation

Max. Benefit (Power, Irrigation) GA = 400.5, HS = 401.3 (GO)

Hydrologic Parameter Calibration I O Wedge Storage = K x (I - O)

S t = K [xI t + (1 - x)Ot ]m

Prism Storage =KO

SSQ =

n

å

t =1

(

O t - Oˆ t

O

Mathematical = 143.60, GA = 38.23, HS = 36.78

)

2

Ecological Conservation

With 24 Sites, SA = 425, HS = 426

Satellite Heat Pipe Design

Satellite Heat Pipe Design BFGS HS Minimize Mass Maximize Conductance BFGS: Mass =25.9 kg, Conductance = 0.3808 W/K HS: Mass = 25.8 kg, Conductance = 0.3945 W/K

Oceanic Oil Structure Mooring

RNA Structure Prediction

Medical Imaging

Radiation Oncology

Astronomical Data Analysis

All that Jazz n n n n n

Robotics Visual Tracking Internet Searching Management Science Et Cetera

0.300 Pitch Adjustment

0.250 Probability

Memory Consideration

0.200

Random Selection

0.150 0.100 0.050 0.000 1

2

3

4

5

6

7

Discrete Variable

8

9

10

Stochastic Partial Derivative of HS n( xi ( k )) n ( xi ( k - m ) ) ¶f 1 = × (1 - HMCR) + × HMCR × (1 - PAR ) + × HMCR × PAR ¶xi K i HMS HMS Pipe 7

Random Selection

Memory Consideration

Pitch Adjustment

Total Gradient

1.000

Probability

0.100

0.010

0.001

0.000 1

2

3

4

6

8

10 12 14 16 18 20 22 24

Pipe Diameter (inch)

Stochastic Co-Derivative of HS xiNew

¶f ¶xi ¶x j

é ¶f ê ¬ xi (k ) where max k ê ¶x ¶x êë i j

= xi = xi ( k ) x j = x j (l )

n( xi (k ) Ù x j (l )) HMS

ù , k = 1, K , K i ú ú xi = xi ( k ) úû x j = x New j

× (1 - PAR )

n(xi (k ) Ù x j (l m m) ) PAR n(xi (k m m) Ù x j (l ) ) PAR × + × + HMS 2 HMS 2

Parameter-Setting-Free HS Overcoming Existing Drawbacks u Suitable for Discrete Variables u No Need for Gradient Information u No Need for Feasible Initial Vector u Better Chance to Find Global Optimum n Drawbacks of MetaMeta-Heuristic Algorithms u Requirement of Algorithm Parameters n

é y11 = Random y12 = Pitch L y1n = Memoryù ú ê 2 2 yn2 = Pitch ú ê y1 = Memory y2 = Memory L ú ê M L L L ú ê HMS HMS HMS úû êëy1 = Memory y2 = Random L yn = Memory

(

n yij = Memory HMCRi = HMS

)

(

n y i j = Pitch PAR i = HMS

)

Wikipedia (Web Encyclopedia)

Books on Harmony Search

Visitor Clustering (As of Jan. 2011)

Paper & Book

Thesis & Syllabus

Citations in Major Literature in tantum ut si priora tua fuerint parva, et novissima tua multiplicentur nimis. Iob 8:7

What is Your Contribution?

Question for Harmony Search? Visit the Website HarmonySearch.info