Competitive Ant Colony Optimisation Marcus Randall School of Information Technology Bond University, QLD 4229, Australia Ph: +61 7 55953361 [email protected]

Abstract. The usual assumptions of the ant colony meta-heuristic are that each ant constructs its own complete solution and that it will then operate relatively independently of the rest of the colony (with only loose communications via the pheromone structure). However, a more aggressive approach is to allow some measure of competition amongst the ants. Two ways in which this can be done are to allow ants to take components from other ants or limit the number of ants that can make a particular component assignment. Both methods involve a number of competitions so that the probabilistic best assignment of component to ant can be made. Both forms of competitive ant colony optimisation outperform a standard implementation on the benchmark set of the assignment type problem, generalised assignment. Keywords: heuristic search, ant colony optimisation.

1

Introduction

The premise of ant colony optimisation (ACO) is that cooperating agents each construct a single solution to a problem in a single iteration [4]. Cooperation (i.e., the sharing of information) is achieved implicitly via a pheromone mechanism. However, this should be balanced with more aggressive competitive strategies to realise greater search effectiveness. It is believed that this is in part may be achieved by allowing ants to compete amongst the members of the colony for the right to use specific solution components. Merkle and Middendorf [10] describe another form of competitive behaviour for ACO. In their model, solutions (ants) within an iteration compete with one another to determine the relative strength with which they can update the pheromone repository for their individual component decisions. Working with the simple test permutation problem showed that the competitive control pheromone system could outperform a standard ACO implementation. There have been some other attempts that have moved away from standard ACO algorithms and challenged the notion of always having a colony of constructive solution producing ants. The most notable of these are the hybrid ant systems. HAS-QAP (Hybrid Ant Systems for the QAP) uses pheromone information to manipulate complete solutions, applying standard local search H.G. Okuno and M. Ali (Eds.): IEA/AIE 2007, LNAI 4570, pp. 974–983, 2007. c Springer-Verlag Berlin Heidelberg 2007 

Competitive Ant Colony Optimisation

975

operators [7] rather than a constructive process. The local search operator used is to swap the facilities at two locations. The first location is chosen at random. The second is chosen such that either the combined pheromone levels of both locations are maximum (corresponding to exploitation) or probabilistically according to the second pheromone amount (exploration). Another hybrid ant system (for the sequential ordering system, HAS-SOP [6]) differs slightly from HAS-QAP. It retains the normal constructive component, but combines this with a sophisticated local search procedure that is tailored to the sequential ordering problem. Gambardella and Dorigo [6] found that this new ACO derivative outperforms one of the best available heuristics for the problem. The paper is organised as follows. Section 2 describes assignment type problems with particular regard to the test problem, the generalised assignment problem (GAP) while Section 3 shows how ACO may be applied to this problem. Section 4 gives an account of how competitive ant colonies differ from standard versions of the algorithm. This description is given in terms of the GAP. Finally, Sections 5 and 6 give the computational results and conclusions respectively.

2

Assignment Type Problems

The competitive approaches described herein are particularly suitable for assignment optimisation problems. Therefore this paper focuses on solving the generalised assignment problem (GAP) as it is the prototypical assignment problem and very suitable for this first implementation. Costa and Hertz [3] refer to these problems as “Assignment Type Problems” (ATPs). These include such problems as graph colouring, quadratic assignment, set covering and generalised assignment. 2.1

ACO for ATPs

There has been a relatively small amount of work on using ACO based techniques specifically for ATPs. The major problems of this type that have been solved are graph colouring and GAP. Louren¸co and Serra [8] present hybrid meta-heuristic search techniques for solving the GAP. The two base meta-heuristics they use are greedy randomised adaptive heuristic (GRAH) and the ant system heuristic (ASH) – based on MAX − MIN Ant System. Both of these have their native internal local search phases removed. The local search methods are replaced by descent and tabu search components having move and ejection chain neighbourhoods. A total of seven methods are tested which include the original MAX − MIN Ant System and GRASP (greedy randomised adaptive search procedures). The methods are differentiated on constructive meta-heuristic, local search technique and neighbourhood operator. On the whole, the ant-based search techniques outperformed the greedy methods on small GAP instances.1 In particular, ASH combined with 1

These instances correspond to the problem sets gap1 to gap12 (5,15 to 10,60 jobs/agents) developed by Chu and Beasley [2].

976

M. Randall

tabu search based on ejection chains performed the best overall. A further exploration of ACO mechanisms for the GAP was undertaken by Randall [11]. This work examined the use of different component selection heuristics as well as local search strategies. It was concluded that provided there is a powerful local search mechanism, it is more important to select solution components (i.e., assignments of jobs to agents) based on satisfying capacity constraints rather than optimising the objective function. Costa and Hertz [3] solve ATPs by generalised evolutionary approaches (based on ACO). The two parts of such generalised algorithms are the self adaptation phase and the cooperation phase in which members of a population build/modify a solution and learn from the other members respectively. Using ant colony optimisation, the authors could satisfactorily solve various instances of the graph colouring problem. 2.2

The Generalised Assignment Problem

The generalised assignment problem [9] is a problem in which jobs are assigned to agents for these agents to perform subject to capacity constraints. Each job may be performed by one agent only. The aim is to minimise the total cost of assigning the jobs to the set of agents. Equations 1-4 give the integer linear programming model of the problem. Minimise

N  M 

cij xij

(1)

i=1 j=1

s.t.

N 

aij xij ≤ bj

∀j

1≤j≤M

(2)

i=1 M 

xij = 1

∀i 1 ≤ i ≤ N

(3)

j=1

xij ∈ {0, 1} ∀i 1 ≤ i ≤ N

∀j

1≤j≤M

(4)

Where: cij is the cost of assigning job i to agent j, aij is the resource required by agent j to perform job i, xij is 1 if job i is assigned to agent j, 0 otherwise, bj is the capacity of agent j, M is the number of agents and N is the number of jobs. Benchmark problem instances are provided by Beasley [1]. In this study, large and tightly constrained instances are solved. These correspond to the A, B, C and D type problem sets and have the following definitions (reproduced from Chu and Beasley [2]):

Competitive Ant Colony Optimisation

977

Type A: aij are integers from U (5, 25), cij are  integers from U (10, 50) and bi = n 0.6( m )15+0.4R where R = maxi∈I j∈J,Ij =i aij and Ij = min[i | cij ≤ ckj , ∀k ∈ I]. Type B: aij and cij are the same as Type A and bi is set to 70% of the value given in Type A.  a Type C: aij and cij are the same as Type A and bi = 0.8 j∈J mij . Type D: aij are integers from U (1, 100), cij =a111 − aij + e where e are integers from U (−10, 10) and bi = 0.8 j∈J mij . where U (a, b) represents a uniform random number, I is the set of agents and J is the set of jobs. Note that n and m correspond to N and M respectively given in Equations 1-4. As can be seen from the above, each problem type is more tightly constrained than the preceding problem type. Each category of problem instances ranges in size from 5 agents, 100 jobs to 20 agents, 200 jobs.

3

Applying Standard ACO to ATPs

The algorithm described here follows the ant colony system (ACS) implementation of the ACO meta-heuristic. The GAP is used to illustrate the algorithm mechanics. In discrete time steps, allow each ant to uniquely assign agents to successive jobs until all have been allocated. After adding a job to the solution, the amount of pheromone on the job-agent assignment is modified. Pheromone is a scalar quantity that allows ants to communicate with one another about the utility of assigning an agent to a job. The accumulated strength of pheromone of job i and agent j is denoted by τ (i, j). The heuristic measure of the suitability of assigning agent j to job i is given by η(i, j). This could either correspond to the resource (the a matrix) or the cost (the c matrix) of assigning agent j to job i. At the beginning of each time step, Equation 5 is used to select agent j for job i. The agents that ensure that the partial solution satisfies the capacity constraints are given by the set Jk (i), where k represents the ant and i is the job/step. If Jk (i) becomes an empty set at any step of an iteration (i.e., there is no agent to which job i can be assigned, as they do not have sufficient capacity), ant k terminates. It is no longer considered for the duration of the iteration. Equation 5 is a selection technique that favours the best combination of pheromone level and the heuristic measure. In order to allow for other combinations of agents and jobs, Equation 6 selects the agent probabilistically. This is essential to avoid premature convergence to a small group of elite solutions. The use of each equation is governed by the q0 parameter. q is a uniform random number that is generated at each step of the ACS process. β governs the relative importance of the heuristic. It is set as a negative value for minimisation problems such as the GAP.    arg maxs∈Jk (i) τ (i, j)[η(i, j)]β if q ≤ q0 j= (5) Equation 6 otherwise

978

M. Randall

 pk (i, j) =



τ (i,j)[η(i,j)]β τ (i,u)[η(i,u)]β

u∈Jk (i)

0

if j ∈ Jk (i)

(6)

otherwise

The pheromone level of the selected agent for a particular job is modified according to the local updating rule in Equation 7 (while the other pheromone values are left unaltered). The equation slightly decreases the amount of pheromone on the job-agent combination so that ants have a measure of diversity between their solutions within a particular colony. τ (i, j) ← (1 − ρ) · τ (i, j) + ρ · τ0

(7)

Where: ρ is the local pheromone decay parameter, 0 < ρ < 1, τ0 is the initial pheromone amount, i is the job and j is the agent. Louren¸co and Serra [8] initialise the pheromone level as τ (i, j) = c1ij , where i is the job and j is the agent. Once all ants have assigned an agent to each job and have hence completed an iteration of the algorithm, the components of the best solution are rewarded with an increase in their pheromone level while other component values are decayed/evaporated (Equation 8). Q if (i, j) ∈ S (8) τ (i, j) ← (1 − γ) · τ (i, j) + γ · L 0 otherwise Where: γ is the global pheromone decay parameter, 0 < γ < 1, L is the cost of the best solution found to date, Q is a constant [5] and S is the set of job-agent pairings of the best solution found since the beginning of the optimisation process.

4

Competitive Ants

In traditional ant colony meta-heuristics, ants are treated as cooperative agents that generate solutions to solve combinatorial optimisation problems. To increase solution quality, it is possible to allow ants to compete for solution components. This will increase the likelihood that the final assignments of components will optimise the objective function. Two approaches to achieve this are outlined. Compete 1: Ants creating partial solutions: In assignment type problems, a solution value is typically assigned to a group. For the GAP each job is assigned to only one agent. Therefore allow each ant in the colony to

Competitive Ant Colony Optimisation

979

represent a single agent (corresponding to a partial solution). The combination of the ants’ partial solutions gives a complete solution. Defining ants in this way allows them to compete for component values (i.e., jobs). Thus in terms of the target application, the number of ants would equal the number of agents (M ). At each step of the revised algorithm, each ant chooses a component value to assign to its group (e.g., a job to an agent). In the first instance an ant evaluates all solution components to work out the best value at this time. This component value must be drawn from either the unallocated values or from those that have been assigned to other ants/agents. In the case of the latter, the ant cannot just reallocate the value to itself. It must be won by means of a competition between the two ants. Given that k represents the current ant/agent and v is the agent to which the job belongs. The probability of ant k winning the competition is given by Equation 9. Of course, the probability of ant/agent v retaining the job is (1 − pk ). pk =

(τkj

τkj × ckj × ckj ) + (τvj × cvj )

(9)

Where: τij is the pheromone strength on the assignment of ant/agent j to job i and c is as defined in Section 2.2. A uniform random number is then generated to determine the winner. If the original ant wins, it keeps the component and the current ant must go through the process again with the next value on its list. Conversely, the value is reassigned to the current ant and the next ant is then allowed to select a component. If an ant at any stage is unable to choose a value (because it does not have the spare capacity to accommodate any assignable value), nothing is assigned. In case of the GAP, this allows agents to have variable length job lists while ensuring they utilise the available capacity. The pseudocode for one iteration is given in Algorithm 1. Compete 2: Ants competing for solution component assignments: Ants that construct full problem solutions may also use the idea of competition. In the case of ATPs, competition can be used to achieve intra-colony diversification (see Randall [12]) by restricting how often an assignment can be made between the ants. For instance, a particular job and agent combination may be permitted to be used by more than one ant (in a single iteration) with a given probability.2 If this fails and the current ant is not automatically allowed to make the assignment, a competition for it (as per Equation 9) must be held. The competition is between the current ant and the ant for which the assignment is of the highest cost (i.e., the weakest ant). The loser will be given a blank assignment instead. At the end of each 2

In the experiments, the probability is given as assignment across the other ants.

1 n

where n is the number of the same

980

M. Randall

Algorithm 1. The competitive ant colonies algorithm 1: while the size of the unallocated pool is not 0 do 2: for each ant do 3: Generate the candidate list of components a in decreasing order of fitness 4: while a component is not chosen and a is not exhausted do 5: v = Choose a candidate component value from a 6: if v is part of another ant’s partial solution then 7: if the current ant wins the competition for v then 8: Add v to the current ant’s partial solution 9: Remove v from the former ant 10: end if 11: else 12: Remove v from the unallocated pool 13: end if 14: end while 15: end for 16: end while

iteration, jobs that have a blank assignment will be allocated an agent by a greedy heuristic. This heuristic and the competitive mechanism add very little computational overhead to the algorithm. 4.1

Local Search

As with all ant colony techniques, it is unlikely that the constructed solutions of the colony will be locally optimal, thus a form of descent or local search is necessary. The two local search operators that have been found to be effective for ACO and the GAP [11] are: swap: Given two jobs, the agent assignments of the two jobs are swapped. change: The agent of a particular job is changed to another agent. Exhaustive neighbourhood searches, terminating when an improved solution cannot be obtained, can be used for both operators.

5

Computational Experiments

The computing platform used to perform the experiments is a 3GHz Pentium 4 based PC. Each problem instance is run across ten random seeds. The experimental programs are coded in the C language and compiled with gcc. Ant Colony System (ACS) [5] is used as the ACO meta-heuristic. The ACS parameter settings are given by the set {β = −2, γ = 0.1, ρ = 0.1, m = 10, q0 = 0.9}. These values have been found to be robust by Dorigo and Gambardella [5]. The test suite of problems is the large-sized set of Chu and Beasley [2]. Both of the new solvers and a standard implementation are run using local search based on swap and change neighbourhoods (see Section 4.1). The standard

Competitive Ant Colony Optimisation

981

Table 1. Cost results of running the three solvers. These are represented as relative percentage differences (RPDs) from the optimal cost. RPD is given as a−b × 100 where b a is the best cost achieved in a run and b is the optimal cost of the problem instance. Note that ‘Min’, ‘Med’ and ‘Max’ denote minimum, median and maximum respectively. Bolded items indicate the best result for the combination of problem and measure. Optimal Compete 1 Compete 2 Problem Cost Min Med Max Min Med Max A5-100 1698 0 0 0 0 0 0 0 0 0 0 0 A5-200 3235 0 0 0 0 0 0 A10-100 1360 0 0 0 0 0 0 A10-200 2623 0 0 0 0 0 0 A20-100 1158 0 0 0.04 A20-200 2339 0.04 0.04 0.04 0 1.14 B5-100 1843 0.43 0.71 0.92 0.65 1 B5-200 3553 0.51 0.65 0.79 0.53 0.66 0.87 0 0.07 B10-100 1407 0.07 0.28 0.43 0 B10-200 2831 1.31 1.45 1.66 1.27 1.54 1.62 B20-100 1166 0.69 1.03 1.11 0.34 0.6 0.77 B20-200 2340 0.64 0.75 0.81 0.43 0.56 0.64 C5-100 1931 0.31 0.57 0.73 0.67 0.85 1.04 C5-200 3458 0.52 0.67 0.81 0.35 0.64 0.75 C10-100 1403 1.14 1.43 1.57 0.93 1.18 1.57 C10-200 2814 0.92 1.33 1.46 1.39 1.58 1.78 C20-100 1244 1.69 1.89 2.49 0.88 1.17 1.53 C20-200 2397 1.63 1.98 2.13 1.34 1.81 2 D5-100 6373 1.99 2.38 2.51 2.97 3.21 3.4 D5-200 12796 2.31 2.5 2.79 3.17 3.26 3.39 D10-100 6379 3.45 3.66 3.87 4.04 4.37 4.61 D10-200 12601 2.77 3.02 3.18 3.56 3.84 3.95 D20-100 6269 3.51 3.55 3.96 3.53 3.88 4.15 D20-200 12452 3.17 3.48 3.7 4.09 4.26 4.41

Control Min Med Max 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.04 3.36 4.72 6.02 0.73 0.89 1.04 0.07 0.14 0.36 1.38 1.7 2.08 0.26 0.69 0.94 0.51 0.68 0.81 0.62 1.09 1.35 0.49 0.78 1.01 1.5 1.75 2.07 1.46 1.74 1.92 1.53 1.65 2.25 1.88 2.07 2.17 3.36 3.88 4.14 3.4 3.56 3.74 4.33 4.45 4.73 3.4 3.92 4.03 3.73 4.07 4.4 4.14 4.33 4.49

implementation of ACS (herein referred to as ‘control’) is the same used in Randall [11]. The two solvers for which ants complete full solutions used three thousand iterations per run. As the other competitive approach only produces one solution per iteration, a larger number of iterations (twenty thousand) are allotted to it per run as being roughly equivalent. A Kruskal-Wallis comparison will be used to determine if there is any discernable difference in solution quality across the approaches. 5.1

Results

The results of running the three solvers are given in Table 1 and graphically in Figure 1. Three trends from this table are evident. All solvers can more or less solve the A type instances to optimality. Compete 2 often finds the best solutions to the middle B and C problems. Compete 1, however, consistently find better solutions on the tightly constrained D problems.

982

M. Randall

Fig. 1. Graphical form of the minimum and maximum RPD results

A statistical analysis shows that there is a significant difference between the methods. It was found that the two competitive ant colony approaches produced statistically significantly better results than the control solver (at the 0.05 level). This therefore represents an improvement on the results for the same test problems used in Randall [11]. There was no significant difference between the two solvers. Additionally, there was little variation in the overall computational runtimes used by the solvers.

6

Conclusions

The balance between competitive and cooperative approaches is an important part of multi-solution (particularly evolutionary) optimisation algorithms. Given the test cases presented herein, two forms of competition added to ACO showed that significantly improved solutions could be found compared to a standard implementation. The nature of the competitions in this paper was quite simple and could be more sophisticated. This will need to be done if a greater variety of problem types are to be solved in this manner. It will also be interesting to see how these notions extend to other multi-solution (population) heuristics, particularly genetic algorithms.

Competitive Ant Colony Optimisation

983

References [1] Beasley, J.: OR-Library (2007) http://people.brunel.ac.uk/mastjjb/jeb/info.html [2] Chu, P., Beasley, J.: A genetic algorithm for the generalised assignment problem. Computers and Operations Research 24, 17–23 (1997) [3] Costa, D., Hertz, A.: Ants can colour graphs. Journal of the Operational Research Society 48, 295–305 (1997) [4] Dorigo, M., Di Caro, G.: The ant colony optimization meta-heuristic. In: Corne, D., Dorigo, M., Glover, F. (eds.) New Ideas in Optimization, pp. 11–32. McGrawHill, London (1999) [5] Dorigo, M., Gambardella, L.: Ant Colony System: A cooperative learning approach to the traveling salesman problem. IEEE Transactions on Evolutionary Computation 1(1), 53–66 (1997) [6] Gambardella, L., Dorigo, M.: HAS-SOP: An hybrid ant system for the sequential ordering problem. Technical Report IDSIA-11-97, IDSIA (1997) [7] Gambardella, L., Taillard, E., Dorigo, M.: Ant colonies for the quadratic assignment problem. Journal of the Operational Research Society 50, 167–176 (1999) [8] Lourenco, H., Serra, D.: Adapative search heuristics for the generalized assignment problem. Mathware and Soft Computing 9, 209–234 (2002) [9] Martello, S., Toth, P.: An algorithm for the generalised assignment problem. In: Proceedings of the 9th IFORS Conference, Hamburg, Germany (1981) [10] Merkle, D., Middendorf, M.: Competetition controlled pheromone update for ant colony optimization. In: Dorigo, M., Birattari, M., Blum, C., Gambardella, L.M., Mondada, F., St¨ utzle, T. (eds.) ANTS 2004. LNCS, vol. 3172, pp. 95–105. Springer, Heidelberg (2004) [11] Randall, M.: Heuristics for ant colony optimisation using the generalised assignment problem. In: Proceedings of the Congress on Evolutionary Computing 2004, Portland, Oregon, pp. 1916–1923 (2004) [12] Randall, M.: Maintaining diversity within individual ant colonies. In: Abbass, H., Bossamaier, T., Wiles, J. (eds.) Recent Advances in Artificial Life. Advances in Natural Computation, vol. 3, pp. 227–238. World Scientific, New Jersey (2005)