The Ant Heuristic “Reloaded”

As a swarm-based optimization heuristic, the Ant System (AS) was proposed to deal with the Traveling Salesman Problem (TSP). Classified as a constructive approach, AS was inspired by the ants’ social behavior.In Practice, its implementation reveals three basic variants that exploit two operating models: a first “natural” model, where ants move and update pheromones, with consideration of distances between towns, to define the Ant-Quantity variant, and without considering distances for the Ant-Density variant. Or an “abnormal” second model, where ants move and delay updates until all ants have completed a full cycle, then consider the tour length which defines the Ant-Cycle that was claimed to be the best variant. We reload the AS and reconsider these three basic ant algorithms and their respective two models. We propose to explore an “overlooked” third model, which consists of further expanding the “abnormal” ants’ attitude, forcing them to move and update pheromones, simultaneously, after every single move, thus conceiving the Ant-Step model. Keeping the option whether or not to consider distances, we propose two new AS basic algorithms: the Ant-Step-Quantity and the AntStep-Density. The aim of this paper is to present and assess these two new basic AS variants in respect to the three existing ones, specially the one considered the best, through an experimental study on various symmetric TSP benchmarks.


Introduction
For a class of Combinatorial Optimization Problems (COP), the exact solution can be found using exact methods, such as: Linear/Dynamic Programming, Simplex Method, Backtracking, Branch and Bound and Branch and Cut. For other problems, known as NP-hard COP, an exact solution can't be reached (either because it is unreachable within a reasonable time or doesn't exist at all), in this case, we just look for an approximate solution using approximate approaches.
Without any guarantee of optimality, approximate methods use appropriate and specific exploration (diversification) and exploitation (intensification) mechanisms to explore the problem's search space. Some of them deal with a single solution and are known as unique solution based methods such as: Local Search, Simulated Annealing, Tabu Search, Variable Neighborhood Search, Iterated Local Search and Global Local Search. Whereas others are population based, dealing with an aggregate of solutions, like: Evolutionary Algorithm, Genetic Algorithm and Swarm Based Algorithms such as: Particle Swarm Optimization, Ant Colony Optimization, Cuckoo Search and Grey Wolf Optimizer.
These approaches can be clustered in two categories: Constructive or Ameliorative ones. A constructive approach starts from scratch and gradually constructs a solution or a set of solutions (such as: Ant Colony Optimization). An In this paper, we define two new variants of the Ant Algorithm: the Ant-Step-Density and the Ant-Step-Quantity Algorithms, where Ants are dotted with an efficient data disseminating mechanism to increase the system's reactivity.
Through Section 2, we introduce, as a background of our work, a brief overview of the application domain: defining the Traveling Salesman Problem, and presenting the principles of the three basic algorithms of the AS heuristic: Ant-Quantity, Ant-Density and Ant-Cycle, as proposed by Dorigo et al.. In Section 3, we present, in our style, the two models used to define the previous three basic variants. Through Section 4, we give some comments on the basic variants, their corresponding models and algorithms. In Section 5, we present our contribution. We signal, first, an "overlooked" choice, when commenting the Dorigo et al.'s work. After that, we give the corresponding models and algorithms of the two proposed new methods: the Ant-Step-Density and the Ant-Step-Quantity.
Throughout Section 6, we assess and validate our new basic variants in respect to the previous existing ones. Section 7 concludes the paper. 284 THE ANT HEURISTIC "RELOADED"

The Traveling Salesman Problem and the Ant Heuristic
Mostly, discrete Combinatorial Optimization Problems (COP) belong to NP-hard class problems. For this class of problems an exact solution is out of reach, forcing the use of heuristic approaches (more details in [13]). The Traveling Salesman Problem (TSP) may be the most famous example of these problems.
This problem consists in a traveling salesman looking for the shortest way to visit n towns without going twice to any town, except starting and ending in the same town. When paths between any given two towns (in case of more than one path) are equal the TSP is symmetric otherwise the problem is asymmetric.
The use of ants for symmetric TSP begins with the use of a novel heuristic based on the ants' social behavior, giving rise to the Ant System (AS) or the Ant Algorithm (AA) of Dorigo et al., with their three basic variants: Ant-Density, Ant-Quantity and Ant-Cycle.
The AS heuristic [7] was always competitive and can be considered as a landmark in solving TSP. Reference [20] presents a comparative study between Ant Colony Optimization (ACO), Genetic Algorithm (GA) and Simulated Annealing (SA).
Dorigo et al. [7,11,12] deal with a symmetric TSP with euclidean distance, and use the following notations: • n: number of towns, m: number of ants; • T abulist k memorizes for Ant k the towns already visited up to time t; • d ij , the euclidean distance between two towns T i and T j : The intensity of pheromone trail on P ath ij at t+1, given by equation (1): ρ the evaporation coefficient, at t=0, τ ij (0) = Q c (Initial pheromone quantity); • η ij = 1 dij the visibility; • The Ant k transition probability from T i to T j , calculated by equation (2): allowed k = n -T abulist k , α and β parameters that control effect of pheromone intensity versus visibility.

The Exploited Ant System Models and the Three Basic Variants
In practice, and without any mathematical referential model, two choices were exploited: • Model 1: Desynchronized updates after every single move.
• Model 2: Synchronized updates after an entire tour.
The first model generates two variants and the second one gives rise to one variant, as presented in the following subsections.

The First Variant: Ant-Density Algorithm
Updates are performed after every single move and distances between towns aren't considered. Ant k going from T i to T j leaves a Q 1 units of pheromone on P ath ij , between t and t+1, according to equation (3): Updates are performed after every single move and distances between towns are considered. Ant k going from T i to T j leaves a quantity Q 2 of pheromone depending on the length of P ath ij , between t and t+1, according to equation (4):

The Third Variant: Ant-Cycle Algorithm
Updates are performed after an entire tour, it is an adaptation of the Ant-Quantity approach. The pheromone update is not computed at every move or step but after a complete tour (n steps/moves). If Ant k goes from T i to T j between t and t+n, it leaves on P ath ij a quantity Q 3 by the length L k of its performed tour, according to equation (5): Equation (1), in this case, becomes: The experimental study on several TSP benchmarks allows the Ant System authors to take the following decisions: • For Ant-Density/Quantity Algorithms, the best parameters set is: α=1, β=5, ρ=0.99; the evaporation is equal to: 1 − ρ = 1 − 0.99 ≈ 0, (no evaporation); • For Ant-Cycle Algorithm, the best parameters set is: α=1, β=5, ρ=0.5; evaporation rate: 1 − ρ = 1 − 0.5 = 0.5, (50% evaporation); The authors concluded that the most performing variant was the Ant-Cycle, due to the use of global information [7,11,12].

Analysis and Comments on the Exploited Ant System Models and the Three Basic Variants
Figures 1 and 2 illustrate and depict the two models used by the three AS basic variants presented in Section 3, respectively, the Ant-Density/Quantity and the Ant-Cycle variants. The obvious observation on these three variants is the fact that the Ant-Cycle variant biases, clearly, the natural ant's behavior. In this case, ants carry out the updates (pheromone's evaporation and deposit) after an entire cycle, 286 THE ANT HEURISTIC "RELOADED" using the realized tour's length. After that, they join the start line and wait for a given next iteration, defining, "undeliberately", an "implicit synchronization point" (See the model Figure 2 and Algorithm 2 as a proposed algorithm for the case).
Whereas Ant-Density and Ant-Quantity reflect the ant behavior in nature, and the latter even more; considering the possible effect of distance on the pheromone evaporation in nature. In this first used model and for both options, ants carry out the updates (pheromone's evaporation and deposit) with each step, after any single move between two towns, either taking in consideration or not the distance between towns (See the model, Figure 1 and the proposed algorithm for the case, Algorithm 1).
Algorithm 1 (Ant-Density/Quantity Algorithm) Initialization % t=0 % The k th Ant For i = 2 to n % Perform a tour Select T j , % Next Town using equation (2) % Best P ij T abulist k [i] = T j % Ant moved to T j Update τ ij % Each ant alone, after any single move using equation (1) and equation (3)

End Ant End Parallel
Get the shortest tour among the m tours End For Get the shortest tour among the t max tours So, how can the Ant-Cycle variant as a "biased" imitation, implementing an "abnormal" ant behavior perform better than an "exact" imitation of nature realized with the Ant-Density or the Ant-Quantity variants? [35].
This "abnormal" behavior adopted by the Ant-Cycle variant, forces ants to "synchronize" the pheromone updates after a whole cycle, all together, just before starting a next iteration (See Figure 2), such behavior, paradoxically, increases the ants' reactivity. Effectively this second model provides, for ants, an efficient disseminating data process in the system by depositing, adequately, the needed data and increases consequently their reactivity.
Update τ ij % All ants together, after an entire cycle using equation (6) and equation (5 Consequently, all improved variants of this heuristic for TSP such as: Ant Colony System (ACS) in [11], MAX-MIN Ant System; MMAS in [30], Rank Based Ant System; AS rank in [3] and Elitist Ant System; AS elit or EAS in [3,10] (Table 1) gained on behalf of the Ant-Cycle model's adequacy (Figure 2). For most cases, the obtained results place the AS approach among the best available heuristics, which encouraged Dorigo to propose the ACO meta-heuristic [12]. We have to highlight here that the application of an ant approach for a given problem is constrained by the following facts and conditions [9]: 1. The problem has to be seen as a graph; 2. The emerging collective behavior is autocatalytic, it defines a positive feedback loop process: the more pheromone trail on a path, the more it is attractive and used by ants; 3. The greedy force to construct a solution and to guide the autocatalytic process: the transition probability between towns; 4. The satisfaction and stop criterion to avoid a pheromone trail combinatorial explosion: the tabu list.

The Overlooked Ant System Model and the Ant-Step Basic Variants
The use of "global information", within the Ant-Cycle model, results from the ants' attitude to update pheromones, all together, after an entire tour, so, if we provide "global information" in the case of the alternative Ant-Density and Ant-Quantity variants, it "can" improve their performance too.
Analogically, using "global information", in the case of Ant-Density or Ant-Quantity, can be performed if the ants "choose" to "wait" for the others to update pheromones, all together, after every single move or step, defining, "deliberately", an "explicit synchronization point".
So when analyzing the two choices explored by Dorigo et al. to define their three AS basic variants, an idea emerges to explore a third "overlooked" choice as follow: • Model 3: Synchronized updates after every single move (See Figure 3 and Algorithm 3).
It is a "combination" of the two previous models: Model 1 and Model 2; the updates are realized after every single move (like in Model 1), all ants synchronize together their updates (such in Model 2) either or not to consider the distances between towns (like offered in Model 1). It generates two new basic variants, as presented in the following subsections.
% The k th Ant For i = 2 to n % Perform a tour Select T j , % Next Town using equation (2) % Best P ij T abulist k [i] = T j % Ant moved to T j Synchronization Update τ ij % All ants together, after any single move using equation (7) and equation (3)

End Ant End Parallel
Get the shortest tour among the m tours End For Get the shortest tour among the t max tours 290 THE ANT HEURISTIC "RELOADED"

The Fourth Variant: Ant-Step-Density Algorithm
Synchronized updates are performed after every single move and distances between towns aren't considered. All ants Ant k going from any given T i to T j leave, simultaneously, a Q units of pheromone on all used P ath ij , between t and t+1 according to equation (3).

The Fifth Variant: Ant-Step-Quantity Algorithm
Synchronized updates are performed after every single move and distances between towns are considered. All ants Ant k going from any given T i to T j leave, simultaneously, a quantity Q of pheromone for every unit of length on all used P ath ij , between t and t+1 according to equation (4).
Equation (1), for these cases, becomes: Table 2 recapitulates features of the three AS basic variants presented in Section 3, and our two new proposed ones (proposed in Section 5).  [36].

Variants
Updates Frequency Ants behavior We recognize and agree that these new variants (Ant-Step-Density Algorithm and Ant-Step-Quantity Algorithm) display, even "more or less", deeper "abnormal" ant behavior, forcing the ants to "wait" every single move between two towns, to perform, at the same time, the pheromone updates, instead of an entire cycle within the Ant-Cycle variant (See the model, Figure 3 and the proposed algorithm for the case, Algorithm 3) [35]. To be more precise, considering the frequency of the "abnormal" ant behavior (every single move), the Ant-Step is more "abnormal", but considering the waiting delay (a whole complete tour), the Ant-Cycle will be, then, more "abnormal".

Experimental Validation
To validate our proposed new algorithms, we perform many executions for each variant implementation (100-500 iterations) on various symmetric TSP benchmarks.
First, we implement the previous three AS basic variants to produce Ant-Density Algorithm, Ant-Quantity Algorithm using Algorithm 1, and the Ant-Cycle Algorithm using Algorithm 2, namely: • A-D A: Ant-Density Algorithm. • A-Q A: Ant-Quantity Algorithm.
• A-C A: Ant-Cycle Algorithm. Table 3 provides a results' synthesis of these three basic AS variants.
After that, we use Algorithm 3 to implement our two proposed variants: the Ant-Step-Density and the Ant-Step-Quantity Algorithms. These last algorithms define the "overlooked" basic AS variants, namely: Table 3. Comparing performed tours' lengths for the Ant-Density/Quantity/Cycle Algorithms on symmetric TSP benchmarks [36].

Variants Benchmarks
A • A-S-D A: Ant-Step-Density Algorithm.
• A-S-Q A: Ant-Step-Quantity Algorithm. Table 4 provides a results' synthesis of the two new proposed variants, compared to the Ant-Cycle Algorithm. Table 4. Comparing performed tours' lengths for the Ant-Step-Density/Quantity and the Ant-Cycle Algorithms on symmetric TSP benchmarks [36]. The first set of tests returned the following results:

Variants Benchmarks A-S-D A A-S-Q A A-C A
1. The Ant-Cycle Algorithm performs better than the Ant-Density / Quantity Algorithms, with 7/10 cases, less performant in 1/10 cases and with equal performance in 2/10 cases (Table 3); 2. The Ant-Step Algorithm (with its two variants: Ant-Step-Density or Ant-Step-Quantity) performs better than the Ant-Cycle Algorithm, in 6/10 cases, less performant in 2/10 cases and with equal performance in 2/10 cases (Table 4).
These results are a consequence of exploiting global information in the AS, by using an efficient data distribution and availability process. That is a result of the simultaneous pheromone updates operated in a "synchronous" manner, after every ants' move. This process provides more efficient data, exactly when and where needed. Consequently, it improves the information dissemination mechanisms and increases the system's reactivity.
Another interesting outcome is the fact that the use of a synchronous ants' updates, such as with the Ant-Cycle Algorithm or with our new variants; Ant-Step-Density and Ant-Step-Quantity, works as a knowledge enhancement, and has to be moderated, neither more nor less than the ant system's need. For that, we have to evaporate the 292 THE ANT HEURISTIC "RELOADED" pheromone, allowing ants to forget part of their previous experiences. A balance has to be maintained between the pheromone deposit and its evaporation to advantage less-used good-path, to be chosen, and to disadvantage the used bad ones. A similar observation had been highlighted in [35], and still needs further investigations.
The pheromone evaporation in the Ant System, as defined by Dorigo et al. (Section 3), is controlled by the evaporation coefficient, ρ, used in equation (1).
We notice that in these three last variants, the synchronized pheromone update process provides more useful knowledge used by ants for their choice of path. In this case, ants must forget part of their previous experience through the evaporation process (Table 5).
On the other side, authors in [7] give some comments on what has been presented at the end of Section 4 about the four conditions to be verified in a given problem to apply an ant approach as follow: 1. A greedy force alone will not be able to construct a solution, but any given tour of towns; 2. The autocatalytic process alone will converge in an exponential manner to a local optimum; The greedy force proposes the best suggestions to the autocatalytic process, guiding it to converge faster to an optimal solution. To help the greedy force to perform this important and crucial job, the pheromone trail must be controlled by evaporation and the synchronized pheromone update process. Thus avoiding misguiding the transaction probability (the greedy force), this confirms our previous statements. At the end, the main contribution of our paper is: • Defining a new AA; the Ant-Step Algorithm with its two variants: Ant-Step-Density Algorithm and Ant-Step-Quantity Algorithm.
The performance shown by these new variants with respect to the previous claimed best one: Ant-Cycle variant, defines a new fact, and encourages us to reconsider the use of the previously best variant in the improvement of AA and try to use, instead, the Ant-Step Algorithms.

Conclusion
Our contribution takes place at an heuristic level and will have consequences at the metaheuristic one. Indeed, we reconsider, in this paper, the Ant System heuristic through an analysis and a study leading to discover an overlooked choice to implement the Ant Algorithms. Exploring this third choice to deal with the AS heuristic, we defined a new alternative: the Ant-Step Algorithms.
The Ant-Step Model increases the ants' reactivity using a data dissemination process, providing, in the system, an efficient information availability/distribution. This process acts as a knowledge enhancement, and has to be watched to keep a balance between attractive bad paths and less attractive good ones. This equilibrium is handled by the evaporation process, allowing ants to forget part of their previously used data to choose the next move.
It improves, also, the heuristic exploration mechanism of the problem's search space to discover a more optimal solution. So, at the heuristic level, the main outcome of our work is the definition of new best AS basic variants: Ant-Step-Density or Ant-Step-Quantity, which we strongly recommend and argue for their use in any improvement of the ant algorithm, instead of the Ant-Cycle variant.
The perspective is to think about "reloading" the Ant Algorithm by considering our new model; Ant-Step Algorithm through its two new best variants; Ant-Step-Density or Ant-Step-Quantity as an improved Computational Model in any future application of Ant Algorithm for TSP or others NP-hard problems. These new applications have to be assessed in respect to the existing ones. Extremely, at this metaheuristic level, the perspective will be to give an improved version of the ACO metaheuristic.