Improving firefly-based Multi-Objective Optimization based on Attraction Law and Crowding Distance

Multi-objective optimization problems are so designed that they simultaneously minimize several objectives functions (which are sometimes contradictory). In most cases, the objectives are in conflict with each other such that optimization of one objective does not lead to the optimization of another ones. Therefore, we should achieve a certain balance of goals to solve these problems, which usually requires the application of an intelligent method. In this regard, use of meta-heuristic algorithms will be associated with resolved problems. In this paper, we propose a new multi-objective firefly optimization method which is designed based on the law of attraction and crowding distance. The proposed methods efficiency has been evaluated by three valid test functions containing convex, nonconvex and multi discontinuous convex Pareto fronts. Simulation results confirm the significant accuracy of proposed method in defining the Pareto front for all three test functions. In addition, the simulation results indicates that proposed algorithm has higher accuracy and greater convergence speed, compared to other well known multi-objective algorithms such as non-dominated sorting genetic algorithm, Bees algorithm, Differential Evolution algorithm and Strong Pareto Evolutionary Algorithm.


Introduction
Most optimization problems in engineering science are non-linear and have many restrictions. For this reasons, finding optimum solutions to solve these problems requires strong, efficient and, yet, simple algorithms [1].Some of the so far discussed optimization problems have been single-objective. Such problems are considered as a scalar optimization problem because the objective function always reaches a global optimal value or a scalar value [2]. However, in the real world, multifunctional objectives should be simultaneously optimized, which is very complex and difficult and defined as multi-objective optimization [4,5].Unlike a single-objective optimization problem, which necessarily minimizes an optimal solution, a multi-objective optimization problem follows and minimizes several objectives and most of the goals may be in conflict with each other [3].When examining a multi-objective optimization problem, we do not insist on an answer. We are looking for different scenarios, meaning that we expect multi-objective problem solving to obtain at least two or more different scenarios. Therefore, there is a level of answers and we will not have answers better than this level, which is called Pareto front. The aim of solving multi-objective optimization problems is providing a set of undefeated answers to a problem [4].
Various methods have been developed for solving multi-objective optimization algorithms; for example, the methods that convert a multi-objective problem into a single-objective problem, which include the methods of weighted sum, goal programming, goal attainment and ϵconstraint. All of these methods, which are not very 230 IMPROVING FIREFLY-BASED MULTI-OBJECTIVE OPTIMIZATION accurate, should be implemented multiple times in order to create a Pareto front [1,3] . Another group includes the methods that solve the problem as it is, such as evolutionary algorithms. These methods are all intelligent and can solve the problem with the least information possible. Total Pareto front points are obtained in these methods, using evolutionary algorithms [5]. In this regard, one of the powerful multi-objective algorithms, which is the founder of population-based multi-objective algorithms, is the multi-objective genetic algorithm [6]. Moreover, multi-objective differential evolution algorithms are extremely powerful and suitable for solving optimization problems [7].This algorithm was developed and presented as a multi-objective adaptive differential algorithm in 2013 [8].]. Most of the meta-heuristic algorithms are based on collective intelligence, and multi-objective particle swarm optimization algorithm (MOPSO) can be a good example in this regard. This algorithm is inspired by the group behavior of birds and fish to find food [9,10]. Another algorithm based on collective intelligence is the firefly algorithm (FA), which is a meta-heuristic algorithm inspired by nature. This algorithm was provided by Yang in 2010, originated from the natural behavior of fireflies of light emission. Although this algorithm is very similar to other swarm intelligence algorithms and its implementation is convenient, this method is very effective and can better solve optimization problems, compared to other conventional algorithms [11].Next, Yang presented multi-objective firefly algorithm in 2013 through developing the firefly algorithm and creating Pareto front. This algorithm has been implemented based on weighted sum model. In this algorithm, an objective function is defined for each firefly, and each firefly is evaluated in relation to other fireflies on the basis of this objective function. Less attractive fireflies are attracted by more attractive fireflies. If no non-dominated answer is found, an optimal answer is obtained using weighted sum method and fireflies are attracted by this point. At the end of the algorithm, a set of non-dominated answers is obtained in the form of optimal Pareto front [12]. During recent years many powerful algorithms for solving multi-objective problems have been proposed.In [14] a new multi-objective algorithm inspired from the navigation of grass hopper swarms in nature, is introduced. Grasshopper Optimization Algorithm (GOA) includes benefits such as effective exploration and fast convergence. The special adaptive mechanism in this algorithm smoothly balances exploration and exploitation. These characteristics make the GOA algorithm potentially able to cope with the difficulties of a multi-objective search space and outperform other techniques. In [15] a multi-objective version of the recently proposed Ant Lion Optimizer (ALO) called Multi-Objective Ant Lion Optimizer (MOALO) is innovated. A repository is first employed to store non-dominated Pareto optimal solutions obtained so far. Solutions are then chosen from this repository using a roulette wheel mechanism based on the coverage of solutions as antlions to guide ants towards promising regions of multi-objective search spaces. this algorithm mimics the hunting mechanism of antlions and the interaction of their favorite prey, ants, with them. Similar to other population-based algorithms, ALO approximates the optimal solutions for optimization problems with employing a set of random solutions. This set is improved based on the principles inspired from the interaction between antlions and ants. In [16] a modified version of MOABC, named evolutionary multi-objective ABC (EMOABC) is also proposed which incorporates the characteristics of simulated evolution (SE) algorithm for improved local search. The effect of control parameters of MOABC is investigated. Comparison of EMOABC with MOABC and the standard ABC (SABC) illustrates the better performance of this algorithm. [17] , proposes an efficient decomposition-based archiving approach (DAA) inspired from the decomposition strategy for dealing with multi-objective optimization. In DAA, the whole objective space is uniformly divided into a number of subspaces according to a set of weight vectors. at each generation, only one non-dominated solution lying in a subspace is chosen to be used for updating the external archive in consideration of its diversity. at each generation, only nondominated solutions that are close to the weight vectors are to be saved in the archive. In [18], a new non-dominated sorting algorithm, termed Hierarchical Non-Dominated Sorting (HNDS) is introduced. Unlike other non-dominated sorting algorithms, HNDS does not need comparing each solution with all others before determining its Pareto rank; it first sorts all solutions by their first objective values in ascending order, which means that the first individual is a non-dominated solution and can never be dominated by any solutions further down the list. In the above works performed in the past two years, new and powerful multi-purpose optimization methods were used for effective Pareto optimal front enhancement and increase of the accuracy and speed of convergence. However, our proposed algorithm is able to find the Pareto optimal front in any shape with a very precise approximation since it used the law of attraction related to the firefly algorithm and a very powerful and efficient criterion called the crowding distance. In this algorithm, all responses are uniformly distributed on all Pareto optimal fronts. Therefore, the convergence accuracy and speed of our proposed algorithm was higher, compared to the mentioned algorithms. Simulation results on test functions and comparison to other efficient multi-purpose algorithms approves the efficiency of the proposed algorithm. This method is suitable for solving very complex engineering problems. Therefore in this article, a new multi-objective firefly-based optimization algorithm is proposed which uses the law of attraction and crowding distance. This algorithm is assessed by three different test functions to verify the capability of defining convex, nonconvex and discontinuous Pareto fronts. The ultimately comparison of proposed algorithm to other multi-objective algorithms shows the impressive improvement in sense of accuracy and convergence speed . The second section of the article deals with definitions and basic principles. In the third section, a brief introduction of the proposed method and also algorithm parameters setting are provided. The fourth section is allocated to the simulation results of the proposed method and comparison of the results with other evolutionary algorithms. Finally, the last section provides the conclusion.

Definitions and Basic Principles
Before introducing the new multi-objective optimization algorithm, which includes quality and regularity together, we define the practical concepts, as follows: Definition 1: non-dominated solution A solution is called non-dominated or Pareto-Optimal if none of the objective functions could obtain better values without regarding some of the other objective values.
Definition 2: domination Considering two components x and y, x dominates y when the following two conditions are established: a) ∀ i: , which means that x is strictly better than y at least in one respect. Definition 3: Pareto front As a set of non-dominated solutions, which are selected as optimal. Considering to figure1, Pareto front includes answers among the possible answers of the problem, which cannot be dominated by any other answer. The purpose of solving a multi-objective optimization problem is to find a set of non-dominated answers to that problem. In the sequel, the crowding distance will be defined. It is a mathematical measure and is used to obtain a solution for assessing a state in which two components do not defeat each other and are not superior to each other. Crowding distance of each particle to the next member, previous member, first member and last member is determined and its greater value is better due to the covering of a larger area and helping the diversity of answers. Therefore, crowding 232 IMPROVING FIREFLY-BASED MULTI-OBJECTIVE OPTIMIZATION distance determines which particle has more importance for us, mathematically defined, as follows: Definition 4: crowding distance Consider two objective functions that should be optimized together, and also let us consider N particles indexed byi=1,.., N as indicated in Figure 2. Crowding distance for the particle i is calculated by the following equation with regard to Figure 2.
In general, crowding distance is calculated from the following equation: d j i is the crowding distance related to the ith answer from the jth objective and di is the crowding distance of the ith particle [13].

Improved Firefly-based Multi-objective Optimization Algorithm
Since the proposed multi-objective algorithm is the modified form of the single-objective firefly algorithm, we first mentioned to the single-objective firefly algorithm.

Firefly Algorithm
The firefly algorithm has been designed based on the optical behavior patterns of fireflies. Accordingly, this algorithm follows the three laws below: First law: Fireflies are unisexual and gender plays no role in attracting them. Second law: Attractiveness of each firefly is proportional to its brightness. For two fireflies, the firefly with less light is attracted by other fireflies. The amount of attractiveness is proportional to brightness and has an inverse Stat., Optim. Inf. Comput. Vol. 8, March 2020 F.SHAYESTEH, R.KARDEHI MOGHADDAM 233 relationship with distance. By increasing the distance between two fireflies, the attractiveness rate decreases. If no firefly is brighter than the others, that particular firefly randomly selects the movement towards another firefly. Third law: The brightness rate of each firefly is specified by determining the objective function value, which should be optimized. For this optimization method, brightness can be simply proportional to the objective function value. As the light intensity and attractiveness affect the movement of fireflies in this algorithm, we should define other parameters. For simplicity, we can always assume that attractiveness of a firefly, which is determined by its brightness, is specified by an objective function. However, the attractiveness rate is relative and should be regarded as an observer among other fireflies. Furthermore, the attractiveness rate will vary based on the distance between two fireflies. Attractiveness of a firefly is obtained from the following equation: In which β 0 is the initial attractiveness in r = 0 ; γ is the light absorption coefficient and r is the distance between two fireflies. The distance between the two fireflies i and j in the situation xi and xj is obtained from the equation below: However, r can be calculated using other distance criteria. Movement of the firefly i, which is attracted by the firefly j, is calculated from the following equation: In the above equation, the next situation of ith firefly is calculated from the sum of three terms; the first term (xi) is the current situation of the firefly; the second term is related to the law of attraction, the firefly with less light moves towards the firefly with more brightness; and the third term is the random movement of the firefly in cases when there is no superiority between two fireflies in terms of light absorption. γ parameter specifies the attractiveness changes, and its value determines the convergence speed and behavior of the firefly algorithm. ϵ i component can have uniform distribution or Gaussian distribution, which follows a longer route in the search space. Its function is similar to a mutation in genetic algorithm. Equation three is named as the absorption formula [19].

The proposed Multi-objective Firefly Algorithm
In the proposed multi-objective optimization algorithm, undefeated answers are obtained using the condition of dominated and non-dominated solutions, the law of attraction of the firefly algorithm. In conditions where the two components (fireflies) do not defeat each other and have no superiority to one another, a secondary measure called the crowding distance is used for evaluation. Moreover, the best answers are considered as the approximate Pareto front by arranging the answers obtained. The process of the multi-objective firefly algorithm is defined, as follows: Second step: setting the parameters and randomly initializing the population of P fireflies with uniform distribution.
Third step: calculating the objective function corresponding to each firefly.

Fourth step:
For each firefly like i, for each firefly like j, if i is not equal to j: The fireflyi is attracted by the fireflyj using the law of attraction (equation eight).
Fifth step: If some answers do not defeat each other (no non-dominated answer was found), the crowding distance should be calculated for each answer with regard to equation five.

IMPROVING FIREFLY-BASED MULTI-OBJECTIVE OPTIMIZATION
Arranging the new population based on the selection of the firefly with greater crowding distance. Controlling to see if the new population is not outside the answer space.
Sixth step:combining the answers obtained from the absorption method and crowding distance. Seventh step: returning to fourth step in case of not meeting the stop condition.
Eighth step: Selection of P non-defeated answers obtained in the form of correct Pareto front. Ninth step: end Flowchart of the proposed multi-objective algorithm for two objective functions has been provided in Figure 3.

Algorithm Parameters Setting:
Each collective intelligence algorithm depends on its parameters. Therefore, setting the algorithmparameters is extremely crucial. Parameters of the firefly algorithm include β 0 ,γ is considered between 0.1 and 20.
In this work, we haveβ 0 = 1, γ = 1, α 0 = 0.25 for all the simulations. The algorithm termination condition can be provided in different ways, the most important of which is reaching an acceptable level of answers, limited run time or limited iteration number. In this work, we use iteration number as the stop condition, which means that after n times of algorithm implementation, if the number of iterations reachesn, the algorithm stops. Algorithm execution speed becomes slower by increasing the number of iterations, whereas the algorithm accuracy increases. Another important parameter that has a great impact on the accuracy and time of the algorithm implementation is the number of the initial population size. Similar to the iteration condition, increasing the population size will be associated with higher algorithm accuracy and, at the same time, lower execution speed. In this work, the iteration number is equal to 100 and during all simulations, the number of the initial population has been considered to be 500.

Numerical Results
The proposed multi-objective firefly algorithm was implemented in MATLAB software and its efficiency was evaluated by a set of multi-objective test functions.

Multi-objective Test Functions
When a new algorithm is proposed, one of the fundamental issues is using a standard method to prove its validity and effectiveness. To this end, numerous test functions have been introduced for single-objective and multi-objective optimization [20]. In this paper, three quite different test functions with convex, non-convex and discontinuous Pareto fronts were used, which were selected from the ZDT family due to their confirmed functions for evaluating the performance of multi-objective optimization methods and finding the Pareto optimal sets. All the test functions of ZDT family consist of two objective functions [21], defined as follows: ZDT1 function, which has an optimal convex Pareto front ZDT2 function, which has an optimal non-convex Pareto front ZDT3 function, which adds a distinctive characteristic for the front. Optimal Pareto front is composed of several discontinuous convex parts and the sinusoidal characteristic of this function has caused this discontinuity but not in all the parameter space. Full description of ZDT1, ZDT2 and ZDT3 functions has been presented in Table 1.
In all three ZDT1, ZDT2 and ZDT3 functions, xi was and the range of each variable was between 0 and 1 [22,23].
The optimal Pareto front of all the functions in Table 1 when g=1 is calculated. The Pareto front obtained for the test function ZDT1 by the proposed algorithm and its comparison with the optimal Pareto front with the population size of 500 and iteration number of 100 are provided in Figure 4.  With regard to Figure 4, it is observed that the front Pareto estimated by the proposed algorithm has well covered all over the optimal Pareto of ZDT1 test function. The Pareto front obtained for the test function ZDT2 by the proposed algorithm and its comparison with the optimal Pareto front with the population size of 500 and iteration number of 100 are provided in Figure 5: As it is illustrated in Figure 5, it is observed that the front Pareto estimated by the proposed algorithm has well covered all over the optimal Pareto of ZDT2 test function. The Pareto front obtained for the test function ZDT3  With respect to Figure 6, we come to the conclusion that the front Pareto estimated by the new MOFA algorithm has well covered all over the optimal Pareto of ZDT3 test function. In figures 4-6, the horizontal axis of the objective function is f1 and horizontal axis of the objective function is f2.

Study of Error Criterion and Convergence Rate
In this section, the distance or error between the estimated Pareto front and optimal Pareto front, denoted by PF estimation error, is defined by the following equation: In equation 9, N is the number of pareto points; P F e is the Pareto front obtained by the algorithm and P F t is the optimal Pareto front. Convergence property can be used to assess the algorithm accuracy. Therefore, it is better to use relative measurement according to the following equation: In which, D g is the convergence rate [3,12]. Figures 7,8,9 revealed the convergence rate of the proposed multiobjective algorithm with the test functions ZDT1, ZDT2 and ZDT3 in each iteration process. As it is observed in Figure 7, the convergence rate reached its minimum amount in the iteration of about 160, which was indicative of the high accuracy of the proposed algorithm. With regard to the Figure 8, the convergence rate reached its minimum amount in the iteration of about 120, which indicates the efficiency of proposed MOFA algorithm. Figure  9 indicated the fact that the convergence rate reached its minimum amount in the iteration of about 260, which revealed the high accuracy of the proposed method. Considering figures 1-9, it can be concluded that the estimated Pareto front covered all over the optimal Pareto front and quickly converged. The amount of PF estimation error for all test functions are provided in Table 2.  7.34e-6 9.95e-5

A Comparative Study of Algorithm
Here, the performance of the proposed multi-objective firefly algorithm was compared to the answers obtained from Yang research, which used the weighted sum model instead of crowding distance. For fair comparison, both algorithms were used on the same population size and number of iteration respectively 500 and 500. The results obtained for both algorithms are presented in Table 3. With regard to Table 3, it was observed that PF estimation error of the proposed algorithm for all test functions was less than the desired algorithm. Nevertheless, the implementation speed of the proposed multi-objective algorithm is lower than the weighted sum algorithm considering the calculation of the crowding distance in the new multi-objective firefly algorithm.
In Figure 10, the convergence rate for both weighted algorithm and crowding distance algorithm are displayed, related to the test function ZDT1. With regard to Figure 10, we observed that in the weighted sum method, the amount of the convergence distance Dg in the iteration of about 310 reached the lowest value. On the other hand, in the crowding distance method, the amount of the convergence distance Dg in the iteration of about 190, it reached the minimum value. This suggests that the error of the crowding distance method was less than the weighted sum method.
In order to compare the performance of the proposed multi-objective firefly algorithm, we selected several evolutionary algorithms performed in previous articles. Multi-objective optimization methods, selected to be compared with the proposed algorithm, include genetic algorithm (NSGAII), Bees algorithm, Differential Evolution algorithm (DE) and Strong Pareto Evolutionary Algorithm (SPEA). Comparison was made through measuring the convergence distance, results of which are presented in Table 4 . [12] It can be concluded from   Table 4 that the results obtained from the proposed multi-objective algorithm for all the test functions was better, compared to the other evolutionary algorithms.

Conclusion
In this article, a new multi-objective firefly optimization method was provided. The efficiency of this algorithm was assessed by three famous test functions, and the proposed algorithm was compared to other multi-objective evolutionary algorithms. The results of comparison demonstrated that the proposed algorithm had higher accuracy and convergence speed, compared to other evolutionary algorithms. Therefore, the proposed algorithm is suitable for solving the optimization problems, in which accuracy is a priority.