An Optimal Strategy for Estimating Weibull distribution Parameters by Using Maximum Likelihood Method

Several methods have been used to estimate the Weibull parameters such as least square method (LSM), weighted least square method (WLSM), method of moments (MOM), and maximum likelihood (MLE). The maximum likelihood method is the most popular method (MLE). Newton-Raphson method has been applied to solve the normal equations of MLE’s in order to estimate the Weibull parameters. The method was used to find the optimal values of the Weibull distribution parameters for which the log-likelihood function is maximized. We tried to find the approximation solution to the normal equations of the MLE’s because there is no close form for get analytical solution. In this work, we tried to carry out a study that show the difference between two strategies to solve the MLE equations using Newton-Raphson algorithm. Both two strategies are provided an optimal solution to estimate the Weibull distribution parameters but which one more easer and which one converges faster. Therefore, we applied both strategies to estimate the Weibull’s shape and scale parameters using two different types of data (Real and simulation). We compared between the results that we got by applying the two strategies. Two studies have been done for comparing and selecting the optimal strategy to estimate Weibull distribution parameters using maximum likelihood method. We used some measurements to compare between the results such as number of steps for convergence (convergence condition), the estimated values for AIC, BIC and the RMSE value. The results show the numerical solution that we got by applying first strategy convergence faster than the solution that we got by applying second strategy. Moreover, the MRSE estimated by applying the first strategy is lower than the MRSE estimated by applying second strategy for the simulation study with different noise levels and different samples size


Introduction
It is important to know which strategy is more accurate and which one is easier to apply.We can use several methods to estimate the Weibull parameters.For instance, least square method (LSM), weighted least square method (WLSM), method of moments (MOM), and maximum likelihood (MLE) [1].The Weibull distribution is one of the most popular distributions used for modeling nonnegative data [2].The distribution widely used in many fields, and the popularity of the distribution comes from its simplicity into estimating the distribution parameters and flexibility of fitting the data [3].The Weibull distribution is used in many real applications such as engineering, biology, and finance [4].The Weibull density function includes two parameters, shape parameter η and scale parameter ν.The MLEs derived from the probability density function of the Weibull distribution to estimate the parameters [2].The parameters estimation using MLE can be done two different ways [5].First, we 1343 need to combine the two simultaneous equations to find one simultaneous equation, and then we need to solve the equation numerically using the Newton algorithm method.In this technique, we need only one guess value to initialize the unknown parameter [6].Second, we apply the Newton-Raphson algorithm with two initial guesses for the Weibull distribution parameters and then come up with another estimate for the parameters that are closer to the previous solution [8].We continue with this process until an optimal solution converges.The optimal solution must be maximize the likelihood function of the Weibull distribution [9].The main objective of this work is to determine which technique of the MLEs provide a better estimation for the parameters.Moreover, the two unknown parameters of the Weibull distribution are derived with the MLE.We discussed which method converges with a low root mean square error (RMSE), Akaike information criterion (AIC), and Bayesian information criterion (BIC) [10].

Weibull Distribution
The non-negative random variable t i , i = 1, 2, 3, . . ., n where n the sample size has a Weibull distribution if the density function given: Where η and ν are shape and scale parameters.The density formula depends on the parameter values.The distributed converges to exponential when (η = 1) and to approximation to normal when (η > 3.4) [11], [12].The measures of central tendency of the time t when t follow Weibull distribution are: As shown in Figure 1, when (η ≥ 1) the Weibull density decreases fast as much as t increases, when η = 1 is decreasing and converges to the exponential density function, and when η > 1 increases for a while and torn to 1344 AN OPTIMAL STRATEGY FOR ESTIMATING WEIBULL DISTRIBUTION PARAMETERS decreases as well as if η ≥ 3.5 convergences to the approximation normal [13], [14], [15], [16].To estimate the Weibull distribution's parameters using MLEs, we find the likelihood function of the sample t 1 , t 2 , t 3 , . . ., t n .

Maximum likelihood Estimation (MLE)
The MLEs which are derived from the probability density function of the Weibull distribution.Using the MLEs we find the optimal values of the shape and scale for the density function of the Weibull distribution [7].The estimated value of the parameters are derived in order to maximize likelihood function [8].To find the MLEs requires computing the first and the second derivatives of the log-likelihood function in respect to the shape and scale parameters [18].The likelihood function using the probability density of the Weibull distribution is given as: Where G(t i , η, ν) = log(L n (t i , η, ν)).Those nonlinear system of equations can be solved using several methods such as Newton-Raphson and separable nonlinear method [19] In order to obtain values of the unknown parameters η and ν that maximize the system.We need to solve the nonlinear system numerically.The nonlinear system can be defined by: maximize The above systems of equations are not easily to solve analytically.So, solving and finding the optimal solution for the previous system can be done using Newton-Raphson algorithm [25].We used the Newton-Raphson method to find the solution of the maximize likelihood function numerically.In order to apply Newton-Raphson method, we need to assume that the second derivative of the likelihood function respect to the parameters exist.We can apply Newton-Raphson method to find the optimal parameters of the likelihood of the Weibull density function in two ways [6], [20], [21].

First Strategy (Shape model η)
The Newton algorithm is often used to solve the system of normal equations that are not easily solved numerically.We can combine the two equations in 4 to get one equation in one unknown parameter η.The relationship is given by ν , also by simplifying and substituting 4 we get Since g (η) is differentiable function with respect to the parameter η and it's not easy to solve manually, then the Newton algorithm method is the best method for finding the estimator η iterative [22].We can suppose the initial value for starting estimate shape parameter η of the Weibull distribution by: , The initial value of the estimator η is obtained based on the best guess.We also use the convergence criteria ηi+1−ηi ηi < δ; δ = 10 −5 .The next algorithm shows how is estimate shape η parameter iterative [23].
First Strategy: algorithm for estimating one parameter of Weibull distribution 1 Initialization a first guess for η = η 0 2 Using Newt method to solve g (η) The solution of the systems of nonlinear equations begins with an initial guess.Moreover, the solution must be numerically estimated using an iterative process.This process continues until it converges to the optimal solution.This solution contains the parameter estimates for which the observed data have the highest probability of occurrence.

Second Strategy (Shape η and scale ν model)
We reformulate an optimization problem in order to find the first and second derivative with respect to the parameters (Gradient and Hessian) [9].To maximize the objective function (Log likelihood function), we need to verify that the matrix of the second partial derivatives is negative definite, and that the solution is the global maximum rather than a local maximum [25].The log likelihood function derivations can be summarized by: where ∆ is step size and i = 0, 1, 2, 3, . .... For more details [24].Newton's method for this case starts with two initial guesses for the shape and scale parameters.We can also use the Fisher information matrix to find the variance of the estimated parameters and the confidence interval of the estimates [25] [26].The variance covariance matrix defined as: Stat., Optim.Inf.Comput.Vol.
8 Go to 2 9 End

Contributions
In this article the main contribution was to optimize one of two strategies that used for solving nonlinear optimization problem of the maximum likelihood function.Because Weibull distribution is a very important distribution to estimate the survival model as well as to estimate regression model, we need a better method or strategy for estimating its parameters.Therefore, we compare between those strategies to determine which strategy is easily for applying, more accurate for results, and fast for converging.

Application Study
We investigate a real-application study to determine the performance of the various maximum likelihood estimators.Two strategies are considered for estimating the Weibull distribution parameters.Using the survival time data, we used the MLE to estimate the parameters of the Weibull distribution.The data used indicated the survival time of 272 patients who were diagnosed with breast cancer [16], [27].The Weibull parameters estimation is done iterative using the survival time of the patients with breast cancer.The estimated parameters of the Weibull distribution from the survival time data are made using the R-Studio program.We established both models, the shape model and the shape and scale model.The results are presented in the Table 1 and Table 2 for each model including the estimation parameters and the corresponding goodness of fit of the model.  1 shows the estimated Weibull parameters obtained from the survival time data using first strategy.Survival time of the patients was used for estimating the shape and scale parameters iterative.In Table 1 we include also the corresponding model goodness of fit and the values of the log-likelihood at each iteration.From the results, we can observe that the optimal values for shape and scale are 2.11 and 9.06 respectively.These estimated values are maximizing the log-likelihood function as well minimizing AIC and BIC for the model.Table 2 demonstrates the results of the estimated values of the Weibull distribution parameters and the model goodness of fit estimated using second strategy for solving the MLEs.The gradient vector and the Hessian matrix which contain the value of the second derivative are Figure 2 shows the contour plots of the log-likelihood function using different values of shape η and scale ν parameters that were estimated using first strategy and second strategy.

Simulation Study
A simulation study was carried out in order to illustrate and compare the estimated parameters of the Weibull distribution using the first strategy and second strategy that used for solving the MLEs.Random samples that size, n=25, 50, and 100 generated the from stander uniform distribution U U nif (0, 1) and (1 − U ) U nif (0, 1).The samples are drawn from the cumulative function of the stander uniform distribution and used to generate Weibull distribution samples with considering the shape parameter η = 2 and scale parameter ν = 3.We use 1 η to generate the random samples.We replicated the experiment using Monte Carlo simulation technique for 1000 times with two levels 5% and 10% of Gaussian noise.The comparisons were done based on the values of the root mean square error (RMSE), Akiak information criteria (AIC), and Bayesian information criterion (BIC).The results of the comparison are presented in Table 3 and Table 4.

Discussion
We used two strategies for solving the maximum likelihood equations using the Newton method.We obtained the estimated parameters of the Weibull distribution through these strategies.For the application study, results are presented in Table 1 and Table 2. From the results, we can observe that the algorithm of the first strategy converges at the second iteration, while the second strategy algorithm needs more iterations to converge.Moreover, Stat., Optim.Inf.Comput.Vol. 12, September 2024  the estimated values of Weibull parameters (shape and scale) using both methods are comparable.The optimal shape and scale parameters using the first strategy are given by 2.11 and 9.06.We can also observe that the optimal parameters are associated with lower values of the model's goodness of fit.The value for AIC was 1496.293, while for BIC was 1503.505.The optimal parameters estimated using the first strategy are maximizing the log-likelihood -746.147.The estimated Weibull parameters using the second strategy are given in Table 2. From the results, we can observe that the estimated Weibull's parameters are comparable with those estimated using the first strategy.The estimated shape and scale parameters are 2.18 and 9.12, respectively.We can see that values 2.18 and 9.12 are the closing values of those estimated using the first strategy.Furthermore, the model's goodness of fit estimated using the second strategy is 1495.701for AIC and 1502.912 for BIC.The log-likelihood value is -745.850.Finally, we can say that the parameters estimated using the second strategy are comparable, even though the second strategy needs more steps for convergence.

Conclusion
The optimal strategy for estimating parameters of the Weibull distribution using the maximum likelihood method involves finding the values that maximize the likelihood function.This method is commonly used in reliability engineering to analyze failure data and predict the lifetime of products or systems.In this paper, we presented two numerical methods that used for estimating the Weibull distribution parameters.The aim of this works is to compare between solving the maximum likelihood equations using two strategies that scientists have been developed before.These strategies are providing an approximal solution for a normal equation of the maximize likelihood function.Our discussion started reviewing results and the strategies.We compared between two strategies using some measurements such number of steps for convergence (convergence condition), the estimated values for AIC, BIC and the RMSE value.Two studies have been done for comparing and selecting the optimal strategy to estimate Weibull distribution parameters using maximum likelihood method.The results show the numerical solution that we got by applying first strategy convergence faster than the solution got using second strategy.Moreover, the MRSE estimated by applying the first strategy is lower than the MRSE estimated by applying second strategy for the simulation study with different noises level and different samples size.

Figure 1 .
Figure 1.Weibull density function with different shape parameters.

Figure 2 .
Figure 2. The log-likelihood values with shape and scale vectors for both strategies (first strategy: left & second strategy: Right).

Figure 3 .
Figure 3.With 5% random noise, the estimated Weibull density and Log-likelihood using two strategies.

Figure 4 .
Figure 4.With 10% random noise, the estimated Weibull density and Log-likelihood using two strategies.
12, September 2024 1346 AN OPTIMAL STRATEGY FOR ESTIMATING WEIBULL DISTRIBUTION PARAMETERS Second Strategy: algorithm for estimating two parameters of Weibull distribution 1 Initialization a first guess for η = η 0 and ν = ν 0 2 Using Newton method to calculate Calculating 2 × 1 gradient 3 Calculating 2 × 2 Hessian matrix 4 If Convergence go to 9 5 Calculating the inverse of Hessian matrix

Table 1 .
Estimated Weibull Parameters and the Corresponding Goodness-of-Fit Measures of the Model

Table 2 .
Estimated Weibull Parameters and the Corresponding Goodness-of-Fit Measures of the Model