The 95% conﬁdence intervals of error rates and discriminant coefﬁcients

Fisher proposed a linear discriminant function (Fisher’s LDF). From 1971, we analysed electrocardiogram (ECG) data in order to develop the diagnostic logic between normal and abnormal symptoms by Fisher’s LDF and a quadratic discriminant function (QDF). Our four years research was inferior to the decision tree logic developed by the medical doctor. After this experience, we discriminated many data and found four problems of the discriminant analysis. A revised Optimal LDF by Integer Programming (Revised IP-OLDF) based on the minimum number of misclassiﬁcation (minimum NM) criterion resolves three problems entirely [13, 18]. In this research, we discuss fourth problem of the discriminant analysis. There are no standard errors (SEs) of the error rate and discriminant coefﬁcient. We propose a k-fold cross-validation method. This method offers a model selection technique and a 95% conﬁdence intervals (C.I.) of error rates and discriminant coefﬁcients.


Introduction
Fisher [3] described the linear discriminant function (Fisher's LDF) and founded the discriminant theory.Following Fisher's LDF, the quadratic discriminant function (QDF) and multi-class discrimination using Mahalanobis distance were proposed.Statistical software packages implement these discriminant functions based on the variance-covariance matrices.However, Fisher never formulated two equations of standard errors (SEs) of error rate and discriminant coefficient of Fisher's LDF.After these discriminant functions, the Framingham study developed a logistic regression.On the other hand, the discriminant functions using the mathematical programming (MP) were proposed.Stam [21] summarized these L p -norm discriminant functions and lamented "Why statistical users rarely used MP-based functions?"honestly.One of the reasons is that there is no examination of real data.Vapnik [24] introduced three kinds of a support vector machine (SVM) such as a hard margin SVM (H-SVM), a soft margin SVM (S-SVM) and a kernel SVM.Statistical users accept SVM examined by the real data.
We used these discriminant functions in many applications, such as medical diagnosis, pattern recognition, genome discrimination, and rating property and bond.However, there are many problems in the discriminant analysis.It may be the simplicity of the discriminant rule: If y i * f (x i ) > 0, x i is classified to class1/class2 correctly.If y i * f (x i ) < 0, x i is misclassified.There are three serious problems hidden in this simplistic scenario [18].Problem 1: We cannot discriminate between cases where x i lies on the discriminant hyperplane (f (x i ) = 0) correctly.We ignored this unresolved problem until now.If some cases lie on the discriminant hyperplane, the numbers of misclassification (NMs) or error rates may not be correct.Many researchers treat these cases belong to class1 without valid reason.Some statisticians believe it is decided by dice because statistics is the study of probability.Both treatments are not logical.Problem 2: H-SVM tells us the discrimination of the linear separable data clearly.However, there is no research about it because of two reasons.H-SVM can recognize the linear separable data, but it can apply only to the linear separable data, and S-SVM sometimes cannot acknowledge the linear separable data.For this reason, there are few linear separable data.IP-OLDF based on MNM criterion finds the Swiss bank note data is linearly separable [4,10,11,12].In addition to this data, the pass/fail determination by exam scores [15] is good research data for the discrimination of linear separable data, because of two reasons.We can easily obtain it, and there is a trivial LDF, MNM of which is zero.Some statisticians believe that the purpose of discriminant analysis is to discriminate overlapping data correctly.However, the definition of overlap is uniquely defined by the condition of "M N M > 0" in the world of LDF.All LDFs except H-SVM and Revised IP-OLDF cannot define the overlap theoretically because they cannot discriminate the linear separable data correctly.Problem 3: We assume that cases vary in statistics.If some variable is constant, we cannot compute the inverse matrices of the variance-covariance matrices.For this reason, most statistical packages exclude these variables from the discriminant analysis.However, JMP [8] adopts the generalized inverse matrix technique and can compute the inverse matrix.However, we found a defect, when the value of a variable belonging to one class is constant, and the value of another class varies.QDF and a regularized discriminant analysis (RDA) [5] misclassify all cases of class2 to class1.After the end of Dec. in 2012, JMP released modified RDA, but QDF enhanced by the generalized inverse matrices cannot discriminate this particular example correctly until 2014.If we add small random noise to the variable with a constant value, third problem is resolved.There is no need to exclude the constant variable in the discriminant analysis.
In this research, we discuss fourth problem that the discriminant analysis is not the inferential statistics such as a regression analysis.Fisher never formulated two equations of SE of the discriminant coefficient and error rate.Until now, no statisticians are successful in formulating these SEs by traditional approach based on the normal distribution.For this reason, we propose the "K-fold cross-validation for small sample" method by the computerintensive approach.This method is a combination of re-sampling technique and k-fold cross-validations.By this new method, we can obtain the 95% confidence intervals (C.I.) of error rates and discriminant coefficients [16,17].Although Konishi and Honda [6] proposed the "bootstrap methods [1] for error rate estimation in discriminant analysis", a new approach is more helpful for researchers who wish to discriminate their small sample.In addition to these results, we discuss the new model selection technique.There are many model selection techniques such as stepwise methods and all possible combination of independent variables and statistics such as AIC, BIC and Cp statistics in the regression analysis.On the other hand, a leave-one-out (LOO) method is used as model selection technique in the discriminant analysis [7].Our approach may be helpful for model selection procedure as follows.
We select the best model that has the minimum mean of error rates (M2) in the validation samples and compare eight LDFs with this model.Some statisticians believe MNM is foolish criterion because it overfits the training sample and overestimates the validation sample.They say the generalization ability of Fisher's LDF is good because it assumes the Fisher's assumption without examination of real data.This claim has been proven to be a complete mistake by our method.We showed that the means of error rates (M1) in the training sample and M2 of Revised IP-OLDF were less than those of Fisher's LDF [20].Revised IP-OLDF had resolved first and second problems [14].In this research, it can resolve fourth problem of the discriminant analysis.

Existing Discriminant Functions
Fisher defined Fisher's LDF to maximize the variance ratio (between/within classes).If we accept Fisher's assumption, the same LDF is obtained in equation ( 1) by the plug-in rule.
This equation defines Fisher's LDF explicitly.For this reason, statistical software packages adopt this equation.Statistician who derived QDF known that most real data did not satisfy Fisher's assumption.When the variancecovariance matrices of two classes are not the same (Σ 1 ̸ = Σ 2 ), QDF can be used.In addition, Mahalanobis distance is used as multi-class discrimination and MT theory [22] in QC.We applied these functions in many areas but cannot obtained the functions if some independent variables remain constant.There are three examples.First, some variables, which belong to both classes, are the same constant.Second, some variables, which belong to both classes, are different but constant.Third, some variable, which belongs to only one class, is constant.Most statistical software packages exclude all variables in these three examples.On the other hand, JMP enhances QDF using the generalized inverse matrix technique.QDF of JMP can treat the first and second examples correctly, but cannot handle the third example properly until 2014.
Recently, the logistic regression in the following equation has been used for the heavy users such as the medical and economic areas: Where p: the probability belongs to class1.
It adapts the maximum likelihood estimation developed by Fisher and offers the 95% C.I. of logistic regression coefficients by numerical approach.If the data is linearly separable, Firth [2] tells us that the estimations are unstable.In this example, JMP output the warning.We can confirm that the data is linearly separable if JMP output the warning and "MNM=0" is confirmed by Revised IP-OLDF.Usually, NM of logistic regression is decided at the level "p=0.5".In this research, we choose the minimum NM (This is not equal to MNM) on the ROC curve by JMP.
Vapnik proposed three different SVM models.H-SVM in equation (3) indicates the discrimination of linear separable data clearly.
Some statisticians misunderstand that the discrimination of linear separable data is very easy.In statistics, there is no technical term for linear separable data.However, the condition of "MNM=0" is the same as being linearly separable.Note that "NM=0" does not imply the data is linearly separable.It is unfortunate that there has been no research on linear separable data because of two reasons.First, H-SVM can apply only to linear separable data, and S-SVM may not analyze it correctly.Second, there is few linear separable research data because H-SVM or Revised IP-OLDF must discriminate all possible models in order to find linear separable models.For this reason, there are few linear separable models until now.Real data are rarely linearly separable.For this reason, S-SVM has been defined in equation ( 4) with two objects.
Where c: penalty c; e i : non-negative decision variable.
These two objects are combined by defining some "penalty c."The Markowitz portfolio model [23] to minimize the risk and maximize the return is the same as S-SVM.However, the return is incorporated as a constraint, and the objective function minimizes the risk.The decision maker chooses a solution on the efficient frontier.On the contrary, S-SVM does not have a rule to determine "c" correctly; nevertheless, it can be solved by an optimization solver.In this research, two S-SVMs such as SVM4 (c = 10 4 ) and SVM1 (c = 1) are examined.We know the "M1 & M2" of SVM4 are almost better than SVM1.

Optimal Linear Discriminant Functions (OLDFs)
At first, IP-OLDF can be defined as in the following equation: Where e i : 0/1 integer decision variable; M: 10,000 (Big M constant).
This notation is defined on p-dimensional coefficient space because the constant is fixed to 1.We can understand the relation between the LDFs and NMs in both data and coefficients space.And n linear equations made by n constraints (H i (b) = y i * ( t x i b + 1) = 0) divide the coefficients space into finite convex polyhedron clearly [11].
This basic understanding leads several new facts of the discriminant theory as follows: 1) LDF corresponding to the interior point of the convex polyhedron has unique NM, and there is no case on the discriminant hyperplane.LDF corresponding to the vertex or edge are not free from the first problem because there may be cases on the discriminant hyperplane.For this reason, error rates and NMs of all LDFs may not be correct because they cannot choose the interior points theoretically.
2) Only Revised IP-OLDF in equation ( 6) are free from the first problem.It can always find the interior point of the optimal convex polyhedron (OCP).Only LDF corresponding to the interior point of the OCP has exact MNM.
Where e i : 0/1 integer decision variable; b 0 : free decision variable.
If e i is a non-negative real variable, we utilize Revised LP-OLDF, which is an L 1 -norm LDF.Its elapsed runtime (CPU time) is faster than that of Revised IP-OLDF.If we choose a large positive number as the penalty c of S-SVM, the result may be almost the same as that given by Revised LP-OLDF.It is the reason the first term of the objective value in the equation (4) becomes meaningless.
Revised IPLP-OLDF is a combined model of Revised LP-OLDF and Revised IP-OLDF.In the first step, Revised LP-OLDF is applied for all cases, and e i is fixed to 0 for cases that are discriminated correctly by Revised LP-OLDF.In the second step, Revised IP-OLDF is used for cases, e i of which are not zero in the first step.It is important that e i changes from real decision variable to binary integer variable.For this reason, Revised IPLP-OLDF can be expected to obtain an estimate of MNM faster than Revised IP-OLDF for large samples [19].SHUICHI SHINMURA

Original Data and Re-sampling Data
Until now, we find three kinds of original data that are linearly separable.Those are the Swiss bank note data, the data of the pass/fail determination by exam scores and the Japanese 44 cars data [18].IP-OLDF found that the Swiss bank note data is linearly separable by 2-variables (X4, X6).For this reason, we consider 16 linear separable models including 2-variables (X4, X6).The 47 models that remain are not linearly separable.Our claim is that we focus on these 16 linear separable models because we need not select the best model among 47 models.In addition to this merit, to compare the results of eight LDFs by these 16 models is more straightforward and clear than to compare eight LDFs by 47 models.
After 2010, we teach the statistical preliminary course for approximately 130 freshmen, attended.Midterm and final exams consisted of 100 questions with ten choices.We consider two discriminations using 100 item scores and four testlet scores as independent variables.If the pass mark is 50 points, we can easily obtain a trivial LDF the students pass or fail the exam, respectively.In this example, students on the discriminant hyperplane pass the exam because their score is exactly 50.This example indicates that there is no first problem because independent variables decide the discriminant rule.
Table 1 shows the discrimination of four testlet scores for a 10% level of the midterm exams.'p' denotes the number of independent variables selected by the forward stepwise method.In 2012, the 2-variables model (T4, T2) was linearly separable.There are 15 discriminant models by all combination of variables, only four LDFs of which are linearly separable.In this research, we focus on four linear separable models such as (T1, T2, T3, T4), (T1, T2, T4), (T2, T3, T4), (T2, T4).In addition, we compare of eight LDFs by these four models.Those comparisons are very clear because four models are linearly separable.On the other hand, when we compare eight LDFs by eleven models, we may not judge the results positively.Some statisticians claim that the purpose of discriminant analysis is to discriminate overlapping data, not linear separable data.Until now, we cannot define the overlapping status in the world of LDF.Users of Fisher's LDF and QDF misunderstand all 15 models are overlapping because NMs are not zero.Only MNMs show four models including (T2, T4) are not overlapping, and 11 models are overlapping.It is difficult for us to define the overlapping status of the world of non-linear discrimination.

K-fold Cross-Validation
Fisher's LDF is re-defined by the plug-in rule under Fisher's assumption.Also, if we consider y i = 1/ − 1 as the object variable and analyze the data by the regression analysis, obtained regression coefficients are proportional to equation ( 1) by second plug-in rule.For this reason, we can use model selection techniques of regression analysis.Some statisticians misunderstand the discriminant analysis is the same as the regression analysis by the reason derived from the normal distribution.We cannot obtain two SEs of error rates and discriminant coefficients.
Although there are many model selection techniques such as stepwise approaches and statistics such as Cp, AIC and BIC in the regression analysis, we can use only the LOO method in the discriminant analysis.In this paper, we propose the "k-fold cross-validation for small sample" method as follows.In addition, we consider the best model that has minimum M2 in the validation sample.In this research, we fix k = 100 in order to obtain the 95% C.I. of error rates and discriminant coefficients.

1)
We copy the original data 100 times.In addition, we add one variable, the value of which is generated by the uniform random number.Now, we can obtain a large re-sampling sample.
2) We consider this data as pseudo-population.We are sorting the data in ascending order in each class.In addition, we add the sub-sample numbers from 1 to 100.This operation is the same effect by re-sampling from pseudopopulation or original sample directly.
3) The sub-samples from 1 to 100 are the 100 training samples.In addition, pseudo-population is the validation sample.This method has two merits.The relation of the validation sample and training samples equals to the relation of population and samples.The validation sample has the same character as the original data that prevents the mistakes.For this reason, we can control the quality of data sets.
This method gives us much information, compared with the LOO methods.We can evaluate eight LDFs by "M1 & M2" and the 95% C.I. of error rates and discriminant coefficients.Until now, users of discriminant analysis cannot use this useful information.

Pass/Fail Determination by Exam Scores
In this research, we discuss the discrimination of linear separable data.We examined the pass/fail determination by midterm exam at the 10% level in 2012.Pass mark is 36 points.For this reason, a trivial LDF such as f=T1+T2+T3+T4-36 is a linear separable model.If f ≥ 0 or f < 0, student pass or fail the exam, respectively.There are 15 discriminant models made by 4-variables such as T1, T2, T3, and T4.Because MNM of (T2, T4) model is zero, four MNMs including (T2, T4) are zero.In addition, other eleven MNMs are not zero.In section 3.1, we focus on four linear separable models.We compare the "M1s & M2s" of eight LDFs including H-SVM.In section 3.2, we discuss the ranges for error rates of four linear separable models and the ranges in four out of eleven models.In section 3.3, we discuss the 95% C.I. of discriminant coefficients of four linear separable models.Revised IP-OLDF analyzes 1,500 training samples as follows.First, we discriminate the 100 training samples (n = 124 cases) by 100 discriminant models of (T1, T2, T3, T4) and obtain 100 NMs.From these NMs, 100 error rates are calculated by dividing n = 124 and the mean of error rate 'M1' is calculated by 100 error rates.In this manner, other fourteen M1s are computed.After optimization, these 1,500 Revised IP-OLDFs are applied to the validation samples (N = 12, 400 cases).In addition, we compute fifteen M2s and the percentiles such as 0%, 2.5%, 50%, 97.5% and 100% of error rates and discriminant coefficients.In this way, LINGO [9] analyzes six MP-based LDFs.JMP [8] script analyzes logistic regression and Fisher's LDF.However, it outputs only 1500 NMs.Excel computes the error rates and percentiles.

The Mean of Error Rates
RIP, H-SVM, SVM4, IPLP, LP, and Logistic can recognize four linear separable models correctly because four M1s are zero.SVM1 and Fisher's LDF cannot recognize three and four linear separable models.
We define a model with a minimum M2 as the best model among fifteen models.RIP, IPLP, LP and Fisher's LDF choose the fourth model (T2, T4) as the best model.H-SVM, SVM4, SVM1, and Logistic select the first model (T1, T2, T3, T4).Minimum values of M2 are 0.863, 0.806, 0.806, 0.806, 0.847, 0.798, 0.774 and 9.913%, respectively.Logistic has the smallest value 0.774%.In this data, we cannot choose the same best models.Nevertheless, the best model of Revised IP-OLDF has the minimum value of M2 among eight LDFs by other data.
"Diff." column is the difference between (M2 -M1).Some statisticians claim a model with a small value of "Diff."has good generalization ability.Although the fourth model of Fisher's LDF has the minimum value 0.372, the values of M1 and M2 are 9.54 and 9.913 those are very wrong.For this reason, this statistics is not useful to check the generalization ability of discriminant models."M1Diff.&M2Diff."columns are the differences of (M1 & M2 of seven LDF -those of RIP).There are ten minus values of "M2Diff." for four linear separable models.This fact tells us ten 'M2' values of Revised IP-OLDF are worse than six LDFs except for Fisher's LDF within 0.1%.On the other hand, four models of Fisher's LDF are 9.05% worse than Revised IP-OLDF.Six ranges of "M2Diff." are [0.395,1.581], [0.524, 1.774], [−0.137, 0.145], [0.403, 1.032], [0.559, 1.625] and [6.238, 9.595], respectively.Only minimum value of Revised IPLP-OLDF is minus.This result may suggest Revised IP-OLDF and Revised IPLP-OLDF are superior to other six LDFs for seven models.CPU times of eight LDFs are in the first column.Until now, MP-solver was slower than statistical software.This slowness increased the research time, by that reducing the feasibility of research using MP-solver.However, the MP-solver is a more powerful tool now.For this reason, we can study discriminant analysis by the computerintensive approach instead of the traditional approach restricted by the theoretical distributions.In 2009, Revised IPLP-OLDF was faster than Revised IP-OLDF when tested on LINGO Ver. 10 [9].Reversal of CPU time occurred for Revised IP-OLDF and Revised IPLP-OLDF, when tested on LINGO Ver.14.We expect that Revised IPLP-OLDF may be faster than Revised IP-OLDF for a larger data set.On the other hand, CPU times of logistic regression and Fisher's LDF are about 12 minutes and 15 minutes as recorded by means of a wrist watch, respectively.Although both statistical LDFs output a large amount of information, they were nonetheless faster than Revised IP-OLDF until 2013.SHUICHI SHINMURA

The Ranges of Error Rates
Table 3 shows the ranges of error rates instead of the 95% C.I. of error rates.Upper and lower rows in each LDF display the ranges of the training and validation samples, respectively.RIP, H-SVM, SVM4, IPLP, and Logistic have the same results.These five LDFs and LP can recognize four linear separable models correctly.SVM1 cannot recognize three linear separable model and Fisher's LDF cannot recognize four linear separable models.The maximum value 3.23% of the fourth model of LP is smaller than other LDFs.We can conclude Fisher's LDF are worse in the training and validation samples.Other seven LDFs almost have the same results.Table 4 shows the ranges of four models among eleven models.Four models are (T1, T3, T4), (T1, T2, T3), (T1, T4), and (T3, T4).We can understand all minimum and maximum values of RIP are less than equal other six LDFs.From Table 2 and Table 3, we do not permit the superiority of RIP.However, RIP may be superior to other LDFs for the overlapping data.We must examine by other data (Future Work 1).

Six MP-Based LDFs
We examine the 95% C.I. of the coefficient about first and fourth models.We discuss which model is the best model.

Fisher's LDF and Logistic Regression
Fisher never formulated the SE of discriminant coefficient.If we use class identifier y i as the object variable in the regression analysis, we can obtain the SE of the regression coefficient.Table 6 shows the 95% C.I. of the regression coefficient calculated by the SE from the validation sample.First two rows show the "2.5% and 97.5%" of regression coefficients.In this research, the validation sample is considered as the pseudo-population.Although the C.I. of pseudo-population may be no meaning, we count the number of 100 regression coefficients by the training samples out of the 95% C.I.The outliers show these figures.We expect these figures are about 5 cases.In eight intervals, the number of outliers is less than eight.In five intervals, the number of outliers is less than five.
For this reason, we can accept this 95% C.I., and judge two Fisher's LDFs are significant.We cannot decide the best model by the 95% C. I. of the Fisher's linear discriminant coefficient.JMP calculates SEs of logistic regression coefficients defined by Hessian matrix that is obtained by the maximum likelihood estimation.The numbers in parentheses in equation (7)  For this reason, JMP outputs a warning message.However, if we find "NM=0" on the ROC by JMP and "MNM=0" by Revised IP-OLDF, we judge it is the linear separable model.In general, an exact logistic regression supported by SAS is recommended.It avoids complex work.We have no idea to decide the best model for logistic regression by the 95% C.I. of the logistic regression coefficient.(7)

Conclusion
In this research, we have discussed the fourth problem of discriminant analysis.Fisher never formulated two SEs of error rate and discriminant coefficient.However, some statisticians believe the discriminant analysis is inferential statistics similar to regression analysis because Fisher's LDF assumes the Fisher's assumption based on the normal distribution.This claim is not logical.Statistical software reflects the common knowledge obtained by statistical research.Statistical users can infer that discriminant analysis is not the same as traditional inferential statistics because a commercial software never display the SE of error rate and discriminant coefficients.There is a research about the error rate by the bootstrap method [6] by the computer-intensive approach.In this study, we propose the "k-fold cross-validation for small sample" method.This method can resolve the fourth problem and evaluates eight LDFs by the M2".The procedure of this method is very straightforward.In addition, there are several merits as follows: 1) It is easy to generate the re-sampling sample by statistical software.
2) It reflects the relation of pseudo-population and samples.The training samples should be a sub-set of pseudopopulation.In addition, we can control the quality of the training and validation samples very easy.
3) This method shows good results as explained in this paper.At least this method is better than the LOO method and displays the following outcomes: 1) Fisher's LDF is worst.SVM1 is second worst.H-SVM, SVM4, Revised IPLP-OLDF, Revised LP-OLDF, and logistic regression show good results for the linear separable models.On the contrary, Revised IP-OLDF is the best among eight LDFs for other eleven models that are not linear separable.This outcome may imply Revised IP-OLDF is superior to other LDFs for non-linear separable models.

2)
We cannot decide the best model with minimum M2 in this data.However, the 95% C.I. of coefficients by six MP-based LDFs recommend 2-variables model of (T2, T4).We must examine this results with other data (Future Work 3).
3) In this research, we conclude that Fisher's LDF is the worst among the eight LDFs.We obtain the same conclusion from the CPD data and the student data [20].We conclude that Fisher's LDF is fragile for discrimination such as the pass/fail determination and medical diagnosis of healthy and ill classes.These data have the same characteristic having many cases near the linear hyperplane.This feature does not satisfy the Fisher's assumption.In the nea future, we shall examine other data that does not satisfy the Fisher's assumption (Future Work 4).We plan to work on four future works using different data, further to understand the results obtained in this study.

Table 1 .
NMs of four discriminant functions by forward stepwise in 2012 midterm exams at the 10% level.
* 1: This table was calculated in 2012.NMs of LDF and QDF remarkably decreased at the end of Dec. in 2014.The NMs in parenthesis and other tables are computed again in 2014.The algorithm of JMP may improved for the linear separable data.

Table 2 (
show the results by 100-fold cross-validations."M1 and M2" columns are the means of error rates in the training and validation samples.The first column shows eight LDFs.Those are RIP (Revised IP-OLDF), H-SVM, SVM4, SVM1, IPLP (Revised IPLP-OLDF), LP (Revised LP-OLDF), Logistic (logistic regression) and Fisher's LDF.Only Fisher's LDF depends on the normal distribution.Four rows from 1 to 4 correspond to four

Table 3 .
The ranges of error rates.

Table 4 .
The ranges of error rates about four non-linear separable models.

Table 5 (
(2)5(2)show the median and 95% C.I. of six MP-based LDFs.Fisher's LDF and logistic regression by JMP script do not output 100 discriminant coefficients.If the 95% C.I. include zero, we can judge the pseudo population coefficient is zero.If the value at 2.5% is greater than 0 or the value at 97.5% is less than 0, we estimate the pseudo population coefficient a positive or negative value.Following this judgment, all coefficients of T2 and T4 are positive values and all constants are negative values.This result implies noteworthy results because most results imply us the coefficients of two variables (T2, T4) are significant at the 5% level.For this reason, we can conclude the two variables model (T2, T4) is the best model selected by RIP, IPLP, LP and Fisher's LDF.We must investigate this fact with other data (Future Work 2).

Table 6 .
The 95% C.I. of the regression analysis.