Responses’ Prediction Standard Error Analysis in Pareto Solutions

A Pareto front provides to the decision-maker a set of optimal solutions from where a unique solution should be selected to solve a multiresponse problem. However, the responses’ prediction standard error for each one of the optimal solutions is different, which impacts on the reproducibility of Pareto solutions. This means that the decision-maker has to take into account the responses’ prediction standard error in the solution selection process. For this purpose, a metric is presented. A bi-objective problem is used as example and results displayed graphically.


Introduction
Industrial problems are multidimensional so the optimization of multiresponse problems has been an active research field.The simultaneous optimization of multiresponses is an often used practice to generate solutions for problems with conflicting responses, which are usual in real-life problems.A desired condition for any candidate solution for these type of problems is that the solution is non-dominated.This means that no better values for one or more responses can be achieved in another solution without degrading the value of, at least, another response.
A representative set of optimal or non-dominated solutions constitutes a Pareto front, and provide the most favorable alternative choices for solving a multiresponse optimization problem.However, assuming that the Pareto front was appropriately generated, the decision-maker faces another issue: to choose an optimal solution from the generated set.This is not a trivial task, because the reproducibility of optimal solutions cannot be ignored by the decision-maker.In fact, there is no guarantee that product or process performs as expected when is run at a chosen input variable location due to the existence of a natural process variability and uncertainty in the estimated responses.This means that the responses' prediction standard error in each optimal solution has to be evaluated and considered in the decision process, which has been ignored so far.Therefore, the objective of this manuscript is to introduce a metric to assess the prediction standard error of each optimal solution to help the decision-maker in making a more informed decision in the solution selection process.

Pareto Optimality-Dominance Relation
A multiresponse optimization (MRO) problem can be defined as follows: In contrast to single response optimization problems, where the optimal solution is defined easily, for MRO problems a solution is more a concept than a definition [1].In practice, a compromise solution has to be selected, because the utopia point (the variables setting that yields a solution with all responses at their target value) for formulation (1) cannot be achieved.
A predominant concept in defining a non-dominated solution is that of Pareto optimality.For a minimization problem like that formulated in Eq. ( 1), a solution (a vector of responses; r ଵ ) dominates another one (r ଶ ), is Pareto optimal, if both the following conditions are true: a) the value of any response in r ଵ is no worse (is lower or equal) than that of r ଶ ; b) the value of at least one response in r ଵ is strictly better (is lower) than those of r ଶ .
Pareto frontiers can be displayed graphically only for problems with two or three responses, and consensus on what qualities a representation of the non-dominated set should possess do not exist.According to [2], quality measures that have been proposed and may be useful to this end are the following: 1-measures of cardinality (which refer to the number of points in a representation); 2-coverage (which refer to the regions of the outcome set that are represented); 3-spacing (which refer to the distance between points in the representation).Hybrid measures which overlap the above three categories have been also introduced in the literature.An example is the hypervolume measure, a well-established indicator of a front's quality [3][4].
A representative set of non-dominated solutions for MRO problems is helpful as it provides a broad overview of alternative solutions.However, in practice, only one solution is selected.This leads to a critical question: which optimal solution must be chosen?

Reproducibility of Non-dominated Solutions
A discrete representation of a Pareto frontier (a collection of solutions distributed along the Pareto front that provide a finite and manageable alternative solutions to the decision-maker) is, in general, sufficient for selecting a solution for MRO problems.Majority of the literature has focused on how to find the Pareto frontier with its most promising choices without providing more insights on how to proceed from those choices to a final decision [5].However, it is important to be aware that some Pareto solutions may lead to operation conditions more hazardous, more costly or more difficult to implement and control than others.In fact, the responses' prediction standard error is not the same for all the Pareto solutions and, if it is ignored, can potentially lead to suboptimal choices and selected solution yield, in practice, unexpected results.To minimize the gap between the theoretical and practical results, the decision-maker must assess the prediction standard error.For this purpose, one can use Eq. ( 2), which is the second term of a two-sided prediction interval for the r-th predicted response at optimal location x 0 .Note that prediction intervals give a range for the yvalue of the next observation given specific x-values, and is defined as where the mean square error MSE = SS E /(n−z), SS E is the error sum of squares, n is the number of experimental runs in the model matrix X, z is the number of terms in the response's model, and D is the significance level.

Case Study
The objective of this example is to determine the settings for reaction time (x 1 ), reaction temperature (x 2 ), and amount of catalyst (x 3 ) to maximize the conversion (y 1 ) of a polymer and achieve a target value for the thermal activity (y 2 The range values for y 1 and y 2 are [80, 100] and [55, 60], respectively.Assuming that y 1 is a Larger-The-Better response type -the estimated response value is expected to be equal or larger than an upper bound, the target value is set equal to 100; y 2 is a Nominal-The-Best response type -the estimated response value is expected to be equal to a target value), the target value is set equal to 57.5.The constraints for the input variables are Figure 1 displays the Pareto front generated with the compromise programming-based criterion proposed in Ref. [6], varying the shape factor 1 ≤ ߱ ≤ 4 in increments of 0.01 and a Sequential Quadratic Programming algorithm.This example deals with the optimization of two responses so it was possible to display the Pareto frontier graphically, and one can see that responses are in conflict: when ߤ̂ଵ increases, ߤ̂ଶ deviates from target and vice-versa.
Figures 2 and 3 show that pick up a solution based on technical, economic or decision-maker preferences may not produce the expected results in practice, because responses' prediction standard errors are not homogeneous.Prediction standard error ranges from 6.5 % to 9.5 % for the polymer conversion and from 2.4 ºC to 3.6 ºC in thermal activity.These values range is not as small as desired and may impact on process or product performance.The normalized responses' prediction standard errors (relative magnitude of the prediction error) for each solution [ ‫.ݏ(‬ ݁. (ߤ̂ ))/ ߤ̂ ] is also presented in Figures 2-3, and it is interesting to note that the values trend in both graphs are similar and analogous to those of ‫.ݏ‬ ݁. (ߤ̂ ).This means that the same decision would be taken from both pair of graphs, ‫.ݏ‬ ݁. (ߤ̂ଵ) versus ‫.ݏ‬ ݁. (ߤ̂ଶ) or ‫.ݏ‬ ݁. (ߤ̂ଵ)/ߤ̂ଵ versus ‫.ݏ‬ ݁. (ߤ̂ଶ)/ߤ̂ଶ.

Results Discussion
A compromise solution to solve a MRO problem must be selected among those of the Pareto front.However, those solutions may not yield the expected practical result, because responses' prediction standard error of a selected solution at optimal variable settings can be (excessively) high.The magnitude of responses' prediction standard error can be displayed graphically, such as shown in Figures 2 and 3, and may help decision-maker in making a more informed optimal solution selection.
Optimal solution selection is more complicated and Pareto front visualization impossible for more than four responses so alternative tools such as tabular lists are necessary to highlight more favorable solutions in an effective manner.Nevertheless, the decision-maker needs to balance the conflict between responses' priority and responses' prediction standard error.For instance, if priority is setting ߤ̂ଶ on target value (ߤ̂ଶ = ߬; see Figure 1), one can see from Figure 3 that lower prediction standard error values for thermal activity ‫.ݏ[‬ ݁. (ߤ̂ଶ)] are found for the first solutions, where ߤ̂ଶ = ߬, as well as for solutions where ߤ̂ଶ is slightly deviated from its target ( ߤ̂ଶ ≠ ߬ ), namely the solutions number 1 up to 12.Moreover, it is interesting to note that ‫.ݏ‬ ݁. (ߤ̂ଵ) values are also low for these solutions.If priority is setting ߤ̂ଵ on target value, Figure 2 shows that ‫.ݏ‬ ݁. (ߤ̂ଵ ) value is undesirably higher for solution number 42 (the solution with the highest ߤ̂ଵ value) than for other solutions, for instance, the thirty-fifth solution, though its ߤ̂ଵ value are slightly worst (lower than 96.5%).Note that ‫.ݏ‬ ݁. (ߤ̂ଶ ) value for solution number 42 is also undesirably higher.This confirms that no conflict exists between ‫.ݏ‬ ݁. (ߤ̂ଵ ) and ‫.ݏ‬ ݁. (ߤ̂ଶ) values if decision-maker assigns a higher priority either to ߤ̂ଵ or ߤ̂ଶ.Nevertheless, this case study shows that only by chance a better solution would be selected from the simple analysis of a list or a graph with a representative set of Pareto (optimal) solutions.Some solutions should be avoided because their prediction standard error is too high.This confirms that a compromise is necessary to select a solution for MRO problems and information about solutions reproducibility ‫.ݏ(‬ ݁. (ߤ̂) or ‫.ݏ‬ ݁. (ߤ̂)/ߤ̂ values) cannot be ignored.In fact, responses are usually in conflict and s.e.values for some solutions in either one or both responses is undesirably high and likely lead to less favorable results in practice.Confirmatory runs can help in the decision-making process, and decision-maker must be aware that solutions whose response values are slightly deviated from target and ‫.ݏ‬ ݁. (ߤ̂ ) or ‫.ݏ‬ ݁. (ߤ̂ )/ ߤ̂ values are lower can be effective alternatives for solving MRO problems.

Conclusions and Future work
This work investigates the Pareto front solutions in terms of responses' prediction standard error, and shows that responses' prediction standard error is different for each solution.From a theoretical point of view, it is desirable to select Pareto solutions with the lowest responses' prediction standard error.However, this may not be possible due to the conflict between the responses' priority and the responses' prediction standard error.In fact, the prediction standard error may be higher for desired response values.Thus, in practice, a compromise between technical and economic considerations will dictate the final decision about the most favorable solution.
To help the decision-maker in making more informed decisions, future work shall investigate the impact of models coefficient uncertainty in Pareto fronts.Optimal solutions built on the worst-case estimated responses must be generated and other metric(s) developed to evaluate optimal solutions.
≤ x ௨where f is a scalar valued function, M represents the set of user-specified or preference parameters (weights, priorities, shape factors, …), and r is the vector of responses ‫ݎ(‬ = 1, … , ‫.)‬ Parameters do not controlled by the decision-maker are denoted by b.Design or input variables, denoted by vector x, with lower bound x and upper bound x ௨ .Constraints are represented by g(x, b) and h(x, b).