New complexity analysis for primal-dual interior-point methods for self-scaled optimization problems
© Choi and Lee; licensee Springer 2012
Received: 30 June 2012
Accepted: 7 November 2012
Published: 26 November 2012
A linear optimization problem over a symmetric cone, defined in a Euclidean Jordan algebra and called a self-scaled optimization problem (SOP), is considered. We formulate an algorithm for a large-update primal-dual interior-point method (IPM) for the SOP by using a proximity function defined by a new kernel function, and we obtain the best known complexity results of the large-update IPM for the SOP by using the Euclidean Jordan algebra techniques.
MSC:90C51, 90C25, 65K05.
1 Introduction and preliminaries
Primal and dual interior-point methods (IPMs) have been well known as the most effective methods for solving wide classes of optimization problems, for example, the linear optimization (LO) problem, the quadratic optimization problem (QOP), the semidefinite optimization (SDO) problem, the second-order cone optimization (SOCO) problem, and the convex optimization problem (CP).
The so-called barrier update parameter θ in algorithms for IPMs plays an important role in both theory and practice of IPMs. Usually, if θ is a constant independent of the dimension of the problem, then the algorithm is called a large-update method. If it depends on the dimension, then the algorithm is said to be a small-update method. Large-update methods are much more efficient than small-update methods in practice , but have a worst-case iteration bound. Such a gap between theory and practice has been referred to as irony of IPMs . Recently, many authors have tried to reduce the gap of the worst-case iteration bound between the large-update IPM and the small-update IPM.
Using self-regular proximity functions instead of a classical logarithmic barrier function, Peng et al. [3–5] improved the complexity of large-update IPMs for the LO problem, the SDO problem, and the SOCO problem. Bai et al.  introduced a new class of eligible kernel functions. The class was defined by some simple conditions on the kernel function and its derivatives. The best iteration bound for the LO problem, which was given by Bai et al. , is . Recently, Wang et al.  obtained the complexity result for the SDO problem based on a simple kernel function. Bai and Wang  obtained the best known complexity result for the SOCO problem based on a parametric kernel function including the classical logarithmic function, the prototype regular kernel function, and the non-self-regular kernel function. Very recently, using the kernel function , Choi and Lee [9, 10] have obtained the complexity results of large-update primal-dual IPMs for SDO and SOCO, and , respectively.
In this paper, we consider a linear optimization problem over a symmetric cone which is defined in a Euclidean Jordan algebra. Nesterov and Todd  proposed first this kind of an optimization problem under the name of convex programming for self-scaled cones and established the polynomial complexity of the primal-dual interior point method using the so-called NT (Nesterov-Todd) direction . We call the linear optimization problem over the symmetric cone the self-scaled optimization problem (SOP).
Faybusovich first studied the SOP in view of a Euclidean Jordan algebra and gave a theoretical background for nondegeneracy assumptions and the uniqueness of solutions for Newton systems in IPMs for the SOP , presented a short-step path-following algorithm for a quadratic programming problem defined on the intersection of a symmetric cone with an affine subspace  and obtained complexity estimates for a long-step primal-dual interior-point algorithm for the optimization problem of the minimization of a linear function on a feasible set obtained as the intersection of an affine subspace and a symmetric cone . SOPs include linear optimization problems, semidefinite optimization problems, second-order optimization problems, and various combinations of these types of problems as special cases. Schmieta and Alizadeh  extended primal-dual interior point algorithms for LOs, SDOs, and SOCOs to SOPs by using logarithmic barrier functions.
Baes raised an open question in his monograph  as follows: The theory of self-regular functions has been created for linear programming by Jiming Peng, Cornelius Roos, and Tamás Terlaky . They subsequently extended it to second-order programming and semidefinite programming separately using implicitly the aforementioned construction. However, the unified treatment of this theory using the Jordan algebraic framework is not accomplished yet.
Choi and Lee  gave primal-dual interior point algorithms by using a very simple self-regular function , for the SOP and gave partial answers for the question of Baes. Very recently, Vieira [19, 20] gave complete answers for the open question of Baes by proving the e-convexity property of eligible kernel functions and, in particular, he presented the iteration complexity results for ten eligible kernel functions. Among ten kernel functions in , the best iteration complexity for a large-update method was obtained for with , and its iteration complexity is , which is the best known one.
In this paper, we define a new eligible kernel function , and for , which was modified from the one in [9, 10], and obtain the best known iteration complexity result for the large-update IPM of the SOP by using the analysis emphasized on the kernel function and the Euclidean Jordan algebra techniques. In our algorithm, we use the well-known lemma for the upper bound of the μ-update (see Lemma 3.1) instead of using Theorem 5.4 in . The lemma makes our analysis in the outer while loop easy. We refer to Theorem 4.9 and Proposition 5.6 in  for complexity analysis. But we use Proposition 3.1 in  obtained from the technique of Sun and Sun  instead of using Proposition 5.7 in .
This paper is organized as follows. In Section 2, we introduce our kernel functions, formulate the Newton system for the SOP, and present a useful inequality for our proximity function. In Section 3, we give an algorithm for the SOP and calculate an upper bound for the proximity function after μ-update. We calculate an upper bound for difference between proximity functions after one step in inner iterations and then determine our default step size for search directions. We present a worst-case iteration bound for our large-update primal-dual interior point method for the SOP.
Now, we give definitions and preliminary properties for a Euclidean Jordan algebra which are found in  and will be used in the next sections.
Definition 1.1 ()
A finite-dimensional real vector space V is called an algebra if a bilinear mapping from to V is defined.
commutativity: for all , ;
Jordan’s axiom: for all , , where .
, equivalently, there exists an inner product on V such that .
for every pair of , there is an invertible linear transformation such that and ;
, where .
Let . Then .
Definition 1.2 ()
Theorem 1.1 (Theorem III.1.2 in )
Square root: if all .
Inverse: if all .
Since V is a Euclidean Jordan algebra, is a scalar product on V (see Proposition III.1.5 in ). The following lemma is called the second Pierce decomposition theorem which will be used in Section 3.
, if ;
, if ;
, for if .
We can check that weak duality between (P) and (D) holds, that is, . From now on, we assume that both (P) and (D) satisfy the interior-point condition (IPC), that is, there exists such that , , , . Then there exists a pair of optimal solutions of (P) and (D), and [11, 23].
Lemma 1.3 ()
Let and p be invertible. Then if and only if .
Proposition 1.1 (Proposition 18 in )
If , then .
Then, for each , the parameterized system (6) has a unique solution [11, 27], which is called a μ-center of (P) and (D). The set of μ-centers, that is, , is said to be the central path of (P) and (D). Therefore, as μ tends to zero, converges to a pair of optimal solutions of (P) and (D) [13, 28].
In general, IPMs for the SOP consist of two strategies. The first one, which is called the inner iteration scheme, is to keep the iterative sequence in a certain neighborhood of the central path or to keep the iterative sequence in a certain neighborhood of the μ-center. And the second one, called the outer iteration scheme, is to decrease the parameter μ to for some .
2 Proximity functions and search directions
Newton’s method is a well-known procedure to solve a system of nonlinear equations. Most IPMs for solving the SOP employ different search directions together with suitable strategies for following the central path appropriately.
We say that the above is called the NT search direction for the SOP. Furthermore, , which is coming from the first and second equations of (10) or from the orthogonality of Δx and Δs.
The following lemma gives a lower bound of σ in terms of .
This completes the proof. □
Also, our new kernel function (11) satisfies the following exponential convexity property.
The following proposition can be found in , but for the completeness, we give its proof.
Proposition 2.1 (Theorem 4.9 in )
3 Algorithm and its complexity analysis
Now, we explain our algorithm for the large-update primal-dual IPM for the SOP. Assuming that a starting point in a certain neighborhood of the central path is available, we can set out from this point. Then, we will go to the outer ‘while loop’. If μ satisfies , then it is reduced by the factor , where . Then, we make use of the inner ‘while loop’, and we repeat the procedure until we find iterates that are ‘close’ to , that is, the proximity . Here, we apply Newton’s method targeting at the new μ-centers to decide a search direction . We return to the outer ‘while loop’. The whole process is repeated until μ is small enough, say until .
The choice of the step size α is another crucial issue in the analysis of the algorithm. It has to be taken so that the closeness of the iterates to the current μ-center can improve by a sufficient amount. In the algorithm, the inner ‘while loop’ is called the inner iteration and the outer ‘while loop’ is called the outer iteration. Each outer iteration consists of an update of the parameter μ and a sequence of (one or more) inner iterations. The total number of inner iterations is the worst-case iteration bound for our algorithm.
3.1 Bound of the proximity function after μ-update
We have before the update of μ with the factor at the start of each outer iteration. After updating μ in an outer iteration, the vector v is divided by the factor , which in general leads to an increase in the value of . Then during the inner iteration, the value of decreases until it passes the threshold τ.
As we mentioned, our kernel function (11) is eligible. To obtain an upper bound for a μ-updated proximity function in each outer iteration in the algorithm, we use the well-known Lemma 3.1, which can be induced from the decreasing part of the kernel function, instead of using theorems which can be obtained from some properties for eligible functions (for example, Theorem 3.2 in  and Theorem 5.4 in ). Both of the following lemmas make our analysis in the outer while loop easy. And we will show a theorem that an upper bound for is expressed with by using the following two lemmas.
This implies . □
the last inequality comes from . □
3.2 Determining a default step size
The main task in the rest of this section is to study the decreasing behavior of .
Lemma 3.3 (Proposition II.3.3 in )
if x and s are invertible.
Lemma 3.4 ()
and have the same eigenvalues.
and have the same eigenvalues.
Then we can find and have the same eigenvalues. Here, . We know that and , by the definition (8) and by (ii) in Lemma 3.4, then and have the same eigenvalues. Therefore, the proximity function satisfies the equality. □
The following lemma is obtained from Lemma 14 in  so that we can get the common lower bound of eigenvalues of and , where α satisfies and .
where σ is a number defined in (15).
The proof of the following proposition can be found in , but for the completeness, we give its detailed proof.
Proposition 3.2 ()
So, the first equality holds.
Thus, we have the conclusion. □
The next result presents an upper bound for the second derivative of which is usable for establishing the polynomial complexity of the algorithm.
In that case, the largest α satisfying (20) is minimal. For our purpose, we need to deal with the worse case, and so we assume that (21) holds.
For the purpose of finding an upper bound of , we need a default step size that is the lower bound of the and consists of σ.
the last inequality comes from and (25). □
Now, we present a lower bound of the value of .
where the inequality follows from and ρ and are monotonically decreasing. Also, by Lemma 3.6, we can complete the proof. □
We will use as the default step size in our algorithm.
3.3 Decrease of the proximity function during an inner iteration
Now, we show that our proximity function Ψ with our default step size is decreasing. It can be easily established by using the following result.
Lemma 3.7 ()
Since , we can obtain the upper bound for the decreasing value of the proximity in the inner iteration by Lemma 3.7.
where the inequality follows from . The theorem is satisfied. □
3.4 Iteration bound
At this stage, we invoke Lemma 14 in .
Lemma 3.8 ()
Letting , and , we can get the following lemma from Lemma 3.8.
where is the value of after the μ-update in the outer iteration.
Now, we estimate the total number of iterations of our algorithm.
Multiplication of this result by the number in the above lemma satisfies the theorem. □
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No. 2012-0006236).
- Andersen ED, Gondzio J, Mészáros C, Xu X: Implementation of interior point methods for large scale linear programming. In Interior Point Methods of Mathematical Programming. Edited by: Terlaky T. Kluwer Academic, Dordrecht; 1996:189–252.View ArticleGoogle Scholar
- Renegar J MPS/SIAM Ser. Optim. In A Mathematical View of Interior-Point Methods in Convex Optimization. SIAM, Philadelphia; 2001.View ArticleGoogle Scholar
- Peng J, Roos C, Terlaky T: Primal-dual interior-point methods for second-order conic optimization based on self-regular proximities. SIAM J. Optim. 2002, 13: 179–203. 10.1137/S1052623401383236MathSciNetView ArticleGoogle Scholar
- Peng J, Roos C, Terlaky T: Self-regular functions and new search directions for linear and semidefinite optimization. Math. Program. 2002, 93: 129–171. 10.1007/s101070200296MathSciNetView ArticleGoogle Scholar
- Peng J, Roos C, Terlaky T: Self-Regularity: A New Paradigm for Primal-Dual Interior-Point Algorithms. Princeton University Press, Princeton; 2002.Google Scholar
- Bai YQ, Ghami ME, Roos C: A comparative study of kernel functions for primal-dual interior-point algorithms in linear optimization. SIAM J. Optim. 2004, 15: 101–128. 10.1137/S1052623403423114MathSciNetView ArticleGoogle Scholar
- Wang GQ, Bai YQ, Roos C: Primal-dual interior-point algorithms for semidefinite optimization based on a simple kernel function. J. Math. Model. Algorithms 2005, 4: 409–433. 10.1007/s10852-005-3561-3MathSciNetView ArticleGoogle Scholar
- Bai YQ, Wang GQ: Primal-dual interior-point algorithms for second-order cone optimization based on a new parametric kernel function. Acta Math. Sin. Engl. Ser. 2007, 23: 2027–2042. 10.1007/s10114-007-0967-zMathSciNetView ArticleGoogle Scholar
- Choi BK, Lee GM: On complexity analysis of the primal-dual interior-point methods for semidefinite optimization problem based on a new proximity function. Nonlinear Anal., Theory Methods Appl. 2009, 71: e2628-e2640. 10.1016/j.na.2009.05.078MathSciNetView ArticleGoogle Scholar
- Choi BK, Lee GM: On complexity analysis of the primal-dual interior-point method for second-order cone optimization problem. J. Korean Soc. Ind. Appl. Math. 2010, 14: 93–111.MathSciNetGoogle Scholar
- Nesterov YE, Tood M: Primal-dual interior-point methods for self-scaled cones. SIAM J. Optim. 1998, 8: 324–364. 10.1137/S1052623495290209MathSciNetView ArticleGoogle Scholar
- Muramatsu M: On a commutative class of search directions for linear programming over symmetric cones. J. Optim. Theory Appl. 2002, 112: 595–625. 10.1023/A:1017920200889MathSciNetView ArticleGoogle Scholar
- Faybusovich L: Linear systems in Jordan algebras and primal-dual interior-point algorithms. J. Comput. Appl. Math. 1997, 86: 149–175. 10.1016/S0377-0427(97)00153-2MathSciNetView ArticleGoogle Scholar
- Faybusovich L: Euclidean Jordan and interior-point algorithms. Positivity 1997, 1: 331–357. 10.1023/A:1009701824047MathSciNetView ArticleGoogle Scholar
- Faybusovich L, Arana R: A long-step primal-dual algorithm for symmetric programming problems. Syst. Control Lett. 2001, 43: 3–7. 10.1016/S0167-6911(01)00092-5MathSciNetView ArticleGoogle Scholar
- Schmieta S, Alizadeh F: Extensions of primal-dual interior-point algorithms to symmetric cones. Math. Program. 2003, 96: 409–438. 10.1007/s10107-003-0380-zMathSciNetView ArticleGoogle Scholar
- Baes, M: Optimization Methods for Convex Symmetric Problems. Monograph, April (2007)Google Scholar
- Choi, BK, Lee, GM: Complexity analysis for primal-dual interior-point methods for self-scaled optimization problems (submitted)Google Scholar
- Vieira, MVC: Jordan algebraic approach to symmetric optimization. Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, The Netherlands (2007)Google Scholar
- Vieira MVC: Interior-point methods based on kernel functions for symmetric optimization. Optim. Methods Softw. 2011, 27: 513–537.MathSciNetView ArticleGoogle Scholar
- Sun D, Sun J: Lowner’s operator and spectral functions in Euclidean Jordan algebras. Math. Oper. Res. 2008, 33: 421–445. 10.1287/moor.1070.0300MathSciNetView ArticleGoogle Scholar
- Faraut J, Korányi A: Analysis on Symmetric Cones. Oxford University Press, London; 1994.Google Scholar
- Nesterov YE, Nemirovskii A SIAM Stud. Appl. Math. 13. In Interior Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia; 1994.View ArticleGoogle Scholar
- Gowda MS, Szajder R, Tao J: Some P-properties for linear transformations on Euclidean Jordan algebras. Linear Algebra Appl. 2004, 393: 203–232.MathSciNetView ArticleGoogle Scholar
- Faybusovich L: A Jordan-algebraic approach to potential-reduction algorithms. Math. Z. 2002, 239: 117–129. 10.1007/s002090100286MathSciNetView ArticleGoogle Scholar
- Lim Y: Applications of geometric means on symmetric cones. Math. Ann. 2001, 319: 457–468. 10.1007/PL00004442MathSciNetView ArticleGoogle Scholar
- Tunçel L: Potential reduction and primal-dual methods. In Handbook of Semidefinite Programming Theory, Algorithms and Applications. Edited by: Wolkowicz H, Saigal R, Vandenberghe L. Kluwer Academic, Boston; 2000:235–265.View ArticleGoogle Scholar
- Alizadeh F, Schmieta S: Symmetric cones, potential reduction methods and word-by-word extensions. In Handbook of Semidefinite Programming, Theory, Algorithms and Applications. Edited by: Wolkowicz H, Saigal R, Vandenberghe L. Kluwer Academic, Boston; 2000:195–233.View ArticleGoogle Scholar
- Horn RA, Johnson CR: Topics in Matrix Analysis. Cambridge University Press, Cambridge; 1991.View ArticleGoogle Scholar
- Baes M: Convexity and differentiability properties of spectral functions and spectral mappings on Euclidean Jordan algebras. Linear Algebra Appl. 2007, 422: 664–700. 10.1016/j.laa.2006.11.025MathSciNetView ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.