Greedy optimization classifiers ensemble based on diversity

• Authors: Shasha Mao;L. C. Jiao;Lin Xiong;Shuiping Gou
• Venue: Pattern Recognition
• Year: 2011

Decreasing the individual error and increasing the diversity among classifiers are two crucial factors for improving ensemble performances. Nevertheless, the ''kappa-error'' diagram shows that enhancing the diversity is at the expense of reducing individual accuracy. Hence, a new method named Matching Pursuit Optimization Ensemble Classifiers (MPOEC) is proposed in this paper in order to balance the diversity and the individual accuracy. MPOEC method adopts a greedy iterative algorithm of matching pursuit to search for an optimal combination of entire classifiers, and eliminates some similar or poor classifiers by giving zero coefficients. In MPOEC approach, the coefficient of every classifier is gained by minimizing the residual between the target function and the linear combination of the basis functions, especially, when the basis functions are similar, their coefficients will be close to zeros in one iteration of the optimization process, which indicates that obtained coefficients of classifiers are based on the diversity among ensemble individuals. Because some classifiers are given zero coefficients, MPOEC approach may be also considered as a selective classifiers ensemble method. Experimental results show that MPOEC improves the performance compared with other methods. Furthermore, the kappa-error diagrams indicate that the diversity is increased by the proposed method compared with standard ensemble strategies and evolutionary ensemble.