Changing the Orientation of the Cutting Plane

1. Step One. Project the correctly classified points onto the cutting plane and leave the incorrectly classified points in place. This creates the matrix Y.
   
   
   
   
   
2. Step Two. Find the Least Squares Line through the Y points. Note that the sum of squared error consists of the sum of squared orthogonal projections from the Y points to the least squares line.
   
   
   The sum of squared distances from the OLS projection shown below is not the same as the sum of squared distances in the figure above. In a simple OLS the sum of squared error is equal to the sum of squared distances from each observation to the regression line. The projection of the point representing an observation is parallel to the dimension representing the dependent variable. In the picture above the projection is orthogonal to the least squares line.
   
   
   
   
3. Here is the 2nd iteration.
   
   
   
   
4. The Cutting Plane Procedure converged in 7 iterations. This animation shows the process.