Sensitivity to learning set reduction

In this experiment we measure the classifier classification rate change when several images are removed from the learning set. We would assume that a highly stable classifier would not change its classification ability rapidly if only a small subset of images is removed from the learning set.

The experiment is performed as follows:

• Step 1. Identify the rank $n$ of the learning set $D$, $n=\vert D\vert$, choose a decreasing constant $\Delta n$
• Step 2. Do the classification for the learning set $D$ and testing set $T$. Obtain classification rate in percents.
• Step 3. Randomly generate a set $S$ of rank $\Delta n$ by selecting members from $D$. Then substract $S$ from $D$
• Step 4. Goto Step 2 if $\vert D\vert>0$
• Step 5. Plot a 2-D graph for every obtained rank of $D$ at any time and corresponding classification rate.