Eigenvectors have a different interpretation than heatmaps. The contribution toward the output is strictly defined by whether it has positive or negative eigenvalue. Strokes of the same color positively interfere while strokes of opposite color negatively interfere. Hence, many strokes can be seen as localized edge-detectors.

Positive eigenvectors show structure that correspond to important strokes or proto-digits. Negative eigenvectors show 'inhibitory' strokes that would adversely affect the classification.