This chapter presents a pattern recognition framework that applies new color features, which are derived from both the primary color (the red component) and the subtraction of the primary colors (the red minus green component, the blue minus green component). In particular, feature extraction from the three color components consists of the following processes: Discrete Cosine Transform (DCT) for dimensionality reduction for each of the three color components, concatenation of the DCT features to form an augmented feature vector, and discriminant analysis of the augmented feature vector with enhanced generalization performance. A new similarity measure is presented to further improve pattern recognition performance of the pattern recognition framework. Experiments using a large scale, grand challenge pattern recognition problem, the Face Recognition Grand Challenge (FRGC), show the feasibility of the proposed framework. Specifically, the experimental results on the most challenging FRGC version 2 Experiment 4 with 36,818 color images reveal that the proposed framework helps improve face recognition performance, and the proposed new similarity measure consistently performs better than other popular similarity measures.