Systems that rely on Face Detection have gained great importance ever, since large-scale databases of thousands of face images are collected from several sources. Thus, the use of an outperforming face detector becomes a challenging problem. Different classification models have been studied and applied for face detection. However, such models involve large scale datasets, which requires huge memory and enormous amount of training time. Therefore, in this paper, we investigate the potency of incrementally projecting data in low variance directions. In fact, in one-class classification, the low variance directions in the training data carry crucial information to build a good model of the target class. On the other hand, incremental learning is known to be powerful, when dealing with dynamic data. We performed extensive tests on human faces, and comparative experiments have been carried out to show the effectiveness and superiority of our proposed method over other face detection methods.