Tracking is recently formulated as a problem of discriminating the object from its nearby background, where the classifier is updated by new samples successively arriving during tracking. Depending on whether labeling the samples or not, the tracker can be designed in a supervised or semi-supervised manner. This paper proposes a novel semi-supervised algorithm for tracking by combining Semi-supervised Discriminant Analysis (SDA) with an online boosting framework. Using the local geometric structure information from the samples, the SDA-based weak classifier is made more robust to outliers. Meanwhile, we design an incremental updating mechanism for SDA so that it can adapt to appearance changes. We further propose an Extended SDA (ESDA) algorithm, which gives better discrimination ability. Results on several challenging video sequences demonstrate the effectiveness of the method.