DeepFake video production and SIFT-based analysis

Efficient detection and reliable matching of image features constitute a fundamental task in computer vision. When real-time operation is required, the solution to this problem becomes a real challenge, because of increased processing requirements. Scale Invariant Feature Transform (SIFT) is considered as a stable and robust algorithm for the extraction of invariant features, however special hardware is required for high frame-rate applications, in order to overcome its computational complexity in real-time. In this paper, a complete processor for SIFT feature matching in video sequences is presented. It performs SIFT feature extraction and matching, as well as rejection of false correspondences using the random sample consensus (RANSAC) algorithm. The processor was evaluated using metrics like response, repeatability and recall. It was found that the proposed fixed-point hardware implementation has a comparable performance with software implementations of SIFT. The processor architecture can process more than 36 fps with resolution 640×480, when it is implemented in Cyclone IV FPGA technology.

The Scale Invariant Feature Transform (SIFT) feature can be used for an image distance measure. SIFT keypoints are mainly located on high-contrast regions of an image, such as object edges and textures, which are often visually important. Therefore, it can be said that if a resized image keeps SIFT features which the corresponding original image has, the resized image is visually acceptable. To the best of our knowledge, there is no resizing method for explicit preservation of SIFT features. In this paper, we propose a resizing method based on mesh warping for explicit preservation of SIFT features. The effectiveness of our resizing method is demonstrated by experiments.

A novel digital image stabilization approach using Harris and Scale Invariant Feature Transform (SIFT) was presented in this article. Using SIFT in digital image stabilization, too many feature points and matches were extracted, but some of them were not so stable. Using these feature points and matches can not only increase the computational effort, but also enhance the wrong matching probability. We proposed to use SIFT to detect feature points and incorporate the Harris criterion to select the most stable feature points in the video sequence where image motion was happened due to vehicle or platform vibration. With these feature points, we use general feature descriptor and matching algorithm to achieve the image stabilization. Experimental results show that the proposed algorithm can not only bring down the probability of wrong matching and get more accurate matches, but also reduce the computation burden than SIFT effectively.

Copy-move forgeries are very common image manipulations that are often carried out with malicious intents. Among the techniques devised by the ‘Image Forensic’ community, those relying on scale invariant feature transform (SIFT) features are the most effective ones. In this paper, we approach the copy-move scenario from the perspective of an attacker whose goal is to remove such features. The attacks conceived so far against SIFT-based forensic techniques implicitly assume that all SIFT keypoints have similar properties. On the contrary, we base our attacking strategy on the observation that it is possible to classify them in different typologies. Also, one may devise attacks tailored to each specific SIFT class, thus improving the performance in terms of removal rate and visual quality. To validate our ideas, we propose to use a SIFT classification scheme based on the gray scale histogram of the neighborhood of SIFT keypoints. Once the classification is performed, we then attack the different classes by means of class-specific methods. Our experiments lead to three interesting results: (1) there is a significant advantage in using SIFT classification, (2) the classification-based attack is robust against different SIFT implementations, and (3) we are able to impair a state-of-the-art SIFT-based copy-move detector in realistic cases.

The SIFT (scale invariant feature transform) has demonstrated its superior performance in identifying transformed images over many other approaches. However, both of its detection and matching stages are expensive, because a large number of keypoints are detected in the scale-space and each keypoint is described using a 128-dimensional vector. We present two possible solutions for feature-point reduction. First is to down scale the image before the SIFT keypoint detection and second is to use corners (instead of SIFT keypoints) which are visually significant, more robust, and much smaller in number than the SIFT keypoints. Either the curvature descriptor or the highly distinctive SIFT descriptors at corner locations can be used to represent corners.We then describe a new feature-point matching technique, which can be used for matching both the down-scaled SIFT keypoints and corners. Experimental results show that two feature-point reduction solutions combined with the SIFT descriptors and the proposed feature-point matching technique not only improve the computational efficiency and decrease the storage requirement, but also improve the transformed image identification accuracy (robustness).

Scale invariant feature transform(SIFT) is an algorithm to extract distinctive and invariant features from images to achieve reliable object matching between different images in variant scales and rotations. However, SIFT's huge computation impedes its real-time implementation. In this paper, a Fast-Gaussian SIFT(FG-SIFT) is proposed. Keypoint detection is optimized in FG-SIFT. SIFT's 2-D difference of Gaussian(DoG) in Gaussian Scale-Space(GSS) is separated into two 1-D DoG in x and y dimensions, and the level of scales in DoG pyramid is also reduced. The experiment shows that FG-SIFT reduces the computational complexity about 95% in GSS construction, also increases the accuracy of keypoint detection. Subsequently, the accuracy of generated features is increased 162%, and the accuracy of matched features is increased 8%. Apart from optimization in algorithm level, a hardware architecture of FG-SIFT's keypoint detection module is proposed. With a parallel architectural incorporating a five-stage pipeline, the execution time of keypoint detection is only 1.42ms@Xilinx Virtex5. Compared to conventional works, the speed is 21% faster than the fastest solution [9](ASIC), and hardware resources are 70% less than the most resources saved solutions [5] [6](Xilinx Virtex5).

Scale invariant feature transform (SIFT), as one of the most popular local feature extraction algorithms, has been widely employed in many computer vision and multimedia security applications. Although SIFT has been extensively investigated from various perspectives, its security against malicious attacks has rarely been discussed. In this paper, we show that the SIFT keypoints can be effectively removed with minimized distortion on the processed image. The SIFT keypoint removal is formulated as a constrained optimization problem, where the constraints are carefully designed to suppress the existence of local extrema and prevent generating new keypoints within a local cuboid in the scale space. To hide the traces of performing SIFT keypoint removal, we then propose to inject a large number of fake SIFT keypoints into the previously cleaned image with minimized distortion. As demonstrated experimentally, our proposed SIFT removal and injection algorithms significantly outperform the state-of-the-art techniques. Furthermore, it is shown that the combined SIFT keypoint removal and injection attack strategy is capable of defeating the most powerful forensic detector designed for SIFT keypoint removal. Our results suggest that an authorization mechanism is required for SIFT-based systems to verify the validity of the input data, so as to achieve high reliability.

Fully pipelined FPGA-based architecture for real-time SIFT extraction

Scale Invariant Feature Transform (SIFT) is used for local features description of images. The proposed technique employs SIFT in order to match binary images. While the employment of SIFT descriptors on binary image databases could be possible, its power is rather limited. The novelty of the proposed algorithm is make use of SIFT for binary image matching by taking the power of Hough Transform (HT) in line detection. HT can be used on any line orientation. Thus, HT investigates lines as criteria for binary image similarity beside SIFT features for local and corner descriptions. The evaluation is achieved and experimental highlights the superiority of this approach for binary images which contains straight lines.

SIFT (scale invariant feature transform) is one of the most robust and the widely used image matching algorithms based on local features. It has been used in the aspect of the cameras target matching. For its high complexity, this paper proposes a target matching method for multiple cameras by combining SIFT and the rotation-invariant LBP (local binary patterns). The object detection is obtained by improved Back-ground subtraction technique, and particle filter is used for region-based object tracking. The descriptor combining SIFT and the rotation-invariant LBP is calculated for every object region. The object matching is obtained by comparing the distance of feature matrix. Experimental results show that the proposed algorithm is more rapid than the standard SIFT algorithm while the performance is favorably compared to the standard SIFT algorithm, and it is robust in various scenes.

With the advent of e-commerce, digital services and social media, scammers have changed their way to gain illegal benefits in various forms such as capturing the credit card information or exploiting personal cloud accounts which is termed as phishing. For this reason, against this cyber crime, last two decades have witnessed a variety of combatting methodologies like HTML content based similarity analysis, URL based classification and recently visual similarity based matching since phishing web pages visually mimic to their legitimate counterparts in order to create an illusion to deceive innocent users. To this end, in this study, we propose a computer vision and machine learning based approach in order to classify whether a suspicious web page is phishing and further recognize its original brand name. In this regard, we have utilized and investigated two different local image descriptors namely Scale Invariant Feature Transform (SIFT) and DAISY. Apart from their common properties such as scale invariance, the aforementioned descriptors have apparent differences such that in addition to rotational invariance, SIFT employs key-point based sampling whereas DAISY applies dense sampling by default. Therefore, we first aimed to investigate the feasibility of these two local image descriptors in addition to revealing the effects of sampling strategy and rotational invariance in problem domain. Furthermore, in order to create a discriminative representation of a web page, we followed the bag of visual words (BOVW) approach having different vocabulary sizes such as 50, 100, 200 and 400. In order to evaluate the proposed approach, we have utilized a publicly available phishing dataset including snapshots of webpages sampled from both 14 different highly phished brands and ordinary legitimate web pages yielding a challenging open-set problem. The aforementioned dataset involves 1313 training and 1539 testing image samples in total. The visual features extracted via SIFT and DAISY were first transformed to a BOVW histogram and fed to three different machine learning methods such as SVM, Random Forest and XGBoost. According to the conducted experiments, based on a 400-D visual vocabulary, SIFT descriptor along with XGBoost has been found as the best descriptor-learner configuration having reached up to 89.34% validation accuracy with 0.76% false positive rate. Moreover, SIFT has outperformed DAISY descriptor in all settings. As a result, it has been shown that SIFT descriptors equipped with BOVW representation can be effectively used for brand identification of phishing web pages.

In this paper, an affine invariance feature detection method based on Scale Invariant Feature Transform (SIFT) and Maximally Stable Extremal Regions (MSER) is proposed. Classical SIFT algorithm is not robust to affine deformations, because it is based on DOG detector which extracts circle regions for keypoint location. In order to overcome this disadvantage, DOG detector in conventional SIFT algorithm is replaced by MSER detector which is robust to affine deformation. Then these regions are normalized and extracted using SIFT. Simulation studies are carried out to show the effectiveness of the proposed method to affine transform in comparison to traditional SIFT algorithm.

얼굴추적은 3차원 공간상에서 머리(head)와 안면(face)의 움직임을 추정하는 기술로, 얼굴 표정 감정인식과 같은 상위 분석단계의 중요한 기반기술이다. 본 논문에서는 AAM 기반의 얼굴추적 알고리즘을 제안한다. AAM은 변형되는 대상을 분할하고 추적하는데 광범위하게 적용되고 있다. 그러나 여전히 여러 가지 해결해야할 제약사항들이 있다. 특히 자체중첩(self-occlusion)과 부분적인 중첩, 그리고 일시적으로 완전히 가려지는 완전중첩 상황에서 보통 국부해에 수렴(local convergence)하거나 발산하기 쉽다. 본 논문에서는 이러한 중첩상황에 대한 AAM의 강인성을 향상시키기 위해서 SIFT 특징을 이용하고 있다. SIFT는 일부 영상의 특징점으로 안정적인 추적이 가능하기 때문에 자체와 부분중첩에 효과적이며, 완전중첩의 상황에도 SIFT의 전역적인 매칭성능으로 별도의 재초기화 없이 연속적인 추적이 가능하다. 또한 추적과정에서 큰 자세변화에 따른 움직임을 효과적으로 추정하기 위해서 다시점(multi-view) 얼굴영상의 SIFT 특징을 온라인으로 등록하여 활용하고 있다. 제안한 알고리즘의 이러한 강인성은 위 세 가지 중첩상황에 대해서 기존 알고리즘들과의 비교실험을 통해서 보여준다. Face tracking is to estimate the motion of a non-rigid face together with a rigid head in 3D, and plays important roles in higher levels such as face/facial expression/emotion recognition. In this paper, we propose an AAM-based face tracking algorithm. AAM has been widely used to segment and track deformable objects, but there are still many difficulties. Particularly, it often tends to diverge or converge into local minima when a target object is self-occluded, partially or completely occluded. To address this problem, we utilize the scale invariant feature transform (SIFT). SIFT is an effective method for self and partial occlusion because it is able to find correspondence between feature points under partial loss. And it enables an AAM to continue to track without re-initialization in complete occlusions thanks to the good performance of global matching. We also register and use the SIFT features extracted from multi-view face images during tracking to effectively track a face across large pose changes. Our proposed algorithm is validated by comparing other algorithms under the above 3 kinds of occlusions.

Extraction and expression of features are critical to improving the recognition rate of ear image recognition. This paper proposes a new ear recognition method based on SIFT (Scale-invariant feature transform) and Forstner corner detection technology. Firstly, Forstner corner points and SIFT keypoints are detected respectively. Then taking Forstner corner into the SIFT algorithm to calculate their descriptor as the image feature vectors. Finally ear recognition based on these feature is carried out with Euclidean distance as similarity measurement. A bi-directional matching algorithm is utilized for improving recognition rate. Experiments on USTB database show that the recognition rate reaches more 94%. The Experimental results prove the effectiveness of the proposed method in term of recognition accuracy in comparison with previous methods. It is robust to rigid changes of ear image and provides a new approach to the research for ear recognition. DOI: http://dx.doi.org/10.11591/telkomnika.v11i12.2760

In allusion to non-rigid registration of medical images, the paper gives a novel algorithm based on improved Scale Invariant Features Transform (SIFT) feature matching algorithm. First, Harris corner detection algorithm is used in the process of scale invariant feature extraction, so the number of right matching points is increased; with regard to the feature points detected in the scale space, an improved SIFT feature extraction algorithm with global context vector is presented to solve the problem that SIFT descriptors result in a lot of mismatches when an image has many similar regions. On this basis, affine transformation is chosen to implement the non-rigid registration, and weighted mutual information (WMI) measure and Particle Swarm Optimization (PSO) algorithm are also chosen to optimize the registration process. The experimental results show that the method can achieve better registration results than the method based on mutual information.