Refaçade: Editing Object with Given Reference Texture

Background subtraction algorithms are widely utilized as a technology for segmentation of background and target objects in images. In particular, the simple background subtraction algorithm is used in many systems for its ease and low cost of implementation. However, because this algorithm relies only on the intensity difference, it has various problems, such as low tolerance for poor illumination and shadows and the inability to distinguish objects from their background when their intensities are similar. In an earlier study we proposed a new statistic, known as radial reach correlation (RRC), for distinguishing similar areas and dissimilar areas when comparing background images and target images at the pixel level. And we achieved a robust background subtraction by evaluating the local texture in images. In this study we extended this method further and developed a method to ensure stable background separation even in cases where the image texture is feeble and the intensity distribution is biased.

Zero shot learning brings new challenges to referring image segmentation, a typical application to find a segmentation mask given a referring expression. Most researches borrow generalization of multi-modal model, such as Contrastive Language Image Pre-training(CLIP), to realize zero shot inference. However, CLIP also suffers the limitation of insensitive to direction and local information, which is incapable of fine-grained region-text matching. To overcome this limitation, we propose a new framework, to leverage multimodal large language models (MLLM) into segmentation. Specifically, augmentations are considered from two aspects. One is to align local image information to comprehensive linguistic information, under the hypothesis that the textual description guiding local target object in image. Comprehensive linguistic information here are considered from local, global and multi-modal generated description. The other one is to enhance the robustness by blurring and captions from multi-modal model. Comprehensive experiments on three datasets prove the effectiveness of proposed approach.

Remote sensing image target object detection and recognition are widely used both in military and civil fields. There are many models proposed for this purpose, but their effectiveness on target object detection in remote sensing images is not ideal due to the influence of climate conditions, obstacles and confusing objects presented in images, image clarity, and associated problems with small-target and multi-target detection and recognition. Therefore, how to accurately detect target objects in images is an urgent problem to be solved. To this end, a novel model, called YOLOv4_CE, is proposed in this paper, based on the classical YOLOv4 model with added improvements, resulting from replacing the backbone feature-extraction network with a ConvNeXt network, replacing the Complete Intersection over Union (CIoU) loss with the Efficient Intersection over Union (EIoU) loss, and adding a coordinate attention mechanism to YOLOv4 as to improve the remote sensing image detection capabilities. The results, obtained through experiments conducted on two open data sets, demonstrate that the proposed YOLOv4_CE model outperforms, in this regard, both the original YOLOv4 model and four other state-of-the-art models, namely Faster R-CNN, Gliding Vertex, Oriented R-CNN, and EfficientDet, in terms of the mean average precision ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mAP</i> ) and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F1 score</i> , by achieving respective values of 95.03% and 0.933 on the NWPU VHR-10 data set, and 95.89% and 0.937 on the RSOD data set.

The purpose of this paper is to examine the relationship between teaching video editing and motion graphics skills and concepts with Adobe Photoshop in a university marketing course and teaching effectiveness. Video editing and motion graphics skills and concepts were taught in a 16-week semester upper division marketing course using Adobe Photoshop. The student evaluation of teaching effectiveness data of this 16-week semester course was analyzed. The results from this analysis suggests students found learning video editing and motion graphics skills and concepts with Photoshop to be a positive learning experience. Marketing educators can teach video editing and motion graphics skills and concepts with Photoshop by following the 16-week semester course outline presented in this paper. This 16-week semester course outline contains the weekly topics and teaching materials that were used to teach the 16-week semester upper division marketing course. Finally, three sections of a master syllabus for this course, the course description, course learning outcomes, and course content are provided to help marketing educators develop and teach a similar course at their institution.

One of the most important problems in Computer Vision is the computation of the 2D projective transformation (homography) that maps features of planar objects in different images and videos. This computation is required by many applications such as image mosaicking, image registration, and augmented reality. The real-time performance imposes constraints on the methods used. In this paper, we address the real-time detection and tracking of planar objects in a video sequence where the object of interest is given by a reference image template. Most existing approaches for homography estimation are based on two steps: feature extraction (first step) followed by a combinatorial optimization method (second step) to match features between the reference template and the scene frame. This paper has two main contributions. First, for the detection part, we propose a feature point classification which is applied prior to performing the matching step in the process of homography calculation. Second, for the tracking part, we propose a fast method for the computation of the homography that is based on the transferred object features and their associated local rawbrightness. The advantage of this proposed scheme is a fast and accurate estimation of the homography.

This paper proposes a key-pose based video summarization system for a video shot facilitated by using a video object segmentation method. Firstly, we detect the camera motion and extract video objects by a 3D graph-based algorithm. Once the objects are obtained, each of them is represented by a shape descriptor. Secondly, in order to find representative frames which preserve scene content as much accurately as possible, the proposed method calculates difference between pairs of frames based on shape descriptors of objects in the video shot. Finally, key-poses (representative frames) are extracted in a global manner by clustering these shapes. Experimental results on motion video shots show that the proposed method outputs satisfactory summarizations.

Sketches, which are simple and concise, have been used in recent deep image synthesis methods to allow intuitive generation and editing of facial images. However, it is nontrivial to extend such methods to video editing due to various challenges, ranging from appropriate manipulation propagation and fusion of multiple editing operations to ensure temporal coherence and visual quality. To address these issues, we propose a novel sketch-based facial video editing framework, in which we represent editing manipulations in latent space and propose specific propagation and fusion modules to generate high-quality video editing results based on StyleGAN3. Specifically, we first design an optimization approach to represent sketch editing manipulations by editing vectors, which are propagated to the whole video sequence using a proper strategy to cope with different editing needs. Specifically, input editing operations are classified into two categories: temporally consistent editing and temporally variant editing. The former (e.g., change of face shape) is applied to the whole video sequence directly, while the latter (e.g., change of facial expression or dynamics) is propagated with the guidance of expression or only affects adjacent frames in a given time window. Since users often perform different editing operations in multiple frames, we further present a region-aware fusion approach to fuse diverse editing effects. Our method supports video editing on facial structure and expression movement by sketch, which cannot be achieved by previous works. Both qualitative and quantitative evaluations show the superior editing ability of our system to existing and alternative solutions.

HKUST Electronic Theses High-fidelity image and video editing with generative models by Chenyang Qi thesis 2024 1 online resource (xv, 91 pages) : illustrations (chiefly color) Recent years have witnessed a growing demand for visual content, such as 2D images…Read more ›

Recent large-scale pre-trained diffusion models have demonstrated a powerful generative ability to produce high-quality videos from detailed text descriptions. However, exerting control over the motion of objects in videos generated by any video diffusion model remains a challenging problem. In this paper, we propose a novel zero-shot moving object trajectory control framework, Motion-Zero, to enable arbitrary single-object-trajectory control for the text-to-video diffusion model. To this end, an initial noise prior module is designed to provide a position-based prior to improve the stability of the appearance of the moving object and the accuracy of position. In addition, based on the attention map of the U-Net, spatial constraints are directly applied to the denoising process of diffusion models, which further ensures the positional consistency of moving objects during the inference. Furthermore, temporal consistency is guaranteed with a proposed shift temporal attention mechanism. Our method can be flexibly applied to various state-of-the-art video diffusion models without any training process. Extensive experiments demonstrate our proposed method can control the motion trajectories of arbitrary objects while preserving the original ability to generate high-quality videos.

The objective of this paper is to build an Object Recognition model using ‘TINY YOLO’ and FPGA. Object Recognition is a CV (i. e., Computer Vision) technique for identifying and detecting objects in images or videos. Object recognition is a key output of DL (i. e., Deep Learning) and ML (i. e., Machine Learning). When we humans look at an image or video, we can readily spot people, object, scene and visual details. The aim of this is to teach a computer to do what comes innately to a human being. Object recognition is done with Image Classification, Object Localization and Object Detection that combines these two tasks and localizes and classifies an object in an image. Object Recognition has become a pivotal technology for driverless cars, security purpose, traffic surveillance etc., It is also used in a various application such as disease detection in bio imaging, industrial examination, and robotic vision. Hence, in a nutshell, ‘Object Recognition is the staple of automation industry’.

INTRODUCTION: Recent deep learning based image editing methods have achieved promising results for removing object in an image but fail to generate appreciable performance for removing large objects of complex nature, especially mask from facial images. Towards this goal the objective of this work is to remove mask objects in facial images. In this study, authors propose a novel approach for face completion using Generative Adversarial Networks (GANs) that utilize masked data. This technology can help in image restoration and preservation, thus enabling us to cherish those memories that are held dear to our hearts.OBJECTIVES: Train a GAN to learn the mapping from incomplete to complete face images by utilizing a masked input image.METHODS: The discriminator is trained to distinguish between face images and full ground truth images. Our results indicate that our technique generates high-quality, realistic facial images that are visually comparable to the ground truth and that it can generalise to new faces that were not encountered during training.RESULTS: Our findings indicate that GANs with masked inputs are a good approach for generating whole face images from partial or masked data.CONCLUSION: Our experimental findings show that our method produces facial images of great quality and realism that are visually equivalent to the actual thing. Our proposed approach can also be applied to fresh faces that weren’t seen. The performance can still be improved further using larger dataset. Also, further investigation into adversial attacks may help in improving performance. This technology can be further utilized for developing realtime mask removal software as well.

Study question Can tracking cell division and predicting human embryo cleavage stages be automated in time-lapse videos (TLV) using AI object detection methods? Summary answer We developed software predicting blastomere count and tracking cell cleavages up until 4-5 stage. The software employs object detection technique called YOLOv5 to detect cells. What is known already Embryo morphology plays an important part in determining viability. Parameters such as number of cells present following fertilization, abnormal cell division (reverse/direct) and evaluating cleavage stages have correlation with pregnancy rates. However, continuous manual evaluation can be time-consuming, and automation will assist in embryo viability assessment. YOLOv5 has proven to accurately detect objects in videos. YOLOv5 uses mean average precision (mAP) as a metric to quantify the portions of frames in videos having the correct count of the objects. Study design, size, duration We have developed a software that uses YOLOv5 to detect cells present in frames of TLV, then marks each cell boundary with different colored circular overlays using OpenCV. We trained YOLOv5 to detect objects: cell, morula and blastocyst using 150 images of different cell-stages, morula, blastocyst. For object cell mAP was 0.65. Annotated location of objects in images and YOLOv5 predictions were reviewed by embryologists. We evaluated the software on TLV from 11 patients. Participants/materials, setting, methods After YOLOv5 detects cells in frames of TLV, our software computes cell count and assigns each cell a different color which is maintained until cell division into daughter cells. Later, daughter cells were also assigned different colors. If the frame has a preceding frame, software calculates detected cells' proximity with each cell in the preceding frame and copies color scheme provided proximity is within some threshold. The software provides TLV with colored overlays as output. Main results and the role of chance In starting frames of TLV with single cell, software accurately detected 1-cell (high precision=0.99, high recall=0.83, high F1-score=0.90). We observed some misclassification between 1-cell and morula. The reason could be that compacted morula looks like 1-cell. Best performance is observed for 2-cells (high precision=0.91, high recall=0.98, high F1-score=0.95). 4-cells were sometimes misclassified with 3 or 5-cells (high precision=0.88, low recall=0.59, high F1-score=0.71). One reason for the misclassification can be that overlapping between cells increases with number of cells. 3-cell and 5-cell are confused with other stages, still cleavage stage detection is better than random: 3-cell (average precision=0.43, high recall=0.83, average F1-score=0.49), 5-cell (average precision=0.44, average recall=0.40, average F1-score=0.40). For cell-stages&amp;gt;5, YOLOv5 detects less cells than actual count and software predicts cleavage later than actual by 9-10 frames on average. The proximity threshold used was 0.10 for cell-count&amp;lt;4 and 0.05 for count&amp;gt;4. In 5 TLV, overlay color for cells changes abruptly between frames, possibly because once YOLOv5 detected a stage, in consecutive frames less cell-number was recorded, and then again reported correct count. Sometimes, software selected the wrong parent for daughter cells (incorrect colored overlay). 2 TLV had direct and reverse cleavages and software could detect these two patterns. Limitations, reasons for caution Overall, our software can precisely detect cells, cell divisions and cleavage stages up to 4-cell stages. We hypothesize that training YOLOv5 on a bigger dataset and including several focal plane information will enable our software to detect overlapping cells and cleavage stages &amp;gt; =5. Wider implications of the findings Object detection proved to be pragmatic for ART and tracking cell division using our software will reduce time consumed in manual annotations, easier prediction of abnormal cleavages and more objective assessments. Qualitative evaluation by embryologists resulted in the overall verdict that this is useful and promising for further development. Trial registration number not applicable

Efficient, interactive foreground/background segmentation in video is of great practical importance in video editing. This paper proposes an interactive and unsupervised video object segmentation algorithm named E-GrabCut concentrating on achieving both of the segmentation quality and time efficiency as highly demanded in the related filed. There are three features in the proposed algorithms. Firstly, we have developed a powerful, non-iterative version of the optimization process for each frame. Secondly, more user interaction in the first frame is used to improve the Gaussian Mixture Model (GMM). Thirdly, a robust algorithm for the following frame segmentation has been developed by reusing the previous GMM. Extensive experiments demonstrate that our method outperforms the state-of-the-art video segmentation algorithm in terms of integration of time efficiency and segmentation quality.

Recent advancements in large‐scale text‐to‐image diffusion models have enabled many applications in image editing. However, none of these methods have been able to edit the layout of single existing images. To address this gap, we propose the first framework for layout editing of a single image while preserving its visual properties, thus allowing for continuous editing on a single image. Our approach is achieved through two key modules. First, to preserve the characteristics of multiple objects within an image, we disentangle the concepts of different objects and embed them into separate textual tokens using a novel method called masked textual inversion. Next, we propose a training‐free optimization method to perform layout control for a pre‐trained diffusion model, which allows us to regenerate images with learned concepts and align them with user‐specified layouts. As the first framework to edit the layout of existing images, we demonstrate that our method is effective and outperforms other baselines that were modified to support this task. Code is available at our project page.

Digital image manipulation has become increasingly prevalent due to the widespread availability of sophisticated image editing tools. In copy-move forgery, a portion of an image is copied and pasted into another area within the same image. The proposed methodology begins with extracting the image's Local Binary Pattern (LBP) algorithm features. Two main statistical functions, Stander Deviation (STD) and Angler Second Moment (ASM), are computed for each LBP feature, capturing additional statistical information about the local textures. Next, a multi-level LBP feature selection is applied to select the most relevant features. This process involves performing LBP computation at multiple scales or levels, capturing textures at different resolutions. By considering features from multiple levels, the detection algorithm can better capture both global and local characteristics of the manipulated regions, enhancing the accuracy of forgery detection. To achieve a high accuracy rate, this paper presents a variety of scenarios based on a machine-learning approach. In Copy-Move detection, artifacts and their properties are used as image features and support Vector Machine (SVM) to determine whether an image is tampered with. The dataset is manipulated to train and test each classifier; the target is to learn the discriminative patterns that detect instances of copy-move forgery. Media Integration and Call Center Forgery (MICC-F2000) were utilized in this paper. Experimental evaluations demonstrate the effectiveness of the proposed methodology in detecting copy-move. The implementation phases in the proposed work have produced encouraging outcomes. In the case of the best-implemented scenario involving multiple trials, the detection stage achieved a copy-move accuracy of 97.8 %.