Key-Frame Selection Strategy Based on Edge Points Classification in 2D-to-3D Conversion

Key frame selection, which aims to choose a small number of frames that well represent the entire video contents, is one of the important research issues in video content analysis. Many existing methods developed so far use some prior information about the contents of video frames based on high-level semantic features, and thus they are applicable only in particular problems. Without any prior information, approaches based on low-level index features such as clusters, correlation, and consecutive frame difference are useful. Key frames found by these approaches are expected to be as much different from each other as possible. Therefore, accurately evaluating the difference among candidate key frames is a key challenge. However, a standard similarity metric such as the squared error and the cross correlation tends to provide redundant key frames. To cope with this problem, we propose to use quadratic mutual information (QMI) as an alternative similarity measure, which allows us to capture higher-order correlation for more than two variables simultaneously. Through experiments, we demonstrate the usefulness of the proposed QMI-based key frame selection method that uses an accurate and computationally efficient least-squares QMI estimator.

This paper presents an objective video quality evaluation method for quantifying the subjective quality of digital mobile video. The proposed method aims to objectify the subjective quality by extracting edgeness and blockiness parameters. To evaluate the performance of the proposed algorithms, we carried out subjective video quality tests with the double-stimulus continuous quality scale method and obtained differential mean opinion score values for 120 mobile video clips. We then compared the performance of the proposed methods with that of existing methods in terms of the differential mean opinion score with 120 mobile video clips. Experimental results showed that the proposed methods were approximately 10% better than the edge peak signal-to-noise ratio of the J.247 method in terms of the Pearson correlation.

A two-stage automatic key frame selection method is proposed to enhance stitching speed and quality for UAV aerial videos. In the first stage, to reduce redundancy, the overlapping rate of the UAV aerial video sequence within the sampling period is calculated. Lagrange interpolation is used to fit the overlapping rate curve of the sequence. An empirical threshold for the overlapping rate is then applied to filter candidate key frames from the sequence. In the second stage, the principle of minimizing remapping spots is used to dynamically adjust and determine the final key frame close to the candidate key frames. Comparative experiments show that the proposed method significantly improves stitching speed and accuracy by more than 40%.

The main objective of this study framework is to help the user to management the interesting video images from the huge storage device. The study can be divided into three main steps is listing as follows: shot boundary detection, key frame selection and evaluation method are designed by analyzing the video content. In this paper we proposed a novel algorithm for shot boundary detection. First, transform the RGB color model into LUV color model according our experimental results. The useful features of each frame are extracted from sequence images by the automatically computation of their color moment-preserving and matching 64-predefined bit planes in order to detect the shot boundary. The excellent shot boundary detection will result in the good key frames selection. Second, the key frame selection is proceed by applying the geometric invariant moments which successful be used in target tracing, pattern recognition, and facial recognition. In our experiments we used five test sequences that belong to five different program categories: movies, advertising, sport game, news, and commercial documentary. The experimental results showed that our method is feasible, effective, and is better than the former approaches.

Video transcoding is usually conducted when the device does not support the current format or has limited storage capacity. Video transcoding is a computation-intensive process that changes one format to another one, and various multimedia applications have made it important in recent years. We present a new motion vector (MV) composition algorithm for arbitrary frame-size video transcoding. The proposed method uses the relation between the prediction error and the required bits when encoding MVs to form an auxiliary function called the Lagrange function. Therefore, MV composition is converted into a constrained optimization problem. Through the Lagrangian optimization, a dominant MV is selected from a set of candidate MVs by minimizing this cost function. The major contribution of the proposed method is that we emphasize the effect of the bits required to encode MVs; therefore, at the same target bitrate, the proposed method provides better coding performance. Experimental results show that the proposed method has better performance in terms of both objective and subjective qualities than other existing methods.

Since 1997, the Video Quality Experts Group (VQEG) has been active in the field of subjective and objective video quality assessment. The group has validated competitive quality metrics throughout several projects. Each of these projects requires mandatory actions such as creating a testplan and obtaining databases consisting of degraded video sequences with corresponding subjective quality ratings. Recently, VQEG started a new open initiative, the Joint Effort Group (JEG), for encouraging joint collaboration on all mandatory actions needed to validate video quality metrics. Within the JEG, effort is made to advance the field of both subjective and objective video quality measurement by providing proper software tools and subjective databases to the community. One of the subprojects of the JEG is the joint development of a hybrid H.264/AVC objective quality metric. In this paper, we introduce the JEG and provide an overview of the different ongoing activities within this newly started group.

Vision-based object positioning is very important in the electronic industry for assembly and inspection tasks. Many methods have been proposed to tackle the problem, either by traditional machine vision or by deep learning (DL) techniques. The traditional methods rely on template matching or feature point correspondence. They are computationally intensive and are easily affected by illumination changes and noise. DL models such as convolutional neural networks (CNNs) are computationally very efficient but are also sensitive against environmental changes. In this article, a deep reinforcement learning (DRL) model based on the Actor-Critic style Proximal Policy Optimization algorithm(s) (AC-PPO) is proposed. The proposed method is applied for the positioning of printed circuit boards (PCBs). The model uses as the current environment the sensed image and the reference template as a guide. It requires only a single manually marked template in the reference image. All possible training images are automatically and randomly generated during the neural network training without human intervention. The proposed reinforcement learning (RL) model is shown to be adaptive to environmental changes, including illumination, noise, de-focusing, and template occlusion, compared with the CNN regressor. Experimental results indicate that the proposed model on average can achieve estimation errors less than 1 pixel in translation and 1° in orientation, with fast evaluation for the real-time PCB positioning task.

This work presents an improved method to estimate disparity maps obtained from light field cameras using a novel edge detection algorithm based on Krawtchouk polynomials. The proposed method takes advantage of these polynomials to determine gradient information and find the edges based on automatically estimated weak and strong thresholds. The calculated edges in the gray scale epipolar plane image representation of a light field are then used to improve the accuracy of object boundaries in the the disparity map. The proposed method achieves better results when compared to other edge detection algorithms, both in terms of objective and subjective quality, specifically by reducing the mean squared error and the artifacts in the object boundaries. Furthermore, on average, the proposed method outperforms the state-of-the-art depth estimation algorithms, in terms of the objective quality of the final disparity map, namely for the commonly used HCI dataset.

Video summarization based on generative adversarial networks (GANs) has been shown to easily produce more realistic results. However, most summary videos are composed of multiple key components. If the selection of some video frames changes during the training process, the information carried by these frames may not be reasonably reflected in the identification results. In this paper, we propose a video summarization method based on selecting keyframes over GANs. The novelty of the proposed method is the discriminator not only identifies the completeness of the video, but also takes into account the value judgment of the candidate keyframes, thus enabling the influence of keyframes on the result value. Given GANs are mainly designed to generate continuous real values, it is generally challenging to generate discrete symbol sequences during the summarization process directly. However, if the generated sample is based on discrete symbols, the slight guidance change of the discrimination network may be meaningless. To better use the advantages of GANs, the study also adopts the video summarization optimization method of GANs under a collaborative reinforcement learning strategy. Experimental results show the proposed method gets a significant summarization effect and character compared with the existing cutting-edge methods.

This paper proposes a 2D-to-3D video conversion network, which aggregate the information of adjacent frames to synthesize the novel view. To solve the occlusion problem in novel view, adjacent frames are used to reproduce the missing data caused by view switching. Especially, a temporal aggregated attention module is proposed to make the network focus on the useful information for synthesizing viewpoints. Mask fusion module is introduced to produce the novel view according to corresponding relationship between each frame and the estimated view. The proposed network is trained by the left-right pairs extracted from 3D movies, without extra real depths. Experimental results show that the proposed method can produce high quality novel view which is superior to the state-of-the-art approach both subjective and objective quality.

ObjectivesTo compare vascular attenuation (VA) of an experimental half iodine-load dual-layer spectral detector CT (SDCT) lower limb computed tomography angiography (CTA) with control (standard iodine-load conventional 120-kilovolt peak (kVp) CTA).MethodsEthical approval and consent were obtained. In this parallel RCT, CTA examinations were randomized into experimental or control. Patients received 0.7 vs 1.4 mL/kg of iohexol 350 mgI/mL in the experimental- vs the control group. Two experimental virtual monoenergetic image (VMI) series at 40 and 50 kiloelectron volts (keV) were reconstructed. Primary outcome: VA. Secondary outcomes: image noise (noise), contrast- and signal-to-noise ratio (CNR and SNR), and subjective examination quality (SEQ).ResultsA total of 106 vs 109 were randomized and 103 vs 108 were analyzed in the experimental vs, control groups, respectively. VA was higher on experimental 40 keV VMI than on control (p < 0.0001), but lower on 50 keV VMI (p < 0.022). Noise was higher on experimental 40 keV VMI than on control (p = 0.00022), but lower on 50 keV VMI (p = 0.0033). CNR and SNR were higher than the control on experimental 40 keV VMI (both p < 0.0001) and 50 keV (p = 0.0058 and p = 0.0023, respectively). SEQ was better on both VMIs in the experimental group than in the control (both p < 0.0001).ConclusionsHalf iodine-load SDCT lower limb CTA at 40 keV achieved higher VA than the control. CNR, SNR, noise, and SEQ were higher at 40 keV, while 50 keV showed lower noise.Clinical relevance statementSpectral detector CT with low-energy virtual monoenergetic imaging performed halved iodine contrast medium (CM) lower limb CT-angiography with sustained objective and subjective quality. This facilitates CM reduction, improvement of low CM-dosage examinations, and examination of patients with more severe kidney impairment.Trial registrationRetrospectively registered 5 August 2022 at clinicaltrials.gov NCT05488899.Key Points• Contrast medium dosage may be halved in lower limb dual-energy CT angiography with virtual monoenergetic images at 40 keV, which may reduce contrast medium consumption in the face of a global shortage.• Experimental half-iodine-load dual-energy CT angiography at 40 keV showed higher vascular attenuation, contrast-to-noise ratio, signal-to-noise ratio, and subjective examination quality than standard iodine-load conventional.• Half-iodine dual-energy CT angiography protocols may allow us to reduce the risk of PC-AKI, examine patients with more severe kidney impairment, and provide higher quality examinations or salvage poor examinations when impaired kidney function limits the CM dose.

The Video Quality Experts Group (VQEG) is preparing a programme of subjective multimedia quality tests. The results from these tests will be used to evaluate the performance of competing objective multimedia quality metrics. The reliability of the subjective test data is of great importance for VQEG's task. This paper provides an overview of VQEG's multimedia ad-hoc group. The work of this group will require subjective tests to be performed by laboratories located in Europe, Asia and North America. For VQEG's multimedia work to be successful, the subjective assessment methodology must be precisely defined and produce reliable and repeatable subjective quality data. Although international standards covering multimedia quality assessment methods are in force, there remains some uncertainty regarding the most effective approach to assessing the subjective quality of multimedia. A review of existing methods is provided. Two experiments are presented investigating the suitability of alternative subjective assessment methods (single-stimulus ACR and SAMVIQ). The results of these experiments are discussed within the context of the VQEG multimedia testing programme.

This paper proposes a new 3D modeling technique based on feature points using spatio-temporal relationship. Normally, the generation of a 3D model from a real scene requires the computation of the depth of the model vertices from a dense correspondence map of the whole image. This is very time consuming, plus it is also quite difficult to achieve an accurate depth. The proposed method can generate a 3D model of an object based on identifying the correspondence of certain feature points without the need for a dense correspondence map. The proposed method consists of three parts: the extraction of an object, the extraction of feature points, and hierarchical 3D modeling using the classified feature points. This method is effective in generating a 3D model and expressing the smoothness of plain regions and sharpness of edges.

This paper proposes a method for detecting moving obstacles on roads, by using a vehicle mounted monocular camera. To detect various moving obstacles, such as vehicles and pedestrians, the ego-motion of the vehicle is initially estimated from images captured by the camera. There are two problems in ego-motion estimation. Firstly, a typical road scene contains moving obstacles. This causes false estimation of the ego-motion. Secondly, roads possess fewer features, when compared to the number associated with background structures. This reduces the accuracy of ego-motion estimation. In our approach, the ego-motion is estimated from the correspondences of dispersed feature points extracted from various regions other than those that contain moving obstacles. After estimating the ego-motion, any moving obstacles are detected by tracking the feature points over consecutive frames. In our experiments, it has been shown that the proposed method is able to detect moving obstacles.

Most existing feature-point matching algorithms rely on photometric region descriptors to distinct and match feature points in two images. In this paper, we propose an efficient feature-point matching algorithm for finding point correspondences between two uncalibrated images separated by small or mid camera baselines. The proposed algorithm does not rely on photometric descriptors for matching. Instead, only the motion smoothness constraint is used, which states that the correspondence vectors within a small neighborhood usually have similar directions and magnitudes. The correspondences of feature points in a neighborhood are collectively determined in such a way that the smoothness of the local correspondence field is maximized. The smoothness constraint is self-contained in the correspondence field and is robust to the camera motion, scene structure, illumination, etc. This makes the entire point-matching process texture-independent, descriptor-free and robust. The experimental results show that the proposed method performs much better than the intensity-based block-matching technique, even when the image contrast varies clearly across images.KeywordsInterest PointFundamental MatrixEpipolar GeometrySmoothness ConstraintFeature CorrespondenceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.