Temporal and Resolution Layering in Advanced Television

The digital micromirror device DMD is widely used in visible light projection, special-purpose spatial light modulation, and infrared scene simulation, due to its high resolution, uniformity and energy concentration. In some applications that require high frame rate scene image display, it is necessary to ensure that the DMD displays high gray level images at a high frame rate. However, the display frame rate of the pulse width modulation (PWM) method is limited by the minimum time required for DMD loading data, unable to achieve high frame rate display. Although the DMD binary display mode can meet the requirements of high frame rate display, it cannot meet the requirements of high gray level. The paper proposes an image display method that uses light source and DMD to synchronize modulation. Decompose the high-order gray image into bit planes according to the gray threshold, and the DMD displays each bit plane in binary mode under the trigger of an sync pulse. The intensity of the illumination laser source is modulated by an acousto-optic modulator to match the bit plane and the radiated laser power. This method makes use of the function of high frame rate display in DMD binary mode cooperate with light intensity modulation of illumination laser source, and realizes high frame rate and high dynamic range image display. With this method, the maximum frame rate of 8-bit gray level image with 1920 ×1080 resolution can reach 2KHz. The experimental system has realized 200Hz frame rate display of 8-bit gray level image with 1920 ×1080 resolution.

Charge coupled device(CCD) is one kind of photoelectric solid-state imaging device with excellent performance. It has the features of large array and good image quality. Working under very complex driving timing, it usually has low frame frequency and can not meet the demand of high imaging frame rate. The method of equivalent high frame rate imaging technique is introduced based on normal CCD in this paper. Some CCDs can be drived via electrodes separately. To meet the imaging demands of high frame rate, high resolution and high quality of two frame imaging technique for pulse light images, the right kind of CCDs should be adopted. The most important is that the CCD electrodes must be controlled directly and respectively. The charge transfer period is a key point while designing CCD driving timing for separate pulse light integration, charge package transfer and readout for two pulse light images. The validation experiments for the principle of equivalent high frame rate exposure of two pulse light images has been accomplished. It is shown that two pulse light images with interval of about 2 μs can be distinguished while all CCD’s inherent excellent performance are retained. A kind of two framing camera system with simple structure can be developed based on this principle.

The digital revolution changed film production and its aesthetics in many ways. Although motion is a defining feature of moving images, it is also one of their most problematic aspects because of blurred images or other signal processing artifacts. An artistic research project was conducted to test the effects of high frame rates (HFR) on visual perception. Typical camera movements were recorded in different frame rates (24 / 48 / 96 fps) to generate test sequences for a cinema experiment. 69 participants were measured with an SMI eye tracking device and a questionnaire in a within subjects design. The results show that the participants valued the higher image quality of HFR, but rated the standard frame rate as more realistic and indicated more presence. Moreover, HFR produced a higher number of fixations than the standard frame rate. Open questions for future research projects on HFR will be discussed.

The use of flat panel detectors in computed tomography (CT) systems can improve resolution, reduce system cost, and add operational flexibility by combining fluoroscopy and radiography applications within CT systems. However, some prior studies have suggested that flat panel detectors would not perform well in CT applications due to their lack of high dynamic range, lag artifacts, and inadequate frame rate. The purpose of this study was to perform a physical evaluation of a prototype flat panel detector capable of high frame rates and extended dynamic range. The flat panel detector used had a pixel size of 194 microns and a matrix size of 2048x1536. The detector could be configured for several combinations of frame rate and matrix size up to 750 frames per second for a 512x16 matrix size with 4x4 binning. The evaluation was performed in terms of the MTF and DQE as a function of frame rate and exposure at the IEC RQA5 (~75 kVp, 21 mm Al) beam quality. The image lag was evaluated in terms of temporal-frequency dependent transfer function. Offset shift were also evaluated. Preliminary results indicate 0.1 MTF at 0.92 cycles/mm and DQE(0) of approximately 0.8, 0.6, 0.4, and 0.22 at 0.144, 0.065, 0.035, and 0.008 mR per frame exposures. The temporal MTF exhibited a low-frequency drop and a value of 0.5 at the Nyquist frequency. Offset shift was negligible. Considering high frame rate capabilities of the new detector, the results suggest that the detector has potential for use in real-time CT applications including CT angiography.

In this paper, we report recent progress that has been made in the development of a high frame rate duplex HF ultrasound system with both B-scan imaging and Doppler flow measurements. A 32-channel HF analog beamformer module with transmitting focusing and dynamic focusing on reception was implemented. High-speed timing circuits were used to achieve high imaging frame rate by reducing the acquisition overhead. Therefore, the frame rate of the system relies only on the field of view. The system also included a 64-channel analog front-end pulser/receiver, a HF pulsed-wave (PW) Doppler module, a PC with a 200MS/s 14-bit PCI A/D card and real-time Labview software for data acquisition and image display. A wire phantom used to evaluate the resolution of the system. High frame rate B-scan images of mouse hearts have been obtained, as well as the PW Doppler blood flow velocity profiles at the specified location. The system is able to acquire real-time B-mode images at a rate greater than 400 frames per second in a 4.8 × 13 mm field of view. In vivo mouse experiment results show a promising future of this system in small animal research. The system will be expanded to support future arrays with more elements.

The purpose of this study was to assess whether or not high frame rate (HFR) videos recorded using high-speed digital imaging (HSDI) improve the visual recognition of the motions of the laryngopharyngeal structures during pharyngeal swallow in fiberoptic endoscopic evaluation of swallowing (FEES). Five healthy subjects were asked to swallow 0.5ml water under fiberoptic nasolaryngoscopy. The endoscope was connected to a high-speed camera, which recorded the laryngopharyngeal view throughout the swallowing process at 4000 frames/s (fps). Each HFR video was then copied and downsampled into a standard frame rate (SFR) video version (30 fps). Fifteen otorhinolaryngologists observed all of the HFR/SFR videos in random order and rated the four-point ordinal scale reflecting the degree of visual recognition of the rapid laryngopharyngeal structure motions just before the 'white-out' phenomenon. Significantly higher scores, reflecting better visibility, were seen for the HFR videos compared with the SFR videos for the following laryngopharyngeal structures: the posterior pharyngeal wall (p=0.001), left pharyngeal wall (p=0.015), right lateral pharyngeal wall (p=0.035), tongue base (p=0.005), and epiglottis tilting (p=0.005). However, when visualized with HFR and SFR, 'certainly clear observation' of the laryngeal structures was achieved in <50% of cases, because all the motions were not necessarily captured in each video. These results demonstrate the use of HSDI in FEES makes the motion perception of the laryngopharyngeal structures during pharyngeal swallow easier in comparison to SFR videos with equivalent image quality due to the ability of HSDI to depict the laryngopharyngeal motions in a continuous manner.

User performance in virtual environments with degraded visual conditions due to low frame rates is an interesting area of inquiry. Visual content shown in a low frame rate simulation has the quality of the original image, but persists for an extended period until the next frame is displayed (so-called high persistence-HP). An alternative, called low persistence (LP), involves displaying the rendered frame for a single display frame and blanking the screen while waiting for the next frame to be generated. Previous research has evaluated the usefulness of the LP technique in low frame rate simulations during a static target acquisition task. To gain greater knowledge about the LP technique, we have conducted a user study to evaluate user performance and learning during a dynamic target acquisition task. The acquisition task was evaluated under a high frame rate, (60 fps) condition, a traditional low frame rate HP condition (10 fps), and the experimental low frame rate LP technique. The task involved the acquisition of targets moving along several different trajectories, modeled after a shotgun trap shooting task. The results of our study indicate the LP condition approaches high frame rate performance within certain classes of target trajectories. Interestingly we also see that learning is consistent across conditions, indicating that it may not always be necessary to train under a visually high frame rate system to learn a particular task. We discuss implications of using the LP technique to mitigate low frame rate issues as well as its potential usefulness for training in low frame rate virtual environments.

The frame rate (FR) of a video plays an important role in affecting the perceptual video quality. Most studies about the effect of FR on the video quality mainly focused on low frame rate, e.g. less than 30 frames per second (fps), at low resolutions like CIF or QCIF. As the video frame rate and resolution advance, we reconsider this issue and investigate the relationship between frame rate and the perceptual video quality under high frame rate and high resolution. In this paper, we discuss the impact of frame rate on the perceptual quality of High Definition (HD) video with high frame rates (up to 120 fps) considered. Firstly, we design and conduct subjective experiment to construct the video dataset, which includes video sequences at different frame rates and the corresponding mean opinion scores (MOS) which represent the perceptual video quality. Based on the MOS results, we analyze how perceptual video quality changes as frame rate varies among different video sequences and propose some meaningful findings. The video dataset will be made publicly available. We deem that this study will enrich video quality assessment and benefit the development of high frame rate and high definition video business.KeywordsVideo quality assessmentSubjective experienceHigh frame rateHigh definition

For gamers, high frame rates are important for a smooth visual display and good quality of experience (QoE). However, high frame rates alone are not enough as variations in the frame display times can degrade QoE even as the average frame rate remains high. While the impact of steady frame rates on player QoE is fairly well-studied, the effects of frame rate variation is not. This paper presents a 33-person user study that evaluates the impact of frame rate variation on users playing three different computer games. Analysis of the results shows average frame rate alone is a poor predictor of QoE, and frame rate variation has a significant impact on player QoE. While the standard deviation of frame times is promising as a general predictor for QoE, frame time standard deviation may not be accurate for all individual games. However, 95% frame rate floor -– the bottom 5% of frame rates the player experiences –- appears to be an effective predictor of both QoE overall and for the individual games tested.

This paper presents our implementation of a multi-view autostereoscopic 3D display system by using time-multiplexing approach, digital micro-mirror devices (DMD), and the optical Wedgereg. The whole system consists of three major parts: high frame rate DMD projector, an active shutter, and an optical Wedgereg. This research investigates and develops the approaches to interface and control DMDs at very high bit rates and ultra-high frame rates to drive a novel flicker-free 3D flat panel device, with a wide viewing area and without need of special viewing glasses or head-tracking system. It needs to design a digital interface circuit capable of receiving video image data in a conventional format and translating it into a form appropriate for display on a DMD light modulator. This approach differs from existing ones in that it facilitates the modulation of the DMD to achieve much high frame rates, though with a trade-off in greyscale resolution. The outcomes of this research will have the great potential for commercialising the autostereoscopic display systems

High frame rate (HFR) videos are becoming increasingly common with the tremendous popularity of live, high-action streaming content such as sports. Although HFR contents are generally of very high quality, high bandwidth requirements make them challenging to deliver efficiently, while simultaneously maintaining their quality. To optimize trade-offs between bandwidth requirements and video quality, in terms of frame rate adaptation, it is imperative to understand the intricate relationship between frame rate and perceptual video quality. Towards advancing progression in this direction we designed a new subjective resource, called the LIVE-YouTube-HFR (LIVE-YT-HFR) dataset, which is comprised of 480 videos having 6 different frame rates, obtained from 16 diverse contents. In order to understand the combined effects of compression and frame rate adjustment, we also processed videos at 5 compression levels at each frame rate. To obtain subjective labels on the videos, we conducted a human study yielding 19,000 human quality ratings obtained from a pool of 85 human subjects. We also conducted a holistic evaluation of existing state-of-the-art Full and No-Reference video quality algorithms, and statistically benchmarked their performance on the new database. The LIVE-YT-HFR database has been made available online for public use and evaluation purposes, with hopes that it will help advance research in this exciting video technology direction. It may be obtained at <uri>https://live.ece.utexas.edu/research/LIVE_YT_HFR/LIVE_YT_HFR/index.html</uri>.

The analysis of knee kinematics and kinematics during high-risk movements associated with anterior cruciate ligament (ACL) injury is essential to studying the mechanism of ACL injury. Motion capture at low frame rates may not always detect the actual peak values during high-speed movements. However, the knee kinetic differences between various frame rate measurements during high-risk movements have not been reported. The purpose of this study was to investigate whether 3D knee kinetics would be measured differently between frame rates such as a high frame rate (1200 Hz), 400 Hz, 240 Hz and the popular low frame rate (120 Hz). Knee kinetics during a single-leg drop landing and side-step cutting under different frame rates were repeatedly measured and statistically compared. Peak knee valgus and tibial internal rotation moments measured at a popular low frame rate were significantly lower than those measured at the other frame rates of 400 Hz and 1200 Hz. The peak anterior and superior forces measured at the highest (1200 Hz) frame rate were significantly higher than those measured at any other frame rates. In addition, the variations in the peak kinetic values were significantly larger at the lowest frame rate capture and trended toward being smaller at the higher frame rates. In conclusion, significant differences in the knee kinetics between the frame rates suggest that the high frequency capturing increases the accuracy of the knee kinetics measurement for high risk maneuvers for the study of ACL injury.

In accordance with advances in camera technology, the requirement of high-quality video has greatly increased. Among the factors required for high-quality video are a high-resolution and a high frame rate. However, the limitation of the pixel transfer rate restricts compatibility of a high-resolution and a high frame rate in commercial camera. We propose a dual-sensor camera that consists of two cameras: one with a high-resolution and a low frame rate and the other with a low-resolution and a high frame rate. The system is capable of capturing two different image sequences: high-resolution images and high-frame-rate images. A sensor calibration for the dual-sensor camera is also proposed.

Effects of Phase Aberration and Noise on Extended High Frame Rate Imaging

In order to realize real-time and high-speed distortion removal of camera images, a new improved high resolution and high frame rate image distortion based on FPGA is proposed. The traditional method uses four random access memory (RAM) to buffer the distorted image respectively, and then remaps it with the double rate synchronous dynamic random access memory (DDR), which will cause the system to be unable to process high-resolution images at high frame rates. A new RAM buffer architecture and an interpolation module more suitable for Field Programmable Gate Array (FPGA) implementation are proposed, which can improve the system’s ability to process high-speed and high-resolution distorted images in real time within the acceptable range of resources. Compared with the traditional FPGA de-distortion algorithm, the proposed method can de-distortion with the frame rate of 100 fps and the resolution of 3072 x 2048 x 8bit.