Real-Time High-Definition Hyperspectral Endoscopy via Spatial-Temporal Low-Frequency-Stochastic Spectral Encoding.

Hyperspectral (HS) imaging provides rich spatial and spectral information and extends image inspection beyond human perception. Existing approaches, however, suffer from several drawbacks such as low sensitivity, resolution and/or frame rate, which confines HS cameras to scientific laboratories. Here we develop a video-rate HS camera capable of collecting spectral information on real-world scenes with sensitivities and spatial resolutions comparable with those of a typical RGB camera. Our camera uses compressive sensing, whereby spatial–spectral encoding is achieved with an array of 64 complementary metal–oxide–semiconductor (CMOS)-compatible Fabry–Pérot filters placed onto a monochromatic image sensor. The array affords high optical transmission while minimizing the reconstruction error in subsequent iterative image reconstruction. The experimentally measured sensitivity of 45% for visible light, the spatial resolution of 3 px for 3 dB contrast, and the frame rate of 32.3 fps at VGA resolution meet the requirements for practical use. For further acceleration, we show that AI-based image reconstruction affords operation at 34.4 fps and full high-definition resolution. By enabling practical sensitivity, resolution and frame rate together with compact size and data compression, our HS camera holds great promise for the adoption of HS technology in real-world scenarios, including consumer applications such as smartphones and drones.

User performance in virtual environments with degraded visual conditions due to low frame rates is an interesting area of inquiry. Visual content shown in a low frame rate simulation has the quality of the original image, but persists for an extended period until the next frame is displayed (so-called high persistence-HP). An alternative, called low persistence (LP), involves displaying the rendered frame for a single display frame and blanking the screen while waiting for the next frame to be generated. Previous research has evaluated the usefulness of the LP technique in low frame rate simulations during a static target acquisition task. To gain greater knowledge about the LP technique, we have conducted a user study to evaluate user performance and learning during a dynamic target acquisition task. The acquisition task was evaluated under a high frame rate, (60 fps) condition, a traditional low frame rate HP condition (10 fps), and the experimental low frame rate LP technique. The task involved the acquisition of targets moving along several different trajectories, modeled after a shotgun trap shooting task. The results of our study indicate the LP condition approaches high frame rate performance within certain classes of target trajectories. Interestingly we also see that learning is consistent across conditions, indicating that it may not always be necessary to train under a visually high frame rate system to learn a particular task. We discuss implications of using the LP technique to mitigate low frame rate issues as well as its potential usefulness for training in low frame rate virtual environments.

Joint variable frame rate and length analysis for speech recognition under adverse conditions

The analysis of knee kinematics and kinematics during high-risk movements associated with anterior cruciate ligament (ACL) injury is essential to studying the mechanism of ACL injury. Motion capture at low frame rates may not always detect the actual peak values during high-speed movements. However, the knee kinetic differences between various frame rate measurements during high-risk movements have not been reported. The purpose of this study was to investigate whether 3D knee kinetics would be measured differently between frame rates such as a high frame rate (1200 Hz), 400 Hz, 240 Hz and the popular low frame rate (120 Hz). Knee kinetics during a single-leg drop landing and side-step cutting under different frame rates were repeatedly measured and statistically compared. Peak knee valgus and tibial internal rotation moments measured at a popular low frame rate were significantly lower than those measured at the other frame rates of 400 Hz and 1200 Hz. The peak anterior and superior forces measured at the highest (1200 Hz) frame rate were significantly higher than those measured at any other frame rates. In addition, the variations in the peak kinetic values were significantly larger at the lowest frame rate capture and trended toward being smaller at the higher frame rates. In conclusion, significant differences in the knee kinetics between the frame rates suggest that the high frequency capturing increases the accuracy of the knee kinetics measurement for high risk maneuvers for the study of ACL injury.

Mobile sign language video conversations can become unintelligible if high video transmission rates cause network congestion and delayed video. In an effort to understand the perceived lower limits of intelligible sign language video intended for mobile communication, we evaluated sign language video transmitted at four low frame rates (1, 5, 10, and 15 frames per second [fps]) and four low fixed bit rates (15, 30, 60, and 120 kilobits per second [kbps]) at a constant spatial resolution of 320 × 240 pixels. We discovered an “intelligibility ceiling effect,” in which increasing the frame rate above 10fps did not improve perceived intelligibility, and increasing the bit rate above 60kbps produced diminishing returns. Given the study parameters, our findings suggest that relaxing the recommended frame rate and bit rate to 10fps at 60kbps will provide intelligible video conversations while reducing total bandwidth consumption to 25% of the ITU-T standard (at least 25fps and 100kbps). As part of this work, we developed the Human Signal Intelligibility Model , a new conceptual model useful for informing evaluations of video intelligibility and our methodology for creating linguistically accessible web surveys for deaf people. We also conducted a battery-savings experiment quantifying battery drain when sign language video is transmitted at the lower frame rates and bit rates. Results confirmed that increasing the transmission rates monotonically decreased the battery life.

The frame rate (FR) of a video plays an important role in affecting the perceptual video quality. Most studies about the effect of FR on the video quality mainly focused on low frame rate, e.g. less than 30 frames per second (fps), at low resolutions like CIF or QCIF. As the video frame rate and resolution advance, we reconsider this issue and investigate the relationship between frame rate and the perceptual video quality under high frame rate and high resolution. In this paper, we discuss the impact of frame rate on the perceptual quality of High Definition (HD) video with high frame rates (up to 120 fps) considered. Firstly, we design and conduct subjective experiment to construct the video dataset, which includes video sequences at different frame rates and the corresponding mean opinion scores (MOS) which represent the perceptual video quality. Based on the MOS results, we analyze how perceptual video quality changes as frame rate varies among different video sequences and propose some meaningful findings. The video dataset will be made publicly available. We deem that this study will enrich video quality assessment and benefit the development of high frame rate and high definition video business.KeywordsVideo quality assessmentSubjective experienceHigh frame rateHigh definition

The preservation of QoS for multimedia traffic through a data network is a difficult problem. We focus our attention on video frame rate and study its influence on speech perception.When sound and picture are discrepant (e.g., acoustic 'ba' combined with visual 'ga'), subjects perceive a different sound (such as 'da'). This phenomenon is known as the McGurk effect. In this paper, the influence of degraded video frame rate on speech perception is studied.It is shown that, when flame rate decreases, correct hearing is improved for discrepant stimuli and is degraded for congruent (voice and picture are the same) stimuli. Furthermore, we studied the case where lip closure was always captured by the synchronization of sampling time and lip position. In this case, frame rate has little effect on mishearing for congruent stimuli. For discrepant stimuli, mishearing is decreased with degraded frame rate. These results indicate that the stiff motion of lips resulting from low frame rate cannot give enough labial information for speech perception. In addition, the effect of delaying the picture to correct for low frame rate was studied. The results, however, were not as definitive as expected, because of compound effects related to the synchronization of sound and picture. Finally, we inspected the still pictures of normal Japanese speech and determined a lower limit of frame rate from the view point of assisting hearing.

Measurements of strain and strain rate, obtained by tissue Doppler might provide new parameters for assessing cardiac function in neonates. The noise-to-signal ratio is high. We investigated the effect of the frequency of the probe used, and the settings for tissue Doppler frame rate, on the noise in the analyses in three series of tissue Doppler images. In the first series, we used the 5S probe, with a frequency of 2.4 MHz, and the default frame rate. We used the10S probe, with a frequency of 8.0 MHz, in the other two series, one with a low and one with the default frame rate. The noise was lower using the 5S rather than the 10S probe, and lower when using the low frame rate rather than the default rate with the 10S probe. Using the settings eligible for two segment analyses with the lowest noise for each series, the noise was from 36 to 42% higher when using the 10S probe at default frame rate, and from 13 to 14% higher when using the 10S probe at low frame rate compared to the 5S probe at default frame rate. There were no differences in peak systolic strain or strain rate between the series. We found, therefore, that use of the 5S probe with the default setting for frame rate, along with a length of 1 mm and width of 2 mm for the region of interest, and a strain length of 10 mm, provided the optimal settings for two-segment analyses in this study.

Effect of acquisition rate on liver and portal vein enhancement with microbubble contrast

Research on the highest frame rate of high-speed videograph with a channelWang Jingping Lou Wenbi Beijing Institute of Technology., Department of Optical EngineeringABSTRACTThe frame rate is an important factor of HSV system. But the application of the scape of HSV is limited by its low frame rate.This paper researches a variety of methods to rise the frame rate. On the basis of a frame signal recorded on a track we analyse the capacity of the magnetic head tape system in all and get the result of 800 frames per second. If normal recorder is used to record high speed video signal, you can get 200 frames per second under the condition of 200 scan lines.1. INTRODUCTIONIn order to study high-speed event, we need not only high speed shutter to catch the instant phnomenon but also a certain number of pictures to show the all process of the event. Now the frame rate has become of an important parameter of HSV. Since its coming out at the beginning of the 70's, the highest frame rate of HSV with a channel has been about 200 frames per second,which is still low for most fast events.SHP technology is the magnification in time, which is that it records the event with higher speed, then replays it with lower speed so that people can observe and research the event in detail. Because the record medium of HSP is sensitization film, information is recorded in the form of parallel as aresult of clear picture and very high photography frequency. In comparison with HSP, HSV is quite different. It transfers the picture by changing pixels of photoelectric device into electric signal. The record and replay of the picture are done according to electricity to magnetics^ magnetics to electricity. Being limited by record capacity of magnetic head/tape, the transfer velocity of information will be limited also. In other words, the video signal bandwidth will be limited, which results in shortages of low clarity and low frame rate. In order to process data obtained, it is required that one magnetic track record a frame signal. In the following sections we shall discuss the highest frame rate of HSV with a channel on the basis of a frame signal recorded on a track.2. VIDEO SIGNAL BANDWIDTHTelevision picture clarity which depends on the capacity of analysing pixels is subjective sensibility of eyes. The greater the analysis is, the clearer the picture is. The television picture consists of many scan lines from its structure. The picture in horizontal direction is described by scan line, but the picture in vertical direction is composed of many scan lines. So the vertical analysis is equal to the number of scan line at most. Considering blanking and kell coefficient, the vertical resolution is smaller than the number of scan line. Because of consequent scan, the horizontal analysis is infinite under the condition of very smalt diameter of electric beam of scan. But the electric: beam has a certain section, the pixels analysed are limited by its diffraction.Now that the horizontal picture is described by scan line, the pixels analysed in level direction will determine the upper limit frequency of the video signal. The most complicated picture that television system transfers is that the picture consists of white black interval. When the size of white/black is equal to pixel, such picture will produce the highest frequency signal at the case of given the scan velocity. This picture can be used to determine the upper limit frequency of the video signal. It is obvious that the smaller the number of white-black is, the lower the upper Limit frequencyis.

The remote, inaccessible location of many rivers in Alaska creates a compelling need for remote sensing approaches to streamflow monitoring. Motivated by this objective, we evaluated the potential to infer flow velocities from optical image sequences acquired from a helicopter deployed above two large, sediment-laden rivers. Rather than artificial seeding, we used an ensemble correlation particle image velocimetry (PIV) algorithm to track the movement of boil vortices that upwell suspended sediment and produce a visible contrast at the water surface. This study introduced a general, modular workflow for image preparation (stabilization and geo-referencing), preprocessing (filtering and contrast enhancement), analysis (PIV), and postprocessing (scaling PIV output and assessing accuracy via comparison to field measurements). Applying this method to images acquired with a digital mapping camera and an inexpensive video camera highlighted the importance of image enhancement and the need to resample the data to an appropriate, coarser pixel size and a lower frame rate. We also developed a Parameter Optimization for PIV (POP) framework to guide selection of the interrogation area (IA) and frame rate for a particular application. POP results indicated that the performance of the PIV algorithm was highly robust and that relatively large IAs (64–320 pixels) and modest frame rates (0.5–2 Hz) yielded strong agreement ( R 2 > 0.9 ) between remotely sensed velocities and field measurements. Similarly, analysis of the sensitivity of PIV accuracy to image sequence duration showed that dwell times as short as 16 s would be sufficient at a frame rate of 1 Hz and could be cut in half if the frame rate were doubled. The results of this investigation indicate that helicopter-based remote sensing of velocities in sediment-laden rivers could contribute to noncontact streamgaging programs and enable reach-scale mapping of flow fields.

Conventional transcranial ultrasound perfusion imaging (UPI) depends on bolus injection and is limited to triggered imaging. To improve our set-ups, we compared two imaging modalities with two different frame rates (FR) and mechanical indices (MI), intending to approach conditions more similar to real time imaging in order to increase parameter precision. Fifteen healthy volunteers were investigated twice with UPI after i. v. administration of 1 ml of SonoVue(R): first, with a high MI (1.6) and a low FR (0.67 Hz)) and second, with a low MI (1.0) and a high FR (5 Hz). Apart from visual analysis, time-intensity curves were calculated from three regions of interest (ROI) and peak intensity (PI), time to PI (TP), and area-under-the-curve (AUC) were compared between the two imaging modalities. Visually, only scarce contrast enhancement was observed in 10/15 probands, and penetration depth was markedly lower at the low MI/high FR setting, while the high MI/low FR setting lead to very intense enhancement in 13/15 individuals. Signal-to-noise-ratio was higher at the low MI/high FR setting. TP was not significantly different between the two set-ups (p > 0.05). PI and AUC were significantly lower at the low MI/high FR setting (p</= 0.001), and both parameters correlated well in corresponding ROIs (p < 0.05 or < 0.01 in 8/9 ROIs). Parameter images reflected the differences of these two semi-quantitative parameters. In patients with excellent bone windows, the new set-up seems to offer highly precise kinetic analysis with an excellent signal-to-noise ratio. In the majority of patients, however, conventional transcranial UPI is limited to high MI and a resulting rather low FR to allow sufficient penetration depth and contrast enhancement.

A temporally scalable video coding algorithm allows extraction of video of multiple frame rates from a single coded stream. This paper compares the rate-distortion performance of three temporally scalable coding techniques: temporal subband coding (TSB), motion-compensated temporal subband coding (MC-TSB), and motion compensated prediction (MCP). The distortions are derived in terms of various temporal pixel correlations and the probability of occlusion. Measured values are used to predict performances at full and lower frame rates, and a comparison with experimental quantitative performances demonstrates that the performance ranking is correctly predicted and bounds the accuracy of the analyses at high rates. MCP and MC-TSB always outperform TSB. For high bit rate full-frame-rate video, MCP and MC-TSB perform equivalently. At lower frame rates, MCP provides better performance.

This paper addresses the issue of data compression in distributed speech recognition on the basis of a variable frame rate and length analysis method. The method first conducts frame selection by using a posteriori signal-to-noise ratio weighted energy distance to find the right time resolution at the signal level, and then increases the length of the selected frame according to the number of non-selected preceding frames to find the right time-frequency resolution at the frame level. It produces high frame rate and small frame length in rapidly changing regions and low frame rate and large frame length for steady regions. The method is applied to scalable source coding in distributed speech recognition where the target bitrate is met by adjusting the frame rate. Speech recognition results show that the proposed approach outperforms other compression methods in terms of recognition accuracy for noisy speech while achieving higher compression rates.

Photodynamic therapy (PDT) combines a photosensitizer such as Photofrin with red laser light (630 nm) to destroy cancer cells. Investigators have reported effectiveness of PDT in the management of patients with recurrent superficial bladder cancer. We retrospectively reviewed our experience in 58 patients to assess the long-term role of PDT in the management of resistant superficial transitional cell carcinoma (TCC) including Ta, T1, and refractory carcinoma in situ (CIS) of the urinary bladder. All 58 patients had failed at least one course of standard intravesical therapy or had contraindication for intravesical chemo- or immunotherapy. Patients with malignancy present (Ta-T1/Grade I-III, CIS) were accepted for ablative PDT. Patients undergoing prophylactic PDT after complete resection were confirmed to be tumor-free by cystoscopy and bladder was cytology before PDT. Post-PDT evaluations included weekly telephone contact to assess acute adverse reactions and assessment of efficacy and bladder toxicity at three months and quarterly thereafter. These 58 patients underwent a single PDT treatment with 2.0 or 1.5 mg/kg of Photofrin and 10-60 J/cm2 light (630 nm). At three months, complete response rates were 84% and 75% for residual resistant papillary TCC and refractory CIS respectively; and 90% of patients treated prophylactically had not had recurrences. At a median followup of 50 months (range 9-110), 59% (34/58) of the responders are alive, with 31/34 still disease-free. PDT using 1.5 mg/kg of Photofrin and 15 J/cm2 of light (630 nm) should be considered a safe and effective treatment for refractory CIS or recurrent papillary TCC.