A UHD MIPI CSI-2 image acquisition system based on FPGA

In the present situation, applications related to multimedia are discovered to be comfortable with the use of video. The number of end consumers who use video continues to rise every day. People are presently searching for videos with better quality among the ones that are currently available there. This results in the launch and dissemination of HD (high definition) videos. Ultra high-definition (UHD) videos are becoming more and more popular as a result of this advancement and need. However, as video communication keeps expanding, there is an upsurge in network traffic because of the limited bandwidth, especially among smart cities. Different advancement codecs have been suggested to deal with the data stream to overcome this hazardous circumstance. However, the fact that modern UHD videos have huge amounts of data makes the available codecs even more complicated. UHD videos can be processed with the latest improvement codec, H.265/High-efficiency video coding (HEVC). Nevertheless, it is impacted by increased power consumption and intricate calculations. Limitations in the codec's functionality confine its use to specific applications, preventing its application in wireless, mobile, or portable settings. Hence, this research concentrates on implementing frame-level quality enhancement through a deep learning network known as FQE-Net. The deep learning convolutional neural network (DLCNN) is specifically crafted to manage films with resolutions up to 16K. Its primary objectives include reducing complexity, minimizing artifacts, enhancing the efficiency of the HEVC codec, and compacting energy consumption. To achieve superior efficiency, it is imperative to replace the DWT transforms within the HEVC codec with a DLCNN model. Additionally, incorporating the Content Block Search Algorithm for Motion Estimation and Compensation, alongside filtering techniques like Sample Adaptive Filter and Deblocking Filter, becomes essential. The simulation results showed that the suggested FQE-Net performed better than the conventional techniques.

The transoral endoscopic thyroidectomy vestibular approach (TOETVA) is the most recently introduced method of minimally invasive thyroid surgery. To our knowledge, no studies have compared TOETVA outcomes using different laparoscopic systems. This study compared outcomes of TOETVA using conventional high definition (HD) and ultra-high definition (UHD) equipment. Medical data and surgery videos of 62 patients who randomized to undergo thyroid lobectomy by TOETVA using an HD or UHD system from August 2018 to April 2019 were retrospectively reviewed. Endoscopic procedures were divided into four phases: flap creation to isthmectomy (phase I); strap muscle division to upper pole ligation (phase II); trimming recurrent laryngeal nerve (phase III); and ligation of Berry's ligament to complete resection (phase IV). Of the 62 patients, 28 underwent TOETVA using an HD device and 34 using a UHD device. The clinical characteristics of the two groups were not different. Mean operating time for lobectomy was similar in the HD and UHD groups (44.19±9.94 versus 43.47±12.19 min, P=0.825). The times required for phases I (15.02±5.33 versus 13.67±5.44 min, P=0.397); II (12.89±2.84 versus 13.17±5.22 min, P=0.816); III (9.85±4.36 versus 9.98±4.55 min, P=0.918); and IV (6.43±3.69 versus 6.65±3.45 min, P=0.840) were also similar in the HD and UHD groups. The numbers of retrieved lymph nodes did not differ significantly in the HD and UHD groups (3.26±2.62 versus 3.45±2.81, P=0.807). Applying a UHD system in TOETVA resulted in outcomes similar to those observed with a conventional HD system. Operation time tended to be lower, especially for flap dissection, and numbers of harvest lymph nodes tended to be higher in the UHD group. Large-scale studies are needed to assess the advantages of the UHD system.

The introduction of a wide-gamut color space in ultrahigh-definition (UHD) television creates a need to match colors on wide-gamut UHD displays with colors on conventional high-definition (HD) displays. Many contemporary color conversion methods that apply to conversion from HD to standard definition (SD), and color conversion methods mandated by current television standards, fail to produce a good color match when converting colors from HD or narrow-gamut UHD color spaces to the wide-gamut UHD color space. This paper illustrates the color errors caused by the application of several of these conversion methods and recommends one method, using display-referred colorimetry, for better color matching between narrow-gamut and wide-gamut displays.

Ultra High-Definition TV (hereinafter referred to as “UHDTV”), featuring 33 megapixels with 22.2 multichannel sound, adds a new sense of reality to TV programs. We are currently developing new video equipment for a UHDTV system, enabling us to shoot and record immersive scenes for sports, nature and art coverage for UHDTV program production. NHK's UHDTV system, generates images with 16 times the spatial sampling of current HDTV, consists of 4layers (G1, G2, B, R) of the Gch diagonal pixel offset structure and each layer is 3840×2160. A new, compact down converter exclusive to UHDTV program production converts 8K (7680×4320) images to 4K (3840×2160) or HDTV images; it's also equipped with an HDTV image cut-out function, a color correction function, and an energy-saving function. Because a second-dimension digital filter process is applied for converting Dual Green UHDTV signals to UHDTV signals, it can excerpt high quality HDTV images smoothly from arbitrary UHDTV images on a live feed, which make a single source effectively multipurpose. This converter is expected to be used for HDTV sport programs such as football games, to pick up specific players from UHDTV images. New lenses are also being developed for UHDTV cameras. New wide-angle and portable zoom lens have been specifically designed for UHDTV imaging. The view field of the wide lens is 100 degrees, the angle at which the sensation of reality becomes saturated. Zoom ratio is 10:1 (18–180mm focal range), while the maximum focal length is triple that of existing lenses and the resolutional power at the center is more than 250 line pairs per millimeter (LP/mm). In addition, a compact camera and recording and playback equipment for UHDTV have been developed. This report provides a UHDTV video equipment development, starting with a consideration of the signal processing of the down converter, and the lenses.

High efficiency Video Coding (HEVC) is the state-of-the-art video coding technique capable of encoding Ultra High Definition (UHD) videos with better compression efficiency and has better reconstruction quality for the same bitrate as compared to its predecessors. Better compression is possible due to its complex partition and prediction methods. These benefits are at the cost of increased computational complexity, which in turn increases resource consumption and processing time. In this work, we design and implement three different architectures, viz: 1) Fully Sequential Architecture (FSA), 2) Semi-parallel Architecture (SPA) and 3) Fully Parallel Architecture (FPA), for the Intra prediction of HEVC on Field Programmable Gate Arrays (FPGA) and discuss the results. These three configurations are tested for the prediction units of sizes 4×4, 8×8 and 16×16. Results show that FSA uses nearly 70% fewer resources than FPA. Also FSA uses 51.73%, 54.33% and 52.2% less resources than SPA for 4×4, 8×8 and 16×16 block sizes, respectively. Also, the FPA implemented for all three pediction unit (PU) sizes is nearly 22 times and 5 times faster than the FSA and SPA, respectively.

High capacity IEEE802.11 networks are increasingly supporting data intensive applications. Most of streaming applications are using high quality variable rate traffic which includes HD (High Definition) and UHD (Ultra High Definition) video streaming sources. Such services demand fair QoE for all end users which requires high network QoS as well as inter and intra service fairness. It is a challenging task to offer service fairness in a random access network due to nondeterministic resource allocation techniques. In this paper, we propose two video packet scheduling algorithms for the 802.11 MAC protocol which can offer high QoS as well as fairness among video streams transmitted on the downlink using the IEEE802.11ac MAC protocol. The first algorithm is the FRATXOP that allocates transmission resources for each flow based on experienced network congestion levels to improve throughput of HD and SD video connections. The second algorithm the EAS (Enhanced Access Scheduler) operates in conjunction with the FRA-TXOP to fairly allocate transmission resources to maintain fairness among all video streams. A ns-3 simulation model was developed to evaluate the performance of the above algorithms and compare with the standard IEEE802.11ac MAC protocol. Simulation results show that proposed algorithms can enhance the high network QoS as well as maintain high QoE fairness among all end video users.

This paper presents Layered Division Multiplexing (LDM), a technology that may be used to provide a flexible multi-layer system transmission by means of spectrum overlay. This technology can be used to simultaneously deliver multiple program streams with different characteristics and robustness for different services (mobile TV, HDTV and UHDTV) in one RF channel. — In Layer Division Multiplex (LDM) the signal to be transmitted consists of a number of different independent signals superimposed together at different injection levels to form a multi-layer signal. Each layer can have its own characteristics. The top layer is the most robust one, which has a negative Signal to Noise Ratio (SNR) system threshold value, and can be used for robust mobile service. The lower layer(s) can be used to provide fixed high data rate services, such as multiple High Definition Television (HDTV) and Ultra High Definition Television (UHDTV). For example, DVB-T2, or an alternative design of a high data rate transmission system can be used for the second layer. The upper layer signal can be a separate program, or be used for delivering supplementary bit rate to be combined with the lower layer signal to provide additional features or achieve higher quality of service (e.g., scalable video coding and placing audio on upper layer for extra robustness).

The video resolution among broadcasting contents is particularly becoming high toward Ultra High Definition (UHD). Due to its massive size, however, UHD contents cannot be delivered over the existing broadcasting system. To solve this problem, we have considered two aspects: techniques of video codec and transmission. In this paper, we propose service architecture for delivering UHD contents encoded hierarchically over heterogeneous networks. They can be divided into two parts and at the sending entity and then transmitted the part of High Definition (HD) content over the broadcasting network and the rest over the communication network. At the receiving entity, they can be merged into the original contents. The proposed architecture is based on Internet Protocol (IP) in order to be compatible with packet-switched networks.

The migration from High Definition (HD) TV to Ultra High Definition (UHD) is already underway. In addition to an increase of picture spatial resolution, UHD will bring more color and higher contrast by introducing Wide Color Gamut (WCG) and High Dynamic Range (HDR) video. As both Standard Dynamic Range (SDR) and HDR devices will coexist in the ecosystem, the transition from Standard Dynamic Range (SDR) to HDR will require distribution solutions supporting some level of backward compatibility. This paper presents a new HDR content distribution scheme, named SL-HDR1, using a single layer codec design and providing SDR compatibility. The solution is based on a pre-encoding HDR-to-SDR conversion, generating a backward compatible SDR video, with side dynamic metadata. The resulting SDR video is then compressed, distributed and decoded using standard-compliant decoders (e.g. HEVC Main 10 compliant). The decoded SDR video can be directly rendered on SDR displays without adaptation. Dynamic metadata of limited size are generated by the pre-processing and used to reconstruct the HDR signal from the decoded SDR video, using a post-processing that is the functional inverse of the pre-processing. Both HDR quality and artistic intent are preserved. Pre- and post-processing are applied independently per picture, do not involve any inter-pixel dependency, and are codec agnostic. Compression performance, and SDR quality are shown to be solidly improved compared to the non-backward and backward-compatible approaches, respectively using the Perceptual Quantization (PQ) and Hybrid Log Gamma (HLG) Opto-Electronic Transfer Functions (OETF).

We are currently researching a next-generation broadcasting system named "Super Hi-Vision", which provides images with much higher resolution and quality than a high definition television, and are developing displays for this system. We have developed a projector which can display the high-resolution images of Super Hi-Vision (7680x4320 pixels) and has a high dynamic range of 1.1 million to 1. This projector features a serial combination of two modulation blocks: the first block for chrominance modulation and the second block for luminance modulation. While a conventional projector has one modulation block that contains three devices to modulate red, green, and blue light, our projector has another modulation block that contains a device for luminance to further modulate the combined RGB modulated light. This configuration enables the projector to output high-resolution color images by combining three low-resolution devices for chrominance modulation and one high-resolution device for luminance modulation. As a high-resolution device, we have newly developed a 1.75-inch liquid crystal on silicon device with 8192x4320 pixels. In addition, the dynamic range is dramatically improved because this dual modulation scheme minimizes the black levels in projected images. We demonstrated that our projector displays Super Hi-Vision color images and that it has an extremely high dynamic range of 1.1 million to 1 and a fine 10-bit tone reproduction.

With a new video coding standard high efficiency video coding (HEVC), the ultra high definition (UHD) TV service with robust video streaming technology is emerging in the TV industry. This paper addresses the system architecture for the UHD video streaming and proposes two main ideas: (i) picture prioritization method, (ii) error concealment mode signaling (ECMS). In the experiments using HEVC reference model conducted, the proposed picture prioritization method shows the gains in video quality from 2.2 to 7.5 dB in Y-PSNR, and the error concealment mode signaling gains from 0.2 to 2.5 dB in Y-PSNR, with corresponding subjective improvements.

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">As video content evolves from traditional television into more complex and varied forms of media, more and more of our content creators capture in higher resolution, frame rate, and dynamic range instead of traditional high-definition (HD) to give themselves creative latitude in post-production. Generating ultrahigh-definition (UHD) content comes with a lot of challenges for a traditional HD infrastructure. For example, which version(s) should be archived: The source, ungraded asset, the HD version, the social media version(s) like square and vertical, or a combination of these? If content is shot in UHD, for example, and then cropped in post, should the final, cropped HD product be the one we archive, or the UHD original? If we archive the original, how do we store information about the cropping parameters? If color and dynamic range are wider in the source than the final render (e.g., SLog3/SGamut3), which 3D look-up table is the right one to use? Should 3D LUT also be archived with the source asset? How do we link a specific asset to a specific 3D LUT for posterity? Also, public broadcasters have legal obligations with regard to content preservation, but these do not necessarily specify the format(s) for archival purposes, so we need to balance obligations, cost, and future value of assets in the process of designing such workflows</i> .

We have been developing an ultra high definition television (UHDTV) system with a 7,680 horizontal by 4,320 vertical pixel resolution and a 60 Hz frame rate. This system, which is called Super Hi-vision (SHV), is expected to serve the next generation of broadcasting services. We have just completed the world's first imaging equipment that is capable of capturing video at a full SHV resolution. In designing this equipment, we decided to develop three new devices, taking into account the camera performance and the ease of implementation. First, we developed a 33-megapixel CMOS image sensor. Its pixel size of 3.8 &mgr;m sq. retained the dynamic range of the sensor above 60 dB even with a 3-transistor pixel structure. Second, a fixed focal length lens was developed to create an adequate MTF right up to the limiting resolution of the sensor. Third, we developed a signal-processing device capable of handling 72 Gbps signals and cascading boards to expand the process. SHV images with a modulation of 20% at the Nyquist frequency were obtained by using these three key technologies.

The Super Hi-Vision (SHV) is an ultra high-definition video system with 4,000 scanning lines. Its video format is 7,680×4,320 pixels, which is 16 times the total number of pixels of high definition television and the frame rate is 60 Hz with progressive scanning. It has been designed to give viewers a strong sense of reality. To make the system suitable for practical use, such as in broadcasting services, a high-efficiency compression coding system is necessary. Therefore, we have developed several Super Hi-Vision codec systems based on MPEG-2 and AVC/H.264 video coding standards. In this chapter, details of these codec systems are described and transmission experiments using the codec systems are introduced.

Layered Division Multiplexing (LDM) has demonstrated increased data rate capacity for wireless applications where the service bitrates being multiplexed are unbalanced. Current Direct-To-Home (DTH) broadcasting satellite standards (DVB-S2 and DVB-S2X) include Variable Coding and Modulation (VCM) mode. VCM provides the opportunity to deliver different services on a Time Division Multiplexing (TDM) basis and in consequence target different reception conditions with a single carrier. This article proposes the combination of the tools provided by DVB-S2X with LDM for satellite broadcast and communication systems. This article describes use cases where multiplexing schemes can be beneficial in comparison with VCM (TDM) approach. The use cases include the delivery of High Definition (HD) and Ultra High Definition (UHD) TV services to customers as well as additional communication services. The performance of the proposal is evaluated with simulations under different channels, including the impact of satellite equipment non-linearities. Analysis shows that LDM provides considerable gain in comparison with VCM in both AWGN and non-linear satellite channels.