An MPEG-4 real-time video decoder software

Due to the high-sped development in semiconductor technology, general-purpose computers can process digital video, audio and graphics easily. Nowadays, the similar functionality has shifted from the general-purpose computer to the so-called media-processor one. Since the processor is targeted on mass-produced consumer electronic devices, the price and the ease of use are the main concern. The latest MPEG-4 video coding standard, which consumes relatively low bit-rate but yields acceptable quality, meets the target exactly. In this paper, by realizing an MPEG-4 codec using both general-purpose processor and media-processor, we investigated some issues related to the use of a media-processor for satisfying various multimedia compression and processing requirements. The investigated issues include the speedup of the MPEG-4 decoder based on a media-processor DSP chip, a multimedia co-processor or hardware accelerator, the very long instruction word (VLIW) and the single instruction multiple data stream (SIMD) programming techniques. Moreover, a media-processor based MPEG-4 solution for set-top box application is also included.

The MPEG-4 standard has already been established, well accepted and is being widely used by content creators, developers, manufacturers, providers of services and end users. Industry leaders have come together to guarantee interoperability. Amongst the video standards, the MPEG-4 video Advanced Simple Profile (ASP) has found wide acceptance and the need for real-time low cost MPEG-4 video ASP encoding solutions isappropriate for certain markets. Embedded multimedia systems, such as programmable VLIW-based chips offer flexibility and functionality, and are very suitable platforms for deployment of real-time multimedia applications. The Equator BSP-15 is one such single chip platform. Enabling technologies for such embedded multimedia systems require very careful and often creative considerations of cost versus performance. To this end, we have designed and developed a real-time MPEG-4 video ASP encoder at Level 5 on an Equator BSP-15 chip. This paper describes our experiences with the design considerations and some implementation issues along with some proposed solutions. The experimental results show the feasibility of these solutions to obtain very good performance on low-cost embedded systems.

This paper provides a realistic case study of using the previously introduced SIMPPL system architectural model, which fixes the physical interface and communication protocols between processing elements (PEs) using PE-specific SIMPPL controllers. The implementation of a real-time MPEG-1 video decoder using SIMPPL provides a practical demonstration of how the complexity of system-level design issues are reduced by enabling rapid system-level integration and on-chip verification. The adaptation of the MPEG-1 PEs into the SIMPPL framework combined with the system-level integration was accomplished in 72.5 hours, which is only 4.5% of the overall system design time, instead of the more typical system integration times that can be as much as 30% of the design time.

We address the problem of improving the scene cut quality in fixed bit-rate real-time video decoding such as is used in the H.263 and MPEG standards. In low bandwidth applications, scene cuts can cause the bits required to encode a single frame to greatly exceed the target average bits per frame, and necessitate the skipping of other frames to provide sufficient time to transmit the scene cut frame. We present an optimal algorithm for minimizing the number of skipped frames and keep the decoding synchronized. Although the algorithm requires additional encoding complexity, there is no change in decoding complexity (in fact, no change to the decoder at all). Experimental results, obtained with a simplified strategy within the framework of H.263+ video encoding, confirm that the method provides an effective alternative to current frame skipping strategies. The overall quality in the presence of scene cuts is improved with respect the TMN-8 rate control. Although the overall bit rate benefits from our method, our focus is to improve the quality of the video where scene cuts occur (by reducing skipped frames and improving decoder synchronization). The approach here can be combined with more sophisticated rate controls, as, for example, the newer rate-distortion optimized TMN-10 and TMN-11.

The current state-of-the-art video coding standard versatile video coding (VVC) offers 50% bit rate savings over its predecessor high efficiency video coding (HEVC) while maintaining the same quality at the cost of a significant increase in the computational complexity. In this work, an Embedded General Purpose Processor (EGPP) with Single Instruction Multiple Data (SIMD) optimisations suite was used to accelerate an open-source software decoder for achieving real-time decoding over an embedded multi-core platform with 8 cores. The decoding speedup has been analyzed with different number of threads/cores, showing that performance increases linearly. This suggests that multi-core based performance for this solution did not reach the saturation point with 8 cores and more parallelism is possible using a higher number of cores. Thanks to the optimization process, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 2$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 2.4$ </tex-math></inline-formula> speedups have been obtained in the decoding time for All Intra and Random Access sequences, respectively. With these results, real-time decoding is achieved for High Definition (HD) sequences. The implementation of a real-time VVC decoder for a low resources embedded platform is the main contribution of this research.

Limited computing resources in portable multimedia devices are an obstacle in real-time video decoding of high resolution and/or high quality video contents. Ordinary H.264/AVC video decoders cannot decode video contents that exceed the limits set by their processing resources. However, in many real applications especially on portable devices, a simplified decoding with some acceptable degradation may be desirable instead of just refusing to decode such contents. For this purpose, a complexity-scalable H.264/AVC video decoding scheme is investigated in this paper. First, several simplified methods of decoding tools that have different characteristics are investigated to reduce decoding complexity and consequential degradation of reconstructed video. Then a complexity scalable H.264/AVC decoding scheme is designed by selectively combining effective simplified methods to achieve the minimum degradation. Experimental results with the H.264/AVC main profile bitstream show that its decoding complexity can be scalably controlled, and reduced by up to 44% without subjective quality loss.

The increased resolution of Quad Full High Definition (QFHD) offers significantly enhanced visual experience. However, the corresponding huge data throughput of up to 530 Mpixels/s greatly challenges the design of real-time video decoder VLSI with the extensive requirement on both DRAM bandwidth and computational power. In this work, a lossless frame recompression technique and a partial MB reordering scheme are proposed to save the DRAM access of a QFHD video decoder chip. Besides, pipelining and parallelization techniques such as NAL/slice-parallel entropy decoding are implemented to efficiently enhance its computational power. The chip supporting H.264/AVC high profile is fabricated in 90 nm CMOS and verified. It delivers a maximum throughput of 4096×2160@60fps, which is at least 4.3 times higher than the state-of-the-art. DRAM bandwidth requirement is reduced by typically 51%, which fits the design into a 64-bit LPDDR SDRAM interface and results in 58% DRAM power saving. Meanwhile, the core energy is saved by 54% by pipelining and parallelization.

This paper presents the design, implementation and experimental analysis of a distributed, real-time MPEG video and audio player. The player is designed for use across the Internet, a shared environment with variable traffic and with great diversity in network bandwidth and host processing speed. We use a novel toolkit approach to build software feedback mechanisms for client/server synchronization, dynamic Quality-of-Service control, and system adaptiveness. Our experimental results show that the feedback mechanisms are effective, and that the player performs very well in the Internet environment.

In the conventional video processing architecture, it is the responsibility of video decoder to decode the raw picture out while it is the responsibility of video post-processor to do operations such as de-interlacing, de-blocking, alpha-blending, as well as the 3:2 pull down detection in DVD playback solution. Our solution mainly focuses on MPEG-2 video decoding, but it can be used more widely in other video format. Conventionally, the video decoder merely provides useful guidance to the decoded picture consumer, the video post-processor. Normally, decisions are made by video post-processor itself to select de-interlacing methods, such as BOB, WAVE and motion adaptive de-interlacing. This paper introduces an interactive method between video decoder and video post-processor. Before providing decoded frame to video post-processor, the video decoder has make decision upon de-interlacing method by analyzing three or more frames in the frame buffers. This method is proved to be memory saving.

There have been increasing market demands to develop more effective schemes for applying MPEG video coding standard to meet the challenges from the emerging applications. We present a real-time multi-pass MPEG video coding sys tem for constant bit rate (CBR) applications, such as Video-on-Demand (VoD) over ADSL. This scheme employs two MPEG encoders. Both encoders operate at the same CBR. The second encoder has a buffer to delay the input by an amount of time relative to the first encoder to create a look-ahead window. In encoding, the first encoder collects statistical and rate-quality characteristic information. An on-line processor then uses the collected information to derive the best coding strategy for the second encoder to encode the incoming frames in the look-ahead window. The second encoder uses the encoding parameters from the processor as the coding guide to execute the coding strategy and produce the compressed bitstream. Results on an IBM MPEG-2 encoder testing system with a 15 frame look-ahead window have shown that the scheme can obtain 0.6 to 1.6dB improvement in PSNR over the conventional single encoder on MPEG test video sources. Moreover, measured by the Tektronix Inc.'s Picture Quality Analysis (PQA) System, this scheme achieved 0.87 ~ 1.35 reduction in PQA index. The visual quality improvment is also very evident.

This paper describes very cost-effective algorithms to detect scene changes and field repetition in video sequences for a real-time MPEG encoder. It also provides a scheme to support a dynamic GOP structure reflecting the detection outcome on the fly. With these features, the encoder can encode video more efficiently in either quality or bitrate aspect. The proposed detection methods only utilize the existing information, field motion vectors and picture coding type, from MPEG coding and are very suitable for (but not limited to) software-based encoders.

In this paper, we propose a fast and effective mode decision (MD) algorithm based on rate distortion optimization (RDO) for AVS high definition video encoder. The fast algorithm is composed of two parts. Firstly, mode pre-selection based on sum of absolute difference (SAD) is employed to reduce modes for candidate so as to alleviate the dramatic throughout burden. Secondly, we adopt 4-way parallel scanning technique to reduce the cycles of each mode decision based on RDO. Theoretical analysis and experimental results show that the proposed fast algorithm can meet the needs of 720P and 1080P real-time high definition AVS video encoding. Besides, the mode pre-selection algorithm provides a similar performance to the all modes enabled algorithm. And the 4-way parallel technique using negligible extra resources increases the speed of coding bits estimation by 3.4 times than traditional techniques.KeywordsAVSmode decisionRDOzigzag scanVLC

3D range geometry video compression with the H.264 codec

We have implemented a so/bvare on(v MPEG-2 Azidio Video Decode (MAVD) system on the Cradle MDSP architectlire arid we highlight the siritobilip of MDSP' architectrire to e.rpluit the data, algorithinic, and pipeline parallelization q/fered hv Video processing algoritlnw like the MPEG2 Video ./or real-time perJiirniance and efficient partitioning a/ Systeiir. Audio and Video Proces.sing 017 a single chip nniltiprocessor. Most e.xisting implementations extract either data or pipeline paralleli.sm along with Instmction Level Paralleli.sm (ILP) in their implementations. We disciics the design of MP@ML MPEG2 video decoding system and MPEG-2 Stereo Decode Svsten? on this shored memon, MDSP plulfbrm. We also highlight how tl7e processor scalahilit?, is exploited as part of the design on this architectirre. Althaiigh simiiltaneous audio-video decode on general-pirrpose processors provides ,/le.rihilip, they are not cost-effective. Most of the media prucesu0r.s exploit hardware acceleration in part or Jirll to alleviate the high-thro~ighprit demands pnt hv these algorithins; thewhy making tl7em inJlerih1e Jar other applications. With the fle.rihility ofired IJV the Cradle platform we cozrld design a video decoder that coiild scale Jiom fbsr MSP.s (Media Stream Proce.ssor that is a clrister ufone RISC arid hvo DSP processors) to eight MSPs and hrrild a single-chip solntian inchiding the 10 interfaces ,for, videa/aridio output. TI7e s,vstem l7a.s heen tested on Cradle's internal CRA20.03 evahiation hoard. Specific contributions inclrrde the mdtiple VLD algorithm and other heiiri.stic approaches like ea!-ly- termination IDCTJbrfast video decoding.

The H.264/AVC Advanced Video Coding standard (AVC) is poised to enable a wide range of applications. However, its increased complexity creates a big challenge for efficient software implementations. This work develops and optimises the H.264/AVC video decoder level two on the TMS320C6416 Digital Signal Processor (DSP) for video conference applications. In order to accelerate the decoding speed, several algorithmic optimisations have been ported to inverse entropy decoding and intra-prediction modules. The parallelism between algorithm execution and data transfers was fully exploited using Enhanced Direct Memory Access (EDMA) engine. Furthermore, based on the DSP architectural features, various core-specific optimisation techniques were adopted leading to an increase in speed by up to 70%. Intensive experimental tests prove that a real-time decoding on TMS320C6416 DSP running at 720 MHz is obtained for Common Intermediate Format resolution (CIF 352 × 288)