Color Normalization for Digital Video Processing

The standard method for making the full content of audio and video material searchable is to annotate it with human-generated meta-data that describes the content in a way that the search can understand, as is done in the creation of multimedia CD-ROMs. However, for the huge amounts of data that could usefully be included in digital video and audio libraries, the cost of producing this meta-data is prohibitive. In the Informedia Digital Video Library, the production of the meta-data supporting the library interface is automated using techniques derived from artificial intelligence (AI) research. By applying speech recognition together with natural language processing, information retrieval, and image analysis, an interface has been produced that helps users locate the information they want, and navigate or browse the digital video library more effectively. Specific interface components include automatic titles, filmstrips, video skims, word location marking, and representative frames for shots. Both the user interface and the information retrieval engine within Informedia are designed for use with automatically derived meta-data, much of which depends on speech recognition for its production. Some experimental information retrieval results will be given, supporting a basic premise of the Informedia project: That speech recognition generated transcripts can make multimedia material searchable. The Informedia project emphasizes the integration of speech recognition, image processing, natural language processing, and information retrieval to compensate for deficiencies in these individual technologies. © 1998 John Wiley & Sons, Inc.

Colour image variations produced by the change of video acquisition devices strongly affect colour appearance based recognition tasks in digital video applications. Up to date, many algorithms have been developed in order to achieve independence from different image formation factors like intensity, orientation and colour of the illumination source. We check the capability of three of these approaches to generate device independent colour representations in order to be used as a normalization step previous to the recognition process. Our previously developed color correction approach for video imagery based on Gaussian mixture models has also been tested. Experimental results show that a simple algorithm like the grayworld correction is enough for appearance based recognition under this particular kind of color variability. Similar recognition rates are obtained using our correction, with the advantage that the visual quality of normalized images is preserved. In digital video library applications, where not only recognition tasks are performed, this approach becomes much more useful.

The number of channels of digital television is increasing, particularly the number that are free-to-air. However due to the nature of broadcasting, this morass of information is not, for the main part, organized---it is principally a succession of images and sound transmitted as multiplexed streams of data. Compare this deluge that terrestrially bombards our homes with the information available in the digital libraries we access over the Internet---stored using software purpose built to help organize carefully curated sets of documents. This project brings together these two seemingly incompatible concepts to develop a software environment that concurrently captures all the available live television channels---so a user does not need to proactively choose what to record---and segments them into files which are then imported into a digital video library with a user interface designed to work from a multimedia remote control. A shifting time-based window of all recordings is maintained---we settled on from the last two weeks so as to be practicably operable on a regular desktop PC. The system leverages off the information contained in the electronic program guide and the video recordings to generate metadata suitable for the digital library. A user evaluation of the developed prototype showed a high level of participant satisfaction across a range of attributes, notably date-based searching.

Video is a challenging material for digital libraries because it contains large amounts of data which must be processed quickly and because video programming has not been as conveniently structured for algorithmic processing as text. Since moving images are used as scholarly material in a number of disciplines, digital libraries must be able to support cataloging, browsing, and navigating through video collections. We have shown that computationally tractable algorithms can be used to solve some of the important problems in digital video libraries. While much further work needs to be done, we believe that signal and image processing algorithms will help us solve other important problems in video libraries.

Technology usage is changing rapidly and is becoming a more mobile, more social and more multimedia-based experience. This is especially true in the area of content creation where mobile social applications used by crowds of people are challenging traditional ways of creating and distributing content, especially for applications like news dissemination. Libraries have traditionally functioned as repositories where the information content of a society is analysed, curated, organised and stored, acting as a permanent record of what is to be remembered from a society. How can this function be achieved by present-day libraries attempting to cope with mobile, social, multimedia content who's nature and utility of which change the type of information we wish to curate and store? This information is both dynamic and organic, posing challenges to the more fixed models of information in digital libraries. In this article we describe two digital library systems that archive video content from the sports domain, and which support user annotations and merging of diverse information sources in an integrated way. We report on analysis of the deployment of these two systems and highlight how they extend the traditional role of a (digital) library.

This paper proposes an abstraction mechanism of the low-level digital video features for the automatic retrievals of the explosion scenes from the digital video library. In the proposed abstraction mechanism, the regional dominant colors of the key frame and the motion energy of the shot are defined as the primary low-level visual features of the shot for the explosion scene retrievals. The regional dominant colors of shot are selected by dividing its key frame image into several regions and extracting their regional dominant colors, and the motion energy of the shot is defined as the edge image differences between key frame and its neighboring frame. Upon the extensive experimental results, we could argue that the recall and precision of the proposed abstraction and detecting algorithm are about 0.8, and also found that they are not sensitive to the thresholds.KeywordsColor HistogramEdge ImageRetrieval AlgorithmEdge PixelDominant ColorThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Accurately describing the effect of lighting on color appearance phenomena is critical for color science education. While it is ideal to conduct in-person tutorials to demonstrate the color appearance fundamentals, laboratory tutorials have been limited due to COVID-19. The limitation of in-person gatherings and the increase popularity of remote teaching help evoke alternative methods to demonstrate color appearance phenomena. Here, a remote tutorial method is described, and results are compared to in-person tutorials. While the remote tutorial had weaker result in representing observers' color experience compared to the in-person lab tutorial, remote demonstrations can be used to demonstrate and discuss the limitations of color imaging, and the difference between the human visual system and digital imaging systems.

In the world as we know it today, collections of landscape paintings are kept in digital libraries for better organization and management. Collection holders such as museums have been increasingly interested in making their collections available to people anytime and anywhere for any Image Retrieval (IR) activities which include browsing and searching. Paintings retrieved by users are accomplished according to their specifications which could be based upon so many concepts. Colour appearance is one of the concepts based on the intensity of colours in image content. In this paper, we investigate the colour appearance categories for whole scene landscape paintings based on human colour perception for Malaysian users. Our first objective is to validate a set of ground-truth images of landscape paintings for six categories of colour appearances determined from previous works using the triangulation technique and secondly is to identify the most commonly used words to describe six categories of colour appearance of landscape paintings. The method used in this work is quantitative survey using purposive sampling which includes two colour blinded (CB) participants. The experimental results show that there is not much of a difference on the triangulations survey; however the results from this work could be more reliable because of the sampling. Six most common words used to describe six categories of colour appearance of landscape paintings for Malaysian users have also been identified. This finding is important for our next research work which focuses on the features of colour appearance categories and classifier for IR application.

Purpose– The purpose of this paper is to report on a user-centred analysis of video digital libraries and to present users’ criteria based on concepts emerging directly from the responses of study participants. As a time-based and multi-channeled format, video warrants considerations for information delivery through interactive tools such as digital libraries.Design/methodology/approach– A survey method was used to collect open-ended responses from participants. Content analysis was performed on the responses, and categories then emerged to form the coding scheme where simple and weighted frequencies were calculated to obtain a ranked set of users’ criteria. Cohen’sκwas 0.87, indicative of high-level of inter-coder reliability. In the end, 81 participants contributed a total of 385 open-ended responses, which were all then analysed and coded.Findings– The emergent coding method and continuous refinements to the coding scheme ultimately produced 28 criteria (subcategories) under four primary categories. Criteria corresponding to “retrieval functionality” of video digital libraries emerged as the highest, or most frequently referenced, primary category, while the “user interface”, “collection qualities”, and “user support” followed, respectively.Research limitations/implications– The primary research contribution is a baseline for video digital libraries that can be applicable throughout design and evaluation. Future studies can utilise findings presented here as guidance for the initial design stages and/or for assessing systems via targeted evaluations.Originality/value– While users’ criteria of digital libraries have been analysed from non-visual contexts, they have not been directly assessed as they pertain to users’ experiences and perceptions with video, which is needed as the nature and structure of video can influence users and use.

The color appearance of a stimulus depends on the context in which it is viewed. Well-known examples of context effects include simultaneous color contrast and color constancy. In simultaneous contrast, the color immediately surrounding a test influences its appearance. In color constancy, the visual system adapts to the ambient illumination to keep surface color approximately constant. Color constancy is a context effect because the way that the visual system interprets the light reaching the eye depends on the context defined by the ambient illumination.To predict accurately the color appearance of stimuli rendered on a CRT monitor, we need a theory of color context effects that applies to such stimuli. Simultaneous contrast suggests that the color perceived at a particular monitor location may vary with the immediate context provided by the image at other locations on the monitor (the monitor context). Color constancy suggests that the color appearance of stimuli displayed on a monitor may vary with the ambient illumination of the room in which the monitor is viewed (the illumination context), at least if images from a monitor are processed by the visual system in the same way as reflective surfaces.In this paper, we report initial experiments designed to measure psychophysically how the monitor context and the illumination context influence the appearance of stimuli presented on CRT color monitors. Our emphasis, however, is on the effect of the illumination context.

The Informedia Digital Video Library Project is developing new technologies for creating full-content search and retrieval digital video libraries. Working in collaboration with WQED Pittsburgh, the project is creating a testbed that will enable K-12 students to access, explore, and retrieve science and mathematics materials from the digital video library. The library will initially contain 1,000 hours of video from the archives of project partners: WQED, Fairfax Co. VA Schools' Electronic BBC-produced video courses. (Industrial partners include Digital Equipment Corp., Bell Atlantic, Intel Corp., and Microsoft, Inc.) This library will be installed at Winchester Thurston School, an independent K-12 school in Pittsburgh.

The automatic extraction and recognition of news captions and annotations can be of great help locating topics of interest in digital news video libraries. To achieve this goal, we present a technique, called Video OCR (Optical Character Reader), which detects, extracts, and reads text areas in digital video data. In this paper, we address problems, describe the method by which Video OCR operates, and suggest applications for its use in digital news archives. To solve two problems of character recognition for videos, low-resolution characters and extremely complex backgrounds, we apply an interpolation filter, multiframe integration and character extraction filters. Character segmentation is performed by a recognition-based segmentation method, and intermediate character recognition results are used to improve the segmentation. We also include a method for locating text areas using text-like properties and the use of a language-based postprocessing technique to increase word recognition rates. The overall recognition results are satisfactory for use in news indexing. Performing Video OCR on news video and combining its results with other video understanding techniques will improve the overall understanding of the news video content.

Age-related anatomical and physiological changes in the human visual system have been reported such as an increase in the density of the lens, a reduction of pupil size and loss of retinal ganglion cells. Although the retinal stimulus and neural signals are strongly affected by such age-related changes, color appearance is stable throughout the life-span because of parallel losses of cone sensitivities and other compensation mechanisms related to color constancy. On the other hand, there is a loss of sensitivity in color discrimination due, in part, to the reduction of retinal illuminance. The results of recent experimental studies by the author and colleagues indicate that this loss in discrimination is not only caused by the reduction of retinal illuminance, but also by a decrease in the signal-to-noise (S/N) ratio in neural pathways processing signals from S-cones and antagonistic signals from L- and M-cones. This change can be considered as a sort of a trade-off between maintaining a constant signal level from the cones through the life-span and decreasing the signal-to-noise ratio in the human visual system. These results suggest some possible methods for making safer and more comfortable visual environments for the elderly.

An automatic disassembly cell requires of a computer vision system for recognition and localization of the products and each of theirs components. The detection of occlusions adds more information to the knowledge base to identify components and products, and generate a trustworthy and precise relational model (generic graph of hierarchic relations among the different components that make up the product). In this paper, a method to detect partial occlusions in assembled components is presented. This method is based on the fusion of regions and edges information, and it offers a certain degree of simplification for the recognition and modelled of the disassembly tasks, of the set of components which compose the product. The proposed approach to detect regions is a hybrid approach between RGB and HSV spaces. The bi-dimensional histogram V/S is employed for the selection of the appropriate thresholds which serve as an aid to diminish the influence of the highlights and shadows in images. The goal of this paper is to present an approach for the detection of occlusions in assembled components from a combination of HSV and RGB spaces, a bi-dimensional histogram and an edge detector.

Many commercial and academic disciplines need digital video libraries. Such libraries must provide the user with the opportunity to manipulate the material non-linearly. This paper discusses digital video libraries and various non-linear services these libraries will provide. The architectural components of such libraries are discussed and a model of user access is developed. Results from caching experiments utilizing that model are reported for several usage patterns and four cache organizations.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.