An automatic video indexing method based on shot classification

The proliferation of computer technology and digital image‐acquisition hardware has led to the widespread use of image data across a variety of applications including astronomy, art, natural resources, engineering design, military, business operation, medicine, education, etc. Major research activities in the digital image databases surged after the U.S. government's Digital Library Initiative (DLI) from 1994 to 1998. The DLI research team at the University of California at Berkeley developed a work‐centered digital information system that contains 450 digital text documents, 200 air photos, and over 11,000 ground photographs. The system allowed a user or a working group to access its own collections of varying data types, and to generate new materials to be added to the collection. The DLI project at the University of California (UC) at Santa Barbara focused on the development of digital databases that contain geographically referenced materials such as maps and aerial photos. The research team developed a number of tools for browsing and retrieving map images at multiple resolution. Two other major efforts in developing digital image databases were at The National Library of Medicine (NLM), a component of the National Institutes of Health (NIH), and at Time Warner. More recently, research scientists at the University of California at Berkeley and the Fine Arts Museums of San Francisco successfully launched on the Internet the largest art image database in the world, the Thinker ImageBase.Automatic image indexing and retrieving technology is fundamental in digital image databases. There are two general strategies in image retrieval, browsing and searching. Browsing is an information retrieval strategy in which a user navigates through an ordered arrangement of images by making selections from the progressive levels of a hierarchy into which the available images have been logically grouped. Image browsing relies on the user's cognitive abilities to recognize images of interest without having to formulate a specific query. Searching is an information retrieval strategy in which the user communicates with an information retrieval system by an interface that requires the user to input a search query.Images can be indexed and retrieved by text‐tag information and/or by image content. Conventionally, images are indexed using text‐tag information, such as title, key words, date of the work, artist, author, photographer, legend, captions, etc. Images are often subject to a wide range of interpretations, and textual descriptions can only begin to capture the richness and complexity of the semantic content of a visual image. In many applications, the complexity of the information embedded in image content such as types of objects, object attributes, and spatial relationship of objects, etc, cannot be synthesized in a few key words. It has been reported that users querying an image collection tend to be much more specific in their requests and information needs than when querying a text database. Furthermore, text indexing requires large human effort in creating the meta‐data that enables visual queries and is language dependent. Large databases containing thousands and millions of digital images that can occupy gigabytes of space are almost impossible for manual indexing and searching. The demand for systems using pictorial information combined with textual description in image retrieval is growing and automatic indexing and retrieval based on image content has become the most promising techniques for large image databases and digital image libraries. This article describes the techniques developed for content‐based retrieval.

When a designer finds a problem during a development process, and effective way of ensuring the security and safety of the product is to identify all the possible causes of the problem by FTA and fix them. Although FTA is an effective method, it is not easy for a designer to construct a complete fault tree without any misunderstanding or oversight. Quantity dimension indexing is the method which verifies consistency of a fault tree automatically by comparison of quantity dimension. This paper describes two result. First, how to verify the correct quantity dimension. Before, there were cases when one phrase had the plural interpretation, a designer had to index the quantity dimension to the phrase manually. The task requires a designer the amount of time and labor. So, this paper describes methods of consistency verification by searching all over the fault tree. Second, new automatic indexing method. FTA often overlook the dynamic fault event. So, it has been proposed the classification about the qualitative fault value, but it is not clear how to indexing automatically. The paper also describes the method of automatic indexing the qualitative fault value.

The trend of learning from videos instead of documents has increased. There could be hundreds and thousands of videos on a single topic, with varying degrees of context, content, and depth of the topic. The literature claims that learners are nowadays less interested in viewing a complete video but prefers the topic of their interests. This develops the need for indexing of video lectures. Manual annotation or topic-wise indexing is not new in the case of videos. However, manual indexing is time-consuming due to the length of a typical video lecture and intricate storylines. Automatic indexing and annotation are, therefore, better and efficient solutions. This research aims to identify the need for automatic video indexing for better information retrieval and ease users navigating topics inside a video. The automatically identified topics are referred to as “Index Points.” 137-layer YoloV4 Darknet Neural Network creates a custom object detector model. The model is trained on approximately 6000 video frames and then tested on a suite of 50 videos of around 20 hours of run time. Shot Boundary detection is performed using Structural Similarity fused with a Binary Search Pattern algorithm which outperformed the state-of-the-art SSIM technique by reducing the processing time to approximately 21% and providing around 96% accuracy. Generation of accurate index points in terms of true positives and false negatives is detected through precision, recall, and F1 score, which varies between 60–80% for every video. The results show that the proposed algorithm successfully generates a digital index with reasonable accuracy in topic detection.

The research presented herein provides exciting findings in the field of automatic affective video indexing. Because advances in multimedia technologies have resulted in extensive digital video libraries, automatic methods are needed to assist users in finding videos with the content they desire. A growing area of research in the field is affective video indexing. This area of research attempts to identify video content that is affective, or emotional in nature. Slapstick comedy is a widely popular humor technique found in videos. Slapstick has features that make it a desirable target for automatic affective video indexing strategies. This research is the first of its kind to use computer-based methods to not only identify slapstick comedy in videos, but to pinpoint the location of the slapstick event. The key break-through of the research is that the targeted content was identified using the low-level video characteristics, rather than relying on human-generated labeling. This study provides positive results along with approaches to future research that are aimed at expanding the field of automatic affective video indexing.

An automatic phrase indexing method based on the term discrimination model is described, and the results of retrieval experiments on five document collections are presented. Problems related to this non-syntactic phrase construction method are discussed, and some possible solutions are proposed that make use of information about the syntactic structure of document and query texts.

Many automatic text indexing and retrieval methods use a term-document matrix that is automatically derived from the text in question. Latent Semantic Indexing, a recent method for approximating large term-document matrices, appears to be quite useful in the problem of text information retrieval, rather than text classification (Deerwester, et al., 90). Here we outline a method that attempts to combine the strength of the LSI method with that of neural networks, in addressing the problem of text classification. In doing so, we also indicate ways to improve performance by adding additional "logical sensors" to the neural network, something that is hard to do with the LSI method when employed by itself. Preliminary results are summarized, but much work remains to be done.

We propose a novel method for automatic annotation, indexing and annotation-based retrieval of images. The new method, that we call Markovian Semantic Indexing (MSI), is presented in the context of an online image retrieval system. Assuming such a system, the users' queries are used to construct an Aggregate Markov Chain (AMC) through which the relevance between the keywords seen by the system is defined. The users' queries are also used to automatically annotate the images. A stochastic distance between images, based on their annotation and the keyword relevance captured in the AMC, is then introduced. Geometric interpretations of the proposed distance are provided and its relation to a clustering in the keyword space is investigated. By means of a new measure of Markovian state similarity, the mean first cross passage time (CPT), optimality properties of the proposed distance are proved. Images are modeled as points in a vector space and their similarity is measured with MSI. The new method is shown to possess certain theoretical advantages and also to achieve better Precision versus Recall results when compared to Latent Semantic Indexing (LSI) and probabilistic Latent Semantic Indexing (pLSI) methods in Annotation-Based Image Retrieval (ABIR) tasks.

Evaluation of a Meta-1-based automatic indexing method for medical documents

With the rapid development of mobile Internet technology and artificial intelligence technology, the digital publishing industry is in urgent need of using intelligent technology to change the current way of content production and service. Most of the e-book resources owned by publishing enterprises are in PDF format, which is not suitable for reading on mobile devices, and it is not convenient to directly extract key information and construct knowledge graph. With this in mind, this article designs a PDF automatic indexing scheme that can identify all the element information in PDF and output structured data automatically and then extract all the key information in it to generate a keyword library with tag weights. The scheme mainly involves two key technical points: parsing PDF based on text features and grammar rules and extracting keywords based on tag weights. The former visualizes the text block in PDF into a rectangular area, divides the elements by clustering algorithm, and, finally, outputs structured data containing all the information. The latter combines the tags and their weights in the structured data and extracts the keywords in it by the inter-word relation algorithm. The structured data and keywords database produced by this scheme can be used to produce intelligent e-book and build knowledge graph, thus helping publishing enterprises to transform from a content service provider to an intelligent knowledge service provider. This transformation can deeply excavate the core value of the content held by the publishing industry and promote the digitization and intelligentization process of the whole industry.

Every day, millions of images are seen through all these mediums-social media, news headlines, advertisements. People have an implicit sense of what those images represent, but for machines, it is possible to generate meaningful insights only through complex algorithms. Captioning images forms one of the most basic applications of AI and pertains to textual descriptions that can help in enabling functionalities like automatic indexing, CBIR, and accessibility. Deep learning models demonstrated potential capabilities to automatically learn features to generate semantically rich captions that are coherent; however, template-based, and retrieval-based approaches find it challenging to implement flexibly to produce ultra-high-detail, context-specific captions. The techniques here, such as CNNs, extract visual features while RNNs and LSTMs generate descriptive text. The higher-level architectures included are the encoder-decoder frameworks and compositional models that provide further enhancement by aligning visual data and textual data. The paper briefly discusses deep learning techniques categorized into structure and application-based categories and tests the performance of benchmark datasets such as Flickr8k, Flickr30k, and MSCOCO. However, much remains to be done in terms of building models robust to complex and diverse visual content; thus, it is observed that there are challenges that carry forward to the work on multimodal integration and attention-based mechanisms to be improved in terms of better quality and accuracy by the captions.

Automatic keyword identification by artificial neural networks compared to manual identification by users of filtering systems

Doc’UMVF: Two search tools to provide quality-controlled teaching resources in French to students and teachers

Legal information certification and secured storage combined with documents electronic signature are of great interest when digital documents security and conservation are in concern. Therefore, these new and evolving technologies offer powerful abilities, such as identification, authentication and certification. The latter contribute to increase the global security of legal digital archives conservation and access. However, currently used cryptographic and hash coding concepts cannot intrinsically enclose cognitive information about both the signer and the signed content. Indeed, an evolution of these technologies may be necessary to achieve full text researches within hundreds or thousands of electronically signed documents. This article aims at describing a possible model along with associated processes to create and make use of these new electronic signatures called "meaningful electronic signatures" as opposed to traditional electronic signatures based on bit per bit computation.

The segmentation and classification of news video into single-story semantic units is a challenging problem. This research proposes a two-level, multi-modal framework to tackle this problem. The video is analyzed at the shot and story unit (or scene) levels using a variety of features and techniques. At the shot level, we employ a Decision Tree to classify the shot into one of 13 predefined categories. At the scene level, we perform the HMM (Hidden Markov Models) analysis to eliminate shot classification errors and to locate story boundaries. We test the performance of our system using two days of news video obtained from the MediaCorp of Singapore. Our initial results indicate that we could achieve a high accuracy of over 95 % for shot classification. The use of HMM analysis helps to improve the accuracy of the shot classification and achieve over 89% accuracy on story segmentation.KeywordsNews Video classificationmulti-modal approachlearning-based approachStory Segmentation

Comparing and combining the effectiveness of latent semantic indexing and the ordinary vector space model for information retrieval