A novel motion-JPEG2000 video transmission system over CDMA environment

In the paper the features of protocols and the basic methods of organization of multimedia information transfer in telecommunication networks and distributed corporate safety networks are represented. The analysis of formats of basic elements of signals used in standards of multimedia information transfer is given.

In this paper, a single speaker tracking mechanism with particle filter is presented for a reliable interaction with robot. To have a smooth tracking result, we use not only audio data but also video data. For sound observation, we use Time Delay Of Arrival (TDOA) between two microphones, and in order to expect high reliability in the likelihood of localization, we apply variable sensitivity values according to the angle of arrival for a triangle arrangement of microphones to calculate the likelihood for audio observation. For video observation, we use face detection routine of OpenCV which is widely available through the Internet. Together with audio and video data, we found out that more reliable source tracking is possible if we apply data fusion mechanism at every possible environment on a robot platform than single information.

Purpose: High-speed videoendoscopy (HSV) is an emerging, but barely used, endoscopy technique in the clinic to assess and diagnose voice disorders because of the lack of dedicated software to analyze the data. HSV allows to quantify the vocal fold oscillations by segmenting the glottal area. This challenging task has been tackled by various studies; however, the proposed approaches are mostly limited and not suitable for daily clinical routine.Method: We developed a user-friendly software in C# that allows the editing, motion correction, segmentation, and quantitative analysis of HSV data. We further provide pretrained deep neural networks for fully automatic glottis segmentation.Results: We freely provide our software Glottis Analysis Tools (GAT). Using GAT, we provide a general threshold-based region growing platform that enables the user to analyze data from various sources, such as in vivo recordings, ex vivo recordings, and high-speed footage of artificial vocal folds. Additionally, especially for in vivo recordings, we provide three robust neural networks at various speed and quality settings to allow a fully automatic glottis segmentation needed for application by untrained personnel. GAT further evaluates video and audio data in parallel and is able to extract various features from the video data, among others the glottal area waveform, that is, the changing glottal area over time. In total, GAT provides 79 unique quantitative analysis parameters for video- and audio-based signals. Many of these parameters have already been shown to reflect voice disorders, highlighting the clinical importance and usefulness of the GAT software.Conclusion: GAT is a unique tool to process HSV and audio data to determine quantitative, clinically relevant parameters for research, diagnosis, and treatment of laryngeal disorders.Supplemental Material S1. Supplemental figures: Figure S1. Neural network training metrics.Figure S2. Neural network evaluation.Figure S3. Example segmentations using different backbones from various source data.Figure S4. Audio analysis platform.Supplemental Material S2. Endoscopic footage with semi-automatic and fully automatic segmentation methods, multiple examples.Supplemental Material S3. GAT manual. Kist, A. M., Gomez, P., Dubrovskiy, D., Schlegel, P., Kunduk, M., Echternach, M., Patel, R., Semmler, M., Bohr, C., Durr, S., Schutzenberger, A., & Dollinger, M. (2021). A deep learning enhanced novel software tool for laryngeal dynamics analysis. Journal of Speech, Language, and Hearing Research. Advance online publication. https://doi.org/10.1044/2021_JSLHR-20-00498

Content Based Video Retrieval (CBVR) is so popular these days, because of the increased utilization of video based analytical systems. Video based analytics is quite effective than image analysis, as a series of actions are captured by the video. This ends up with better decision making ability. The CBVR systems play an important role in boosting the human–computer interaction. This paper presents a multimodal CBVR that takes both the visual and audio information into account for retrieving relevant videos to the user. Two modules are employed by this work to deal with video and audio data. The video data is processed to detect the significant frame from shots and is achieved by Lion Optimization Algorithm (LOA). The features are extracted from the visual data and with respect to the audio data, MHEC and LPCC features are extracted. The extracted features are clustered by Kernelized Fuzzy C Mean (KFCM) algorithm. Finally, the feature database is formed and is utilized in the query matching process during the testing phase. The performance of the proposed work is tested in terms of precision, recall, F-measure and time consumption rates. The proposed CBVR system proves better performance than the existing approaches and is evident through attained results.

Fibre channel is an emerging high-speed digital network technology that combines to make inroads into the avionics arena. The suitability of fibre channel for such applications is largely due to its flexibility in these key areas: Network topologies can be configured in point-to-point, arbitrated loop or switched fabric connections. The physical layer supports either copper or fiber optic implementations with a Bit Error Rate of less than 10<SUP>-12</SUP>. Multiple Classes of Service are available. Multiple Upper Level Protocols are supported. Multiple high speed data rates offer open ended growth paths providing speed negotiation within a single network. Current speeds supported by commercially available hardware are 1 and 2 Gbps providing effective data rates of 100 and 200 MBps respectively. Such networks lend themselves well to the transport of digital video and audio data. This paper summarizes an ANSI standard currently in the final approval cycle of the InterNational Committee for Information Technology Standardization (INCITS). This standard defines a flexible mechanism whereby digital video, audio and ancillary data are systematically packaged for transport over a fibre channel network. The basic mechanism, called a container, houses audio and video content functionally grouped as elements of the container called objects. Featured in this paper is a specific container mapping called Simple Parametric Digital Video (SPDV) developed particularly to address digital video in avionics systems. SPDV provides pixel-based video with associated ancillary data typically sourced by various sensors to be processed and/or distributed in the cockpit for presentation via high-resolution displays. Also highlighted in this paper is a streamlined Upper Level Protocol (ULP) called Frame Header Control Procedure (FHCP) targeted for avionics systems where the functionality of a more complex ULP is not required.

Recent advances in artificial intelligence combined with behavioral sciences have led to the development of cutting-edge tools for recognizing human emotions based on text, video, audio, and physiological data. However, these data sources are expensive, intrusive, and regulated, unlike plants, which have been shown to be sensitive to human steps and sounds. A methodology to use plants as human emotion detectors is proposed. Electrical signals from plants were tracked and labeled based on video data. The labeled data were then used for classification., and the MLP, biLSTM, MFCC-CNN, MFCC-ResNet, Random Forest, 1-Dimensional CNN, and biLSTM (without windowing) models were set using a grid search algorithm with cross-validation. Finally, the best-parameterized models were trained and used on the test set for classification. The performance of this methodology was measured via a case study with 54 participants who were watching an emotionally charged video; as ground truth, their facial emotions were simultaneously measured using facial emotion analysis. The Random Forest model shows the best performance, particularly in recognizing high-arousal emotions, achieving an overall weighted accuracy of 55.2% and demonstrating high weighted recall in emotions such as fear (61.0%) and happiness (60.4%). The MFCC-ResNet model offers decently balanced results, with AccuracyMFCC-ResNet=0.318 and RecallMFCC-ResNet=0.324. Regarding the MFCC-ResNet model, fear and anger were recognized with 75% and 50% recall, respectively. Thus, using plants as an emotion recognition tool seems worth investigating, addressing both cost and privacy concerns.

Real-time operating system services are required to support multimedia systems that rely heavily on the workstation processor for control of the audio and video processors and movement of audio and video data. The requirements for each service are described, together with the YARTOS kernel, an operating system kernel that provides real-time communication and computation services. The programming model supported by YARTOS is an extension of Wirth's discipline of real-time programming. In essence it is a message-passing system with a semantics of interprocess communication that specifies the real-time response that an operating system must provide to a message receiver. This allows a programmer to assert an upper bound on the time to receipt and processing of each message. The YARTOS kernel supports the notion of guaranteed processing rates. The desired processing rate of each task is made known to the kernel, and the kernel provides a guaranteed response time to each task that is sufficient for ensuring that the required processing rate is achieved. >

The primary goal of this study is to create a robot that can monitor human activity by using wireless night vision cameras and Using an Android Bluetooth application, robots can be manually controlled. A night vision wireless camera built into the robot can send videos of people. This robot’s job is to go around and collect audio and video data from the environment it is in, then communicate that data to the user. In this project, a mobile device is used to control the robot. This robot can send data to a distant IoT cloud database after collecting it from a faraway location. By transmitting commands via the Bluetooth software on our Android phone, we can direct the robot’s movement. In the proposed system, we are going to construct a low-cost Microcontroller Based Android operated Robot that can move in any of the directions of Forward, Left, Right, Backward, and Stop. The robot will travel in the directions indicated by the mobile device, including forward, backward, left, and right. There are numerous uses for this system. The goal of our project is to create a robotic vehicle with a wireless camera capable of night vision for remote monitoring/spying applications. Even in dimly lit areas, the night vision camera enables the transmission of real-time night vision footage. The camera’s footage can be watched on a PC for reference.

In the pandemic situation of COVID-19 education institutions have change their mode from offline to online for providing education facility. Allowing to learn and improve themselves at their own pace With this advantage an disadvantage can also be seen concern with examination taken to check excellence of student is that malpractice or cheating in examination. To eliminate this effect a solution is proposed called Online Proctoring Using AI Technology. Online Proctoring System (henceforth called as OPS), in general make use of online tools to maintain sanctity of the examination. There are various psychological, cultural and technological parameters need to be considered while developing AI-based Proctored System. In this paper, we present a Online Proctoring System that performs automatic online exam proctoring. The system hardware includes one webcam and a microphone, for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes five basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, gaze estimation and phone detection. By using the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. The system includes five basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, gaze estimation and phone detection. To check how efficient our proposed system is we collected multimedia (audio and video data) representing various malpractice action’s while taking online exams. The results determines how accurate, robust, and efficient our online exam proctoring system is.

Emotions are a crucial aspect of human life and the researchers have tried to build an automatic emotion recognition system that helps to provide important real-world applications. The psychologists have shown that emotions differ across culture, considering this fact, we provide and describe the first audio-visual Arabic emotional dataset which called (AVANEmo). In this work we aim to fill the gap between studies of emotion recognition for Arabic content and other languages by provided an Arabic dataset which is a major and fundamental part of build emotion recognition application. Our dataset contains 3000 clips for video and audio data, and it covers six basic emotional labels (Happy, Sad, Angry, Surprise, Disgust, Neutral). Also, we provide some baseline experiments to measure the primitive performance for automated audio and visual emotion recognition application using the AVANEmo dataset. The best accuracy that we achieved was 54.5% and 57.9% using the audio and visual data respectively. The data will be available for distribution to researchers.

Depression is a severe mental illness that impairs a person&#x0027;s capacity to function normally in personal and professional life. The assessment of depression usually requires a comprehensive examination by an expert professional. Recently, machine learning-based automatic depression assessment has received considerable attention for a reliable and efficient depression diagnosis. Various techniques for automated depression detection were developed; however, certain concerns still need to be investigated. In this work, we propose a novel deep multi-modal framework that effectively utilizes facial and verbal cues for an automated depression assessment. Specifically, we first partition the audio and video data into fixed-length segments. Then, these segments are fed into the Spatio-Temporal Networks as input, which captures both spatial and temporal features as well as assigns higher weights to the features that contribute most. In addition, Volume Local Directional Structural Pattern (VLDSP) based dynamic feature descriptor is introduced to extract the facial dynamics by encoding the structural aspects. Afterwards, we employ the Temporal Attentive Pooling (TAP) approach to summarize the segment-level features for audio and video data. Finally, the multi-modal factorized bilinear pooling (MFB) strategy is applied to fuse the multi-modal features effectively. An extensive experimental study reveals that the proposed method outperforms state-of-the-art approaches.

With the widespread application of deep learning methods, multimodal techniques have also achieved rapid development. Since single-modal speech recognition may affect the accuracy of recognition results in noisy environments, multimodal fusion recognition gradually replaces the traditional single-modal recognition methods. In this paper, we mainly strengthen and pre-process audio and video data first, and use LSTM recurrent neural network for deep feature extraction of audio and video streams, which effectively solves the problem of long-term forgetting of general neural networks. The audio and video feature vectors are then fused by a fully connected neural network with linear connections. Compared with the speech recognition technique alone, this audiovisual fusion recognition method has a better recognition effect in the case of noise interference. Compared with the traditional audiovisual recognition method, the model simplifies the recognition work. Recognition experiments on the LRS2-BBC dataset show that the recognition accuracy of this method improves to a certain extent over that of other methods in a clean environment and greatly improves in noisy conditions.

Deep learning (DL) is a powerful classification technique that has excellent performance in processing audio, image, and video data. In this paper, we use DL in communication systems, especially for modulation classification. Convolutional neural network (CNN) is then utilized to complete the classification task. The deep learning methods are implemented in an OFDM system using Software Defined Radio (SDR). Experimental results indicate that the proposed CNN based modulation classification achieves comparable classification accuracy without the necessity of manual feature selection.

BackgroundRespectful, equitable, and supportive health care team interactions with patients and their companions are a key part of quality health care services. Despite the importance of communication and other aspects of engagement during inpatient postpartum care, little is known about how care is experienced during this period from families’ perspectives.MethodsThis study collected video and audio data with 15 birthing families (n = 9 English-speaking and n = 6 Spanish-speaking) and their health care team members during inpatient postpartum care in a southeastern United States academic medical center. This analysis quantifies health care team member presence in family hospital rooms, assesses linguistic appropriateness of health care team member verbal communication, and describes birthing parent and companion verbal reactions to health care team member interactions. A behavioral taxonomy was applied to identify these codes in the 12 h prior to hospital discharge. Additionally, we transcribed the birthing parent and companion verbal reactions for 10-minutes each time the health care team member departed their room. This content was inductively coded to identify topics and develop themes.ResultsA total of 160 h of video and audio data were coded across 15 participating families for this analysis. There were 19.9 h of missing data in the 12 h prior to hospital discharge due to equipment being turned off across five participants. At least one health care team member was present in the postnatal unit rooms within the observation period in 200 instances (median 13 times, range 5 to 19 times per participating family). Communication with Spanish-speaking birthing parents was linguistically appropriate for between 20.0% and 75.0% of interactions. Following health care team member presence, birthing parent and companion reactions were indeterminate 1 time, no verbal reaction 107 times, positive 25 times, confused 38 times, and negative 52 times. Many parents expressed more than one reaction. Reaction topics included clarity of information provided, postpartum pain management, hospital discharge coordination, health care team member conduct, and access to supplies.ConclusionThere are strengths in postpartum patient engagement as well as areas for health care team improvement. More appropriate communication including setting of clearer expectations surrounding various aspects of services on the postnatal unit may improve patient experiences. As a part of this improved quality of inpatient postpartum care, consistent utilization of interpretation services could decrease patient confusion, enable shared decision-making, and promote positive patient-provider relationships.

High-speed videoendoscopy is an important tool to study laryngeal dynamics, to quantify vocal fold oscillations, to diagnose voice impairments at laryngeal level and to monitor treatment progress. However, there is a significant lack of an open source, expandable research tool that features latest hardware and data analysis. In this work, we propose an open research platform termed OpenHSV that is based on state-of-the-art, commercially available equipment and features a fully automatic data analysis pipeline. A publicly available, user-friendly graphical user interface implemented in Python is used to interface the hardware. Video and audio data are recorded in synchrony and are subsequently fully automatically analyzed. Video segmentation of the glottal area is performed using efficient deep neural networks to derive glottal area waveform and glottal midline. Established quantitative, clinically relevant video and audio parameters were implemented and computed. In a preliminary clinical study, we recorded video and audio data from 28 healthy subjects. Analyzing these data in terms of image quality and derived quantitative parameters, we show the applicability, performance and usefulness of OpenHSV. Therefore, OpenHSV provides a valid, standardized access to high-speed videoendoscopy data acquisition and analysis for voice scientists, highlighting its use as a valuable research tool in understanding voice physiology. We envision that OpenHSV serves as basis for the next generation of clinical HSV systems.