A new foreground extraction scheme for video streams

Recent research in computer vision is increasingly focusing on developing systems to understand people’s appearance, movements and activities, provide advanced interfaces for interacting with people, create human models. For any of these systems to work, they need methods to identify people from a particular input image or video. Today, real-time object detection and sizing of objects is an important issue in many areas of the industry. This is a vital issue of computer vision problems. With Covid-19's healing process, it will be very important to maintain social distance. In this research and development, it is aimed to maintain social distance with proposed big data architecture. This article provides an advanced technique to detect objects in video streams in real time and calculate their distance. The system composed research and developments to perform a stream from the camera, such as video stream, distance and object detection model, incoming data stream, data stream collection and report generation. The video stream from the camera is processed with GStream. The frames from the video stream are taken by OpenCV, YOLOV3 is trained by distance and object detection model and developed by Python. Video streaming data trained with Kubeflow is published with Apache Kafka and Apache Spark. It uses HDFS used to store published data. It is used to query and analyze data in Hive, Impala, Hbase HDFS. After that Analytical reports are created. E-mail notifications can be created according to the data in the database using by Apache Oozie. Through the proposed real time big data architecture, people can be safe in closed areas.

The development of mathematical models for detecting moving objects in real-time video streams is a significant task, especially in light of the growing demands for security and process automation. This paper proposes a detection system based on the Faster Region-Based Convolutional Neural Network (Faster R-CNN) model, utilizing various backbone architectures, including Residual Network-50 (ResNet-50), Feature Pyramid Network (FPN), MobileNet Version 3 Large, and Efficient Network-B0 (EfficientNetB0) with a Self-Attention mechanism. The model extracts object features by combining deep neural networks and scaling mechanisms, enabling adaptation to various operating conditions. Experimental results demonstrate that the proposed approach achieves high accuracy, with the MobileNet Version 3 Large model achieving 95.70% accuracy and the Residual Network-50 model reaching 100% accuracy within the first three training cycles (epochs). The Faster Region-Based Convolutional Neural Network model using Efficient Network-B0 with the Self-Attention mechanism reaches 100% accuracy by the third training cycle, maintaining consistent performance throughout the remaining training cycles. The model with the Residual Network-50 and Feature Pyramid Network backbone architecture shows an average loss reduction from 0.0922 in the first training cycle to 0.0102 by the 15th training cycle, confirming its high stability and performance. Future research could focus on optimizing the proposed approach to handle even more complex and dynamic video sequences and improving data processing to reduce computational costs and increase processing speed.

In variable-bit-rate video coding, the video is preprocessed to collect sequence-level statistics, which are used for global bit allocation in the actual encoding stage to obtain a smoothed video presentation quality. However, in real-time video recording and network streaming, this type of two-pass encoding scheme is not allowed because the access to future frames and global statistics is not available. To address this issue, we introduce the concept of low-pass filtering of rate-distortion functions and develop a smoothed rate control (SRC) framework for real-time video recording and streaming. Theoretically, we prove that, using a geometric averaging filter, the SRC algorithm is able to maintain a smoothed video presentation quality while achieving the target bit rate automatically. We also analyze the buffer requirement of the SRC algorithm in real-time video streaming, and propose a scheme to seamlessly integrate robust buffer control into the SRC framework. The proposed SRC algorithm has very low computational complexity and implementation cost. Our extensive experimental results demonstrate that the SRC algorithm significantly reduces the picture quality variation in the encoded video clips.

21 Following advances in hardware and computing power, interactive conducting systems have grown in complexity and capability. Modern conducting systems incorporate research from a variety of disciplines, including motion tracking, gesture recognition and interpretation, and digital signal processing. Frameworks have emerged in recent years to enable rapid development of such complex systems, including Max/MSP for manipulating and processing synthesized and sampled audio (Puckette 2002) and EyesWeb for gesture tracking (Camurri, Mazzarino, and Volpe 2003). Today’s computers are, moreover, able to handle large chains of complex filters and other operations on digitally sampled audio and video streams in real time. In contrast, modern computer-music systems often do not take full advantage of these capabilities by continuing to use synthesized music, usually MIDI-based; even fewer incorporate video. Using synthesized music over digitally sampled audio streams affords a higher level of semantic access to the data, such as beats, notes, and voicings. However, digital audio and video recordings can offer a higher level of fidelity and realism: today’s synthesizing technology is still unable to reproduce, for example, the unique character of the Vienna Philharmonic playing in their Golden Hall of Vienna’s Musikverein. Part of the problem can be attributed to the difficulty of working with time-based effects in current multimedia frameworks such as Apple’s QuickTime, Microsoft’s DirectShow/DirectSound, or Max/MSP. We have encountered some of these difficulties in our own work, which includes a series of interactive conducting systems that incorporate audio and video recordings. These systems have been well-received as museum exhibits around the world. Personal Orchestra, which was coordinated by Max Muhlhauser

Recent advances in mobile technologies have enabled a plethora of new applications. The hardware capabilities of mobile devices, however, are still insufficient for real-time stream data processing (e.g., real-time video stream). In order to process real-time streaming data, most existing applications offload the data and computation to a remote cloud service, such as Apache Storm or Apache Spark Streaming. Offloading streaming data, however, has high costs for users, e.g., significant service fees and battery consumption. To address these challenges, we design, implement and evaluate Mobile Storm, the first stream processing platform for mobile clouds, leveraging clusters of local mobile devices to process real-time stream data. In Mobile Storm, we model the workflow of a real-time stream processing job and decompose it into several tasks so that the job can be executed concurrently and in a distributed manner on multiple mobile devices. Mobile Storm was implemented on Android phones and evaluated extensively through a real-time HD video processing application. The result shows that Mobile Storm effectively processes HD Video Stream in a mobile cloud, which would be impossible on a single mobile device.

A complexion-based face detection algorithm is proposed. Firstly, an inputted image is transformed to Ycbcr color space. Its illumination is compensated. Gaussian complexion model is used to detect face. Each complexion field is detected to be face field or not by the oval rule. A face detection platform is constructed with the proposed algorithm to detect face from video streams in real time. Experiment results show that face detection rate is above 90% under normal illumination conditions, and is above 85% under dark illumination.

We present Vinci, a vision-language system designed to provide real-time, comprehensive AI assistance on portable devices. At its core, Vinci leverages EgoVideo-VL, a novel model that integrates an egocentric vision foundation model with a large language model (LLM), enabling advanced functionalities such as scene understanding, temporal grounding, video summarization, and future planning. To enhance its utility, Vinci incorporates a memory module for processing long video streams in real time while retaining contextual history, a generation module for producing visual action demonstrations, and a retrieval module that bridges egocentric and third-person perspectives to provide relevant how-to videos for skill acquisition. Unlike existing systems that often depend on specialized hardware, Vinci is hardware-agnostic, supporting deployment across a wide range of devices, including smartphones and wearable cameras. In our experiments, we first demonstrate the superior performance of EgoVideo-VL on multiple public benchmarks, showcasing its vision-language reasoning and contextual understanding capabilities. We then conduct a series of user studies to evaluate the real-world effectiveness of Vinci, highlighting its adaptability and usability in diverse scenarios. We hope Vinci can establish a new framework for portable, real-time egocentric AI systems, empowering users with contextual and actionable insights. Including the frontend, backend, and models, all code of Vinci is publicly available at https://github.com/OpenGVLab/vinci.

The article proposes an improved method for searching and recognising objects in a video stream in real time using the calculation of interframe changes (deltas) and a neural classifier. The main goal of the study is to achieve high performance and reduce the computational load on system resources while maintaining acceptable accuracy. An experimental comparison with the basic SSD (Single Shot MultiBox Detector) method was carried out, which measured the following indicators: average frame processing time, RAM and video memory usage, CPU and graphics load, and recognition accuracy. Unlike SSDs, the proposed approach provides a higher processing speed (up to 35% increase) with a slight decrease in accuracy (less than 4%), which is compensated for by further adaptation of the model. At the same time, the use of the CPU and RAM increases by only 0.5-5%, while the amount of video memory consumed decreases by 57%. The study confirms the feasibility of using the improved delta classification method in video analytics systems with limited resources. This method can be integrated into applied security, video surveillance, and real-time intelligent monitoring systems.

Object detection is one of the most active research and application areas of neural networks. In this article we combine FPGA and neural networks technologies to solve the real-time object recognition problem. The article discusses the integration of the YOLOv3 neural network on the DE10-Nano FPGA. Slightly worse indicators of the main metrics (mAP, FPS, inference time) when operating a neural network on a De10-Nano board in comparison with more expensive solutions based on GPUs, are offset by differences in the cost and dimensions of the FPGA board used. Based on the results of the study of various methods for converting neural networks to FPGA, it was concluded that this architecture is applicable for solving problems of detecting objects on a video stream in real time.

The problem of short-term channel failure in transmission of real-time video streams in Wi-Fi networks is considered. An overview of existing solutions to this problem has shown that there is currently no effective solution that could be successfully applied in real equipment. Also discussed is the transfer of real-time video streams over the Wi-Fi network using the random access method to several recipients in the context of short-term channel failures for some of them. Using the standard algorithm of FIFO queuing in the conditions of short-term channel failure, leads to the problem of blocking the queue with the main packet, resulting in the quality of transmission of video streams decreases for recipients with poor quality of connections, and for recipients whose connection quality remained unchanged. To solve this problem a new algorithm for queuing is developed. The application of this discipline avoids a decrease in the quality of the transmission of video streams to recipients for whom the quality of the connections has not deteriorated, as well as to reduce the decrease in the quality of the transfer of video streams to recipients who have been refused channel.

Remote laboratories are an inevitable necessity for Internet enabled education in Science, Technology, Engineering, and Math (STEM) fields due to their effectiveness, flexibility, cost savings, and the fact that they provide many benefits to students, instructors and researchers. Accordingly, real-time experiment live video streaming is an essential part of remote experimentation operation. Nevertheless, in the development of real-time experiment video transmission, it is a key and difficult issue that the video is transferred across the network firewall in most of the current remote laboratory solutions. To address the network firewall issue, we developed a complete novel solution via HTTP Live Streaming (HLS) protocol and FFMPEG that is a powerful cross-platform command line video trans-code/encoding software package on the server side. In this paper, a novel, real-time video streaming transmission approach based on HLS for the remote laboratory development is presented. With this new solution, the terminal users can view the real-time experiment live video streaming on any portable device without any firewall issues or the need for a third party plug-in., This new solution also significantly benefits remote laboratory development in the future.

5G mobile communication infrastructure attracts increasing attention of stakeholders, going beyond only interconnection of people and increasingly serve to connect and manage 5G-enabled IoT devices. Among others, unmanned aerial vehicles (UAVs) or systems (UAS) already rely on 5G communication infrastructure and in the near future are envisioned to use it even more. 5G systems already support a wide-variety of different applications, such as remote healthcare, self-driving ground vehicles, virtual or augmented reality, drones, surveillance and many more. Among these is the high-resolution video surveillance using drones for different purposes. In road traffic analysis, one of the most important public safety applications, the low compressed or uncompressed video stream can greatly improve the analysis and traffic incident detection performance. On the other hand it is challenging to transmit high-resolution video stream in real time due to its data size. This paper provides an overview of the current research in the area of UAV command and control using 5G systems describing basic concepts and challenges. We review some of the latest research in regard to real-time high-resolution video transfer. A brief discussion of experiments on the 5G private campus network communication between a drone and a ground analytical system is presented.

Real-time and accurate measurement of blast furnace (BF) stockline is important for charging control and BF safety. However, traditional non-contact methods for stockline detection face many challenges such as low accuracy, poor anti-interference ability, and stability in harsh BF environments. The high-temperature novel industrial endoscope (NIE), a visible light detection equipment for BF burden surface (BBS), has captured real-time and continuous video streams of BBS, which provides a brand-new means and ideas for non-contact stockline detection. Based on this, a novel non-contact stockline detection method based on real-time burden surface video streams is proposed. First, an edge detection network based on bootstrap sampling and eigenvector coding clustering (ECC) is established to extract accurate edge distribution of burden surface video streams. Then, an actual imaging region model of the NIE is developed by the latitude circle method to quantify the relationship between the actual imaging region and the BF stockline. Finally, the edge of burden surface video streams is correlated with the corresponding actual imaging region through the NIE imaging model, and the non-contact stockline detection model is derived to realize the BF stockline detection based on video streams. Experiments and industrial applications show that the proposed method is superior to the traditional non-contact stockline detection methods in terms of detection accuracy, stability, and real-time performance.

Optimization methods are considered, such as the Kalman method and the Monte Carlo method for solving the problem of tracking objects on video. The analysis of the application of algorithms for threshold processing, precinct cultivation and graph optimization when dividing an image into separate parts in order to extract more detailed information is carried out. The main areas of application of algorithms for analyzing and processing video streams in real time in robotics were highlighted. The metrics used for processing the video stream are presented and a comparative analysis of the considered algorithms is performed. Recommendations are given on the choice of methods and algorithms for processing a video stream for a specific task.

Our tool allows artists to create living paintings or stylize a live video stream using their own artwork with minimal effort. While an artist is painting the image, our framework learns their artistic style on the fly and transfers it to the provided live video stream in real time.