High Availability Server Using Raspberry Pi 4 Cluster and Docker Swarm

The programming foundation course which based on Python has become an important basic course for different disciplines in universities. In experimental teaching, the main difficulty is the environment configuration and rapid deployment of Python. Virtualization technology is a common method to solve this problem. However, there are some problems, such as low resource utilization, insufficient support for concurrent, slow start, and so on. So, a new method for constructing the teaching experiment platform of the course using Docker container technology is proposed. This method can quickly realize the dynamic allocation, deployment and management of the running environment image according to the needs of students. Experimental results show that this method has obvious advantages in terms of resource utilization, concurrent users, response time, construction cost and so on, compared with virtual machine technology

SummaryIn recent years, Docker container technology is being applied in the field of cloud computing at an explosive speed. The scheduling of Docker container resources has gradually become a research hotspot. Existing big data computing and storage platforms apply with traditional virtual machine technology, which often results in low resource utilization, a long time for flexible scaling and expanding clusters. In this paper, we propose an improved container scheduling algorithm for big data applications named Kubernetes‐based particle swarm optimization(K‐PSO). Experimental results show that the proposed K‐PSO algorithm converges faster than the basic PSO algorithm, and the running time of the algorithm is cut in about half. The K‐PSO container scheduling algorithm and algorithm experiment for big data applications are implemented in the Kubernetes container cloud system. Our experimental results show that the node resource utilization rate of the improved scheduling strategy based on K‐PSO algorithm is about 20% higher than that of the Kube‐scheduler default strategy, balanced QoS priority strategy, ESS strategy, and PSO strategy, while the average I/O performance and average computing performance of Hadoop cluster are not degraded.

Cloud Computing is online technology where computing resources like hardware, software and applications are available as per the user's needs. A cloud computing architecture microservices-based application involves multiple microservices deployed, updated, and redeployed on lightweight virtualization technology called docker container rather than hypervisor-based virtualization. Docker Swarm, Kubernetes and Apache Mesos are container orchestration tool for scheduling and managing individual Container for microservice application within a cluster of private cloud and public cloud. Docker container orchestration can include creating and scheduling Container, availability of container and the host machine, rescheduling of failed Container, scaling of Container to balance the workload on infrastructure and securing the interaction between Container. This survey provides a complete description of docker container orchestration approaches with containers, analyzing the framework and classification of container orchestration management.KeywordsDockerContainerOrchestrationMicro-services

The concept of containerization and virtualization is getting traction in the cloud based IT environments. Docker engine is popular implementation for simplifying and streamlining containerization technology. IT industry realizes numerous automation and acceleration features and facilities through the embracement of the Docker-sponsored containerization paradigm, which is an operating system (OS)-level and lightweight virtualization. However the security issues affect the widespread and confident usage of Docker platform. In this paper, we have discussed important security issues of the Docker containers as well as the related work that is being carried out in this area. Also we have proposed security algorithms and methods to address DoS attacks related issues in the Docker container technology. The preliminary experiments and testing of the security methods are promising.

The Government of India proposed "Clean India Mission" in 2014 that has led to the establishment of many public restrooms across the country. The purpose of this mission is to provide hygienic toilets to prevent open defecation and eradicate water-borne diseases. These restrooms are assigned human resources. However, the majority of them are in bad condition for usage. For increasing the availability of toilets, there is a requirement of a smart sanitation system for its automated monitoring. Moreover, rural and semi-urban restroom locations suffer from limited network connectivity. Thus, restroom state analysis needs to be performed near the site for seamless monitoring. This paper proposes an edge analytics framework based on the Internet of Things (IoT) and fog computing to analyze restroom data at the network edge and provide intelligent decisions. The machine learning model runs over Docker containers for analyzing data for the resource-constraint fog nodes. The choice of a machine learning model for the analysis depends on the model's performance in classification and on Docker containers over Raspberry Pi. Results show Adaptive Neuro-Fuzzy Inference System (ANFIS) is suitable over Docker containers for handling restroom data at the edge of the network.

Containerization technology utilizes operating system level virtualization to package applications to run with required libraries and are isolated from other processes on the same host. Lightweight and quick deployment make containers popular in many data centers. Running distributed applications in data centers usually involves multiple clusters of machines. Docker Swarm is a container orchestration tool for managing a cluster of Docker containers and their hosts. However, Docker Swarm’s scheduler does not consider resource utilization when placing containers in a cluster. This paper first investigated performance interference in container clusters. Our experimental study showed that distributed applications’ performance can be degraded when co-located with other containers which aggressively consume resources. A new scheduler is proposed to improve performance while keeping high resource utilization. The experimental results demonstrated that the proposed prototype with machine learning based clustering algorithms could effectively improve distributed applications’ performance by up to 14.5% with an average at around 12%. This work also provides theoretical bounds for the container placement problem.

Teaching practicum is a learning phase that aims to complete the process of learning in higher education. In some educational institutions, teaching laboratories are not well facilitated, lack of tools, lack of maintenance staff, space and time which induce some problems in the implementation of the learning process. In this paper, a remote laboratory module for a grating experiment was introduced to solve some problem in maintaining the conventional laboratories. The remote laboratory module consists of a set of Practicum workbench, a grating (80 lines per mm), a laser source (645–655 nm, 3,0 mW), a camera and a Raspberry Pi. The Raspberry Pi was configured as a web server, workbench controller, and image pattern acquisition system. A user could choose the distance between screen and grating through practicum website page that developed using a bootstrap module. The grating pattern (dark and light) was captured using a camera and displayed as intensity versus distance (mm) graphic in practicum website page instantly. The proposed practicum module was evaluated to performed diffraction grating experiment to compute the laser wavelength by varying the screen and grating distance. The measurement results from the remote laboratory module are then validated with the data from conventional practicum which provides 0.5% error. Finally, the proposed remote laboratory module could be expanded for another practicum module in various field.

Recently, Internet of Things (IoT) has drawn a great deal of attention and is envisioned in various sectors in the near future due to its promising benefits. However, the constant and rapid growth in IoT devices also brings new challenges due to constrained power and resources associated to them. One of the challenges is to provide seamless connectivity in mobile IoT. Secondly, IoT devices may stream enormous amount of data; hence, providing a solution that can effectively reduce service cost of data transfer. Finally, there are challenges in management and deployment of services running at mobile IoT Edge Gateway. In this context, containerized virtualization solution could play a key role in support of efficient management and deployment of microservices to provide seamless connectivity. This paper proposes a lightweight container-based virtualization technology for IoT, which employs Docker container-based microservices architecture for effectively deploying applications in a virtualized ecosystem. We evaluated the performance of the proposed solution on real IoT testbed using Raspberry Pi 3 as a mobile IoT Edge Gateway for network handover decision making among various alternatives, such as Wi-Fi, Radio, and Satellite. The results demonstrated better performance compared with the native environment, i.e., the one without introduction of a virtualization layer. The results also showed that the Docker container produces negligible resource overhead and can be used on resource constrained mobile IoT Edge Gateway devices like Raspberry Pi 3 for efficiently managing IoT application and services.

The paper proposes improvised Vehicular Networks (VANETs) in an environment that is constantly subject to changes that give various problems in VANETs, like topology replacement, and also to equalizing small routing paths. The remedy for such a problem is clubbing. The proposed work is about to study five different routing procedures using the “Dynamic Ant flying Cluster (DAFC)” method and Dynamic Self-Configured Weights to meet minimum clubbing, more accuracy, less time, low cost, and Quality of Service (QoS) by choosing efficient cluster-head derived from dynamic clustering. In this study, by correcting the steps for updating the setting evaporation rate we applied a very large improvement on basic DAFC. In this aspect for better performance evaluation of the proposed routing procedure, two different phases of experiments are conducted. In the initial phase, the classic “Ant Colony Optimization (ACO), DAFC” which is a Clustering method for Vehicular Ad Hoc Networks (VANET) was examined and compared. Subsequently, in the second stage, clustering systems such as “Centre Position and Mobility (CPM)”, “Angle-based Clustering Algorithm (ACA), and Highest-Degree algorithm (HD)” are evaluated using MATLAB and SUMO simulation tools. Through this process, we observed the anticipated behaviour and demonstrated that our proposed method achieves enhanced node connectivity and cluster stability compared to existing approaches. Before delving into the original condition, it is important to note that assessing the efficiency of routing algorithms plays a challenging role in evaluating the performance of routing protocols in VANETs. This study contributes to understanding the statistical “Design of Experiments (DOE)” techniques as an advanced alternative to the “One Factor at a Time (OFAT)” approach for assessing and demonstrating the VANET routing protocol concert. The study employed the 2-level full factorial technique, the Placket–Burman method, and the Taguchi method, comparing and applying them with three design experiment methods. Four key factors were considered in this work: black hole attacks, the number of connections, node density, and wormhole spasms. Their impacts on four measured outputs, namely “throughput, packet loss ratio, average end-to-end delay (EED)”, and routing overhead of the AODV routing protocol, were simultaneously examined and analyzed.

In big data computing application, there are jobs of multiple computing modes mainly based on MapReduce. Therefore, compute-intensive cluster needs to maintain multiple computing modes. The utilization of virtual computing resources is not high because of the change of computing load. In order to optimize the resource utilization of virtual cluster, a dynamic deployment model is designed with the support of lightweight Docker container technology. The deployment model can adjust the form of virtual cluster according to the change of the resource request of job, mainly changing the type and size of computing nodes in real time. Simulation experiments show that the dynamic deployment model can optimize the utilization of virtual resources, with CPU utilization increased by 5.2%, and the execution efficiency of computing jobs is optimized. The dynamic deployment model can be applied to cloud environment and large-scale computing clusters, not only to achieve peak-taggering computing of user jobs, but also to achieve the purpose of dynamic customization of job execution environment.

As a low-cost and high-performance distributed computing platform, we have studied the User-PC Computing (UPC) system based on the master-worker model. Docker container technology is adopted to run various application programs or jobs on heterogeneous PC environments for workers. Some jobs, such as physics simulations and neural networks, require long CPU time, which increases the probability of failure of running workers. The automatic backup of the job running state and migration to other worker will be essential to reduce the job completion delay. In this paper, we implement the job running backup function in the UPC system. Checkpoint-Restore in Userspace (CRIU) is periodically applied to capture the job running state of the running job at a worker. When the master detects the failure, it automatically migrates the job to another worker. To evaluate the function, we conducted experiments using the testbed UPC system with 14 jobs and six workers of different specifications, and confirmed that the proposal successfully resumes the job running from the interrupted point at another worker.

BackgroundMetagenomics caused a quantum leap in microbial ecology. However, the inherent size and complexity of metagenomic data limit its interpretation. The quantification of metagenomic traits in metagenomic analysis workflows has the potential to improve the exploitation of metagenomic data. Metagenomic traits are organisms’ characteristics linked to their performance. They are measured at the genomic level taking a random sample of individuals in a community. As such, these traits provide valuable information to uncover microorganisms’ ecological patterns. The Average Genome Size (AGS) and the 16S rRNA gene Average Copy Number (ACN) are two highly informative metagenomic traits that reflect microorganisms’ ecological strategies as well as the environmental conditions they inhabit.ResultsHere, we present the ags.sh and acn.sh tools, which analytically derive the AGS and ACN metagenomic traits. These tools represent an advance on previous approaches to compute the AGS and ACN traits. Benchmarking shows that ags.sh is up to 11 times faster than state-of-the-art tools dedicated to the estimation AGS. Both ags.sh and acn.sh show comparable or higher accuracy than existing tools used to estimate these traits. To exemplify the applicability of both tools, we analyzed the 139 prokaryotic metagenomes of TARA Oceans and revealed the ecological strategies associated with different water layers.ConclusionWe took advantage of recent advances in gene annotation to develop the ags.sh and acn.sh tools to combine easy tool usage with fast and accurate performance. Our tools compute the AGS and ACN metagenomic traits on unassembled metagenomes and allow researchers to improve their metagenomic data analysis to gain deeper insights into microorganisms’ ecology. The ags.sh and acn.sh tools are publicly available using Docker container technology at https://github.com/pereiramemo/AGS-and-ACN-tools.

The rapid expansion of the group of students with financial difficulties from their families in higher education schools and the increasing financial support from the state and schools year by year, how to ensure that these two aspects can be matched reasonably and effectively, is a high standard and high accuracy requirement for the identification of students with financial difficulties from their families in higher education schools every academic year. The aim of this paper is to build a cloud-based platform to study an accurate financial aid system for university students. A comparison between Docker and OpenStack is made, and the actual physical equipment situation of this paper is combined with the choice of building a suitable private cloud i.e. building a private cloud with Docker container technology. Then, the modules of the university student financial aid system were developed according to the design plan. Finally, after the development of the university student financial aid system was completed, the performance of the platform was tested in detail and comprehensively. The test results show that the performance of the system has met the design of the project and can meet the basic needs of the users.

Analysis of SR-IOV in Docker containers using RTT measurements

Apache Spark is a data processing engine for large data sets. Apache Spark is much faster (up to 100 times faster in memory) than Apache Hadoop MapReduce. In cluster mode, Spark applications run as independent processes coordinated by the SparkContext object in the driver program, which is the main program. The SparkContext may connect to several types of cluster managers to allocate resources to Spark applications. The supported cluster managers include the Standalone cluster manager, Mesos and YARN. Apache Spark is designed to access data from varied data sources including the HDFS, Apache HBase and NoSQL databases such as Apache Cassandra and MongoDB. In this chapter we shall use the same CDH Docker image that we used for several of the Apache Hadoop frameworks including Apache Hive and Apache HBase. We shall run an Apache Spark Master in cluster mode using the YARN cluster manager in a Docker container.