Understanding Data Characteristics and Access Patterns in a Cloud Storage System

In the HDFS cloud storage environment, the existing replica strategy is not able to adopt the optimal replica according to the current system environment dynamically, which affects the performance of the Cloud Storage system. This paper aims at the replica selection strategies in the load balancing problem, proposes a Replica Strategy based on grey prediction theory in Cloud Storage and predicts the node load rate of different replicas at a moment, selecting the minimum load node dynamically, eventually to load balance.

We study the replication transition problem (RTP) in the Grid systems. Most distributed systems replicate data to increase data access efficiency. A replication strategy dictates where the replicas are stored in respond to data access pattern, and a good strategy can improve data access efficiency. However, the access pattern in a distributed system is constantly changing. As a result a good replication strategy must evolve accordingly. The replication transition problem is to seek an efficient transition from one replication strategy to another in order to cope with the dynamic data access pattern. This paper focuses on the RTP problem for Grid systems in four communication models that have different communication capabilities, i.e., whether message forwarding is allowed and whether network capacity is uniform among different links. We show that there exists a polynomial time algorithm that provides optimal solution for the RTP problem when forwarding is not allowed and the communication links are uniform. We also propose heuristic algorithms for solving variants of the RTP problem and conduct experiments to evaluate their performances. The experimental results indicate that our proposed heuristics are very effective.

Cloud storage system can reshape the way how kinds of contents are generated, stored and distributed on today's Internet. It is intriguing to study the service provision of cloud storage system for the sake of providing global users with satisfactory service. This paper conducts modeling analysis of a cloud storage system, and proposes a Markov decision process modeling framework to analyze the reliability of the storage system. Based on the modeling analysis, the optimal strategy can be got after executing two solution algorithms. By incorporating these techniques into cloud storage system design, the authors can significantly improve the replication performance and reliability of the system in this paper. Our numerical results further demonstrate the effectiveness and superiority of our technique against traditional static approaches.

In today’s World there exists various source of data in various formats (file formats), different structure, different types and etc. which is a hug collection of unstructured over the internet or social media. This gives rise to categorization of data as unstructured, semi structured and structured data. Data that exist in irregular manner without any particular schema are referred as unstructured data which is very difficult to process as it consists of irregularities and ambiguities. So, we are focused on Intelligent Processing Unit which converts unstructured big data into intelligent meaningful information. Intelligent text extraction is a technique that automatically identifies and extracts text from file format. The system consists of different stages which include the pre-processing, keyphase extraction techniques and transformation for the text extraction and retrieve structured data from unstructured data. The system consists multiple method/approach give better result. We are currently working in various file formats and converting the file format into DOCX which will come in the form of the un-structure Form, and then we will obtain that file in the structure form with the help of intelligent Pre-processing. The pre-process stages that triggers the unstructured data/corpus into structured data converting into meaning full. The Initial stage is the system remove the stop word, unwanted symbols noisy data and line spacing. The second stage is Data Extraction from various sources of file or types of files into proper format plain text. The then in third stage we transform the data or information from one format to another for the user to understand the data. The final step is rebuilding the file in its original format maintaining tag of the files. The large size files are divided into sub small size file to executed the parallel processing algorithms for fast processing of larger files and data. Parallel processing is a very important concept for text extraction and with its help; the big file breaks in a small file and improves the result. Extraction of data is done in Bilingual language, and represent the most relevant information contained in the document. Key-phase extraction is an important problem of data mining, Knowledge retrieval and natural speech processing. Keyword Extraction technique has been used to abstract keywords that exclusively recognize a document. Rebuilding is an important part of this project and we will use the entire concept in that file format and in the last, we need the same format which we have done in that file. This concept is being widely used but not much work of the work has been done in the area of developing many functionalities under one tool, so this makes us feel the requirement of such a tool which can easily and efficiently convert unstructured files into structured one.

The performance achieved by an application is limited by architectural features such as program data access and processing patterns. Parallelization approaches exhibit dissimilar performance and have a direct impact in application execution time. Additionally, developing parallel code involves additional complexity and productivity for programmers to accelerate or rewrite the program. In this paper, we present a comparative performance evaluation of a CPU, GPU, and many-core (Xeon Phi KNL) architectures for the Non-Local Means filter. We asses the effect of different data access and processing patterns in two computational optimizations developed for the aforementioned filter. We follow a top-down approach in terms of the parallelization approach chosen, starting from performance primitives as a first step to give easy drop-in acceleration and then compiler directives with frameworks such as OpenMP and OpenACC as an intermediate step to map computing tasks to the underlying hardware. Results show that both libraries and directives are effective at accelerating code with a combination of both being necessary to overcome performance bottlenecks.

To solve the problem of low security in cloud storage system under the background of rapid prevailing cloud technology, this paper studies the design and implementation of User Autonomous Encryption Cloud Storage System based on Dokan. With the help of Dokan's file development model, a specific file system intended for curtain group of users can be easily customized. In order to overcome the limitations existing in traditional cloud storage systems, this system manages to establish the environment with embedded AES encryption algorithms on local clients and cloud storage servers based on OpenStack. The experimental results show that the User Autonomous Encryption Cloud Storage System based on Dokan is of higher security when compared with the traditional cloud storage systems.

In modern computer architectures, access latency varies considerably between different levels in the memory hierarchy. Consequently, applications with data access patterns that don't reuse much data in fast levels of the hierarchy incur additional delays. To improve the performance of complex, data-intensive applications, developers need tools that help them understand the causes of poor memory hierarchy utilization. While most performance tools associate metrics with functions or statements, in this paper we explore data-centric analyses that associate metrics not only with data accesses but also with data objects themselves. Our contributions are three-fold. First, we propose several refinements to existing data-centric techniques that enable accurate and low-overhead measurements. Second, we combine data-centric analysis with call path profiling; this combination of techniques relates inefficient access patterns back to data objects across complete dynamic call chains. Third, we developed a graphical user interface that gracefully presents our analysis results using a multiplicity of views, which helps users identify problematic accesses and data structures. We demonstrate the utility of our approach by showing how our tool identifies problematic data access patterns in several HPC applications and a pair of the SPEC CPU2006 benchmarks.

Software transactional memory (STM) has been proposed as a promising programming paradigm for shared memory multi-threaded programs as an alternative to conventional lock based synchronization primitives. Typical STM implementations employ a conflict detection scheme, which works with uniform access granularity, tracking shared data accesses either at word/cache line or at object level. It is well known that a single fixed access tracking granularity cannot meet the conflicting goals of reducing false conflicts without impacting concurrency adversely. A fine grained granularity while improving concurrency can have an adverse impact on performance due to lock aliasing, lock validation overheads, and additional cache pressure. On the other hand, a coarse grained granularity can impact performance due to reduced concurrency. Thus, in general, a fixed or uniform granularity access tracking (UGAT) scheme is application-unaware and rarely matches the access patterns of individual application or parts of an application, leading to sub-optimal performance for different parts of the application(s). In order to mitigate the disadvantages associated with UGAT scheme, we propose a Variable Granularity Access Tracking (VGAT) scheme in this paper. We propose a compiler based approach wherein the compiler uses inter-procedural whole program static analysis to select the access tracking granularity for different shared data structures of the application based on the application's data access pattern. We describe our prototype VGAT scheme, using TL2 as our STM implementation. Our experimental results reveal that VGAT-STM scheme can improve the application performance of STAMP benchmarks from 1.87% to up to 21.2%.

High-performance computing (HPC) has enabled advancements in computation speed and resource cost by utilizing all available server resources and using parallelization for speedup. This computation scheme encourages simulation model development, massive data collection, and AI computation models, all which store and compute on massive amounts of data. Data compression has enhanced the performance of storing and transferring this HPC application data to enable acceleration, but the benefits of data compression can also be transferred to the active allocated memory used by the application. In-line compression is a compression method that keeps the application memory compressed in allocated memory, decompressing memory as its data is needed by the algorithm. The actively decompressed data size in allocated memory can be limited by grouping and compressing the data into blocks informed by the application's computational kernel's data access patterns and a selected compressor. This research explores factors such as block size and compressor choice on the runtime and memory usage of the Matrix Multiplication kernel. Matrix multiplication (MM) is a fundamental HPC algorithm that exemplifies the memory access patterns and use cases present in other HPC kernels. MM kernels provide a baseline for evaluating in-line compression due to MM's row-based data access patterns and the usage of several matrices to compute the resulting matrix. Trade-offs between memory size and the compression runtime necessitate tuned parameters for each in-line compression-enabled kernel. The results of this research demonstrate essential parameter trends, trade-offs, and the importance of locality in the kernel's data access patterns.

This study aims to enhance the RC5 algorithm to improve encryption and decryption speeds in devices with limited power and memory resources. These resource-constrained applications, which range in size from wearables and smart cards to microscopic sensors, frequently function in settings where traditional cryptographic techniques because of their high computational overhead and memory requirements are impracticable. The Enhanced RC5 (ERC5) algorithm integrates the PKCS#7 padding method to effectively adapt to various data sizes. Empirical investigation reveals significant improvements in encryption speed with ERC5, ranging from 50.90% to 64.18% for audio files and 46.97% to 56.84% for image files, depending on file size. A substantial improvement of 59.90% is observed for data files sized at 1500000kb. Partitioning larger files notably reduces encryption time, while smaller files experience marginal benefits. Certain file types benefit from both strategies. Evaluation metrics include encryption execution time and throughput, consistently demonstrating ERC5's superiority over the original RC5. Moreover, ERC5 exhibits reduced power consumption and heightened throughput, highlighting its multifaceted benefits in resource-constrained environments. ERC5 is developed and tested on various file types and sizes to evaluate encryption speed, power consumption, and throughput. ERC5 significantly improves encryption speed across different file types and sizes, with notable gains for audio, image, and large data files. While partitioning smaller files only slightly improves encryption time, larger files partitioning yields faster results. Future research could explore ERC5 optimizations for different computing environments, its integration into real-time encryption scenarios, and its impact on other cryptographic operations and security protocols.

One ongoing problem with digital image file sizes is the cost of transferring data over a network. Larger image file sizes result in larger webpage file sizes. Larger webpage file sizes take more time to load and users incur higher data transfer costs when opening these larger webpages. It is cheaper and faster to utilize smaller compressed files than to use larger uncompressed files. Compression usually reduces the size of files for improved storage and online transmission of the files. Some types of files will compress better than other types, for example certain text files and BMP image files may even be compressed by up to 90%. This paper is part of a research work carried out to analyze and compare the efficiency of Adaptive Huffman coding with Arithmetic coding compression algorithms using various image files to assess the efficiency of the two different compression methods and to assess possible combinations of the compared compression algorithms which effectively reduce the image file size with minimal time complexity overhead. The research design focused on experiments such that a set of image files were sampled and systematically compressed. After progressively and systematically compressing a subset of sample files with the two selected general compression methods; it was observed that a fast Arithmetic coding (AC) variant consistently yielded fast compression times and high space-savings (file size reduction) results for image files and audio files. Additionally, reducing a BMP image's range of bytes (number of different bytes within the data of the file), using vector quantization, would also increase the space-savings result of compressing that BMP file with the fast AC variant.

More and more embedded devices, such as mobile phones, tablet PCs and laptops, are used in every field, so huge files need to be stored or backed up into cloud storage. Optimizing the performance of cloud storage is very important for Internet development. This paper presents the performance evaluation of the open source distributed storage system, a highly available, distributed, eventually consistent object/blob store from OpenStack cloud computing components. This paper mainly focuses on the mechanism of cloud storage as well as the optimization methods to process different sized files. This work provides two major contributions through comprehensive performance evaluations. First, it provides different configurations for OpenStack Swift system and an analysis of how every component affects the performance. Second, it presents the detailed optimization methods to improve the performance in processing different sized files. The experimental results show that our method improves the performance and the structure. We give the methods to optimize the object-based cloud storage system to deploy the readily available storage system.

The rapid adoption of cloud computing has transformed the way data is stored, shared, and accessed across organizationsand individuals. Despite its scalability and cost efficiency, cloud storage introduces serious security and privacy challenges, particularly when sensitive data is outsourced to third-party cloud service providers. Traditional access control mechanisms often rely on the cloud provider’s trustworthiness, which may expose data to unauthorized access, insider threats, and data breaches. To address these concerns, CryptCloud+ is proposed as a secure and expressive data access control framework that enforces fine-grained authorization directly through cryptographic techniques. CryptCloud+ focuses on ensuring data confidentiality while enabling flexible and policy-driven access control over encrypted cloud data. The system leverages encryption-based access control, where data is encrypted before being uploaded to the cloud, and only authorized users possessing valid cryptographic credentials can decrypt and access the data. This approach eliminates the need to trust the cloud service provider, as all sensitive operations related to data access are handled at the client side. A key contribution of CryptCloud+ is its support for expressive access policies, allowing data owners to define complex rules based on user attributes such as roles, departments, access levels, or time constraints. These policies enable fine grained access control without requiring data owners to manage individual permissions manually. The framework ensures that users can access only the data they are authorized for, while unauthorized users, including cloud administrators, are unable to infer any meaningful information from the encrypted data. The proposed system is implemented using Python, taking advantage of its robust cryptographic libraries, simplicity, and integration capabilities. Python-based modules handle key generation, encryption, decryption, and policy enforcement, making the system modular and easy to extend. The implementation demonstrates how secure key management and attribute-based access enforcement can be achieved efficiently in a real-world cloud environment. CryptCloud+ also addresses scalability and performance concerns by minimizing computation overhead during data access. Encryption and decryption operations are optimized to ensure that system performance remains practical even as the number of users and data files increases. Experimental results indicate that the framework maintains strong security guarantees while providing acceptable response times for authorized users. In addition to security, the system enhances usability by allowing seamless data sharing among multiple users without re encrypting data for each individual. This significantly reduces management complexity for data owners while preserving strict access control policies. The system design ensures compatibility with existing cloud storage services, making itadaptable to various deployment scenarios. Overall, CryptCloud+ presents a secure, flexible, and effiient solution for cloud data access control. By combining cryptographic enforcement with expressive policy definition and Python-based implementation, the frameworkstrengthens data security, protects user privacy, and improves trust in cloud storage systems. The proposed model demonstrates the feasibility of practical, cryptography-driven access control for modern cloud environments.

Cloud storage systems allows users to access their data from anywhere. They can download or upload their files or share them with others through the internet. Companies are competing to provide cloud storage services to users with varying features. This paper introduces criteria to compare cloud storage systems. It demonstrates the comparison between Dropbox and OneDrive in terms of their security, download/upload speed, application integration, pricing and storage, synchronization performance and focus on differences between them. The comparison conducts depend on six criteria that are: security, pricing and storage, integration of application, content sharing and synchronization. Dropbox shows the best performance in security, pricing and storage, synchronization and the download and upload of small files with small sizes. However, OneDrive shows a good performance in speed of download and upload big size of file. The comparison is intended to guide users and organizations when choose between the two services.

In modern cloud-based analytics systems, compaction plays a crucial role in optimizing storage and query performance for high-volume datasets. However, static compaction schedules lead to inefficiencies in resource utilization and cost, especially in variable load environments. This paper introduces AutoCompactionAI, a modular and serverless framework that uses adaptive learning to intelligently trigger compaction jobs based on data volume trends and access patterns. Built on AWS Glue, Lambda, and Athena, our framework integrates time series analysis and reinforcement learning to optimize file sizes and reduce redundant operations. Compared to traditional interval-based compaction, AutoCompactionAI achieves a 65% reduction in API costs and a 3x improvement in query latency across multiple domains, including digital marketing, healthcare telemetry, and e-commerce logs. We demonstrate its extensibility through domain-specific adapters and present rigorous evaluations that validate its scalability and generalizability.