Convergent encryption enabled secure data deduplication algorithm for cloud environment

Abstract: Cloud storage is one of the most important services of cloud computing, offering significant benefits to users by allowing them to outsource their data for storage and sharing with authorized parties. Secure deduplication has been extensively researched in cloud storage, aiming to eliminate redundancy in encrypted data and reduce storage space. Our proposed solution focuses on uploading files to the cloud storage system. When a file is uploaded, it is checked to see if an identical file already exists. If a duplicate is found, the system displays an entry indicating that the file already exists. This secure and efficient data deduplication scheme ensures that users can store their data securely in the cloud, with access limited to authorized parties. By eliminating redundancy, more space is available for uploading different files. To ensure data security, we utilize the MD5 algorithm for hashing purposes and AES for encryption. Our system offers a practical solution for organizations seeking to securely store their data in the cloud, while also preserving user privacy and data ownership

Deduplication is an important technology in the cloud storage service. For protecting user privacy, sensitive data usually have to be encrypted before outsourcing. This makes secure data deduplication a challenging task. Although convergent encryption is used to securely eliminate duplicate copies on the encrypted data, these secure deduplication techniques support only exact data deduplication. That is, there is no tolerance of differences in traditional deduplication schemes. This requirement is too strict for multimedia data including image. For images, typical modifications such as resizing and compression only change their binary presentation but maintain human visual perceptions, which should be eliminated as duplicate copies. Those perceptual similar images occupy a lot of storage space on the remote server and greatly affect the efficiency of deduplication system. In this paper, we first formalize and solve the problem of effective fuzzy image deduplication while maintaining user privacy. Our solution eliminates duplicated images based on the measurement of image similarity over encrypted data. The robustness evaluation is given and demonstrates that this fuzzy deduplication system is able to duplicate perceptual similar images, which optimizes the storage and bandwidth overhead greatly in cloud storage service.

Data redundancy consumes huge storage space while setting up or employing cloud and fog storage. The dynamic cloud nature primarily focuses on the static environments which must be revised. Data deduplication solutions help minimize and control this issue by eradicating duplicate data from cloud storage systems. Since it might improve storage economy and security, data deduplication (DD) over encrypted data is a crucial problem in computing and storage systems. In this research, a novel approach to building secure deduplication systems across cloud and fog environments is developed. It uses MCDD and convergent cryptographic algorithms. The two most significant objectives of such systems are the focus of the suggested approach. Data redundancy must be minimized, but it also needs to be secured using a robust encryption method, which needs to be devised. The suggested approach is ideally suited for tasks like a user uploading new data to cloud storage or the fog. The proposed method might eliminate data redundancy by detecting redundancy at the block level. The testing results indicate that the recommended methodology can surpass a few cutting-edge techniques regarding computing effectiveness and security levels. The file is encrypted twice, once with the modified cryptographic model for deduplication (MCDD) and once with convergence encryption (CE).

A secure and lightweight cloud data deduplication scheme with efficient access control and key management

As the demand for data sharing and complex access-control policies continues to grow, traditional encryption mechanisms, which are generally established using a Public Key Infrastructure, face the problem of massive processing overheads and huge network bandwidth consumption. Attribute-Based Encryption (ABE) schemes have been proposed as a potential means of addressing these issues and have attracted great attention in recent years. Most previous studies on ABE focus on issues such as the revocation mechanism, multi-authority, the access structure design, and traceability. However, very few studies consider the storage efficiency problem and the present study proposes a novel data deduplication scheme based on ciphertext-policy ABE with convergent encryption and block-level data. The scheme can be deployed in third-party semi-trusted environments, and not only provides flexible, fine-grained access control over encrypted data, but also allows for the in-line elimination of redundancies in order to save cloud storage space. The experimental results show that the proposed scheme has an acceptable computational overhead and provides a promising solution for real-world data cloud storage and access scenarios.

Day by day the use of memory is increases rapidly. A lot of data created every day so the data management is become a critical task. Most of time much amount of memory wastage because of same copies present on various locations. The process of eliminating the repeated or duplicates copies of data is called as Data deduplication. This data deduplication process is widely used in cloud storage to decrease storage space and upload bandwidth. However there is only one copy of each file stored even if such a file is owned by huge number of users. By using, deduplication system progress of storage utilization and reliability is increases. In addition, the dare of privacy for sensitive data also take place when they are outsourced by users to cloud. In this we use new distributed deduplication systems with upper dependability in which the data chunks are distributed from corner to cornering multiple cloud servers. The safety needs of data privacy and tag stability are also accomplish by introducing a deterministic secret sharing scheme in distributed storage systems, instead of using convergent encryption as in previous deduplication systems. A deduplication technique, on the other hand, can reduce the storage cost at the server side and save the upload bandwidth at the user side.

Cloud storage is the decisive service for the organization, hospitals etc., for storing huge amount of data within their budget. But, Cloud Service Providers (CSPs) are facing the major problem of duplicate and redundant data being accumulated by Data Owners (DO) in the cloud storages. Deduplication is one of the major technique to solve the above problem but faces the major drawback in the form of security flaws. Secure deduplication is a very promising solution proposed by the researchers for addressing this issue. In this paper, problems like revocation of DOs, brute force attack and duplicate faking attack are addressed as part of secure deduplication. The proposed approach is having two protocols, namely, Proof of Identity (PoI) and Proof of Data Integrity (PoDI) which solves the above-mentioned problems. The proposed approach improves the efficiency of secure deduplication, enhances data privacy and addresses Data Owner (DO) revocation problem.

Cloud computing provides a scalable platform for large amount of data and processes that work on various applications and services by means of on-demand service. The storage services offered by clouds have become a new profit growth by providing a comparable cheaper, scalable, location-independent platform for managing users’ data. The client uses the cloud storage and enjoys the high end applications and services from a shared group of configurable computing resources using cloud services. It reduces the difficulty of local data storage and maintenance. But it gives severe security issues toward users’ outsourced data. Data Redundancy promotes the data reliability in Cloud Storage. At the same time, it increases storage space, Bandwidth and Security threats due to some server vulnerability. Data Deduplication helps to improve storage utilization. Backup is also less which means less Hardware and Backup media. But it has lots of security issues. Data reliability is a very risky issue in a Deduplication storage system because there is single copy for each file stored in the server which is shared by all the data owners. If such a shared file/chunk was missing, large amount of data becomes unreachable. The main aim of this work is to implement Deduplication System without sacrificing Security in cloud storage. It combines both Deduplication and convergent key cryptography with reduced overhead.

Nowadays, with the increase in cyber-attacks, hacking, and data theft, maintaining data security and confidentiality is of paramount importance. Several techniques are used in cryptography and steganography to ensure their safety during the transfer of information between the two parties without interference from an unauthorized third party. This paper proposes a modern approach to cryptography and steganography based on exploiting a new environment: bases and protein chains used to encrypt and hide sensitive data. The protein bases are used to form a cipher key whose length is twice the length of the data to be encrypted. During the encryption process, the plain data and the cipher key are represented in several forms, including hexadecimal and binary representation, and several arithmetic operations are performed on them, in addition to the use of logic gates in the encryption process to increase encrypted data randomness. As for the protein chains, they are used as a cover to hide the encrypted data. The process of hiding inside the protein bases will be performed in a sophisticated manner that is undetectable by statistical analysis methods, where each byte will be fragmented into three groups of bits in a special order, and each group will be included in one specific protein base that will be allocated to this group only, depending on the classifications of bits that have been previously stored in special databases. Each byte of the encrypted data will be hidden in three protein bases, and these protein bases will be distributed randomly over the protein chain, depending on an equation designed for this purpose. The advantages of these proposed algorithms are that they are fast in encrypting and hiding data, scalable, i.e., insensitive to the size of plain data, and lossless algorithms. The experiments showed that the proposed cryptography algorithm outperforms the most recent algorithms in terms of entropy and correlation values that reach −0.6778 and 7.99941, and the proposed steganography algorithm has the highest payload of 2.666 among five well-known hiding algorithms that used DNA sequences as the cover of the data.

Cloud computing has empowered the individual user by providing seemingly unlimited storage space and availability and accessibility of data anytime and anywhere. Cloud service providers are able to maximize data storage space by incorporating data deduplication into cloud storage. Although data deduplication removes data redundancy and data replication, it also introduces major data privacy and security issues for the user. In this paper, a new privacy-preserving framework that addresses this issue is proposed. Our framework uses an efficient deduplication algorithm to divide a given file into smaller units. These units are then encrypted by the user using the combination of a secure hash function and a block encryption algorithm. An index tree of hash values of these units is also generated and encrypted using an asymmetric search encryption scheme by the user. This index tree will enable the cloud service provider to search through the index and return the requested units. We will show that our proposed framework will allow cloud service and storage providers to employ data deduplication techniques without giving them access to either the users' plaintexts or the users' decryption keys.

Data deduplication is a valuable technique for compressing and minimizing data duplication during data transfers, especially in cloud environments. By eliminating redundant data, it optimizes transmission capacity and reduces memory usage. To ensure the integrity of sensitive data, encryption is applied throughout the deduplication process. The SHA algorithm is commonly used for storing text data during deduplication. It generates security bits by padding the text and computes a hash consisting of hexadecimal, string, and integer data. Hash-based deduplication involves hashing the entire file and treating the hash values of text data as unique identifiers. This allows clients to identify duplicate data within the cloud. The Proof-of-Work (PoW) algorithm is widely utilized in blockchain networks like Bitcoin and Ethereum. Its primary function is to verify transactions through a process called mining. Miners engage in competition to solve complex mathematical problems, and the first one to find a solution is granted the right to add a new block to the blockchain. PoW relies on cryptographic hashing, such as the SHA-256 hashing function, to validate and secure transactions. On the other hand, the Proof of Retrievability (PoR) algorithm finds application in cloud computing systems. It serves as a consensus mechanism, ensuring that cloud storage providers store and retrieve data accurately. PoR enables cloud providers to demonstrate to consumers that their files can be fully recovered. This algorithm incorporates cryptographic proofs that validate the integrity and availability of the stored data. In cloud storage, deduplication is often implemented using the Memory mapping technique (MPT). This allows multiple data owners to store the same data in a single copy, enhancing storage efficiency. To maintain data security, encryption is applied both before and during the deduplication process, ensuring that sensitive information remains protected. In summary, data deduplication is a powerful method for compressing data, minimizing duplication, and optimizing transmission capacity. With encryption techniques and hash-based identification, it provides secure and efficient deduplication in cloud environments, while memory deduplication and MPT support further enhance performance and storage efficiency.

An application of Cloud Computing (CC) technologies for interconnecting data as well as applications served as of manifold geographic locations is the distributed cloud. For distributed CC, managing a vast measure of data along with maintaining security are vital aspects. Though lots of techniques were introduced recently to manage and protect data on the distributed cloud, those have not rendered any desirable security. Thus, this paper proposes a secure Data Deduplication (DD) and Data Portability (DP) for a distributed cloud environment. To enhance the user data’s security, primarily, the inputted files are bifurcated into blocks and after that, an appropriate Cloud Server (CS) is chosen utilizing Hybrid Forest Genetic Algorithm (HFGA) centred upon split file features together with CS features. In a DD phase, the Whirlpool algorithm converts the split data into Hash Code (HC), and then, the hash chaining technique securely removes the duplicated data. Then, that split data is comp ressed, encrypted, together with amassed in the selected CS. Next, the Levy Flight – Wind Driven Optimization (LF-WDO) performs the DP to enhance the cloud data’s security. Investigational outcomes exhibit the proposed work's potential.

The rapid adoption of cloud computing has transformed the way data is stored, shared, and accessed across organizationsand individuals. Despite its scalability and cost efficiency, cloud storage introduces serious security and privacy challenges, particularly when sensitive data is outsourced to third-party cloud service providers. Traditional access control mechanisms often rely on the cloud provider’s trustworthiness, which may expose data to unauthorized access, insider threats, and data breaches. To address these concerns, CryptCloud+ is proposed as a secure and expressive data access control framework that enforces fine-grained authorization directly through cryptographic techniques. CryptCloud+ focuses on ensuring data confidentiality while enabling flexible and policy-driven access control over encrypted cloud data. The system leverages encryption-based access control, where data is encrypted before being uploaded to the cloud, and only authorized users possessing valid cryptographic credentials can decrypt and access the data. This approach eliminates the need to trust the cloud service provider, as all sensitive operations related to data access are handled at the client side. A key contribution of CryptCloud+ is its support for expressive access policies, allowing data owners to define complex rules based on user attributes such as roles, departments, access levels, or time constraints. These policies enable fine grained access control without requiring data owners to manage individual permissions manually. The framework ensures that users can access only the data they are authorized for, while unauthorized users, including cloud administrators, are unable to infer any meaningful information from the encrypted data. The proposed system is implemented using Python, taking advantage of its robust cryptographic libraries, simplicity, and integration capabilities. Python-based modules handle key generation, encryption, decryption, and policy enforcement, making the system modular and easy to extend. The implementation demonstrates how secure key management and attribute-based access enforcement can be achieved efficiently in a real-world cloud environment. CryptCloud+ also addresses scalability and performance concerns by minimizing computation overhead during data access. Encryption and decryption operations are optimized to ensure that system performance remains practical even as the number of users and data files increases. Experimental results indicate that the framework maintains strong security guarantees while providing acceptable response times for authorized users. In addition to security, the system enhances usability by allowing seamless data sharing among multiple users without re encrypting data for each individual. This significantly reduces management complexity for data owners while preserving strict access control policies. The system design ensures compatibility with existing cloud storage services, making itadaptable to various deployment scenarios. Overall, CryptCloud+ presents a secure, flexible, and effiient solution for cloud data access control. By combining cryptographic enforcement with expressive policy definition and Python-based implementation, the frameworkstrengthens data security, protects user privacy, and improves trust in cloud storage systems. The proposed model demonstrates the feasibility of practical, cryptography-driven access control for modern cloud environments.

Data deduplication is a special type of resource usage optimisation. It leads to reduction of the used storage space and network bandwidth by eliminating duplicate copies of the same data file. Convergent encryption, as the state-of-art approach, has been widely adopted to perform secure deduplication in the cross-user scenario. However, all prior solutions do not support user traceability: there is no way to trace the identities of malicious users in case of duplicate faking attacks. To cope with this problem, we propose a deduplication scheme called TrDup. It realises traceability of malicious user's identity by incorporating traceable signatures with message-locked encryption technique. The TrDup construction is followed by its formal security analysis.

By only storing a unique copy of duplicate data possessed by different data owners, deduplication can significantly reduce storage cost, and hence is used broadly in public clouds. When combining with confidentiality, deduplication will become problematic as encryption performed by different data owners may differentiate identical data which may then become not deduplicable. The Message-Locked Encryption (MLE) is thus utilized to derive the same encryption key for the identical data, by which the encrypted data are still deduplicable after being encrypted by different data owners. As keys may be leaked over time, re-encrypting outsourced data is of paramount importance to ensure continuous confidentiality, which, however, has not been well addressed in the literature. In this paper, we design SEDER, a SEcure client-side Deduplication system enabling Efficient Re-encryption for cloud storage by (1) leveraging all-or-nothing transform (AONT), (2) designing a new delegated re-encryption (DRE), and (3) proposing a new proof of ownership scheme for encrypted cloud data (PoWC). Security analysis and experimental evaluation validate security and efficiency of SEDER, respectively.