Classification and Grading Storage Method Based on Terminal Scanning

High performance and energy-efficient storage systems are essential for mobile data-intensive applications such as remote surgery and mobile data center. Existing mobile storage systems consist of an array of independent small form factor hard disks connected to a host by a storage interface in a mobile computing environment. Although hard disks are cost-effective and can provide huge capacity and high-throughput, they have some intrinsic limitations such as long access latencies, high annual disk replacement rates, fragile physical characteristics, and energy-inefficiency. Compared with hard disk drives, solid state disks (SSD) are much more energy-efficient, and can offer much faster access times. A major concern on current solid state disk is its relatively higher price. In this paper, we developed hybrid mobile disk architecture HyBuM that integrates an array of solid state disks with buffer disks and an array of mobile disks. The most recently used data will be cached in the solid state disks, the second most popular data will be stored in the buffer disks, and the least used data sets will be stored in the mobile disk array. Experimental results demonstratively show that HyBuM provides significant energy saving for mobile storage systems such as laptops, mobile phones, and PDA compared to two existing non hybrid architecture.

In this paper, we propose a fast and simple system emulator, called a system performance emulator (SPE), to evaluate long read operations. The SPE estimates how much system-wide performance is enhanced by using a faster solid state disk (SSD). By suspending a CPU for a certain time during direct memory access (DMA) transfer and subtracting this suspended time from the total DMA time, the SPE estimates the improvement in system performance expected from an enhanced SSD prior to its manufacture. We also examine the relation between storage performance and system performance using the SPE.

Multi-tiered storage systems today are integrating Solid State Disks (SSD) on top of traditional rotational hard disks for performance enhancement due to the significant IO improvements in SSD technology. It is widely recognized that automated data migration between SSD and HDD plays a critical role in effective integration of SSD into multi-tiered storage systems. Furthermore, effective data migration has to take into account of application specific workload characteristics, deadlines, and IO profiles. An important and interesting challenge for automated data migration in multi-tiered storage systems is how to fully release the power of data migration while guaranteeing the migration deadline is critical to maximizing the performance of SSD-enabled multi-tiered storage system. In this paper, we present an adaptive look ahead data migration model that can incorporate application specific characteristics and I/O profiles as well as workload deadlines. Our adaptive data migration model has three unique features. First, it incorporates a set of key factors that may impact on the performance of look ahead migration efficiency in our formal model develop. Second, our data migration model can adaptively determine the optimal look ahead window size, based on several parameters, to optimize the effectiveness of look ahead migration. Third, we formally and experimentally show that the adaptive data migration model can improve overall system performance and resource utilization while meeting workload deadlines. Through our trace driven experimental study, we compare the adaptive look ahead migration approach with the basic migration model and show that the adaptive migration model is effective and efficient for continuously improving and tuning of the performance and scalability of multi-tier storage systems.

The performance gap between magnetic disks and data processing on HPC systems has become that huge that an efficient data processing can only be achieved by introducing non-volatile memory (NVRAM) as a new storage tier. Although the benefits of hierarchical storage have been adequately demonstrated to the point that the newest leadership class HPC systems will employ burst buffers, critical questions remain for supporting hierarchical storage systems, including: How should we present hierarchical storage systems to user applications, such that they are easy to use and that application code is portable across systems? How should we manage data movement through a storage hierarchy for best performance and resilience of data? How do the particular I/O use cases mandate the way we manage data? There have been many efforts to explore this space in the form of file systems, with increasingly more implemented at the user level. This is because it is relatively easy to swap in new, specialized user-level file systems for use by applications on a case-by-case basis, as opposed to the current mainstream approach of using general-purpose, system-level file systems which may not be optimized for HPC workloads and must be installed by administrators. In contrast, file systems at the user level can be tailored for specific HPC workloads for high performance and can be used by applications without administrator intervention. Many user-level file system developers have found themselves “having to reinvent the wheel” to implement various optimizations in their file systems. Thus, a main goal of this meeting was to bring together experts in I/O performance, file systems, and storage, and collectively explore the space of current and future problems and solutions for I/O on hierarchical storage systems in order to begin a community effort in enabling user-level file system support for HPC systems. We had a lively week of learning about each other’s approaches as well as unique I/O use cases that can influence the design of a community-driven file and storage system standards. The agenda for this meeting contained talks from participants on the following high level topics: HPC storage and I/O support today; what do HPC users need for I/O; existing userlevel file system efforts; object stores and other alternative storage systems; and components for building user-level file systems. The talks were short and intended to be conversation starters for more in-depth discussions with the whole group. The participants engaged in lengthy discussions on various questions that arose from the talks including: Are we ready to program to a memory hierarchy versus block devices? Are the needs of HPC users reflected in our existing file systems and storage systems? Should we drop or keep POSIX moving forward? What do we mean when we say “user-level file system”? Do we all mean the same thing? How should the IO 500 benchmark be defined so it is fair and useful? and How are stage-in and stage-out actually going to work? The report for this seminar contains a record of the talks from the participants as well as the resulting discussions. Our hope is that the effort initiated during this seminar will result in long-term collaborations that will benefit the HPC community as a whole.

Parallel circle frameworks are equipped for satisfying quickly expanding requests for both substantial stockpiling limit and high I/O execution. Be that as it may, it is trying to fundamentally increment circle I/O transfer speed for information escalated workloads because of the unwavering quality and moment handling of information solicitations under element workload conditions and the ideal tradeoff between framework adaptability and information dependability in information serious frameworks. To build the figuring execution and diminish power utilization, we proposed a novel half and half construction modeling that comprises of heterogeneous registering gadgets that comprises of Graphics Processing Unit (GPU) and Digital Signal Processor (DSP), Hard Disk Drive (HDD) exhibit and Solid State Disk (SSD). The most as often as could reasonably be expected utilized information will be put away as a part of the Solid state circles making the hard plates drive in a rest mode, however much as would be prudent to decrease the vitality utilization. We directed broad analyses to quantitatively assess the execution of the proposed construction modeling. Exploratory results demonstrate that coordinating GPU and DSP with the HDD and the SSD altogether lessen the force utilization over utilizing the GPU just without scarifying the execution.

A large write amplification ratio degrades the program/erase cycles (P/Es) of NAND Flashes and reduces the endurance and performance of solid state disks (SSDs). The lack of a practical way to measure write amplification for SSDs motivates us to propose a novel measuring method called RB-Explorer at the SSD level rather than the NAND Flash level. The goal of RB-Explorer is two-fold: (1) to accurately measure the write amplification of SSDs to quantify SSD endurance and (2) to study the impacts of I/O techniques on write amplification of SSDs. RB-Explorer incorporates a Ready/Busy (R/B) signal of one of the NAND Flashes in an SSD in a proposed write amplification model for SSDs with four full-parallelism levels (i.e., the channel, chip, die, and plane levels). RB-Explorer takes two steps toward measuring write amplification. First, RB-Explorer quantifies the number of page programs using the low R/B signal level, the duration of which varies with the different operation (i.e., read, program, and erase) in NAND Flash. Second, RB-Explorer measures data volume written to NAND Flashes by considering parallelisms at four levels. Data volume written to a die in a NAND Flash is obtained as a product of the number <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX"> ${\rm N_{p}}$</tex><mathgraphic fileref="sun-ieq1-2308207.gif" graphicformat="GIF"/></formula> of programs and page size <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">${\rm P_{a}}$</tex> <mathgraphic fileref="sun-ieq2-2308207.gif" graphicformat="GIF"/></formula> . Given the number <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">${\rm N_{channel}}$</tex> <mathgraphic fileref="sun-ieq3-2308207.gif" graphicformat="GIF"/></formula> of channels, the number <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">${\rm N_{chip}}$</tex> <mathgraphic fileref="sun-ieq4-2308207.gif" graphicformat="GIF"/></formula> of chips per channel, and the number <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">${\rm N_{die}}$</tex> <mathgraphic fileref="sun-ieq5-2308207.gif" graphicformat="GIF"/></formula> of dies per chip, one can obtain the data volume written to NAND Flashes as a product of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">${\rm N_{p}}, {\rm P_{a}}, {\rm N_{die}}, {\rm N_{chip}}$</tex><mathgraphic fileref="sun-ieq6-2308207.gif" graphicformat="GIF"/></formula> , and <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">${\rm N_{channel}}$</tex> <mathgraphic fileref="sun-ieq7-2308207.gif" graphicformat="GIF"/></formula> . RB-Explorer is applied to analyzing write amplification ratios of SSDs to track SSD endurance. Furthermore, we implement a real-world SSD (i.e., SSD-v) and employ a fine-tuned SSD simulator (i.e., SSDsim) to validate the accuracy of RB-Explorer. Our experimental results show that RB-Explorer improves on the accuracy of SSDsim—the state-of-the-art SSD simulator—in most tested cases. We conduct a series of measurements using micro-benchmarks and I/O traces to demonstrate how RB-Explorer may be applied to investigate SSDs.

Various key-value (KV) stores are widely employed for data management to support Internet services as they offer higher efficiency, scalability, and availability than relational database systems. The log-structured merge tree (LSM-tree) based KV stores have attracted growing attention because they can eliminate random writes and maintain acceptable read performance. Recently, as the price per unit capacity of NAND flash decreases, solid state disks (SSDs) have been extensively adopted in enterprise-scale data centers to provide high I/O bandwidth and low access latency. However, it is inefficient to naively combine LSM-tree-based KV stores with SSDs, as the high parallelism enabled within the SSD cannot be fully exploited. Current LSM-tree-based KV stores are designed without assuming SSD's multi-channel architecture.To address this inadequacy, we propose LOCS, a system equipped with a customized SSD design, which exposes its internal flash channels to applications, to work with the LSM-tree-based KV store, specifically LevelDB in this work. We extend LevelDB to explicitly leverage the multiple channels of an SSD to exploit its abundant parallelism. In addition, we optimize scheduling and dispatching polices for concurrent I/O requests to further improve the efficiency of data access. Compared with the scenario where a stock LevelDB runs on a conventional SSD, the throughput of storage system can be improved by more than 4X after applying all proposed optimization techniques.

The data transfer rate is the traditional criterion for evaluating the overall performance of a hard disk. In this paper, the data transmission performance of hard disk was characterized based on fractal theory. The traditional mechanical hard disks and solid state disks with different capacities and specifications were tested and the corresponding signal graph of data transfer was analysed for the calculation of fractal dimension D. The results indicate that the process of hard disk data transfer has fractal characteristics; the fractal dimension D increases with the increase of the average transfer rate and the rotating speed of mechanical hard disk. However, there seems no direct correspondence between the fractal dimension D and the capacity of the hard disk; the larger the floating range of data transmission, the greater value of fractal dimension D.

Solid State Disks (SSDs) are becoming increasingly popular due to enhancement in speed and reduction in size, power and cost. It is an obvious choice for many applications including personal computers, embedded systems, medical and healthcare devices etc. However, the reliability of SSD has now become a critical issue and this is an area of active research. SSD uses Flash technology which is a fast, efficient nonvolatile storage chip and offers cost-effective solution for SSDs. However, with all these features, flash memory defines a limit of write-erase on its blocks. Wearleveling policies extend SSD life time by ensuring even wearleveling of the blocks in a flash memory. Several algorithms have been proposed to improve wearleveling of SSD including Hot-Cold Swapping, Dual pool Algorithm (DP), Kim and Lee algorithm (KL) and Cost Age and Times Algorithm (CAT). This paper discusses all these wearleveling policies and introduces a Block Aging Prevention (BAP) technique which has been tested on 8Gb Flash. The comparison of all these algorithms show that the proposed technique has reduced the erasure count deviation of NAND flash blocks by 6.71% and 20% compared to DP and CAT respectively. It has also reduced overhead by 4% comparatively to DP.

Recent studies have shown that moderate to high data redundancy exists in primary storage systems, such as VM-based, enterprise and HPC storage systems, which indicates that the data deduplication technology can be used to effectively reduce the write traffic and storage space in such environments. However, our experimental studies reveal that applying data deduplication to primary storage systems will cause space contention in main memory and data fragmentation on disks. This is in part because applying data deduplication introduces significant index memory overhead to the existing system and in part because a file or block is split into multiple small data chunks that are often located in non-sequential locations on disks after deduplication. This fragmentation of data can cause a subsequent read operation to invoke many disk I/O requests, thus leading to performance degradation. The existing primary data deduplication schemes, such as iDedup[1], are to leverage spatial locality in that they only select the large requests to deduplicate and exclude the small requests (e.g., 4KB, 8KB or less) because the latter only account for a tiny fraction of the storage capacity requirement[2]. Moreover, these schemes tend to overlook the importance of cache management, leading them to manage the index cache and the read cache separately. However, previous workload studies on primary storage systems have revealed that small I/O requests dominate in the primary storage systems (more than 50%) and are at the root of the system performance bottleneck. Furthermore, the accesses in primary storage systems exhibit obvious I/O burstiness. The existing primary-storage data deduplication schemes fail to consider these workload characteristics in primary storage systems from the performance's perspective. We argue that, primary-storage data deduplication schemes should take the workload characteristics of primary storage into the design considerations. To address the two problems and take the primary-storage workload characteristics into considerations, we propose a Performance-Oriented I/O Deduplication approach, POD, to improving the I/O performance of primary storage systems in the Cloud. POD takes a two-pronged deduplication approach to improve primary storage systems, a request-based I/O and data deduplication scheme, called Select-Dedupe, aimed at alleviating data fragmentation and an adaptive memory management scheme, called iCache, to ease the main memory contention. More specifically, the former takes the workload characteristics of small-I/O-request domination into the design considerations. It deduplicates all the write requests if their write data is already stored sequentially on disks, including the small write requests that would otherwise be excluded from by the capacity-oriented deduplication schemes. For other write requests, Select-Dedupe does not deduplicate their redundant write data to maintain the performance of the subsequent read requests to these data. iCache takes the I/O burstiness characteristics into the design considerations. It dynamically adjusts the cache space between the index cache and the read cache according to the workload characteristics, and swaps these data between memory and backend storage devices accordingly. During the write-intensive bursty periods, iCache enlarges the index cache size and shrinks the read cache size to detect much more redundant write requests, thus improving the write performance. The read cache size is enlarged to cache more hot read data to improve the read performance during the read-intensive bursty periods. The prototype of the POD scheme is implemented as an embedded module at the block-device level with the fixed-size chunking method. Preliminary evaluations driven by the real traces conducted on our lightweight POD prototype implementation show that POD significantly outperforms iDedup in improving the performance of primary storage systems in the Cloud.

Analysis of six optimization techniques grouped under three categories (hardware, back-end, and front-end) is done to study the reduction in average user response time for Modular Object Oriented Dynamic Learning Environment (Moodle), a Learning Management System which is scripted in PHP5, runs on Apache web server and utilizes MySQL database software. Before the implementation of these techniques, performance analysis of Moodle is performed for varying number of concurrent users. The results obtained for each optimization technique are then reported in a tabular format. The maximum reduction in end user response time was achieved for hardware optimization which requires Moodle server and database to be installed on solid state disk.

Flash-based solid state disks (SSDs) are an alternative form of storage device that promises to deliver higher performance than the traditional mechanically rotating hard drives. While SSDs have seen utilization in embedded, consumer, and server computer systems, there has been little understanding of its performance effects with scientific I/O workloads. This paper provides a trace driven performance evaluation of scientific I/O workloads on SSDs. We find that SSDs only provide modest performance gains over mechanical hard drives due to the write-intensive nature of many scientific workloads. Other workloads (like read-mostly web servers) would likely see much larger gains. Additionally, we observe that the concurrent I/O (when multiple parallel processes simultaneously access a single storage device) may significantly affect the SSD performance. However, such effects appear to be dependent on specific SSD implementation features and they are hard to predict in a general fashion. These results suggest that abundant cautions are needed when supporting high-performance scientific I/O workloads on Flash-based SSDs.

A hybrid flash translation layer design for SLC–MLC flash memory based multibank solid state disk

This paper proposes a design procedure that exploits storage devices with different cost/bandwidth and cost/capacity ratios to build a hierarchical storage system for video-on- demand with minimum cost. The storage system is assumed to have two levels of hierarchy. The level-1 storage devices feature a low cost/bandwidth ratio and a high cost/capacity ratio. On the other hand, the level-2 storage devices feature a high cost/bandwidth ratio and a low cost/capacity ratio. The proposed procedure determines, with respect to overall system cost, which level of the hierarchy each program should be placed into. Based on the decisions, the designer then can figure out the appropriate configuration of the hierarchical storage system. The optimization target is to minimize the overall cost of the storage system.

Storage systems are the next frontier for providing protection against intrusion. Since storage systems see changes to persistent data, several types of intrusions can be detected by storage systems. Intrusion detection (ID) techniques can be deployed in various storage systems. In this paper, we study how intrusions can be detected at the block storage level and in SAN environments. We propose novel approaches for storage-based intrusion detection and discuss how features of state-of-the-art block storage systems can be used for intrusion detection and recovery of compromised data. In particular we present two prototype systems. First we present a real time intrusion detection system (IDS), which has been integrated within a storage management and virtualization system. In this system incoming requests for storage blocks are examined for signs of intrusions in real time. We then discuss how intrusion detection schemes can be deployed as an appliance loosely coupled with a SAN storage system. The major advantage of this approach is that it does not require any modification and enhancement to the storage system software. In this approach, we use the space and time efficient point-in-time copy operation provided by SAN storage devices. We also present performance results showing that the impact of ID on the overall storage system performance is negligible. Recovering data in compromised systems is also discussed.