GumTree: Data reduction

This Phase II research project was focused on constructing and testing a facility for the measurement of the structure of hot solid and liquid materials under extreme conditions using neutron diffraction. The work resulted in measurements at temperatures of 3300 K, the highest ever performed in a neutron beam. Work was performed jointly by Containerless Research, Inc. and Argonne National Laboratory with significant interactions with engineers and scientists at the under construction-SNS facility in Oak Ridge, TN. The work comprised four main activities: Design and construct an advanced instrument for structural studies of liquids and hot solids using neutron scattering. Develop and test a software package for instrument control, data acquisition and analysis. Test and demonstrate the instrument in experiments at the GLAD beamline at IPNS. Evaluate requirements for performing experiments at the SNS. Develop interest from the potential user base and identify potential support for Phase III. The objectives of the research were met. A second-generation instrument was developed and constructed. The instrument design drew on the results of a formal design review which was held at Argonne National Laboratory during the Phase I research [1]. The review included discussion with potential instrument users, SNS scientists and engineers and various scientists involved with materials, glass, ceramics, and geological sciences. The instrument combines aerodynamic levitation with pulsed neutron diffraction in a controlled atmosphere. An important innovation was the use of pure vanadium levitation nozzles that effectively eliminated contributions from the sample environment to the measured data. The instrument employed a 250 Watt CO2 laser that was configured for Class I laser operation. The use of Class I laser configuration meant that operators could work with the equipment with minimal restrictions and so concentrate on the research activities. Instrument control and data acquisition software was developed and implemented. As part of a larger initiative at IPNS, PC-based programs are being developed for acquisition and processing of neutron data. The PC-based beamline data handling system will enable compatibility with the levitator software. The instrument was bench tested at CRI and operated in three campaigns at the GLAD beamline at IPNS. Samples approximately 3.5 mm in diameter were levitated for periods up to 6 hours and at temperatures up to 3300 K. Structure factors were obtained for liquid oxide materials and hot solids. Details are given in this report and in published or submitted papers. During the course of the Phase I and Phase II projects, technical presentations were made at the Materials Research Society meeting in Boston, November, 2001, the American Conference on Neutron Scattering in Knoxville, TN, June, 2002, the Gordon Research Conference on High Temperature Chemistry (poster) in Waterville, ME, August 2002, the ACNS meeting in Baltimore, MD, June, 2004 and the Non-crystalline Materials-9 meeting in Corning NY, July, 2004. Two manuscripts were prepared, one is published, one is in review. The presentations have resulted in contact with the user community and we have received several requests to use the instrument. As a result, we are seeking support for collaborative research and plan to offer beamline instruments for commercial sale.

As the number of pre-trained machine learning (ML) models is growing exponentially, data reduction tools are not catching up. Existing data reduction techniques are not specifically designed for pre-trained model (PTM) dataset files. This is largely due to a lack of understanding of the patterns and characteristics of these datasets, especially those relevant to data reduction and compressibility. This paper presents the first, exhaustive analysis to date of PTM datasets on storage compressibility. Our analysis spans different types of data reduction and compression techniques, from hash-based data deduplication, data similarity detection, to dictionary-coding compression. Our analysis explores these techniques at three data granularity levels, from model layers, model chunks, to model parameters. We draw new observations that indicate that modern data reduction tools are not effective when handling PTM datasets. There is a pressing need for new compression methods that take into account PTMs' data characteristics for effective storage reduction. Motivated by our findings, we design Elf, a simple yet effective, error-bounded, lossy floating-point compression method. Elf transforms floating-point parameters in such a way that the common exponent field of the transformed parameters can be completely eliminated to save storage space. We develop Elves, a compression framework that integrates Elf along with several other data reduction methods. Elves uses the most effective method to compress PTMs that exhibit different patterns. Evaluation shows that Elves achieves an overall compression ratio of 1.52×, which is 1.31×, 1.32× and 1.29× higher than a general-purpose compressor (zstd), an error-bounded lossy compressor (SZ3), and the uniform model quantization, respectively, with negligible model accuracy loss.

ABSTRACTA general virtual experiment for the time-of-flight neutron reflectometer is introduced to simulate the experimental data collection and processing, including instrument simulation, sample modeling, detector simulation, and data reduction. The process mimics the experimental data processing and reduction routines. The reduced data agree well with the theoretical calculation, confirming the reliability of the data reduction algorithms. The methodology and algorithms developed in this study pave the way for the future development of data processing and data reduction software for the time-of-flight neutron reflectometer of China Spallation Neutron Source.

Frequently, at scientific meetings proteome researchers meet after the formal program has ended to discuss the state of the field and their own situation over some drinks. Typically these discussions center around instrumentation, primarily mass spectrometers, with a faster scan rate, higher mass accuracy or sensitivity, and the results generated by these powerful instruments. Statements like, “this is the future,” “this will revolutionize proteomics,” or “this has changed what we do in the lab” can be heard around the table. These sentiments are certainly justified. The progress in mass spectrometric instrumentation in the short history of proteomics has been nothing but astounding. Just a decade ago virtually all proteomic data were generated on ion trap instruments with low resolution, low mass accuracy, and limited scan speed and sensitivity, compared with the instruments presently in common use. Today, a significant fraction of the proteomic data is collected on high performance hybrid mass spectrometers with high scan speed, high mass accuracy and resolution, and high sensitivity of detection. The effects of these dramatic improvements are apparent in the manuscripts being published. However, its importance notwithstanding, data collection is only half of the whole proteomic story. The capacity to annotate, analyze, store, manage, and distribute the many gigabytes of data collected is the other half of the proteomic technology tandem. Long gone are the days when the expert analyst pretended to have manually verified all the data supporting the conclusions in a proteomic paper. Therefore, proteomics today, particularly mass spectrometry-based proteomics, depends as much on computers and the software tools they use as it does on high tech instruments to generate the data. Surprisingly, at the sessions in the bar, no one ever seems to boast about the data analysis part of the proteomic enterprise. Statements such as “ this program just recovered 20% more true positive peptide identification from my data set,” “ this allowed me to analyze my data ten times faster and at a credible FDR,” or “this statistical model allowed me to identify statistically significant quantitative changes in my dataset” are seldom if ever uttered. It is unlikely that things in the realm of electronics and software are inherently less appealing or attractive than high performance devices because the same individuals boasting about their mass spectrometers are likely to also share interesting new iPhone apps. It is certainly also not the case that advances on the software side of proteomics have had a lower impact than publications describing new hardware. In fact some of the most highly cited proteomics papers are about software. I therefore propose the likely controversial thesis that the software world of proteomics is less generally discussed because few can keep up and know about the field in some depth. This view is supported by several observations. First, in a recent study (1), in which a standardized protein sample was analyzed by a number of laboratories, it was clearly apparent that the analysis of the collected data was a major weakness. Second, the adoption of new software tools for proteomic data analysis takes much longer than the adoption of new instruments or wet-lab protocols. To raise the level of awareness in the crucial area of proteomic software tools, Molecular and Cellular Proteomics will publish a series of mini reviews that summarize the current state of specific aspects of this often bewildering field. These are not tutorials, i.e. the reader still will have to figure out how to use the tools and which tools to use. The intent of this series is to discuss the significance of a particular type of analysis, the question(s) addressed, the currently available solutions and their strengths and weaknesses, and the availability and utility of germane software tools or computational resources. We expect that these mini reviews will help proteome researchers to extract more and better information from the data generated by the wonderful instruments available today and to increase the excitement about the topic so that statistical models, search engines, and XML schema can also claim the stature in bar discussions that they deserve.

BackgroundAs high-throughput sequencing applications continue to evolve, the rapid growth in quantity and variety of sequence-based data calls for the development of new software libraries and tools for data analysis and visualization. Often, effective use of these tools requires computational skills beyond those of many researchers. To ease this computational barrier, we have created a dynamic web-based platform, NASQAR (Nucleic Acid SeQuence Analysis Resource).ResultsNASQAR offers a collection of custom and publicly available open-source web applications that make extensive use of a variety of R packages to provide interactive data analysis and visualization. The platform is publicly accessible at http://nasqar.abudhabi.nyu.edu/. Open-source code is on GitHub at https://github.com/nasqar/NASQAR, and the system is also available as a Docker image at https://hub.docker.com/r/aymanm/nasqarall. NASQAR is a collaboration between the core bioinformatics teams of the NYU Abu Dhabi and NYU New York Centers for Genomics and Systems Biology.ConclusionsNASQAR empowers non-programming experts with a versatile and intuitive toolbox to easily and efficiently explore, analyze, and visualize their Transcriptomics data interactively. Popular tools for a variety of applications are currently available, including Transcriptome Data Preprocessing, RNA-seq Analysis (including Single-cell RNA-seq), Metagenomics, and Gene Enrichment.

The students' over reliance upon formulas and routine use of technique in problem solving too often lead to poor performance in advanced courses and a high attrition rate in the engineering, technology, and science programs. The students' lack of comprehension of mathematical concepts results in time wastage during laboratory experiments, misinterpretations of lab data and underachievement in standardized science and engineering tests that stress the fundamentals. This problem can be effectively addressed by improving the student's conceptual understanding and comprehension of the topics covered in introductory science and technology courses. One way to achieve this is through interactive learning and teaching and upgrading the existing laboratories with modern equipment. This will require increased funding and resources. But in recent years there is a decrease in resource allocation making it increasingly difficult to modernize the laboratories to provide adequate levels of laboratory and course work and universities are under pressure to look for alternative cost effective methods. One way to achieve this is through interactive learning and teaching through the use of software packages like LabVIEW (Virtual Instruments) and Java/Visual Basic (Object Oriented Programming).LabVIEW and Java/Visual Basic are currently used in a number of engineering schools and industries for simulation and analysis. By introducing virtual instrumentation (LabVIEW) and object oriented programming (Java/Visual Basic) to the existing laboratory facilities and course(s) the students can be well trained with the latest design techniques and computer aided instrumentation, design and process control used throughout industry. This will also allow the students greater interaction with the subject matter and improve his/her skills in the use of number of applied engineering software packages.LabVIEW is based on graphical programming and easy to use. It is an interactive problem-solving environment, where the students can analyze, visualize, and document real-world science and engineering problems. A number of books are available covering the use and application of LabVIEW.LabVIEW also provides an extensive library of virtual instruments and functions to help in programming. It also contains application specific libraries for data acquisition, GPIB and serial instrument control, data analysis, and file input/output with conventional program debugging tools that can be used to set breakpoints, single-step through the program, and animate the execution so that the flow of data can be observed.At South Carolina State University (SCSU), the PC and VI based system concept is used to design and develop a laboratory called Computer Based Virtual Engineering Laboratory (CBVEL). The CBVEL consists of IBM compatible computers with appropriate software and hardware from National Instruments (NI), Simulation software such as C++, Visual Basic, Java, PSPICE, etc., and is connected to School of Engineering Technology and Sciences (SETS) network and existing equipment. Virtual Instrument (VI) and Object Oriented Programming modules for different courses and research areas are currently developed and used to teach various courses. Examples of some of these VIs are Circuit Analysis, Electronics, Communications, Digital Signal processing, Networking, and Digital Filters.This paper discusses design and development of some of the modules such as Number Systems, Internet Protocol Addresses, Address Classes, Subnet Masks, and Sub-netting using LabVIEW and Visual Basic/Java.

The degree of cure and temperature are consistent variables used in models to describe the state of material behaviour development for a thermoset during cure. Therefore, the validity of a cure kinetics model is an underlying concern when combining several material models to describe a part forming process, as is the case for process modeling. The goals of this work are to identify sources of uncertainty in the decision-making process from cure measurement by differential scanning calorimeter (DSC) to cure kinetics modeling, and to recommend practices for reducing uncertainty. Variability of cure kinetics model predictions based on DSC measurements are investigated in this work by a study on the carbon-fiber-reinforced-plastic (CFRP) T800H/3900-2, an interlaboratory Round Robin comparison of cure studies on T800H/3900-2, and a literature review of cure models for Hexcel 8552. It is shown that variability between model predictions can be as large as 50% for some process conditions when uncertainty goes unchecked for decisions of instrument quality, material consistency, measurement quality, data reduction and modeling practices. The variability decreases to 10% when all of the above decisions are identical except for the data reduction and modeling practices. In this work, recommendations are offered for the following practices: baseline selection, balancing heats of reaction, comparing data over an extensive temperature range (300 K), choosing appropriate models to describe a wide range of behaviour, testing model reliability, and visualization techniques for cure cycle selection. Specific insight is offered to the data reduction and analysis of thermoplastic-toughened systems which undergo phase separation during cure, as is the case for T800H/3900-2. The evidence of phase separation is a history-dependent Tg-α relationship. In the absence of a concise outline of best practices for cure measurement by DSC and modeling of complex materials, a list of guidelines based on the literature and the studies herein is proposed.

本文以电气安全检测实验室原有测试设备，在增加少许控制仪器和数据采集仪器的基础上，利用计算机编程语言，设计开发一种储水式电热水器能耗测试系统，可灵活设定控制参数，实现多标准要求下对测试过程的控制、测试数据的采集、测试数据的计算等功能，特别适用复杂测试过程中自动化控制的要求。本文在减少测试成本，提高测试精度等方面做出了积极研究，其论述的系统已在实践中应用，为国内外相关测试实验室进行储水式电热水器能耗的测试提供了一种高效低成本的解决方案。 Based on the testing equipment in electrical safety testing laboratory, this paper increased a little control instruments and data acquisition instruments and used computer programming language to develop one kind of energy testing system for electric storage water-heaters. This testing system can be flexibly set to meet the multi-standards testing requirements and to achieve the purpose of controlling the tested process, data collection, data calculation and the other functions. The testing system also can satisfy the requirements of complex testing in automated control process. The paper has made a positive research to reduce testing costs and improve testing accuracy. The testing system has been applied in practice and has provided a high efficient and low-cost solution plan for domestic and foreign laboratories which have engaged in the energy consumption testing of electric storage water-heaters.

Researchers from myriad fields frequently seek to quantify and classify concentrations of carbonaceous aerosols as organic carbon (OC) or elemental carbon (EC). This is commonly accomplished using thermal-optical OC/EC analyzers (TOAs), which enable measurement via controlled thermal pyrolysis and oxidation under specific temperature protocols and within constrained atmospheres. Several commercial TOAs exist, including a semi-continuous instrument that enables on-line analyses in the field. This instrument employs an in-test calibration procedure that requires relatively frequent calibration. This article details a calibration protocol for this semi-continuous TOA and presents an open-source software tool for data analysis and rigorous Monte Carlo quantification of uncertainties. Notably, the software tool includes novel means to correct for instrument drift and identify and quantify the uncertainty in the OC/EC split point. This is a significant improvement on the uncertainty estimation in the manufacturer’s software, which ignores split point uncertainty and otherwise uses fixed equations for relative and absolute errors (generally leading to under-estimated uncertainties and often yielding non-physical results as demonstrated in several example data sets). The demonstrated calibration protocol and new software tool enabling accurate quantification of combined uncertainties from calibration, repeatability, and OC/EC split point are shared with the intent of assisting other researchers in achieving better measurements of OC, EC, and total carbon mass in aerosol samples.

The ideas which crystallised into the ORAC project developed gradually at the JAC during the mid-1990s. Key to the process was probably Alan Bridger, head of the UKIRT software group, who had been thinking for some time about how to radically improve UKIRT’s end-to-end software system to make it easier to maintain and upgrade. In this he was not alone. He says ‘The ideas/thoughts behind what would become ORAC had been evolving while I was still at the JAC, the semi-automation of observing, the evolution of CGS4/IRCAM_PREP into _, the CGS4_DR system and so on. Various things had been steadily coming together over a few years, and from a number of directions and individuals’. There was also the issue of the control, user interface and data reduction for the new instruments which were on the way. For example, UFTI was being produced under the then popular ‘faster, cheaper, better’ philosophy and was not expected to arrive with its own data reduction pipeline. There was also the question of Michelle, which would operate as both an imager and spectrometer and would need various data reduction packages able to work reliably in the tough thermal-infrared regime with its high data rates and the need to nod, chop and perhaps mosaic images. Gillian Wright says she was ‘Absolutely totally convinced that in the long- term to get the best out of Michelle UKIRT would need some way of doing the 10 micron work on the best nights. Plus we saw Michelle as being like CSG4 in that it would be so new technically that to get publications out quickly would mean providing observers with publication quality data in real or almost real time’.

A great deal of progress has been made in improving the data reduction experience for the SANS instruments at the SNS and HFIR at ORNL. The existing data reduction toolset, drtsans, makes it possible to integrate data analysis and visualization tools into the data reduction scripts, thereby providing new opportunities for more automated data processing for users of the SNS and HFIR. Here, the first set of tools developed is described with usage examples.

Data reduction for large-scale linear regression is one of the most important tasks in this era of data deluge. Exact model information is however not often available for big data analytics. Therefore, we propose a framework for big data sketching (i.e., a data reduction tool) that is robust to possible model mismatch. Such a sketching task is cast as a Boolean min-max optimization problem, and then equivalently reduced to a Boolean minimization program. Capitalizing on the block coordinate descent algorithm, a scalable solver is developed to yield an efficient sampler and a good estimate of the unknown regression coefficient.

A feature selection method for multimodal multispectral LiDAR sensing

Motivation: Rapid advances in live-cell imaging analysis and mathematical modeling have produced a large amount of quantitative data on spatiotemporal dynamics of biological objects ranging from molecules to organisms. There is now a crucial need to bring these large amounts of quantitative biological dynamics data together centrally in a coherent and systematic manner. This will facilitate the reuse of this data for further analysis.Results: We have developed the Systems Science of Biological Dynamics database (SSBD) to store and share quantitative biological dynamics data. SSBD currently provides 311 sets of quantitative data for single molecules, nuclei and whole organisms in a wide variety of model organisms from Escherichia coli to Mus musculus. The data are provided in Biological Dynamics Markup Language format and also through a REST API. In addition, SSBD provides 188 sets of time-lapse microscopy images from which the quantitative data were obtained and software tools for data visualization and analysis.Availability and Implementation: SSBD is accessible at http://ssbd.qbic.riken.jp.Contact:sonami@riken.jp

Hewlett-Packard's marketing research over the past several years reveals that the chief concern of computer-aided test (CAT) managers is to reduce the cost and effort of program development. CAT software tools and techniques for rapidly developing high-performance test programs at lower costs and with less effort are discussed. Functional test manager (FTM) and interactive test generator (ITG) software packages work together to provide a very easy functional test software solution. FTM provides a wide range of capability designed to help automate functional test applications. ITG is a very easy-to-use tool for generating instrument control and measurement procedure software. It makes it possible to generate the instrument control portion of functional test programs without having to search through manuals looking for instrument control mnemonics.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>