Crowdsourcing Without a Crowd

The idea of the wisdom of the crowd is that integrating multiple estimates of a group of individuals provides an outcome that is often better than most of the underlying estimates or even better than the best individual estimate. In this paper, we examine the wisdom of the crowd principle on the example of spatial data collection by paid crowdworkers. We developed a web-based user interface for the collection of vehicles from rasterized shadings derived from 3D point clouds and executed different data collection campaigns on the crowdsourcing marketplace microWorkers. Our main question is: how large must be the crowd in order that the quality of the outcome fulfils the quality requirements of a specific application? To answer this question, we computed precision, recall, F1 score, and geometric quality measures for different crowd sizes. We found that increasing the crowd size improves the quality of the outcome. This improvement is quite large at the beginning and gradually decreases with larger crowd sizes. These findings confirm the wisdom of the crowd principle and help to find an optimum number of the crowd size that is in the end a compromise between data quality, and cost and time required to perform the data collection.

A HIDDEN MARKOV MODEL’S AGREEMENT WITH CLINICAL DIAGNOSES AND ITS INDICATION OF ADDITIONAL PRECLINICAL COGNITIVE DEFICITS IN THE WISCONSIN REGISTRY FOR ALZHEIMER’S PREVENTION

Galaxy Zoo: Clump Scout is a web-based citizen science project designed to identify and spatially locate giant star forming clumps in galaxies that were imaged by the Sloan Digital Sky Survey Legacy Survey. We present a statistically driven software framework that is designed to aggregate two-dimensional annotations of clump locations provided by multiple independent Galaxy Zoo: Clump Scout volunteers and generate a consensus label that identifies the locations of probable clumps within each galaxy. The statistical model our framework is based on allows us to assign false-positive probabilities to each of the clumps we identify, to estimate the skill levels of each of the volunteers who contribute to Galaxy Zoo: Clump Scout and also to quantitatively assess the reliability of the consensus labels that are derived for each subject. We apply our framework to a data set containing 3561 454 two-dimensional points, which constitute 1739 259 annotations of 85 286 distinct subjects provided by 20 999 volunteers. Using this data set, we identify 128 100 potential clumps distributed among 44 126 galaxies. This data set can be used to study the prevalence and demographics of giant star forming clumps in low-redshift galaxies. The code for our aggregation software framework is publicly available at: https://github.com/ou-astrophysics/BoxAggregator

Crowd accidents – defined as situations where mass gatherings of people lead to deaths or injuries – have become a frequent occurrence on a global scale. Given the recurring nature of these accidents, it is essential that their characteristics are analyzed. To this end, an important step would be documenting these records. Here, a database of crowd accidents is developed for the period of 1900–2019 through a comprehensive investigation of the press and media reports. The analyses focus mainly on temporal trends of their frequency and injury/casualty in each accident, as well as their geographical distribution and classification based on the purpose of gathering. Results show that the frequency of crowd accidents has been unambiguously on the rise over the last 120 years. Also, there was no indication that larger crowd sizes increase the risk of injury or death per person. In fact, the opposite was the case, although a causal relation between crowd size and risk of injury/death is impossible to establish. Over time, the share of sport events in crowd accidents has declined, and instead, religious gatherings have become more notably present in the statistics. An interesting observation is the association of accident rates to the income level of the countries where they happen, with low-and-middle-income countries being more represented in the records. India and (to a lesser extent) West Africa, in particular, appear to be hot spots for crowd accidents. Finally, it is argued that the exponential increase in crowd accidents of the last century was only partially real, with technology also playing a role in making information more accessible for recent accidents. After the internet (and SNS) became widespread, the trend for reported crowd accidents does not show anymore an exponential increase although it is difficult to conclude whether their frequency is stable or not. The insights obtained from this study can pave the way for developing diagnostic knowledge and raising awareness about the ubiquity of crowd accidents.

Accurate spatiotemporal information on crowds is a necessity for a better management in general and for the mitigation of potential security risks. The large numbers of individuals involved and their mobility, however, make generation of this information non-trivial. This paper proposes a novel methodology to estimate and map crowd sizes using mobile Bluetooth sensors and examines to what extent this methodology represents a valuable alternative to existing traditional crowd density estimation methods. The proposed methodology is applied in a unique case study that uses Bluetooth technology for the mobile mapping of spectators of the Tour of Flanders 2011 road cycling race. The locations of nearly 16,000 cell phones of spectators along the race course were registered and detailed views of the spatiotemporal distribution of the crowd were generated. Comparison with visual head counts from camera footage delivered a detection ratio of 13.0 ± 2.3%, making it possible to estimate the crowd size. To our knowledge, this is the first study that uses mobile Bluetooth sensors to count and map a crowd over space and time.

We present an experimental investigation of the pressure dynamics during the flow of self-propelled particles through narrow passages. When the ensemble is flowing, pressure fluctuates around a constant value that does not depend on the crowd size, suggesting that the orifice locally determines the dynamics in this scenario. On the contrary, when the system clogs, pressures are higher for larger crowd sizes, highlighting the importance of the whole collectivity in the process. Then, by correlating the pressure evolution with the exit time of the self-propelled particles, we discover that when a clog is resolved, pressure suddenly drops as a consequence of system reorganization. After this dramatic event, there is a sustained pressure growth over time that shows a square-root dependence, compatible with the structural aging that has been proposed to be behind the broad tail distributions of clogging times.

Introduction In general, the suitability of spatial (GIS) environmental data sources for analytical use in ecological modeling (Hunsaker et al ., 1993) has not been given as much attention as other aspect of the SDM problem, such as the species data (Chapter 4) and modeling methods (Chapters 6–8). A few studies have explicitly examined issues of spatial resolution and data quality of predictors on SDMs (Aspinall & Pearson, 1996), have shown that including satellite-derived climate variables with land cover as predictors improves SDMs (Suarez-Seoane et al ., 2004), and have illustrated the impact of predictors derived from digital elevation models on SDMs via error propagation (Van Niel & Austin, 2007). Daly (2006) provided guidelines for assessing the quality of interpolated climate maps used in environmental modeling. However, these efforts, while important, do not give comprehensive guidelines for selecting data to represent environmental gradients related to species distributions. Some software systems developed for SDM include global environmental datasets at 0.01- (mostly 0.5-) 1.0 degree resolution (Hijmans et al ., 2001; Stockwell, 2006), but this is the exception rather than the rule. Usually it is the job of the modeler to select and assemble appropriate predictor data, and consider issues of data quality and resolution; the effort involved is non-trivial. Chapter 3 outlined conceptual models of factors driving species distributions, and in this chapter, I will describe the types of environmental data typically used in SDM to represent those factors, including climate, topography, substrate, land cover and vegetation, disturbance maps, remote sensing-based land surface characterizations, measures of landscape pattern, and information about other species (biotic interactions).

Online prediction of network-level public transport demand based on principle component analysis

BackgroundData quality assessment is important but complex and task dependent. Identifying suitable measurement methods and reference ranges for assessing their results is challenging. Manually inspecting the measurement results and current data driven approaches for learning which results indicate data quality issues have considerable limitations, e.g. to identify task dependent thresholds for measurement results that indicate data quality issues.ObjectivesTo explore the applicability and potential benefits of a data driven approach to learn task dependent knowledge about suitable measurement methods and assessment of their results. Such knowledge could be useful for others to determine whether a local data stock is suitable for a given task.MethodsWe started by creating artificial data with previously defined data quality issues and applied a set of generic measurement methods on this data (e.g. a method to count the number of values in a certain variable or the mean value of the values). We trained decision trees on exported measurement methods’ results and corresponding outcome data (data that indicated the data’s suitability for a use case). For evaluation, we derived rules for potential measurement methods and reference values from the decision trees and compared these regarding their coverage of the true data quality issues artificially created in the dataset. Three researchers independently derived these rules. One with knowledge about present data quality issues and two without.ResultsOur self-trained decision trees were able to indicate rules for 12 of 19 previously defined data quality issues. Learned knowledge about measurement methods and their assessment was complementary to manual interpretation of measurement methods’ results.ConclusionsOur data driven approach derives sensible knowledge for task dependent data quality assessment and complements other current approaches. Based on labeled measurement methods’ results as training data, our approach successfully suggested applicable rules for checking data quality characteristics that determine whether a dataset is suitable for a given task.

Smart manufacturing applications increasingly rely on time-series data from Industrial IoT sensors, yet these data streams often contain data quality issues (DQIs) that affect analysis and disrupt production. While traditional Machine Learning methods are difficult to apply due to the small amount of data available, the knowledge-based approach of Case-Based Reasoning (CBR) offers a way to reuse previously gained experience. We introduce the first end-to-end Case-Based Reasoning (CBR) framework that both detects and remedies DQIs in near real time, even when only a handful of annotated fault instances are available. Our solution encodes expert experience in the four CBR knowledge containers: (i) a vocabulary that represents sensor streams and their context in the DataStream format; (ii) a case base populated with fault-annotated event logs; (iii) tailored similarity measures—including a weighted Dynamic Time Warping variant and structure-aware list mapping—that isolate the signatures of missing-value, missing-sensor, and time-shift errors; and (iv) lightweight adaptation rules that recommend concrete repair actions or, where appropriate, invoke automated imputation and alignment routines. A case study is used to examine and present the suitability of the approach for a specific application domain. Although the case study demonstrates only limited capabilities in identifying Data Quality Issues (DQIs), we aim to support transparent evaluation and future research by publishing (1) a prototype of the Case-Based Reasoning (CBR) system and (2) a publicly accessible, meticulously annotated sensor-log benchmark. Together, these resources provide a reproducible baseline and a modular foundation for advancing similarity metrics, expanding the DQI taxonomy, and enabling knowledge-intensive reasoning in IoT data quality management.

Chapter 11 - Stage 5: Consolidating Knowledge in Builds

Numerical data quality in IS research and the implications for replication

Harmonizing National Topographic Data Using an Automated Quality Validation Process

Due to the dramatically increasing amount of available data, effective and scalable solutions for data organization and search are essential. Distributed solutions naturally provide promising alternatives to standard centralized approaches. With the computational power of thousands or millions of computers in clusters or P2P systems, the challenges that arise are manifold, ranging from efficient resource discovery to load balancing issues and distributed ranking problems. This 2008 Large Scale Distributed Systems for Information Retrieval Workshop (LSDS-IR08) is a forum for researchers to discuss these problems and to define new directions for the work on distributed information retrieval. This year's program features ten papers on emerging work in the areas of search in structured and unstructured networks, network organization schemes, issues of data quality, search for different data types, result ranking techniques and query routing algorithms. In addition, our keynote, entitled "Search Engine Architectures from Conventional to P2P," will be delivered by Torsten Suel of Polytechnic University in New York.

In this article, we have investigated how researchers use the data provided by the Brazilian Information System of Primary Care . We also searched, for the first time, studies that evaluated the quality and reliability of the information provided by the Primary Care Information System. An integrative review of the literature was performed using the keywords 'information systems, primary care and SIAB' on search databases, and 53 of 174 articles were selected. These publications were classified into two large subgroups: those using the Primary Care Information System as 'data source' and those that took it as the 'object of study'. The first group included 35 studies, 18 of which used demographic and social health data records, and nine described data about diseases, specifically hypertension and diabetes. These data were used by researchers for association with health indicators (20%) or comparison with other information systems (17%), sample or population calculus (9%), estimation of prevalence and characterization of the epidemiological profile of a population (26%) or, more generally, to carry out the assessment of health status (29%). The Primary Care Information System as the 'object of study' group included 18 works, describing the knowledge and practices of professionals in relation to the information system. These researchers pointed out issues in the process of production and information consolidation, mainly due to the lack of training and supervision of community health workers and bureaucratization of their work process. Although some issues in the quality of data provided by the Primary Care Information System were reported by researchers, these findings were not corroborated by two studies that assessed the reliability of information disclosed by this system. Despite changes in the Brazilian health policies, the issue of data quality in health information systems continues to be a challenge preventing data from being used for decision-making and knowledge production.