Effective multiple feature fusion using topic model for social image visualization

<p indent="0mm">Volunteer geographic data, especially from social media, has emerged as a crucial resource for disaster mitigation. Particularly the included imagery can enhance situational awareness during emergencies. However, current applications of this data for disaster relief are primarily limited by a scarcity of annotated images for training, which hinders the efficacy of techniques like machine learning and deep learning, consequently restricting the data′s capacity to convey disaster information. Starting from the perspective of practical application, this paper focuses on earthquake disasters as the research subject. By integrating data acquisition, automatic deduplication, semantic annotation, and data augmentation methods, we developed a dataset of semantic segmentation in earthquake disaster detection based on social media images. This dataset mainly consists of original images collected from social media and pseudo-color images with manual semantic annotations. To ensure data quality, multiple annotators work in rotation and cross-check each other′s work. This dataset plays a significant role in improving the efficiency of social media data usage and enhancing awareness of disaster reduction efforts.

The influence of architectural design on well-being is increasingly recognised, underscoring the need to understand how humans evaluate their surroundings. This study examines the impact of low-level visual features and high-level architectural elements in images of house facades generated by a generative adversarial network (GAN). We introduce a novel method to create controlled image datasets using the style mixing component of StyleGAN2-ADA, enabling the combination of specific architectural features with styles representing low-level image properties, such as brightness or colour. Our dataset consists of 900 images encompassing five high-level features (house size, garage door visibility, number of windows, entrance door visibility, and roof shape) and 16 low-level visual features. These GAN-generated images were rated in an online experiment by 303 participants on six dimensions: facelikeness, hominess, relaxation, invitingness, safety, and price. Linear mixed-effects models identified significant predictors, including the number of floors as a high-level feature and green pixel percentage, saturation (SD), brightness (SD), and contrast as low-level features. This indicated that houses with two floors and images with an overall higher amount of green pixel, as well as diversity of saturation and brightness and a lower contrast were rated as, e.g., more inviting and safer. The analysis showed that high- and low-level features explained up to 54% of the variance in house price ratings, with high-level features accounting for the larger share and highlighting their relevance for house facade evaluations. This work advances the use of GANs and offers a method to modify visual elements that shape architectural evaluations.

An image is worth a thousand words, conveying information that goes beyond the mere visual content therein. In this paper, we study the intent behind social media images with an aim to analyze how visual information can facilitate recognition of human intent. Towards this goal, we introduce an intent dataset, Intentonomy, comprising 14K images covering a wide range of everyday scenes. These images are manually annotated with 28 intent categories derived from a social psychology taxonomy. We then systematically study whether, and to what extent, commonly used visual information, i.e., object and context, contribute to human motive understanding. Based on our findings, we conduct further study to quantify the effect of attending to object and context classes as well as textual information in the form of hashtags when training an intent classifier. Our results quantitatively and qualitatively shed light on how visual and textual information can produce observable effects when predicting intent. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>

The popularity of social media has grown considerably over the past few years and has taken a foothold in society. This study examined the connection between extensive social media use and the way people view their bodies in relation to what is seen in the images and messages portrayed on social media. Emphasis is placed on social media and whether it has a negative impact on body image and if it leads to body dissatisfaction. A mixed method approach was taken to collect data on females residing in Tobago and how they perceive body image. Survey participation was both voluntary and anonymous. Participants were females above the age of 15. Additionally, focus groups were conducted among Tobago women ranging between ages 16- 56 to further supplement the statistical data collected. The study and the data collected examined how social media influenced the way that women view themselves. The data collected and the research took into consideration how the participants were affected by their exposure to the various social media platforms and their messages. While the information gathered shows there is a correlation between social media and negative body image, there is no evidence to prove that social media has a direct impact on one’s body image.

Social media is a routine part of every-day life for millions of people worldwide. How does engaging with social media shape enduring memories for that experience? This question is important given the popularity of certain types of content on social media platforms, such as content widely known as “fitspiration”. Two experiments involving 510 US adults (mean age = 36.82) examined memory for food and fitness-related social media images that individuals write comments about, as well as memory for other images in the context. We demonstrate that commenting on social media images boosts memory for them and weakly affects memory for conceptually related images in the same context. Exploratory analyses revealed correlations between self-reported disordered eating symptomology and effects of commenting on memory. These findings demonstrate that how people engage with social media has implications for the enduring memories of that content and may relate to behaviors and attitudes in offline lives, such as eating and body image.

Floods are among the most frequent and catastrophic natural disasters and affect millions of people worldwide. It is important to create accurate flood maps to plan (offline) and conduct (real-time) flood mitigation and flood rescue operations. Arguably, images collected from social media can provide useful information for that task, which would otherwise be unavailable. We introduce a computer vision system that estimates water depth from social media images taken during flooding events, in order to build flood maps in (near) real-time. We propose a multi-task (deep) learning approach, where a model is trained using both a regression and a pairwise ranking loss. Our approach is motivated by the observation that a main bottleneck for image-based flood level estimation is training data: it is difficult and requires a lot of effort to annotate uncontrolled images with the correct water depth. We demonstrate how to efficiently learn a predictor from a small set of annotated water levels and a larger set of weaker annotations that only indicate in which of two images the water level is higher, and are much easier to obtain. Moreover, we provide a new dataset, named DeepFlood, with 8145 annotated ground-level images, and show that the proposed multi-task approach can predict the water level from a single, crowd-sourced image with ≈11 cm root mean square error.

Urban land use on a building instance level is crucial geo-information for many applications yet challenging to obtain. Steet-level images are highly suited to predict building functions as the building façades provide clear hints. Social media image platforms contain billions of images, including but not limited to street perspectives. This study proposes a filtering pipeline to yield high-quality, ground-level imagery from large-scale social media image datasets to cope with this issue. The pipeline ensures all resulting images have complete and valid geotags with a compass direction to relate image content and spatial objects.We analyze our method on a culturally diverse social media dataset from Flickr with more than 28 million images from 42 cities worldwide. The obtained dataset is then evaluated in the context of a building function classification task with three classes: Commercial, residential, and other. Fine-tuned state-of-the-art architectures yield F1 scores of up to 0.51 on the filtered images. Our analysis shows that the quality of the labels from OpenStreetMap limits the performance. Human-validated labels increase the F1 score by 0.2. Therefore, we consider these labels weak and publish the resulting images from our pipeline and the depicted buildings as a weakly labeled dataset.

Image retrieval is one of the most interesting and fastest growing research areas in all fields. It is an effective and efficient tool for managing large image databases. In most Content-Based Image Retrieval (CBIR) systems, an image is represented by a set of low-level visual features; hence a direct correlation with high-level semantic information will be absent. Therefore, a gap exists between high-level information and low-level features, which is the main reason that hinders the improvement of the image retrieval accuracy. In this work, main focus is on the semantic based image retrieval system using Gray Level Co-occurrence Matrix (GLCM) for texture feature extraction. Based on the texture features, semantic interpretations are given to the extracted textures. The images are retrieved according to user satisfaction and thereby reduce the semantic gap between low level features and high level features.

In an article in this issue of Sex Roles, I proposed a transactional model of social media effects on body image concerns that emphasizes the role played by individual vulnerability characteristics, gratifications sought from social media, and a host of mediating psychological processes. Commentators offered thoughtful responses to the model, pointing to strengths, but also targeting a number of areas for additional emphasis and ameliorative attention. This article, with a focus on the U.S. context, reviews each of the papers. It summarizes their main suggestions, and pulls together the host of forward-looking ideas, including the role played by cultural forces, the interplay between social and conventional mass media, and an emphasis on the salutary effects of social media on body image processes.

In our paper we tried to classify food images using convolutional neural network. Convolutional neural network extracts spatial features from images so it is very efficient to use convolutional neural network for image clasification problem. Recently people are sharing food images in social media and writing review on food. So there is a lot of food image in the social media but some image may not be labeled. It will be very helpful for restaurants if they can advertise their food to those people who is looking similar kind of foods they offer. Food classification system can help social media platform to identify food. Food classification system can enable an opportunity for social media platform to offer advertisement service for restaurants and beverage companies to their targeted users. It will be financially beneficial for both social media platform and beverage companies. Food classification is very difficult task because there is high variance in same category of food images. We developed a convolutional neural network model to classify food images in food-11 dataset. We also used a pre-trained Inception V3 convolutional neural network model to classify food images.

This article examines how political leaders use social media to shape their image during elections. Social media and digital platforms have changed political communication, allowing politicians to craft their image carefully. Social media has changed politics, yet misinformation and public opinion manipulation are concerns. Politicians' online avatars may misrepresent their goals and policies, overshadowing public policy discussions. This study uses a comprehensive analysis of the literature from the Scopus database to analyze how political elites shape their social media image during elections in different nations. This research examines how political leaders use social media to shape their public persona during election campaigns in several countries. The results will reveal political elites' social media image-building strategies during elections, including personalized methods, visual cues' impact on engagement, and preferred platforms for visual communication. This extensive study of the long-term consequences of visual communication tactics shows that politicians’ use of social media to convey stories has a lasting impact on public perception and engagement in politics.

Purpose/Rationale The study examines how consumers look at sponsorship messaging embedded within Instagram images on the International Volleyball Federation’s official beach volleyball account, investigating how social media consumers dedicate visual attention to the photographs. Design The study followed an experimental design using eye-tracking technology with a within-subjects protocol. Researchers recorded the gaze behavior of participants (n = 41) on social media images (n = 18) featuring beach volleyball athletes who played on the 2023 Volleyball World Beach Pro Tour Elite16. Findings The study’s results revealed that consumers dedicated most of their visual attention to the athletes in the images compared to the sponsorship messaging. In addition, participants were less likely to look at sponsors when the athletes were in defensive positions compared to offensive positions or posed positions. Practical implications The lack of visual attention to sponsorship messaging reinforces the need for social media marketers to be strategic in how they select images for Instagram and incorporate sponsors into their social media feeds. Research contribution Results highlight the need to better understand how consumers dedicate visual attention to sponsors embedded within social media posts. Originality/Value This article widens the methodological diversity in sports and social media research by using eye-tracking technology.

Promises and pitfalls of using computer vision to make inferences about landscape preferences: Evidence from an urban-proximate park system

Disaster response preparedness, risk mitigation, and effective post-event response are critical for national security, public health, and economic integrity. For areas impacted by extreme events, timely and accurate impact assessments of lifeline infrastructures are critical. Traditional disaster management and situational awareness platforms are not capable of handling the massive influx of available information from traditional and non-traditional sources. Next-generation analytic assessment approaches, where use of dynamically sourced, heterogeneous data with near real-time availability are integrated into a continually-updated analytics framework. The objective of this research was to develop a foundational multi-formalism modeling platform to integrate novel data sources for use in improving situational awareness after a disaster event. Specifically, to 1) provide an architecture for machine-learning based data fusion various multi-sensor datasets; 2) classify, geolocate, and temporally bin novel data sources for use in spatially-enabled machine-learning based data fusion; and 3) derive confidence-based observation data with machine-learning based prediction to fill spatial and temporal observation gaps. This project developed a reusable cloud-based pipeline architecture for ingesting, storing, and analyzing relevant data, executing a range of deep-learning models, and visualize the information geographically. The pipeline development was informed by previous PNNL investments (StreamKit) and built upon leading-edge technologies consisting of: 1) a raw datastore; 2) a data enrichment pipeline with custom and third-party analytics; 3) enriched data storage; and 4) a custom UI to display the enriched data. Machine learning/deep learning models were selected and tested to classify social media images for relevancy and presence of flood waters. Our custom trained relevance models were run on a validation dataset and the overall accuracy of the relevance classifiers ranged from 65% to 90%. We found that convolutional neural network (CNN) models were both faster and more accurate than more traditional machine learning models. We applied the relevance classifier to remove non-relevant images and then undertook a limited training of CNN models to detect flood on ~1,800 labeled images. The testing accuracy of the flood detection models were ~88%. To improve the geographic location of both social media and oblique aerial images we tested a multi-step workflow built on a pre-trained deep-learning model. To identify similarity features from test and reference scenes, we applied the Xception and Faiss models to a set of test images with inaccurate spatial locations, as well and precisely-located reference scenes from the general area. Matching reference scenes to the test scenes with the closest similarity index had limited success in improving the precision of test scene location and additional methods have been identified for testing. Data fusion between authoritative geographic data (satellite imaging) and non-authoritative crowdsourced data is an underlying theme throughout this research and methods are tested bring these data to a common geographic form through feature matching and transformations. The automated extraction of supporting information for disaster response from oblique aerial images or ground-level social media images is accomplished through semantic image segmentation and transformation. The non-authoritative data are used to validate the authoritative data collections and processing and optionally, t+0 flood models. In addition, the common form geographic data are combined with other static landscape features to develop spatially-explicit training vectors used in deep generative statistical inference models to develop complex non-linear relationships in static and dynamic data and ultimately derive a spatially continuous and probabilistic assessment of flood damage with the event domain.

With the development of theories and technologies in medical imaging, most of the tumors can be detected in the early stage. However, the nature of ovarian cysts lacks accurate judgement, leading to that many patients with benign nodules still need Fine Needle Aspiration (FNA) biopsies or surgeries, increasing the physical pain and mental pressure of patients as well as unnecessary medical health care costs. Therefore, we present an image diagnosis system for classifying the ovarian cysts in color ultrasound images, which novelly applies the image features fused by both high-level features from deep learning network and low-level features from texture descriptor. Firstly, the ultrasound images are enhanced to improve the quality of training data set and the rotation invariant uniform local binary pattern (ULBP) features are extracted from each of the images as the low-level texture features. Then the high-level deep features extracted by the fine-tuned GoogLeNet neural network and the low-level ULBP features are normalized and cascaded as one fusion feature that can represent both the semantic context and the texture patterns distributed in the image. Finally, the fusion features are input to the Cost-sensitive Random Forest classifier to classify the images into "malignant" and "benign". The high-level features extracted by the deep neural network from the medical ultrasound image can reflect the visual features of the lesion region, while the low-level texture features can describe the edges, direction and distribution of intensities. Experimental results indicate that the combination of the two types of features can describe the differences between the lesion regions and other regions, and the differences between lesions regions of malignant and benign ovarian cysts.