CanvasEmb: Learning Layout Representation with Large-scale Pre-training for Graphic Design

It is expensive and time-consuming to obtain a large number of labeled synthetic aperture radar (SAR) images. In the task of small training data size, the results of target detection on SAR images using deep network approaches are usually not ideal. In this study, considering that optical remote sensing images are much easier to be labeled than SAR images, we assume to have a large number of labeled optical remote sensing images and a small number of labeled SAR images with the similar scenes, propose to transfer knowledge from optical remote sensing images to SAR images, and develop a domain adaptive Faster R-CNN for SAR target detection with small training data size. In the proposed method, in order to make full use of the label information and realize more accurate domain adaptation knowledge transfer, an instance level domain adaptation constraint is used rather than feature level domain adaptation constraint. Specifically, generative adversarial network (GAN) constraint is applied as the domain adaptation constraint in the adaptation module after the proposals of Faster R-CNN to achieve instance level domain adaptation and learn the transferable features. The experimental results on the measured SAR image dataset show that the proposed method has higher detection accuracy in the task of SAR target detection with small training data size than the traditional Faster R-CNN.

Classification of vehicles is one of the most important tasks in intelligent transportation systems (ITS). While there are various types of sensors for measuring vehicle characteristics, this paper is focused on an image-based vehicle classification system. Most traditional approaches for image-based vehicle classification are computationally extensive and typically require a large amount of data for model training. This paper investigates whether it is possible to transfer the learning of a highly accurate pre-trained model for classifying truck images based on body type. Results show that using a pre-trained model to extract low-level features of images increases the accuracy of the model significantly, even with a relatively small size of training data. Furthermore, a convolutional neural network (CNN) is shown to outperform other types of models to classify trucks based on the extracted features.

The aim of this research is maximizing differentiation quality of skin neoplasms by Raman Spectroscopy (RS) and Autofluorescence (AF) using conventional Machine Learning (ML) algorithms (which means excluding Neural Networks). Thus, a basic task of this research consists of making and testing ML algorithm ensembles on Raman spectra, that were obtained in vivo in Samara Oncology Clinical Center. The data (spectra) have been obtained in a form of text files containing the identifier of a patient, as well as a Raman spectrum in the form of pair values – a wavelength and an appropriate value. All data (964 spectra) has been divided for two classes: Tumor and Skin. Further preprocessing of the input data and the analysis of models of machine learning for a problem of classification have been carried out. We used the following ML tools, namely Python 3.7.3, an open source ML libraries Scikit-learn v0.21.2, NumPy 1.17.0, and Pandas 1.0.1, IDE Anaconda Enterprise 5.3 and cloud service Google Colaboratory with an interactive environment Jupyter Notebook. Machine learning models that show a high accuracy result include Classification and Regression Tree (CART), Support Vector Classification (SVC), Logistic Regression (LR), K-nearest neighbors algorithm (KNN). These classifiers show high quality classification on standard parameters already. The Soft Voting Classification module was selected as an ensemble, that allows us to use several models of classifiers that are not similar to each other at once, combine them into one classifier. Results of this ensemble testing from these ML algorithms showed that the classification accuracy, unlike the best qualifier, has not been increased. However, the metrics of classification quality show that the model has become stabler and steady. Results: specificity - 93%, sensitivity - 88, harmonic mean between precision and recall (F1 score) - 90%. An analysis of validation and training curves indicated a small size of training data and, for some cases, a high complexity of the model, which led to a decrease in the classification accuracy.

Digital breast tomosynthesis (DBT) is a pseudo-3D breast image, which is a collection of 2D slice images. It is increasingly being used for detection and diagnosis of breast cancer. In comparison to mammography (2D breast image), DBT provides higher sensitivity (true-positive rate). Recently, there have been many state-of-the-art methods for the detection of masses and calcifications in DBT. However, these previous studies can identify just only one type of breast lesions (abnormality class), while there can be multiple breast lesions simultaneously (multi-label classification); each of them requires different treatment procedures. In this paper, we present an end-to-end multi-label classification approach that takes into account the pseudo-3D nature of DBT and able to automatically detect two major types of malignant lesions: soft tissue and calcifications. The proposed network is a real 3D convolutional network (ConvNet) with two additional strategies: global kernel and global average pooling. Also, an inversion augmentation is invented; it does not only alleviate a small size of training data, but also help an occlusion overlapping issue. Such system can be used to support radiologists in DBT analysis by prompting suspicious locations. Our in-house dataset consists of 115 DBT volumes, including 91 volumes of cancer detected cases (contained biopsy-proven malignant lesions) and 24 volumes of normal cases. The experimental results show that our approach yields a promising result for the classification of malignant lesions: 72% accuracy with f1-score at 0.842. Moreover, a multi-label classification network is able to detect concurrent lesions in 3 of 6 volumes in the testing set.

Traditionally, the normalized cross correlation (NCC) based or shape based template matching methods are utilized in machine vision to locate an object for a robot pick and place or other automatic equipment. For stability, well-designed LED lighting must be mounted to uniform and stabilize lighting condition. Even so, these algorithms are not robust to detect the small, blurred, or large deformed target in industrial environment. In this paper, we propose a convolutional neural network (CNN) based object localization method, called C-CNN: cascaded convolutional neural network, to overcome the disadvantages of the conventional methods. Our C-CNN method first applies a shallow CNN densely scanning the whole image, most of the background regions are rejected by the network. Then two CNNs are adopted to further evaluate the passed windows and the windows around. A relatively deep model net-4 is applied to adjust the passed windows at last and the adjusted windows are regarded as final positions. The experimental results show that our method can achieve real time detection at the rate of 14FPS and be robust with a small size of training data. The detection accuracy is much higher than traditional methods and state-of-the-art methods.

Authorship attribution refers to the task of identifying the authors of a set of documents. Early studies in this area either used book length texts or assumed that there were a large number of training documents. The focus of modern authorship attribution has been shifted to the analysis on small online texts. This is realistic since in the real life it is hard to collect the training texts. However, the small size of training data makes the authorship attribution much more difficult. In this paper, we present a novel co-training method to iteratively recognize a few unlabeled data to augment the training set. Specifically, each document is first partitioned into two distinct views, i.e., lexical and syntactic view. And then, a two view semi-supervised method, co-training, is adopted to exploit the large amount of unlabeled documents. Our experiment results based on real data show that the proposed method can effectively exploit unlabeled data to improve the classification performance.

Pre-training and fine-tuning have become the de facto paradigm in many natural language processing (NLP) tasks. However, compared to other NLP tasks, neural machine translation (NMT) aims to generate target language sentences through the contextual representation from the source language counterparts. This characteristic means the optimization objective of NMT is far from that of the universal pre-trained models (PTMs), leading to the standard procedure of pre-training and fine-tuning does not work well in NMT. In this paper, we propose a novel framework to deep fuse the pre-trained representation into NMT, fully exploring the potential of PTMs in NMT. Specifically, we directly replace the randomly initialized Transformer encoder with a pre-trained encoder and propose a layer-wise coordination structure to coordinate PTM and NMT decoder learning. Then, we introduce a partitioned multi-task learning method to fine-tune the pre-trained parameter, reducing the gap between PTM and NMT by progressively learning the task-specific representation. Experimental results show that our approach achieves considerable improvements on WMT14 En2De, WMT14 En2Fr, and WMT16 Ro2En translation benchmarks and outperforms previous work in both autoregressive and non-autoregressive NMT models.

In this paper we explore the capabilities of two state-of-the-art machine learning techniques, transfer learning with convolutional neural networks (CNN) and support vector machines (SVM) to distinguish between 10 cloud genera. We will evaluate these methods using images acquired by the satellite Landsat 8. The classification of cloud genera is of high general relevance for remote sensing applications such as the surveillance of atmospheric or meteorological processes. Transfer learning is of advantage because it exploits neural networks, which are known to perform well, and enables the adaption to a specific problem with relatively small training data size. We will utilize Landsat 8 images for evaluating the examined machine learning approaches because these image data are freely available in large amounts. Downscaling the Landsat 8 images utilized for training to a resolution of about 300 meters per pixel will allow for keeping the CNN and SVM size reasonably low, such that our training data set can be restricted to a moderate size.

Recently it has shown that the policy-gradient methods for reinforcement learning have been utilized to train deep end-to-end systems on natural language processing tasks. What's more, with the complexity of understanding image content and diverse ways of describing image content in natural language, image captioning has been a challenging problem to deal with. To the best of our knowledge, most state-of-the-art methods follow a pattern of sequential model, such as recurrent neural networks (RNN). However, in this paper, we propose a novel architecture for image captioning with deep reinforcement learning to optimize image captioning tasks. We utilize two networks called "policy network" and "value network" to collaboratively generate the captions of images. The experiments are conducted on Microsoft COCO dataset, and the experimental results have verified the effectiveness of the proposed method.

Various exquisite graphic designs that can be seen everywhere beautify people’s vision and influence people’s visual aesthetics subtly. This paper starts with the visual aesthetic elements of graphic design and discusses the construction of visual aesthetic element systems in graphic design by analyzing the composition elements and aesthetic characteristics. Secondly, it proposes to use the biological visual perceptual machine model to process information through two visual pathways in a hierarchical manner. The features of graphic design images are expressed, stored, and extracted using the visual information model. Finally, the effectiveness of the bio-visual perceptual model applied to the construction of visual aesthetic elements of graphic design is verified by the Caltech5 database, Caltech256 database, and Scene15 database. The results show that the classification accuracy of the bio-visual perceptron model is 94.28%, 91.36%, and 99.75% in Caltech5, Caltech256, and Scene15 databases, respectively, which is 23.12% higher than the accuracy of other neural network models on average. The bio-visual perceptron model proposed in this paper can accurately detect the elemental features in complex graphic design, which provides a new idea and method for rapid detection and recognition for graphic design workers.

This study explores the role of language as a visual strategy in the construction of graphic product identity within the creative industry. In the context of visual communication design, language is commonly understood as a verbal element used to convey information. However, in contemporary graphic product design, language undergoes a transformation into a visual element that contributes to the formation of visual identity, branding, and product differentiation. This research aims to analyze how language functions as a visual strategy in graphic product design, particularly in shaping product identity and strengthening its position in the creative industry. The study employs a mixed-methods approach by combining qualitative visual analysis and quantitative consumer perception analysis. The qualitative approach is used to examine typographic forms, visual composition, and the integration of language with other visual elements, while the quantitative approach is used to measure consumer perceptions of visual identity, product uniqueness, and purchase intention. The findings indicate that language, when treated as a visual element, plays a significant role in constructing graphic product identity. Through typographic exploration and visual composition, language contributes to the creation of visual uniqueness, cultural representation, and product differentiation. Furthermore, the use of language enhances consumer perception in terms of visual appeal, identity recognition, and emotional connection to cultural values. This study contributes to the development of visual communication design by proposing the concept of language as a visual strategy in graphic product identity. It highlights the role of graphic product design not only as a visual artifact but also as a strategic medium for identity construction, branding, and differentiation in the creative industry.

Relation classification is one of the important tasks in natural language processing, which aims to determine the relationship between entities given a sentence and their respective positions. Most of the existing methods for relation classification are based on neural networks using pre-trained models such as BERT. In recent years, models based on pre-trained models like BERT have achieved excellent results in relation classification tasks in general domains. However, these methods often struggle when applied to specialized domains due to the limitations of the corpora used for BERT pre-training. Most pre-trained models are trained on text corpora from general sources like Wikipedia, which cover a wide range of domains. As a result, the content of these corpora in specific domains is limited and lacks the necessary expertise, leading to subpar performance of relation classification models in specific domains. While providing a large number of domain-specific corpora to pre-trained models could potentially address this issue, it comes with limitations such as increased computational requirements and insufficient training of specialized vocabulary. This paper proposes a method inspired by the K-BERT pre-training model to incorporate triplet knowledge from domain knowledge graphs into sentence sequences. The triplets are transformed into sentence trees and then fed into the BERT pre-trained model using absolute and relative indices. Our model has achieved an accuracy of 93.06%, which is markedly higher than that of any other baseline approach. This approach allows the incorporation of domain knowledge without significantly increasing the computational complexity. Additionally, the paper introduces a partial input method that enables the computer to understand input sentences from multiple dimensions and hierarchical levels. Experimental results on a medical domain dataset for relation classification, which includes type labels, demonstrate significant advantages over other relation classification models in terms of Accuracy. Received: 24 January 2024 | Revised: 19 March 2024 | Accepted: 3 April 2024 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in [Github] at https://github.com/Soso6666/Classification-data. Author Contribution Statement Boxuan Chen: Methodology, Software, Validation, Resources, Data curation, Writing - original draft, Visualization. Guan Yuan: Conceptualization, Methodology, Formal analysis, Investigation, Resources, Data curation, Writing - review & editing, Supervision, Project administration.

Cross-lingual transfer, where a high-resource transfer language is used to improve the accuracy of a low-resource task language, is now an invaluable tool for improving performance of natural language processing (NLP) on low-resource languages. However, given a particular task language, it is not clear which language to transfer from, and the standard strategy is to select languages based on ad hoc criteria, usually the intuition of the experimenter. Since a large number of features contribute to the success of cross-lingual transfer (including phylogenetic similarity, typological properties, lexical overlap, or size of available data), even the most enlightened experimenter rarely considers all these factors for the particular task at hand. In this paper, we consider this task of automatically selecting optimal transfer languages as a ranking problem, and build models that consider the aforementioned features to perform this prediction. In experiments on representative NLP tasks, we demonstrate that our model predicts good transfer languages much better than ad hoc baselines considering single features in isolation, and glean insights on what features are most informative for each different NLP tasks, which may inform future ad hoc selection even without use of our method. Code, data, and pre-trained models are available at https://github.com/neulab/langrank

In recent years, deep learning technology has been widely used and developed. In natural language processing tasks, pre-training models have been more widely used. Whether it is sentence extraction or sentiment analysis of text, the pre-training model plays a very important role. The use of a large-scale corpus for unsupervised pre-training of models has proven to be an excellent and effective way to provide models. This article summarizes the existing pre-training models and sorts out the improved models and processing methods of the relatively new pre-training models, and finally summarizes the challenges and prospects of the current pre-training models.

The purpose of the study is to identify, systematise and critically analyse the psycho-visual mechanisms used in contemporary graphic design to manage perception in conditions of information overload. Research methodology. An interdisciplinary approach was used, combining the principles of cognitive psychology, neuroaesthetics, Gestalt theory and design analytics. The primary research methods include: critical visual analysis, cognitive-perceptual interpretation, philosophical conceptualisation of visual experience, and content analysis of examples of visual communication in conditions of information saturation. Scientific novelty. The article offers an interdisciplinary analysis of psycho-visual mechanisms for managing perception through graphic design in conditions of information overload, examining the intersection of cognitive aesthetics and visual semiotics. The application of psycho-visual consistency as a factor in effective communication in overloaded conditions is clarified. A typology of cognitive functions of forms is proposed, which allows visual elements to be considered as carriers of perceptual influence. Psycho-visual mechanisms of perception control, facilitated by graphic design, are generalised in conditions of information overload. The results are of practical importance for the design of visual communications, can be used as analytical tools in the creation of visual strategies, and can be incorporated into the training of future graphic designers. Conclusions. Research into the psycho-visual mechanisms of perception control through graphic design confirms their leading role in structuring visual information in conditions of information overload. It is essential to focus not on visual complexity, but on the informativeness and controllability of attention, which provide grounds for creating adaptive and intuitive design solutions. The graphic designer acts as an intermediary between complex information flows and the user’s cognitive abilities, while graphic design performs regulatory and meaning-making functions. The effectiveness of graphic design depends on the graphic designer’s ability to apply Gestalt principles, visual accents, rhythm, and compositional balance, which ensure cognitive economy and facilitate perception.