Learning deep discriminative representations with pseudo supervision for image clustering

Increasing numbers of software vulnerabilities are discovered every year whether they are reported publicly or discovered internally in proprietary code. These vulnerabilities can pose serious risk of exploit and result in system compromise, information leaks, or denial of service. We leveraged the wealth of C and C++ open-source code available to develop a largescale function-level vulnerability detection system using machine learning. To supplement existing labeled vulnerability datasets, we compiled a vast dataset of millions of open-source functions and labeled it with carefully-selected findings from three different static analyzers that indicate potential exploits. Using these datasets, we developed a fast and scalable vulnerability detection tool based on deep feature representation learning that directly interprets lexed source code. We evaluated our tool on code from both real software packages and the NIST SATE IV benchmark dataset. Our results demonstrate that deep feature representation learning on source code is a promising approach for automated software vulnerability detection.

Recent research has shown the initial success of sparse coding (Sc) in solving many computer vision tasks. Motivated by the fact that kernel trick can capture the nonlinear similarity of features, which helps in finding a sparse representation of nonlinear features, we propose kernel sparse representation (KSR). Essentially, KSR is a sparse coding technique in a high dimensional feature space mapped by an implicit mapping function. We apply KSR to feature coding in image classification, face recognition, and kernel matrix approximation. More specifically, by incorporating KSR into spatial pyramid matching (SPM), we develop KSRSPM, which achieves a good performance for image classification. Moreover, KSR-based feature coding can be shown as a generalization of efficient match kernel and an extension of Sc-based SPM. We further show that our proposed KSR using a histogram intersection kernel (HIK) can be considered a soft assignment extension of HIK-based feature quantization in the feature coding process. Besides feature coding, comparing with sparse coding, KSR can learn more discriminative sparse codes and achieve higher accuracy for face recognition. Moreover, KSR can also be applied to kernel matrix approximation in large scale learning tasks, and it demonstrates its robustness to kernel matrix approximation, especially when a small fraction of the data is used. Extensive experimental results demonstrate promising results of KSR in image classification, face recognition, and kernel matrix approximation. All these applications prove the effectiveness of KSR in computer vision and machine learning tasks.

In this paper, we have proposed a novel feature representation on multi-scale receptive fields for objection recognition. The method is based on a modified convolutional neural networks (CNN), named network-in-network (NIN), which has shown a good performance in some computer vision tasks. However, applying NIN to some specific applications may encounter a few problems. First, the NIN removes the fully connected layers, which makes it unsuited to use in large-scale face recognition due to lack of an efficient feature representation, even though it brings a lot of performance benefits. Second, some lowerlayer features, which can make the feature representation more discriminative, is unused. In the pure forward architecture, these features are unseen to the classifier. To solve the two problems, we present a multi-scale receptive fields (MSRF) representation learning scheme. Based on a well trained NIN, we add a pathway to top layer and design a feature vector as final representation. In our experiments, we compare the result of our multi-scale receptive fields with standard NIN architecture. The results show our method can obtain a more explicit feature representation and improvements in performance.

Learning data representations directly from the data itself is an approach that has shown great success in different pattern recognition problems, outperforming state-of-the-art feature extraction schemes for different tasks in computer vision, speech recognition and natural language processing. Representation learning applies unsupervised and supervised machine learning methods to large amounts of data to find building-blocks that better represent the information in it. Digitized histopathology images represents a very good testbed for representation learning since it involves large amounts of high complex, visual data. This paper presents a comparative evaluation of different supervised and unsupervised representation learning architectures to specifically address open questions on what type of learning architectures (deep or shallow), type of learning (unsupervised or supervised) is optimal. In this paper we limit ourselves to addressing these questions in the context of distinguishing between anaplastic and non-anaplastic medulloblastomas from routine haematoxylin and eosin stained images. The unsupervised approaches evaluated were sparse autoencoders and topographic reconstruct independent component analysis, and the supervised approach was convolutional neural networks. Experimental results show that shallow architectures with more neurons are better than deeper architectures without taking into account local space invariances and that topographic constraints provide useful invariant features in scale and rotations for efficient tumor differentiation.

Visual computing in medicine involves handling the generation, acquisition, processing, analysis, exploration, visualization, and interpretation of medical visual information. Machine learning has become a prominent tool for data analytics and problem-solving, which is the process of enabling computers to automatically learn from data and obtain certain knowledge, patterns, or input–output relationships. The tasks involving visual computing in medicine often could be transformed into tasks of machine learning. In recent years, there has been a surge in research focusing on machine-learning-based visual computing. However, there are few reviews comprehensively introducing and surveying the systematic implementation of machine-learning-based vision computing in medicine, and in relevant reviews, little attention has been paid to the use of machine learning methods to transform medical visual computing tasks into data-driven learning problems with high-level feature representation, while exploring their effectiveness in key medical applications, such as image-guided surgery. This review paper addresses the above question and surveys fully and systematically the recent advancements, challenges, and future directions regarding machine-learning-based medical visual computing with high-level features. This paper is organized as follows. The fundamentals and paradigm of visual computing in medicine are first concisely introduced. Then, aspects of visual computing in medicine are delved into: (1) acquisition of visual information; (2) processing and analysis of visual information; (3) exploration and interpretation of visual information; and (4) image-guided surgery. In particular, this paper explores machine-learning-based methods and factors for visual computing tasks. Finally, the future prospects are discussed. In conclusion, this literature review on machine learning for visual computing in medicine showcases the diverse applications and advancements in this field.

目的 随着实际应用场景中海量数据采集技术的发展和数据标注成本的不断增加，自监督学习成为海量数据分析的一个重要策略。然而，如何从海量数据中抽取有用的监督信息，并该监督信息下开展有效的学习仍然是制约该方向发展的研究难点。为此，提出了一个基于共识图学习的自监督集成聚类框架。方法 框架主要包括3个功能模块。首先，利用集成学习中多个基学习器构建共识图；其次，利用图神经网络分析共识图，捕获节点优化表示和节点的聚类结构，并从聚类中挑选高置信度的节点子集及对应的类标签生成监督信息；再次，在此标签监督下，联合其他无标注样本更新集成成员基学习器。交替迭代上述功能块，最终提高无监督聚类的性能。结果 为验证该框架的有效性，在标准数据集（包括图像和文本数据）上设计了一系列实验。实验结果表明，所提方法在性能上一致优于现有聚类方法。尤其是在MNIST-Test （modified national institute of standards and technology database）上，本文方法实现了97.78%的准确率，比已有最佳方法高出3.85%。结论 该方法旨在利用图表示学习提升自监督学习中监督信息捕获的能力，监督信息的有效获取进一步强化了集成学习中成员构建的能力，最终提升了无监督海量数据本质结构的挖掘性能。;Objective Clustering is focused on machine learning-related data segmentation for multiple datasets. Its applications are in relevant to such domains like image segmentation and anomaly detection. In addition，to simplify complex tasks optimize its performance，clustering is used in data preprocessing tasks of those are data sub-blocks segmentation， pseudo-labels generation，and abnormal points-removal. Self-supervised learning has become an essential technique for massive data analysis. However，it is challenged to extract effective supervision information and analyze the input data. Method A consensus graph learning based self-supervised ensemble clustering（CGL-SEC）framework is developed. It consists of three main modules：1）to construct the consensus graph based on several ensemble components（i. e. ，the basic clustering methods）. 2）to extract the supervision information by learning the consensus graph representation，and 3）its node clustering results，where the subset of nodes with the high-confidence are selected as labeled samples. To optimize the ensemble components and the corresponding consensus graph，t basic clustering methods are re-trained in related the option of samples-labeled and other samples-unlabeled. The final clustering results can be optimized iteratively until the learning process converges. Result A series of experiments are carried out on benchmarks，including both image and textual datasets. Especially，CGL-SEC is 3. 85% over baseline in terms of clustering evaluation metric on themodified national institute of standards and technology database（MNIST-Test）. First，to optimize data representation and cluster assignment at the same time，deep embedding clustering can be focused on data itself as the supervision information and auto-encoder with the reconstruction loss is pre-trained. The soft cluster assignment of features-embedded is then calculated，and the KL（Kullback-Leibler）divergence is minimized between the soft cluster assignment and the auxiliary target distribution. To improve the performance of the model further，following deep clustering network（DCN）can use hard clustering instead of soft allocation，and local constraints are applied by improved deep embedding clustering（IDEC）. The pseudo-label strategy is implemented as a self-supervised learning method that uses the prediction results of the neural network as the label to simulate the supervision information compared to using data itself as the supervision information. Deepcluster-based K-means clustering is used to generate pseudo-labels to guide the training of convolutional networks. However，the generated pseudo-labels have lower confidence and are prone to trivial solutions in the initial stage of network training. Deep embedding clustering with data augmentation（DEC-DA）and MixMatch-based prediction of data-enhanced samples are used as the supervision information of the original data，which improves the accuracy of the supervision information to a certain extent，but this method is difficult to extend to text and other fields. Deep adaptive clustering-based high-confidence pseudo-label subsets-selected are iteratively trained the network in the prediction results，but lowconfidence samples-involved data distribution information is ignored. Pseudo-semi-supervised clustering votes are used to select a subset of high-confidence pseudo-labels，and all samples are used to train semi-supervised neural network. Although the ensemble strategy can improve the confidence of the pseudo-label，the voting strategy is concerned of category representation only without the feature representation of the sample itself，which can reduce the clustering performance in some cases. The ensemble learning is regarded as a representative machine learning method that reflects the ability of group intelligence，whereas a learning method can improve the overall prediction performance via multiple base learners training and their coordinated prediction results. In pseudo-label-based clustering tasks，it can coordinate multiple base learners to obtain high-confidence pseudo-labels. However，the effectiveness of the supervision information acquisition is still to be resolved. The category information of the sample is considered for current pseudo-label-based ensemble clustering method only when the label is captured and some effective information are ignored like the feature representation of the sample itself and the clustering structure between samples. Conclusion Graph neural network is composed of content information of nodes and the structural information between nodes at the same time. To design a self-supervised ensemble clustering method based on consensus graph representation learning，it is required to make full use of sample features and relationships between samples in ensemble learning. To obtain higher confidence pseudo-labels as supervised information and improve the performance of self-supervised clustering，it is necessary to mine global and local information at the same time. We illustrate a learnable data ensemble representation through graph neural network. The confidence of pseudo-labels is improved，and the entire model is trained in self-supervision iteratively. To be summarized：1）Commonly-used consensus graph learning-integrated clustering framework is developed，which can use multi-level information like clusteringintegrated sample characteristics and category structure. 2）Self-supervision method is proposed，which uses graph neural network to mine the global and local information of the consensus graph，and high-confidence pseudo-labels are obtained as supervised information. 3）Experiments are demonstrated that the consensus graph learning ensemble clustering method has its potentials on image and text datasets.

Image cyberbullying detection and recognition using transfer deep machine learning

The unmanned aerial vehicles (UAVs) have been extensively used in providing intelligence such as target tracking. In our field experiments, a pre-trained deep neural network (DNN) is deployed on the UAV to identify a target from the captured video frames and enable UAV to keep tracking. However, tracking in real time by the DNN requires a lot of computational resources. This motivates us to consider offloading this type of machine learning (ML) tasks to a mobile edge computing (MEC) server. Specifically, we propose a novel hierarchical ML tasks distribution framework for the UAV tracking system, where the UAV is embedded with lower layers of the pre-trained convolutional neural network (CNN) model due to its limited computing capability, while the MEC server with rich computing resources will handle the higher layers of the CNN model. An optimization problem is formulated to minimize the CNN inference delay while taking into account the communications delay, computing time, and ML error. Insights are provided to understand the tradeoff between communications and ML computing in offloading decisions. Numerical results demonstrate the effectiveness of the proposed ML tasks distribution framework with the optimized offloading strategy.

Among the most popular tasks in computer vision, pedestrian re-identification (person re-identification, re-ID) is a technology that uses computer vision knowledge to determine if a particular person is present in an image or video sequence. Person re-ID has attracted substantial interest in the field of computer vision due to the growing demand for intelligent security. Person re-ID is often perturbed by factors such as occluded targets, unconstrained poses, and varying feature scales, and the application of convolutional neural network to extract global features of pedestrians for recognition often fails to produce suitable results. To obtain richer target features, it is advantageous to extract local features; Unfortunately, these features sometimes fail to align, which negatively impacts on the model. In this context, we propose a multi-branch neural network that combines global and local feature learning in order to obtain a more accurate representation of pedestrian features and enhance the recognition capacity of the model. We design a feature alignment module to further improve the performance of the model by implementing a spatial transformation of the feature maps generated by the network at different depths to achieve feature alignment. According to the experiments, the rank-1 and mAP indices on the Market-1501 dataset are 95.32% and 87.64%, respectively. the method achieves results on DukeMTMC-reID dataset with 88.21% rank-1 and 77.62% mAP. The results demonstrate that our method performs more efficiently against the state-of-the-art methods on two widely used person re-ID datasets.

This paper presents an effective approach to robot arm control by integrating computer vision with machine learning techniques.A 5-DOF physical robot arm equipped with a two-finger gripper, controlled via an Arduino and servo motors (MG996R and SG90), was used for the implementation.A mobile phone camera served as the primary vision sensor for detecting ArUco markers, while a laptop handled the computer vision and machine learning tasks using OpenCV and scikit-learn libraries.The core objective was to train the robot arm to predict and follow target positions using a Linear Regression model, a type of supervised learning used for regression tasks.The training phase involved collecting (x, y) camera coordinates of ArUco markers along with corresponding servo angles().This dataset was used to train the model to learn the mapping from 2D image coordinates to continuous joint angle values.Once trained, the model was able to predict the appropriate servo angles for any new location within the camera's view, enabling precise trajectory planning and pick-and-place operations.The system architecture included a camera-based GUI for collecting calibration data, training the regression model, and executing real-time predictions to control the robot arm.Servo commands were sent to the Arduino through serial communication, enabling the arm to accurately reach and manipulate target objects based on camera input.

Feature representation learning for classification of multiple graphs is a problem with practical applications in many domains. For instance, in chemoinformatics, the learned feature representations of molecular graphs can be used to classify molecules which exhibit anti-cancer properties. In many previous work, including discriminative subgraph mining and graphlet-based approaches, a graph representation is derived by counting the occurrence of various graph sub-structures. However, these representations fail to capture the co-occurrence patterns that are inherently present among different sub-structures. Recently, various methods (e.g., DeepWalk, node2vec) have been proposed to learn representations for nodes in a graph. These methods use node co-occurrence to learn node embeddings. However, these methods fail to capture the co-occurrence relationship between more complex sub-structures in the graph since they were designed primarily for node representation learning. In this work, we study the problem of learning graph representations that can capture the structural and functional similarity of sub-structures (as evidenced by their co-occurrence relationship) in graphs. This is particularly useful when classifying graphs that have very few sub-structures in common. The proposed method uses an encoder-decoder model to predict the random walk sequence along neighboring regions (or sub-structures) in a graph given a random walk along a particular region. The method is unsupervised and can be used to obtain generic feature representations of graphs making it applicable with various types of graphs. We evaluate the learned representations using several real-world datasets on the binary graph classification task. The proposed model is able to achieve superior results against multiple state-of-the-art techniques.

Image clustering is a crucial but challenging task in multimedia machine learning. Recently the combination of clustering with deep learning has achieved promising performance against conventional methods on high-dimensional image data. Unfortunately, existing deep clustering methods (DC) often ignore the importance of information fusion with a global perception field among different image regions for clustering images, especially complex ones. Additionally, the learned features are usually not clustering-friendly in terms of dimensionality and are based only on simple distance information for the clustering. In this regard, we propose a deep embedded image clustering TDEC, which for the first time to our knowledge, jointly considers feature representation, dimensional preference, and robust assignment for image clustering. Specifically, we introduce the Transformer to form a novel module T-Encoder to learn discriminative features with global dependency while using the Dim-Reduction block to build a friendly low-dimensional space favoring clustering. Moreover, the distribution information of embedded features is considered in the clustering process to provide reliable supervised signals for joint training. Our method is robust and allows for more flexibility in data size, the number of clusters, and the context complexity. More importantly, the clustering performance of TDEC is much higher than that of recent competitors. Extensive experiments with state-of-the-art approaches on complex datasets demonstrate the superiority of TDEC.

Well log curves, acquired from downhole logging tools during well logging, are pivotal for reservoir characterization and formation evaluation in oil and gas exploration and production. However, manual feature extraction from raw curves remains essential for constructing effective machine learning models, presenting time-consuming challenges and stringent labeling requirements. Concurrently, the transformer architecture, prevalent in NLP and computer vision, offers promise for representation learning. This paper proposes a self-supervised transformer based methodology for extracting well log curves representations, aiming to expedite downstream model development.While transformer models have gained prominence in handling text and image data, well log curves present a distinct challenge as they resemble time series data. Despite the nascent development of time series transformer models, we conducted an extensive review of current progress and adopted the best-performing time series transformer model for extracting representations from well log curves. Importantly, given the challenges posed by factors such as borehole conditions and instrument failure, certain types of well log curves may occasionally be missing or distorted. To address this issue, our proposed methodology introduces an adaptive masking mechanism, which selectively applies masking to patches of curves where data quality is poor, thereby effectively mitigating data quality concerns.Data from 2000 wells are utilized for model training, with an additional 100 wells reserved for validation purposes. Our study observed a consistent decrease in both training and test losses until convergence during the training stage. Initially, mean squared error (MSE) and mean absolute error (MAE) are employed to quantify reconstruction errors between reconstructed curves and raw curves, low values of MSE (0.08) and MAE (0.07) indicate effectiveness of the learned representations. Subsequently, a downstream task involving oil and gas identification is undertaken, wherein a classification model is developed based on representations learned by the transformer model. Performance comparison between models utilizing learned representation and those employing statistical features highlights the superior performance of the former (98% accuracy), emphasizing the efficacy of our representation learning methodology. This paper introduces a novel self-supervised methodology based on transformer architecture for well log curve representation learning. The method automates information extraction without requiring logging expertise and substantially enhances downstream machine learning model performance.

Person re-identification aims to match person of interest under non-overlapping camera views. Therefore, how to generate a robust and discriminative representation is crucial for person re-identification. Mining local clues from human body parts to describe pedestrians has been extensively studied in existing methods. However, existing methods locate human body parts coarsely and do not consider the relations among different local parts. To address the above problem, we propose a Part-based Structured Representation Learning (PSRL) for better exploiting local clues to improve the person representation. There are two important modules in our architecture: Local Semantic Feature Extraction and Structured Person Representation Learning. The Local Semantic Feature Extraction module is designed to extract local features from human body semantic regions. After obtaining the local features, the Structured Person Representation Learning is proposed to fuse the local features by considering the person structure. To model the underlying person structure, a graph convolutional network is employed to capture the relations of different semantic regions. The generated structured feature encodes underlying person structure information, and local semantic feature can solve the misalignment problem caused by pose variations in feature matching. By combining them together, we can improve the descriptive ability of the generated representation. Extensive evaluations on four standard benchmarks show that our proposed method achieves competitive performance against state-of-the-art methods.

Local discriminative representation is needed in many medical image analysis tasks such as identifying sub-types of lesion or segmenting detailed components of anatomical structures. However, the commonly applied supervised representation learning methods require a large amount of annotated data, and unsupervised discriminative representation learning distinguishes different images by learning a global feature, both of which are not suitable for localized medical image analysis tasks. In order to avoid the limitations of these two methods, we introduce local discrimination into unsupervised representation learning in this work. The model contains two branches: one is an embedding branch which learns an embedding function to disperse dissimilar pixels over a low-dimensional hypersphere; and the other is a clustering branch which learns a clustering function to classify similar pixels into the same cluster. These two branches are trained simultaneously in a mutually beneficial pattern, and the learnt local discriminative representations are able to well measure the similarity of local image regions. These representations can be transferred to enhance various downstream tasks. Meanwhile, they can also be applied to cluster anatomical structures from unlabeled medical images under the guidance of topological priors from simulation or other structures with similar topological characteristics. The effectiveness and usefulness of the proposed method are demonstrated by enhancing various downstream tasks and clustering anatomical structures in retinal images and chest X-ray images.KeywordsUnsupervised representation learningLocal discriminationTopological priors