Private‐encoder: Enforcing privacy in latent space for human face images

The disclosure of face image features can seriously threaten the security of user information, which limits the application of face recognition technology in Internet of Vehicles. This paper proposes a new method of generating and restoring private face images based on semantic features and adversarial examples. The Segnet network first segments the face images semantically, then a generative adversarial network generates adversarial examples and perturbs the semantic features of the face image. The perturbation position can be accurately controlled through a coefficient matrix as the identity tag of the face image is concealed steganographically. A restoration network is trained to extract the real identity tag from the private face image using a discriminator against the generation network, then it restores the private face image to its original state. Compared to other state-of-the-art methods, private face images generated by the proposed method experimentally show high detection resistance, better quality, and stronger median filtering defense.

Recently, deep learning has become a mainstream method of image inpainting. It can not only restore the image texture, obtain high-level abstract features of images, but also restore semantic images such as human face images. Among these methods, generative adversarial networks (GANs) using autoencoder as the generator have become the promising model for image inpainting. These models implement the end-to-end image inpainting and also generate visually reasonable and clear image structures and textures. However, GANs often have problems with gradient vanishing and model collapse during training, so we propose a Generative Adversarial Network with Evolutionary Generators (EG-GAN) and apply it in face image inpainting. To stabilize the model training process, EG-GAN trains the generator network by evolution, combines two mutation functions as a training objective to update the parameter of generator networks, and produces offspring generators through crossover, using the matcher assists the discriminator to criticize the generated image. Experiments on various face image datasets such as CelebA-HQ and CelebA show that EG-GAN successfully overcomes the gradient vanishing problem, achieves stable and efficient training, and generates visually reasonable images.

The rapid advancement of Visual Language Models (VLMs) has enabled sophisticated analysis of visual content, leading to concerns about the inference of sensitive user attributes and subsequent privacy risks. While technical capabilities of VLMs are increasingly studied, users’ understanding, perceptions, and reactions to these inferences remain less explored, especially concerning videos uploaded on the social media. This paper addresses this gap through a semi-structured interview (N=17), investigating user perspectives on VLM-driven sensitive attribute inference from their visual data. Findings reveal that users perceive VLMs as capable of inferring a range of attributes, including location, demographics, and socioeconomic indicators, often with unsettling accuracy. Key concerns include unauthorized identification, misuse of personal information, pervasive surveillance, and harm from inaccurate inferences. Participants reported employing various mitigation strategies, though with skepticism about their ultimate effectiveness against advanced AI. Users also articulate clear expectations for platforms and regulators, emphasizing the need for enhanced transparency, user control, and proactive privacy safeguards. These insights are crucial for guiding the development of responsible AI systems, effective privacy-enhancing technologies, and informed policymaking that aligns with user expectations and societal values.

Today's technology and communications (ICT) facility has seen the creation of various social media platforms, chat application and even online entertainment platforms. So, more and more generations are accessing these platforms in search of entertainment, information and even communicating with others. This situation actually opens the door to cyber risk when it is not used wisely and responsibly. As a result, in recent times, various incidents have been reported in adolescents that involve cyber risk, affecting psychological, mental and physical health. These include cyberbullying, digital games and social media addiction, pornography, grooming, cybersex and the misuse of personal information. The purpose of this research is to investigate the online activities among school students and to examine their online risks experienced by Malaysian secondary school students. A total of 1376 form four students (16 years old) were involved in this research. The research data was collected using a set of questionnaire. Descriptive data analysis was performed using SPSS version 20 software. The findings show a high trend of online student activities for communicating, socializing, watching videos and finding information. Although students' experiences of cyber risk are low, steps to control the situation need to be taken in order to prevent the any risks when they are online. Therefore, this research paper also discusses the role of all parties in ensuring that the cyber world is used wisely and responsibly.

Human face recognition is still a challenging task for computer-based machines mainly due to the similarity of human faces and unpredictable variations such as illumination, pose, expression, occlusion etc. A lot of algorithms have been developed to deal with these challenging issues in face recognition. One of the difficult problems is the geometric variation of face images. In some face recognition applications, only face images with uncontrolled format are available. Face images obtained from face detection stage may be directly used. Therefore, alignment or normalization is required to achieve better recognition performance. Eye detection technique is normally used to normalize face images. Invariant representations are used to improve the generalization of facial features. Feature-based and modelbased automatic face alignment can also be used to localize human faces more precisely. Illumination variation is another tough problem because human face is a 3D model and practical lighting conditions are complicated and unpredictable. Illumination normalization based on preprocessing, illumination invariant feature extraction and face modelling have been employed to reduce illumination variations or render faces under varying lighting conditions. The geometric on-the-plane rotation of a face image can be normalized by aforementioned methods. However, a face image with rotation in depth is not easily adjusted to a certain view. It is difficult to recognize faces in all poses with only training face images in limited views. One solution is to use 2D or 3D generative models to generate novel views of the face. Another method is based on the view-subspace learning. For facial expression variations,

<span lang="EN-IN">There is a growing need for an image encryption scheme, for huge amount of social media data or even the medical data to secure the privacy of the patients or the user. This study introduces a ground-breaking deep learning architecture named crypto generative adversarial networks (CryptoGAN), a novel architecture for generating cipher images. This architecture has the ability to generate both encrypted and decrypted images. The CryptoGAN system consists of an initial encryption network, a generative network that verifies the output against the desired domain, and a subsequent decryption phase. The generative adversarial networks (GAN) are utilised as the learning network to generate cipher images. This is achieved by training the neural network using images encrypted from a conventional image encryption scheme such as advanced encryption standards (AES), and learning from the resulting losses. This enhances security measures when dealing with a large dataset of photos. The assessment of the performance metrics of the encrypted image, including entropy, histogram, correlation plot, and vulnerability to assaults, demonstrates that the suggested generative network may get a higher level of security.</span>

Generative Adversarial Networks (GANs) are prominent in unsupervised learning for their exceptional data-generation capabilities. GANs utilize backpropagation and a competitive process between a Generative Network (G) and a Discriminative Network (D). In this setup, G generates artificial images while D distinguishes real from artificial ones, enhancing G's ability to create realistic images. Deep Convolutional Generative Adversarial Networks (DCGAN) are particularly notable, using a convolutional architecture to produce high-quality human face images. This study trains DCGAN on the CelebFaces Attributes Dataset (CelebA), demonstrating its ability to generate human faces from unlabeled data and random noise. Evaluation is done quantitatively using the Structural Similarities Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR) to assess image quality. Additionally, this abstract will compare the effectiveness of GANs and DCGANs in human face generation.

Style synthesis attracts great interests recently, while few works focus on its dual problem "style separation". In this paper, we propose the Style Separation and Synthesis Generative Adversarial Network (S3-GAN) to simultaneously implement style separation and style synthesis on object photographs of specific categories. Based on the assumption that the object photographs lie on a manifold, and the contents and styles are independent, we employ S3-GAN to build mappings between the manifold and a latent vector space for separating and synthesizing the contents and styles. The S3-GAN consists of an encoder network, a generator network, and an adversarial network. The encoder network performs style separation by mapping an object photograph to a latent vector. Two halves of the latent vector represent the content and style, respectively. The generator network performs style synthesis by taking a concatenated vector as input. The concatenated vector contains the style half vector of the style target image and the content half vector of the content target image. Once obtaining the images from the generator network, an adversarial network is imposed to generate more photo-realistic images. Experiments on CelebA and UT Zappos 50K datasets demonstrate that the S3-GAN has the capacity of style separation and synthesis simultaneously, and could capture various styles in a single model.

The Privacy Journal (2003), a print newsletter and Web site devoted to privacy matters, defines the present-day use of the word privacy as “the right of individuals to control the collection and use of personal information about themselves.” Similar definitions are provided by law specialists (Gavison, 1980; Warren & Brandies, 1890). The networked society changes the way in which privacy rights are defined, used and interpreted, because: a. The IT-enabled channels of communication change the rules of personal and commercial interaction; b. The participation in the networked society implies a diminishing of individual privacy rights. The fundamental principle of the networked society is information sharing and processing (Kling & Allen, 1996). Advances in computing technology—that represents the infrastructure of the networked society—make possible to collect, store, analyze, and retrieve personal information created in the process of participation. The manifestation and the protection of individual privacy rights represent the field of conflict between various disciplines and social events. The heterogeneous nature of this phenomenon is mirrored in this paper, which aims to present the complex nature of privacy rights in the context of the networked society. The study proposes a negotiating model of online privacy rights, and analyses the necessary conditions for the implementation of this model on the Internet. The new economy is redefined on the basis of information entrepreneurism (Kling & Allen, 1996; Zwick & Dholakia, 1999). This cultural paradigm emphasizes the use of data-intensive analysis techniques for designing and implementing effective marketing and management strategies. This has as a direct consequence the use of an information superpanopticon–a concept derived from Foucault’s panopticon, a system of perfect surveillance and control. Online privacy is a major concern for Internet users (Ackerman, Cranor, & Reagle, 1999). For the individual Internet user, the privacy threats fall into two main categories: a. Web tracking devices that collect information about the online behavior of the user (e.g., cookies); b. The misuse of the personal information provided by the online user in exchange of specific benefits: increased personalization, Web group membership, etc. The databases, intelligent agents and tracking devices are surrounding the Internet users with a Web of surveillance, which is often hidden and unknown to the users. The surveillance is initiated by the simple act of presence on the Internet. Specialized software applications, such as cookies are tracking the online behavior of Internet users, feeding the data into databases, which create and permanently update a profile of online consumers. These profiles are then used for segmenting the market and targeting the most profitable consumers. A company can use cookies for various valid reasons: security, personalization, marketing, customer service, etc., however, there is an important distinction between cookies, which are active only within a specific Web site, and the ones that can track the user’s activity across unrelated Web sites. Recently, some aggregator networks have deployed hidden ‘pixel beacon’ technology that allows ad-serving companies to connect unrelated sites and overcome the site-specific nature of traditional cookies (Mabley, 2000). Additionally, some companies are now connecting this aggregated data with offline demographic and credit card data. Eventually, these resulting databases can be used or sold as powerful marketing tools. Exercising control of information, after it was voluntarily released, presents another critical problem. The misuse of personal information covers many possible aspects, which can be defined as any use which is not explicitly defined in the company’s privacy disclaimer, or which is not approved by the informed customer. For example, in 2000, Toysurus.com was subject to intense debate and controversy, when it was discovered that shoppers’ personal information was transferred through an unmarked Internet channel to a data processing firm, for analysis and aggregation. This operation was not disclosed in the company’s privacy disclaimer, and therefore, online customers were not aware of it. Regulators and legislators have addressed the controversial privacy issue quite differently across the world (Nakra, 2001). The USA, the largest world’s financial and Internet market, has not yet adopted a national, standard-setting privacy law (Jarvis, 2001). U.S. privacy statutes have primarily focused so far on protecting consumers’ financial data, health information, and children’s personal information (Desai, Richards, & Desai, 2003; Frye, 2001). In comparison with the American official opinion that online privacy protection is a matter of voluntary self-regulation by market-driven companies, the Europeans consider that it is more effective to enforce specific legislation regarding this issue. The current European approach is based on three basic tenets: 1. Individuals have the right to access any data relating to them and have it kept accurate and up-to-date; 2. Data cannot be retained for longer than the purpose for which it was obtained, nor used or disclosed “in a matter incompatible with that purpose”, and must be kept only for “lawful purposes”; 3. Those who control data have “a special duty of care” in relation to the individuals whose data they keep. Data commissioners oversee these rights in each European country and require most “data controllers”—people who handle data—to register with them to track what information is being collected and where. They are charged also with investigating all complaints from citizens. These principles have been incorporated in the European Data Directive, which came into effect in 1998, and more recently, in the European Directive on Privacy and Electronic Communications, adopted in 2002. Despite these legislative efforts, it is not yet clear how effective are the measures implemented by EU States. The direct involvement of governmental institutions can be considered as a form of censorship that can undermine the freedom and the flexibility of the Internet domain.

The perception of a meaningful facial pattern on a nebulous stimulus-face pareidolia-is a typical human experience. Neuroimaging and electrophysiological studies have generally shown similarities in the spatio-temporal responses to typical faces and objects eliciting face pareidolia, that is, facelike objects. However, the extent to which facelike objects engage the same neural basis as human faces remains unclear. To address this issue, we used direct measures of brain activity from intracerebral electrodes implanted in the ventral occipito-temporal cortex (VOTC) of a large group of patients (n = 44). Face selectivity was determined by contrasting a large set of naturalistic face or facelike object images with non-face object categories. High signal-to-noise ratio face-selective and facelike object-selective responses were objectively identified and quantified with frequency tagging and compared in space and time throughout the VOTC. Selective activity to facelike objects was found in all key regions of the human cortical face network, extending to the previously unexplored anterior temporal lobe (ATL). Although category-selective activity was markedly reduced for facelike objects compared with human faces, consistent with previous findings, 89% of facelike object-selective contacts spatially overlapped with human face-selective contacts, while the remaining spatially scattered contacts recorded negligible responses. Furthermore, the amplitude of the two face-selective neural signals showed high correlations across regions, recording contacts and time courses as well as concurrent early onset, challenging the view that facelike objects are interpreted as faces through feedback from higher order brain regions. Together, our findings demonstrate that the pareidolic perception of face in facelike objects engages the same ventro-temporal neural circuitry, with the same temporal dynamics, as human faces.

Talking head video generation aims to produce a synthetic human face video that contains the identity and pose information respectively from a given source image and a driving video. Existing works for this task heavily rely on 2D representations (e.g. appearance and motion) learned from the input images. However, dense 3D facial geometry (e.g. pixel-wise depth) is extremely important for this task as it is particularly beneficial for us to essentially generate accurate 3D face structures and distinguish noisy information from the possibly cluttered background. Nevertheless, dense 3D geometry annotations are prohibitively costly for videos and are typically not available for this video generation task. In this paper, we introduce a self-supervised face-depth learning method to automatically recover dense 3D facial geometry (i.e. depth) from the face videos without the requirement of any expensive 3D annotation data. Based on the learned dense depth maps, we further propose to leverage them to estimate sparse facial keypoints that capture the critical movement of the human head. In a more dense way, the depth is also utilized to learn 3D-aware cross-modal (i.e. appearance and depth) attention to guide the generation of motion fields for warping source image representations. All these contributions compose a novel depth-aware generative adversarial network (DaGAN) for talking head generation. Extensive experiments conducted demonstrate that our proposed method can generate highly realistic faces, and achieve significant results on the unseen human faces. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://github.com/harlanhong/CVPR2022-DaGAN

Generative models based on latent variables, such as generative adversarial networks (GANs) and variational auto-encoders (VAEs), have gained lots of interests due to their impressive performance in many fields. However, many data such as natural images usually do not populate the ambient Euclidean space but instead reside in a lower-dimensional manifold. Thus an inappropriate choice of the latent dimension fails to uncover the structure of the data, possibly resulting in mismatch of latent representations and poor generative qualities. Toward addressing these problems, we propose a novel framework called the latent Wasserstein GAN (LWGAN) that fuses the Wasserstein auto-encoder and the Wasserstein GAN so that the intrinsic dimension of the data manifold can be adaptively learned by a modified informative latent distribution. We prove that there exist an encoder network and a generator network in such a way that the intrinsic dimension of the learned encoding distribution is equal to the dimension of the data manifold. We theoretically establish that our estimated intrinsic dimension is a consistent estimate of the true dimension of the data manifold. Meanwhile, we provide an upper bound on the generalization error of LWGAN, implying that we force the synthetic data distribution to be similar to the real data distribution from a population perspective. Comprehensive empirical experiments verify our framework and show that LWGAN is able to identify the correct intrinsic dimension under several scenarios, and simultaneously generate high-quality synthetic data by sampling from the learned latent distribution. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Non-intrusive load monitoring (NILM) provides fine-grained consumption information at the appliance level by analyzing the terminal voltage and total current measured. It shows prospective applications in demand side management, such as demand response, energy efficiency, and home energy management system. However, most cutting-edge NILM models have a critical assumption that switching events are triggered by known appliances in the training set, which may be unrealistic. In reality, new appliances are constantly being added, reducing the effectiveness of current methods for load monitoring. In this paper, we propose a conditional generative adversarial network (CGAN) to correctly classify all the known appliances while simultaneously detecting unknown ones using the V-I trajectory features. We integrate variational autoencoder and capsule networks in the generator network, the capsule features of the same known appliance class are forced to match a pre-defined Gaussian and a group of Gaussian priors (one for each appliance class) as the posterior distribution approximately. Furthermore, using an additional encoder network maps the generated V-I trajectory to its latent representation, minimizing the distance between the latent representations and the feature vectors of the generator aids in learning the data distribution for known appliance samples during the training process. In these ways, we can improve the ability of the model to detect unknown appliances. Experimental results on two public datasets demonstrate the effectiveness and superiority of our method.

Biometrics is the study of methods of recognizing humans based on their behavioral and physical characteristics or traits. Face recognition is one of the biometric modalities that received a great amount of attention from many researchers during the past few decades because of its potential applications in a variety of security domains. Face recognition however is not only concerned with recognizing human faces, but also with recognizing faces of non-biological entities or avatars. Fortunately, the need for secure and affordable virtual worlds is attracting the attention of many researchers who seek to find fast, automatic and reliable ways to identify virtual worlds’ avatars. In this work, I propose new techniques for recognizing avatar faces, which also can be applied to recognize human faces. Proposed methods are based mainly on a well-known and efficient local texture descriptor, Local Binary Pattern (LBP). I am applying different versions of LBP such as: Hierarchical Multi-scale Local Binary Patterns and Adaptive Local Binary Pattern with Directional Statistical Features in the wavelet space and discuss the effect of this application on the performance of each LBP version. In addition, I use a new version of LBP called Local Difference Pattern (LDP) with other well-known descriptors and classifiers to differentiate between human and avatar face images. The original LBP achieves high recognition rate if the tested images are pure but its performance gets worse if these images are corrupted by noise. To deal with this problem I propose a new definition to the original LBP in which the LBP descriptor will not threshold all the neighborhood pixel based on the central pixel value. A weight for each pixel in the neighborhood will be computed, a new value for each pixel will be calculated and then using simple statistical operations will be used to compute the new threshold, which will change automatically, based on the pixel’s values. This threshold can be applied with the original LBP or any other version of LBP and can be extended to work with Local Ternary Pattern (LTP) or any version of LTP to produce different versions of LTP for recognizing noisy avatar and human faces images.

Human face detection plays an important role in applications such as video surveillance, human computer interface, face recognition, and face image database management. We propose a face detection algorithm for color images in the presence of varying lighting compensation technique and we segment the skin color of the face image with the improved skin color model which is in the space of YCrCb, and then find the human face in a small area using the template matching method. The application on the skin color information is add to the face detection algorithm, the purpose of which is to pre-judge whether the image exist a face, thus we can exclude non-skin regions to reduce the space to search and improve the efficiency of the face detection algorithm. This paper proposed various adaptive templates to overcome the shortcomings of poor adaptability, the scale of the template can be seized to adjust the size of the area, so the adaptability of template can be increased.