AI-powered image enhancement for accelerated defect analysis of 3D integration technologies utilized in quantum-information processing

New semiconductor chip technologies and technologies for 3D integration require information’s of packaging and interface defects in 3 dimensions, that means the lateral dimension of the defect and the location inside the device or package must be defined. In this paper, new methodical approaches for non destructive failure analysis on 3D integrated TSV samples are introduced. The concepts combine improved scanning acoustic microscopy (SAM) imaging hardware with unique software solutions for defect identification and quantitative analysis of mechanical properties using scanning acoustic investigations. In case of MEMS 3D integration, e.g. based on direct bonding, related interface defects must be investigated by SAM. With respect to 3D integration applications, the potential of recent SAM improvements applying specifically adapted hardware and custom-made signal processing algorithms will be discussed. Examples of SAM-based failure detection techniques for the application in 3D integration are demonstrated. New technologies are shown to improve the through put of fully wafer scanning using scanning acoustic microscopy. To improve the defect resolution, a new transducer design was developed to increase defect resolution and signal to noise for interface characterisation.

One promising approach for pixel-wise semantic segmentation is based on higher-order Conditional Random Fields (CKFs). We aim to selectively choose segments for the higher-order CRFs in semantic segmentation. To this end, we formulate the selection as an optimization problem. We propose three optimization criteria in relation to the selected segments, namely: a) averaged goodness, b) coverage area and c) non-overlapped area. Essentially, we desire to have best segments with maximum coverage area and maximum non-overlapped area. We apply two evolutionary optimization algorithms, namely: the genetic algorithm (GA) and the particle swarm optimization (PSO). The goodness of segments is estimated using the Latent Dirichlet Allocation approach. Experiment results show that semantic segmentation with GA-or-PSO-selected segments yields competitive semantic segmentation accuracy in comparison to that of naively using all segments. Moreover, the fewer number of segments used in semantic segmentation speeds up its computation time up to six times faster.

In this study, we propose a segmentation model based on convolutional neural networks (CNNs) to address image segmentation challenges in computer vision. Prior to designing the model, the activation function and other modules of the convolutional neural network were optimized to meet specific requirements. The segmentation task was transformed into binary classification problem to simplify network calculations and improve efficiency. Additionally, the model utilized a mask map obtained from the semantic segmentation model to aid in instance segmentation. Class activation technology was introduced to extract feature mapping maps. The corresponding thermal maps were obtained to achieve target instance segmentation. To further validate the effectiveness of the segmentation model, simulation experiments were conducted on semantic segmentation and instance segmentation respectively. The results show that the accuracy of the basic semantic segmentation model reached 87.58%, while the average accuracy of the entire class of the optimized instance segmentation model reached 97.9%. Therefore, the research and design of image segmentation models demonstrate high accuracy and good robustness.

The field of continual deep learning is an emerging field and a lot of progress has been made. However, concurrently most of the approaches are only tested on the task of image classification, which is not relevant in the field of intelligent vehicles. Only recently approaches for class-incremental semantic segmentation were proposed. However, all of those approaches are based on some form of knowledge distillation. At the moment there are no investigations on replay-based approaches that are commonly used for object recognition in a continual setting. At the same time while unsupervised domain adaption for semantic segmentation gained a lot of traction, investigations regarding domain-incremental learning in an continual setting is not well-studied. Therefore, the goal of our work is to evaluate and adapt established solutions for continual object recognition to the task of semantic segmentation and to provide baseline methods and evaluation protocols for the task of continual semantic segmentation. We firstly introduce evaluation protocols for the class- and domain-incremental segmentation and analyze selected approaches. We show that the nature of the task of semantic segmentation changes which methods are most effective in mitigating forgetting compared to image classification. Especially, in class-incremental learning knowledge distillation proves to be a vital tool, whereas in domain-incremental learning replay methods are the most effective method.

Supervised object detection and semantic segmentation require object or even pixel level annotations. When there exist image level labels only, it is challenging for weakly supervised algorithms to achieve accurate predictions. The accuracy achieved by top weakly supervised algorithms is still significantly lower than their fully supervised counterparts. In this paper, we propose a novel weakly supervised curriculum learning pipeline for multi-label object recognition, detection and semantic segmentation. In this pipeline, we first obtain intermediate object localization and pixel labeling results for the training images, and then use such results to train task-specific deep networks in a fully supervised manner. The entire process consists of four stages, including object localization in the training images, filtering and fusing object instances, pixel labeling for the training images, and task-specific network training. To obtain clean object instances in the training images, we propose a novel algorithm for filtering, fusing and classifying object instances collected from multiple solution mechanisms. In this algorithm, we incorporate both metric learning and density-based clustering to filter detected object instances. Experiments show that our weakly supervised pipeline achieves state-of-the-art results in multi-label image classification as well as weakly supervised object detection and very competitive results in weakly supervised semantic segmentation on MS-COCO, PASCAL VOC 2007 and PASCAL VOC 2012.

Semantic segmentation and activity classification are key components to create intelligent surgical systems able to understand and assist clinical workflow. In the operating room, semantic segmentation is at the core of creating robots aware of clinical surroundings, whereas activity classification aims at understanding OR workflow at a higher level. State-of-the-art semantic segmentation and activity recognition approaches are fully supervised, which is not scalable. Self-supervision can decrease the amount of annotated data needed. We propose a new 3D self-supervised task for OR scene understanding utilizing OR scene images captured with ToF cameras. Contrary to other self-supervised approaches, where handcrafted pretext tasks are focused on 2D image features, our proposed task consists of predicting relative 3D distance of image patches by exploiting the depth maps. By learning 3D spatial context, it generates discriminative features for our downstream tasks. Our approach is evaluated on two tasks and datasets containing multiview data captured from clinical scenarios. We demonstrate a noteworthy improvement in performance on both tasks, specifically on low-regime data where utility of self-supervised learning is the highest. We propose a novel privacy-preserving self-supervised approach utilizing depth maps. Our proposed method shows performance on par with other self-supervised approaches and could be an interesting way to alleviate the burden of full supervision.

Introduction: Synchrotron radiation micro-computed tomography (SRμCT) has been used as a non-invasive technique to examine the microstructure and tissue integration of biodegradable bone implants. To be able to characterize parameters regarding the disintegration and osseointegration of such materials quantitatively, the three-dimensional (3D) image data provided by SRμCT needs to be processed by means of semantic segmentation. However, accurate image segmentation is challenging using traditional automated techniques. This study investigates the effectiveness of deep learning approaches for semantic segmentation of SRμCT volumes of Mg-based implants in sheep bone ex vivo.Methodology: For this purpose different convolutional neural networks (CNNs), including U-Net, HR-Net, U²-Net, from the TomoSeg framework, the Scaled U-Net framework, and 2D/3D U-Net from the nnU-Net framework were trained and validated. The image data used in this work was part of a previous study where biodegradable screws were surgically implanted in sheep tibiae and imaged using SRμCT after different healing periods. The comparative analysis of CNN models considers their performance in semantic segmentation and subsequent calculation of degradation and osseointegration parameters. The models’ performance is evaluated using the intersection over union (IoU) metric, and their generalization ability is tested on unseen datasets.Results and discussion: This work shows that the 2D nnU-Net achieves better generalization performance, with the degradation layer being the most challenging label to segment for all models.

Augmenting RGB data with measured depth has been shown to improve the performance of a range of tasks in computer vision, including object detection and semantic segmentation. Although depth sensors such as the Microsoft Kinect have facilitated easy acquisition of such depth information, the vast majority of images used in vision tasks do not contain depth information. In this paper, we show that augmenting RGB images with estimated depth can also improve the accuracy of both object detection and semantic segmentation. Specifically, we first exploit the recent success of depth estimation from monocular images and learn a deep depth estimation model. Then, we learn deep depth features from the estimated depth and combine with RGB features for object detection and semantic segmentation. In addition, we propose an RGB-D semantic segmentation method, which applies a multi-task training scheme: semantic label prediction and depth value regression. We test our methods on several data sets and demonstrate that incorporating information from estimated depth improves the performance of object detection and semantic segmentation remarkably.

Advanced Driver Assistance System (ADAS) is a very important part of an up to date vehicle. For achieving highlevel objectives in such ADAS functionality like LKA (lane keeping assistance), LDW (lane departure warning system), FCW (forward collision warning) the quality of the algorithms under the hood must be extremely high. In the last few years, it is common that these algorithms are based on DNNs (deep neural networks) applied to the tasks of semantic and instance segmentation, 2D/3D object detection and visual object tracking. Recent state-of-the-art DNN models as usual solve only one single task from the listed above and running several neural networks is rather computationally expensive and even impossible due to the lack of the GPU memory. One of the approaches used to overcome such a problem is a shared backbone (also called feature extractor or encoder). The backbone consumes most of the computing resources thus the model with a shared backbone achieves better inference performance. Unfortunately, the training procedure for a shared backbone model has several difficulties. The first one is the lack of datasets with all the required and uniform annotation types. The second problem is a more sophisticated backpropagation procedure. In this paper, we consider several methods for multi-task neural network training and present the results of such training procedures on several public datasets with dissimilar annotation types. The shared backbone is applied to the following three tasks performed simultaneously on the road scene: semantic segmentation, 2D object detection and 3D object detection. While the performance of the DNNs with shared backbone increased significantly, we obtained the quality evaluation results, which are quite close to the original separate state-of-the-art DNNs and even outperforms them in some evaluation indices.

Object detection and semantic segmentation are two fundamental techniques of various applications in the fields of Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Early studies separately handle these two problems. In this paper, inspired by some recent works, we propose a deep neural network model for joint object detection and semantic segmentation. Given an image, an encoder-decoder convolution network extracts a set of feature maps, these feature maps are shared by the detection branch and the segmentation branch to jointly carry out the object detection and semantic segmentation. In the detection branch, we design a PriorBox initialization mechanism to propose more object candidates. In the segmentation branch, we use the multi-scale atrous convolution to explore the global and local semantic information in traffic scenes. Benefiting from the PriorBox Initialization Mechanism (PBIM) and Multi-Scale Atrous Convolution (MSAC), our model presents the competitive performance. In the experiments, we widely compare with several recently-proposed methods on the public Cityscapes dataset, achieving the highest accuracy. In addition, to verify the robustness and generalization of our model, the extension experiments are also conducted on the well-known VOC2012 dataset.

Over the past two decades, 3D techniques combined with heterogeneous integration, combining different functions and materials, have emerged as a key enabler for overcoming the physical and scaling limits of traditional 2D integration. These approaches are now expanding across multiple domains, including CMOS, memories, photonics, imagers, and RF systems, with the aim of delivering next-generation integrated systems for advanced computing (AI), analog/RF, and imaging applications. In this presentation, we will explore the fundamental techniques, recent advancements, and some challenges associated with achieving integration. Special emphasis will be placed on the role of hybrid bonding and sequential 3D integration schemes, as well as their implications for energy efficiency, device performance, and system-level innovations.3D integration has transitioned from a niche research area to a cornerstone technology in microelectronics, driven by its potential to address high density interconnect and traditional 2D scaling bottlenecks for several applications and enable heterogeneous integration at unprecedented scales. At the heart of this technological evolution are advanced schemes such as wafer thinning, Through-Silicon Vias (TSVs), hybrid bonding, and sequential 3D integration techniques. These techniques have been pivotal in shrinking the interconnect length, reducing latency, and enhancing data bandwidth in computing systems. In parallel, developments in wafer-to-wafer, die-to-wafer, and chiplet integration are enabling the seamless merging of diverse materials enabling functionalities such as logic, memory, photonics, and RF. Compact form factors are achieved enabling to boost performances of the most demanding applications such as Artificial Intelligence or Tb/s transceiver applications [1].Recent studies highlight significant progress in hybrid bonding technology, which achieves high-density interconnects at the submicron scale, thereby optimizing power delivery and thermal management in stacked devices [2]. Meanwhile, sequential 3D integration [3][4] where layers of transistors are fabricated sequentially on the same substrate, is emerging as a game-changing paradigm. It promises ultra-high-density integration with minimal parasitics, creating new opportunities for energy-efficient and high-performance systems. Several applications are foreseen including C-FETS, memories and imagers [5] .Photonic interconnects will enter in next generation 3D IC integration, addressing challenges in data bandwidth, power consumption and signal integrity and enabling photonic interposers for next generation of components for data centers and communication systems [7]. Similarly, RF [8] and analog domains [9] benefit from 3D stacking by improving front-end module integration with specific materials (III-V, HR, ...), leading to enhanced performance and higher frequencies.Despite these achievements, significant challenges remain. Thermal dissipation is a critical bottleneck [10] , particularly in densely 3D stacked architectures where hotspots can severely impact reliability and performance. Moreover, the mechanical stresses induced during bonding and processing can affect device yield and long-term stability. Our recent work in advanced thermal management and stress engineering underscores the need for interdisciplinary approaches to address these challenges. Solutions such as microfluidic cooling, novel materials for thermal interface layers and specific bonding technologies are being actively explored. Furthermore, while heterogeneous integration is currently feasible at the wafer scale—such as wafer-to-wafer hybrid bonding, which is already in mass production—expanding application fields will require advancements in die-to-wafer integration techniques.Looking ahead, the convergence of 3D integration with emerging paradigms such as neuromorphic computing, quantum technologies, multispectral and smart imagers, and advanced packaging promises to redefine system design at all levels. The synergy between 3D technologies and heterogeneous integration of advanced materials, such as 2D semiconductors, III-V, ferroelectric or oxide semiconductor [11] materials, further expands the horizon for innovation.In conclusion, 3D integration has proven to be a transformative force in microelectronics, offering pathways to overcome traditional limitations and unlock new performance dimensions. Intensive researches worldwide continues contribute to push the boundaries of what is possible, paving the way for smarter, more efficient, and highly integrated systems.

Object Detection has been a great challenge over the years. The reason behind is that, it is applied for numerous real time applications like vision based control, traffic control, video surveillance, sports analysis, etc. But, object detection in a video sequence is a highly challenging task. It has various problems like occlusion, fast moving objects, shadow, poor lighting, color contrast and other static background objects. This reason brought the object detection to be a thrust research area in the field of image processing. In the previous researches the conventional methods of object detection like Frame Difference, Mixture of Gaussian (MoG), Optical Flow etc., still have the above problems. Hence the research focuses on a different approach in object detection using Metaheuristic algorithm for the video sequence of volley ball player in the practice session. In this research three Metaheuristic Algorithms, namely Firefly, Teaching and Learning Based Optimization (TLBO) and Cuckoo Search Algorithm are used. These algorithms are evaluated and compared with the parameters like accuracy, precision, and recall. The result shows Cuckoo Search Algorithm is best suited to object detection especially in this application.

3D Packaging Technologies and Applications, Latest Challenges and Supply Chain ActivitiesSemiconductor industry, for more than four decades, has rigorously followed Moore's Law in scaling down the CMOS technologies. Although several new materials and processes are being developed to address the challenges of future technology nodes, in the coming years they will be limited with respect to functionalities that future devices will require. As a consequence a clear trend of moving from CMOS to package and system architecture can be observed. Three-dimensional (3D) technology using the well-known Through Silicon Via (TSV) interconnect is one the emerging options, considered today the most advanced technology, that could enable various heterogeneous integration. 3D Integration is not limited to the CMOS scaling in itself, it is rather based on bringing more functionalities by stacking different type of devices (Logic, Memory, Analog, MEMS, Passive components) while reducing the form factor of the packaging. This functional diversification is also known as More-than-Moore. In addition, considering Known Good Die approach, each component of the 3D package could have a different manufacturer using different wafer sizes and node technology, thus bringing more complexity but also more opportunities and responsibilities to the supply chain. There are several business models identified, either using vertical integration or collaborative approach, if a dominant one will emerge or several tactics will co-exist, it is still remains a key question that will have to be answered. The supply chain interaction and key players will be addressed in this presentation, including current and future standardization needs. This is today a key for the manufacturing of advanced 3D devices. 3D integration is considered today a new paradigm for the semiconductor industry, since it will drive evolution for packages for the coming decades. Due to several advantages that TSV technology can bring, several platforms have started. 3D WLCSP, 2.5D interposers & 3DIC are the main platforms that will be studied in this paper. Market forecasts in terms of wafer starts, market revenue, segments and end-products as well as supply chain activities and major player interactions will be presented. The industry has enthusiastically been waiting for mass production of 3D ICs. Although some small level of production has already been reported, the adoption rate in high volume manufacturing (HVM) is still low due to unresolved challenges that the industry still needs to address. Process technology is not fully mature, there are still many challenges in bonding and de-bonding, testing as well as thermal management that have to be overcome. Furthermore, design tools have to be fully released to enable proper 3D integration design. Looking at the time to market it is foreseen that device such as the Hybrid Memory Cube, combining high-speed logic with multiple stacks of TSV bonded memories, will come into high volume production in 2014. This will definitely change the world of the memory market and will significantly speed up the adoption of 3D technologies. Technology roadmaps for 3D integration will also be included in the manuscript and reviewed during the presentation.

In several studies, fuzzy classifiers (FCs) have been shown to achieve higher generalization ability when learned through support vector machines (SVMs) compared to learning through neural networks that aim to minimize only the training error. This chapter introduces the learning of FCs using linear SVMs. Two types of FCs with zero-order and high-order Takagi-Sugeno (TS)-type consequents are considered. Given a number of rules, the antecedent parameters in the two FCs are determined using a self-splitting clustering (SSC) algorithm. Regarding the consequent parameter optimization, this chapter first describes the basic concept of linear SVMs followed by its application to the learning of the consequent parameters to endow the FCs with high generalization ability. These two types of FCs are subsequently applied to object detection and shape extraction using a red-green-blue-depth (RGB-D) camera. In this application, after the detection of an object using a color-feature-based FC, the depth information from the camera is used to extract the shape of an object. A histogram-based shape feature is proposed for improving the object detection performance. The performance of the proposed classification approach is evaluated through the detection of different objects and comparisons with various object detection approaches.

In this paper, we address the problems of contour detection, bottom-up grouping, object detection and semantic segmentation on RGB-D data. We focus on the challenging setting of cluttered indoor scenes, and evaluate our approach on the recently introduced NYU-Depth V2 (NYUD2) dataset (Silberman et al., ECCV, 2012). We propose algorithms for object boundary detection and hierarchical segmentation that generalize the $$gPb-ucm$$gPb-ucm approach of Arbelaez et al. (TPAMI, 2011) by making effective use of depth information. We show that our system can label each contour with its type (depth, normal or albedo). We also propose a generic method for long-range amodal completion of surfaces and show its effectiveness in grouping. We train RGB-D object detectors by analyzing and computing histogram of oriented gradients on the depth image and using them with deformable part models (Felzenszwalb et al., TPAMI, 2010). We observe that this simple strategy for training object detectors significantly outperforms more complicated models in the literature. We then turn to the problem of semantic segmentation for which we propose an approach that classifies superpixels into the dominant object categories in the NYUD2 dataset. We design generic and class-specific features to encode the appearance and geometry of objects. We also show that additional features computed from RGB-D object detectors and scene classifiers further improves semantic segmentation accuracy. In all of these tasks, we report significant improvements over the state-of-the-art.