Cycle Time Perspectives for Small Transfer Batch Size

Deep learning architectures are usually proposed with millions of parameters, resulting in a memory issue when training deep neural networks with stochastic gradient descent type methods using large batch sizes. However, training with small batch sizes tends to produce low quality solution due to the large variance of stochastic gradients. In this paper, we tackle this problem by proposing a new framework for training deep neural network with small batches/noisy gradient. During optimization, our method iteratively applies a proximal type regularizer to make loss function strongly convex. Such regularizer stablizes the gradient, leading to better training performance. We prove that our algorithm achieves comparable convergence rate as vanilla SGD even with small batch size. Our framework is simple to implement and can be potentially combined with many existing optimization algorithms. Empirical results show that our method outperforms SGD and Adam when batch size is small. Our implementation is available at https://github.com/huiqu18/TRAlgorithm.

Transfer Batch Sizing in Flexible Manufacturing Systems

Transfer batch sizing in flexible manufacturing systems

Automation first made an impact in the automotive and other mass production markets. This was followed by developments such as computer numerical control that introduced automation to small and medium batch sizes. Until recently, the one‐off and very small batch sizes encountered in toolrooms and prototype production remained the domain of manual machine tools. Describes the Prototrak system which was developed to address this market.

Normalization layers play an important role in deep network training. As one of the most popular normalization techniques, batch normalization (BN) has shown its effectiveness in accelerating the model training speed and improving model generalization capability. The success of BN has been explained from different views, such as reducing internal covariate shift, allowing the use of large learning rate, smoothing optimization landscape, etc. To make a deeper understanding of BN, in this work we prove that BN actually introduces a certain level of noise into the sample mean and variance during the training process, while the noise level depends only on the batch size. Such a noise generation mechanism of BN regularizes the training process, and we present an explicit regularizer formulation of BN. Since the regularization strength of BN is determined by the batch size, a small batch size may cause the under-fitting problem, resulting in a less effective model. To reduce the dependency of BN on batch size, we propose a momentum BN (MBN) scheme by averaging the mean and variance of current mini-batch with the historical means and variances. With a dynamic momentum parameter, we can automatically control the noise level in the training process. As a result, MBN works very well even when the batch size is very small (e.g., 2), which is hard to achieve by traditional BN.

Today many manufacturing firms are focusing on cycle or throughput time reduction to gain a competitive advantage. In many instances, cycle times at individual workstations are adversely affected by unscheduled machine breakdowns. In this paper, existing closed-form approximations for the expected cycle time at a parallel machine workstation with general process and interarrival times and for the variability in departures from the workstation are used to demonstrate how the mean and variance of machine repair or downtime affects cycle times at a given workstation and downstream workstations. Results from the machine interference literature are used to characterize the dependence of repair time on a base stock inventory policy used for the replenishment of spare components. Expressions for the mean and approximations for the variance of repair times are obtained for machines with either a single critical component or a series of critical components when such components fail exponentially. Numerical examples are given to illustrate the effect that spare component inventory levels have on the average station cycle time and departure variability. A methodology for base stock parameter selection that considers the trade-off between reduced cycle times and inventory investment is also presented.

By extrapolating technological trends, one can expect future fabs to consist largely of highly reliable cluster tools. Such trends raise the issues of how cluster tools should be configured (i.e., how modules should be grouped together), and the impact of configuration on optimal lot size. This paper generates equation-based models to compare the throughput and cycle time of different systems that perform the same sequence of process steps, with the same total number of modules grouped in different ways into multiple cluster tools. The two extremes of the spectrum of possible cluster tool configurations are considered: the serial configuration, in which a wafer visits each module in the cluster tool, and the parallel configuration, in which all of the modules in the cluster tool are identical. The models predict the following for typical cluster tool parameters and process times. When all process step times are identical, systems of serial-configured tools can achieve the same throughputs with lower cycle times, as their parallel-configured counterparts. However, if process times in the sequence significantly differ from one another, then systems of parallel-configured tools may achieve lower cycle times than their serial-configured counterparts at high throughputs. Finally, the serial configuration significantly reduces the lot size required to achieve total throughput capacity.

Batching production is to process products in lots rather than by the piece and most activities in repetitive construction projects are performed by batching production. Batch size is a key factor which affects duration of construction project and it is recommended to use small batch size for early completion of construction projects. However, it is possible that batch sizes are different each other between activities, and no research about impacts of batch size matching has been conducted. This paper is to examine the impacts of batch size matching on construction duration. A computer simulation model which includes two activities was developed. Two hypotheses were tested under the Monte-Carlo simulation method: 1) small batch sizes which are not matched each other can reduce time depending on productivities and 2) synchronized batch sizes leads to earlier completion of project than unsynchronized batch sizes irrespective of productivities. The results identify an advantage of matching batch sizes with regard to project duration and show that batch size matching needs consideration of compatibility between productivities also.

Batch size is a key hyperparameter in training deep learning models. Conventional wisdom suggests larger batches produce improved model performance. Here we present evidence to the contrary, particularly when using autoencoders to derive meaningful latent spaces from data with spatially global similarities and local differences, such as electronic health records (EHR) and medical imaging. We investigate batch size effects in both EHR data from the Baltimore Longitudinal Study of Aging and medical imaging data from the multimodal brain tumor segmentation (BraTS) challenge. We train fully connected and convolutional autoencoders to compress the EHR and imaging input spaces, respectively, into 32-dimensional latent spaces via reconstruction losses for various batch sizes between 1 and 100. Under the same hyperparameter configurations, smaller batches improve loss performance for both datasets. Additionally, latent spaces derived by autoencoders with smaller batches capture more biologically meaningful information. Qualitatively, we visualize 2-dimensional projections of the latent spaces and find that with smaller batches the EHR network better separates the sex of the individuals, and the imaging network better captures the right-left laterality of tumors. Quantitatively, the analogous sex classification and laterality regressions using the latent spaces demonstrate statistically significant improvements in performance at smaller batch sizes. Finally, we find improved individual variation locally in visualizations of representative data reconstructions at lower batch sizes. Taken together, these results suggest that smaller batch sizes should be considered when designing autoencoders to extract meaningful latent spaces among EHR and medical imaging data driven by global similarities and local variation.

Fixtures are used to locate, hold and support workpieces in manufacturing operations such as machining, inspection, and assembly. Designing and fabricating fixtures can take up to 10-20% of the total cost of a manufacturing system. Fixture planning is a complex activity restricted by the extreme diversity of workpiece, product batch size, product geometry, part accessibility, and working force. Computer-Aided Fixture Planning (CAFP) has been used to improve the fixture design for over 20 years. CAFP contributes to the reduction of lead-time and human interaction in fixture planning. It helps verification of the fixture quality and integration of fixture design with CAD/CAM systems. This paper summarizes constraints of fixture planning and four phases of CAFP. Various approaches to CAFP are surveyed. Application systems of CAFP including some recent patents are reviewed. The paper concludes with the research trend of CAFP. The global competition and customer demand change lead to greater product variety, smaller product batch size, shorter production time and higher product quality. Industrial organizations have to pursue flexibility and automation in manufacturing systems to remain competitive and profitable. Flexible Manufacturing System (FMS) developed in the past decades is to meet such challenge. FMS usually consists of one or more CNC machines which are capable of machining a variety of workpieces. With the development of Computer- Aided Design and Computer-Aided Manufacturing (CAD/ CAM) technology, product design and machining program- ming have been greatly computerized. However, one of the difficulties in FMS is the fixture design for locating and securing a workpiece during manufacturing operations. Fixture planning is time-consuming as it is customized for workpieces. Computer-aided fixture planning (CAFP) aims to reduce the lead-time and human interaction in fixture planning. It can verify fixture quality to integrate fixture design with CAD/CAM systems by automating the reasoning process. The integration of CAFP with CAD/CAM has greatly reduced the production lead-time and cost for an overall optimal solution in product design and manufacturing (1). Fixtures are used to locate, hold and support workpieces in manufacturing operations such as machining, inspection, and assembly. Fixture design and manufacturing is costly due to the tight tolerance and restricted machining opera- tions. Designing and fabricating fixtures can take up to 10- 20% of the total cost of a manufacturing system. Dedicated fixtures are mostly applied in mass production to gain high efficiency. For smaller batch size, the fixture flexibility becomes more important than efficiency due to the workpiece variety and machining requirements. Flexible and

This chapter presents a multicriteria heuristic model to solve the plant layout problem taking into consideration the quantitative and qualitative factors affecting plant layout in a fuzzy environment. The fuzzy approach integrated with a multicriteria decision making method, Analytical Hierarchy Process (AHP), has been used in the model. An attempt has been made to make the model practical by taking into consideration the product demand, transfer batch size and multiple non-consecutive visits of parts to the same facility.KeywordsAnalytic Hierarchy ProcessLayout DesignLayout ProblemFacility LayoutOperation SequenceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Design of layouts for Cellular Manufacturing Systems (CMSs) has not received attention by researchers as much as cell formation in cellular manufacturing. In this paper a mathematical model, formulated as a multicriteria Quadratic Assignment Problem (QAP) is proposed for the design of layouts for CMSs with the objective of minimising the material handling cost (quantitative) and maximising the closeness rating (qualitative). A Genetic Algorithm (GA)-based tool called FUGEN, which makes use of crossover, mutation (inversion) and direct entry (elitism) with embedded fuzzy logic and Analytic Hierarchy Process (AHP) models, is developed for the design of layouts for CMSs. An attempt is made to make the model practical by considering the production volume, transfer batch size and operational sequence of parts. The proposed tool is validated using an example from the literature.

Contrastive learning has been used to learn useful low-dimensional state representations in visual reinforcement learning (RL). Such state representations substantially improve the sample efficiency of visual RL. Nevertheless, existing contrastive learning-based RL methods have the problem of unstable training. Such instability comes from the fact that contrastive learning requires an extremely large batch size (e.g., 4096 or larger), while current contrastive learning-based RL methods typically set a small batch size (e.g., 512). In this paper, we propose an approach of discrete information bottleneck (DIB) to address this problem. DIB applies the technique of discretization and information bottleneck to contrastive learning in representing the state with concise discrete representation. Using this discrete representation for policy learning results in more stable algorithm training and higher sample efficiency with a small batch size. We demonstrate the advantage of discrete state representation of DIB on several continuous control tasks in the DeepMind Control suite. In the experiments, DIB outperforms prior visual RL methods, both model-based and model-free, in terms of performance and sample efficiency.

Small batch sizes are a consequence of more personalized medicine and reflect a trend in the biopharmaceutical industry. Freeze drying of vials nested in a rack system is a tool used in new flexible pilot scale processing lines. Understanding of heat transfer mechanisms in the rack loaded with vials not in direct contact with each other is necessary to ensure high quality. Lyophilization in the rack vial system enables a homogeneous drying with a reduced edge-vial-effect and shielding against radiation from surrounding components, e.g., the chamber wall. Due to the separation effect of the rack, direct shelf contact contributes approx. 40% to the overall energy transfer to the product during primary drying. Hence overall the rack is a flexible, robust tool for small batch production, which ensures a controlled heat transfer resulting in a uniform product.

High-end manufacturing sectors such as aerospace, marine vessels, rail transportation, energy power, and information electronics impose extremely demanding requirements on manufacturing technologies, and face challenges including multi-variety production, small batch sizes, high reliability, high consistency, and complex manufacturing processes. Meanwhile, production anomalies occur throughout the entire manufacturing flow, and traditional anomaly management methods—plagued by data fragmentation, delayed feedback, and inflexible scheduling—are increasingly inadequate to meet the practical needs of high-end equipment manufacturing. To address this bottleneck, a framework of digital-intelligent anomaly control in the design and manufacturing of high-end equipment is proposed. This framework establishes a key technological system for a collaborative closed loop of “design-scheduling-control” tailored to complex manufacturing requirements, which includes: a large language model(LLM)-driven design approach; a knowledge- and data-fusion driven proactive-reactive scheduling method for smart workshops; and a cloud-edge-device collaborative digital twin control method. By reshaping conventional anomaly management processes through digital-intelligent technologies, this model effectively supports flexible production and high-precision control in multi-variety, small-batch manufacturing scenarios, thereby enhancing the batch production capability of complex products.