Impacts of Matched Batch Sizes on Time Reduction in Construction Projects

Batch size is a key hyperparameter in training deep learning models. Conventional wisdom suggests larger batches produce improved model performance. Here we present evidence to the contrary, particularly when using autoencoders to derive meaningful latent spaces from data with spatially global similarities and local differences, such as electronic health records (EHR) and medical imaging. We investigate batch size effects in both EHR data from the Baltimore Longitudinal Study of Aging and medical imaging data from the multimodal brain tumor segmentation (BraTS) challenge. We train fully connected and convolutional autoencoders to compress the EHR and imaging input spaces, respectively, into 32-dimensional latent spaces via reconstruction losses for various batch sizes between 1 and 100. Under the same hyperparameter configurations, smaller batches improve loss performance for both datasets. Additionally, latent spaces derived by autoencoders with smaller batches capture more biologically meaningful information. Qualitatively, we visualize 2-dimensional projections of the latent spaces and find that with smaller batches the EHR network better separates the sex of the individuals, and the imaging network better captures the right-left laterality of tumors. Quantitatively, the analogous sex classification and laterality regressions using the latent spaces demonstrate statistically significant improvements in performance at smaller batch sizes. Finally, we find improved individual variation locally in visualizations of representative data reconstructions at lower batch sizes. Taken together, these results suggest that smaller batch sizes should be considered when designing autoencoders to extract meaningful latent spaces among EHR and medical imaging data driven by global similarities and local variation.

Normalization layers play an important role in deep network training. As one of the most popular normalization techniques, batch normalization (BN) has shown its effectiveness in accelerating the model training speed and improving model generalization capability. The success of BN has been explained from different views, such as reducing internal covariate shift, allowing the use of large learning rate, smoothing optimization landscape, etc. To make a deeper understanding of BN, in this work we prove that BN actually introduces a certain level of noise into the sample mean and variance during the training process, while the noise level depends only on the batch size. Such a noise generation mechanism of BN regularizes the training process, and we present an explicit regularizer formulation of BN. Since the regularization strength of BN is determined by the batch size, a small batch size may cause the under-fitting problem, resulting in a less effective model. To reduce the dependency of BN on batch size, we propose a momentum BN (MBN) scheme by averaging the mean and variance of current mini-batch with the historical means and variances. With a dynamic momentum parameter, we can automatically control the noise level in the training process. As a result, MBN works very well even when the batch size is very small (e.g., 2), which is hard to achieve by traditional BN.

A GA optimization model for workgroup-based repetitive scheduling (WoRSM)

This paper presents a multiobjective optimization model for the planning and scheduling of repetitive construction projects. The model enables construction planners to generate and evaluate optimal construction plans that minimize project duration and maximize crew work continuity, simultaneously. The computations in the present model are organized in three major modules: scheduling, optimization, and ranking modules. First, the scheduling module uses a resource-driven scheduling algorithm to develop practical schedules for repetitive construction projects. Second, the optimization module utilizes multiobjective genetic algorithms to search for and identify feasible construction plans that establish optimal tradeoffs between project duration and crew work continuity. Third, the ranking module uses multiattribute utility theory to rank the generated plans in order to facilitate the selection and execution of the best overall plan for the project being considered. An application example is analyzed to illustrate the use of the model demonstrate its new capabilities in optimizing the planning and scheduling of repetitive construction projects.

Deep learning architectures are usually proposed with millions of parameters, resulting in a memory issue when training deep neural networks with stochastic gradient descent type methods using large batch sizes. However, training with small batch sizes tends to produce low quality solution due to the large variance of stochastic gradients. In this paper, we tackle this problem by proposing a new framework for training deep neural network with small batches/noisy gradient. During optimization, our method iteratively applies a proximal type regularizer to make loss function strongly convex. Such regularizer stablizes the gradient, leading to better training performance. We prove that our algorithm achieves comparable convergence rate as vanilla SGD even with small batch size. Our framework is simple to implement and can be potentially combined with many existing optimization algorithms. Empirical results show that our method outperforms SGD and Adam when batch size is small. Our implementation is available at https://github.com/huiqu18/TRAlgorithm.

This article focuses on enhancing the performance of supervisory consultants in construction projects through assessments conducted in the LKPP's SIKaP application. Supervision plays a crucial role in ensuring the smooth execution of construction activities and addressing potential obstacles. The quality and safety of construction operations have become increasingly important, necessitating effective supervision. The article highlights the importance of conducting assessments to measure the success of supervisory consultants in meeting project objectives and adhering to established standards. It emphasizes the evaluation of various performance criteria, including human resources, equipment, materials, costs, time, and quality. The assessment process enables the identification of strengths and weaknesses, facilitating corrective measures and improving future performance. The article also emphasizes the interconnectedness of construction supervision and project management, with project management playing a pivotal role in achieving project goals. Communication management is highlighted as a crucial aspect for effective project execution. The regulatory framework governing technical supervision activities in construction projects is discussed, along with the roles and responsibilities of construction management and construction supervision service providers. The article emphasizes the importance of on-site inspections, project administration, progress monitoring, and quality management in construction supervision. Lastly, the article discusses the evaluation of providers' performance in goods/services procurement, emphasizing the importance of quality, cost efficiency, and adherence to standards. The implementation of the SIKaP application is seen as a means to enhance transparency, accountability, and project quality in the construction industry, fostering a culture of continuous improvement and industry benchmarking.

Deep-learners have many hyper-parameters including learning rate, batch size, kernel size - all playing a significant role toward estimating high quality models. Discovering useful hyper-parameter guidelines is an active area of research, though the state of the art generally uses a brute force, uniform grid approach or random search for finding ideal settings. We share the preliminary results of using an alternative approach to deep learner hyper-parameter tuning that uses an evolutionary algorithm to improve the accuracy of a deep-learner models used in satellite imagery building footprint detection. We found that the kernel and batch size hyper-parameters surprisingly differed from sizes arrived at via a brute force uniform grid approach. These differences suggest a novel role for evolutionary algorithms in determining the number of convolution layers, as well as smaller batch sizes in improving deep-learner models.

The efficient utilization of resources in repetitive construction projects is crucial to optimizing schedules and reducing costs. This study investigates the application of Discrete Event Simulation (DES) to enhance resource utilization in scheduling repetitive construction activities, specifically the construction of Type-36 houses in Palu City, Indonesia. Traditional project-based planning methods, such as the Critical Path Method (CPM), are compared with production-based planning approaches. The study shows that although CPM can effectively determine project completion times, it falls short in optimizing resource allocation, particularly in scenarios involving repetitive tasks. By employing the DES tool SIMUL8, the study models the construction process and simulates various resource allocation scenarios to identify the optimal resources required. The results indicate that DES provides a more dynamic and accurate analysis, allowing real-time adjustments and improved resource management. The study concludes that integrating DES into construction project management can significantly enhance efficiency, reduce project duration, and optimize resource utilization.

Test Time Adaptation (TTA) has garnered significant attention in recent years, with the research focus on addressing distribution shifts during test time. As one fundamental component of many TTA methods, the Batch Normalization (BN) layer plays a crucial role in enabling the model adaptability. However, existing BN strategies can prove detrimental when the batch size is (extremely) small. In numerous real-world scenarios, limited hardware resources or just-in-time demand often necessitates adjusting models with very small batch sizes, making existing methods less practical. In this paper, we first showcase and thoroughly analyze the negative transfer phenomenon in previous TTA methods encountering extremely small batch sizes. Subsequently, we propose a novel batch size-agnostic method called NanoAdapt to effectively mitigate the negative transfer even with batch size 1. NanoAdapt is composed of three key components: a dynamic BN calibration strategy that leverages historical information and the Taylor series to refine the statistics estimations, an entropy-weighted gradient accumulation strategy that uses the entropy of each sample's label prediction to weigh and accumulate the loss for backpropagation, and a novel proxy computation graph to capture the sample interactions. Extensive experiments are conducted to validate the superiority of NanoAdapt, showing its consistent efficacy in improving existing TTA methods.

System development for non-unit based repetitive project scheduling

A Multi Criteria Decision Making Model for Prioritizing the Non-Critical Activities in Construction Projects

Construction projects contribute to economic activities, leading to multiplier effects that boost an economy. Nevertheless, construction activities have contributed to increasing air pollution emissions, leading to challenges for the site workers and residents dwelling around the construction site. This paper provides a systematic literature review on the types of air pollutants emitted during construction project processes, the air emission sources, and the effects of air pollution on exposed individuals. Various secondary sources of data were used, published from 2000 to 2023. Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and systematic literature review were used in the literature selection. This study established that the most predominant air pollutant of concern released during construction project activities is particulate matter. Other air pollutants, such as carbon monoxide and oxides of nitrogen, were also identified. The main sources of air pollution from construction project processes were demolition works, quarry sites, machinery/equipment, and on-road traffic. It is essential for construction industry stakeholders to be conscious of air pollution effects caused by construction processes and to proffer solutions to curtail this menace. This will assist in reducing the devastating air pollution effect on people and ensure sustainable construction processes.

Contrastive learning has been used to learn useful low-dimensional state representations in visual reinforcement learning (RL). Such state representations substantially improve the sample efficiency of visual RL. Nevertheless, existing contrastive learning-based RL methods have the problem of unstable training. Such instability comes from the fact that contrastive learning requires an extremely large batch size (e.g., 4096 or larger), while current contrastive learning-based RL methods typically set a small batch size (e.g., 512). In this paper, we propose an approach of discrete information bottleneck (DIB) to address this problem. DIB applies the technique of discretization and information bottleneck to contrastive learning in representing the state with concise discrete representation. Using this discrete representation for policy learning results in more stable algorithm training and higher sample efficiency with a small batch size. We demonstrate the advantage of discrete state representation of DIB on several continuous control tasks in the DeepMind Control suite. In the experiments, DIB outperforms prior visual RL methods, both model-based and model-free, in terms of performance and sample efficiency.

In the just-in-time (JIT) manufacturing system, the vendors ship in small batch sizes. Hence the buyers are at an advantageous position as they can reduce their inventory holding costs. The practice of shipping in smaller batch sizes often results in the vendors incurring high costs as the number of production setups and the number of shipments increases for the vendors. The vendors could only switch to JIT mode when there is an assurance from the buyer of providing some compensation for the increased costs. This compensation from the buyer to the vendors is called ‘surcharge price’. In our work, we have studied the surcharge pricing mechanism in the presence of backorder when there is a powerful vendor and have shown that the it can coordinate a decentralized supply chain. Our study incorporates both the cases of the buyer holding full information about the vendors’ costs as well as the case of information asymmetry. In the case of full information, we have shown that the optimal order quantity in the presence of surcharge pricing is less than that of the vendor dominated scenario. At the same time, the optimal order quantity is higher than that in the buyer dominated scenario. Further, the total supply chain costs are also minimized when a surcharge pricing contract is offered in comparison to both the buyer and the vendor dominated scenarios. For the information asymmetry case, we have framed the optimal set of screening contracts and have shown that an optimal surcharge pricing contract to the vendor is free from the probability distribution of the vendor type.

Artificial neural network training with gradient descent can be destabilized by ‘bad batches’ with high losses. This is often problematic for training with small batch sizes, high order loss functions or unstably high learning rates. To stabilize learning, we have developed adaptive learning rate clipping (ALRC) to limit backpropagated losses to a number of standard deviations above their running means. ALRC is designed to complement existing learning algorithms: Our algorithm is computationally inexpensive, can be applied to any loss function or batch size, is robust to hyperparameter choices and does not affect backpropagated gradient distributions. Experiments with CIFAR-10 supersampling show that ALCR decreases errors for unstable mean quartic error training while stable mean squared error training is unaffected. We also show that ALRC decreases unstable mean squared errors for scanning transmission electron microscopy supersampling and partial scan completion. Our source code is available at https://github.com/Jeffrey-Ede/ALRC.