A Reinforcement Learning - Great-Deluge Hyper-Heuristic for Examination Timetabling

Hyper-heuristics can be defined as search method for selecting or generating heuristics to solve difficult problem. A high level heuristic therefore operate on a set of low level heuristics with the overall aim of selecting the most suitable set of low level heuristics at a particular point in generating an overall solution. In this work, we propose a set of constructive hyper-heuristics for solving attribute reduction problems. At the high level, the hyper-heuristics (at each iteration) adaptively select the most suitable low level heuristics using roulette wheel selection mechanism. Whilst, at the underlying low level, four low level heuristics are used to gradually, and indirectly construct the solution. The proposed hyper-heuristics has been evaluated on a widely used UCI datasets. Results show that our hyper-heuristic produces good quality solutions when compared against other metaheuristic and outperforms other approaches on some benchmark instances.


 
 
 
 Abstract:
 Aim: This study is on tackling Examination Timetabling Problem (ETP) of the Faculty of Economics And Administrative Sciences (FEAS) of the Ankara HBV University summer school, where the courses of fall and spring semesters are offered simultaneously and regulations on restricting enrollments in inter-department electives or in-department courses of distinct years are relaxed. Thus, the complexity of the nature of the ETP problem is exacerbated. The direct heuristics based on successive assignments that the university normally adopts was proven inadequate for assuming standard regulations hence, another approach we explain in this paper was needed.
 Design / Research methods: The ETP was formulated as a Linear Mixed-Integer Program (LMIP) and decomposed into three stages; timetabling exams, room assignment, student allocation. To manage the conflict between the stakeholders of the examination procedure, a lexicographic optimization process based on the priority of the parties was undertaken.
 Conclusions / findings: After a recursive timetabling process based on a trial-and-error method a clash-free timetable was generated and, a room assignment plan that minimizes the total number of proctoring duties, usage of higher floor rooms and total crowdedness of rooms respectively was put into action. Therefore no student group experienced any clashing exams, the faculty members saved time that can be spent on research instead, since the room usage was better planned the costs (elevator usage, lighting, air conditioning, the labor of the janitors) were assumed to be decreased.
 Originality / value of the article: Each examination period bares a different ETP due to its problem-specific nature (number of courses offered, structure of student enrollments, availability of rooms, etc.). Summer schools provide a more irregular structure that demands special attention, a trial-and-error reformulation of the ETP in our case. In addition, the traditional formulations of the ETP, to the extent we have been able to scan, do not include the minimization of the crowdedness of the rooms. Thus, in creating a more comfortable environment, easier to monitor exams and, ability in handling unexpected dysfunctionalities (broken classroom equipment, etc.) this study is novel.
 Limitations of the research: The algorithms to solve an ETP formulated as an LMIP are of high complexity therefore, we are not able to assert the optimality of our suggested solutions acquired within time limitations.
 
 
 
 
 Keywords: examination timetabling, group decision making, lexicographic optimization, linear mixed-integer programming
 JEL: C44, C61, M12
 
 
 

A double evolutionary pool memetic algorithm is proposed to solve the examination timetabling problem. To improve the performance of the proposed algorithm, two evolutionary pools, that is, the main evolutionary pool and the secondary evolutionary pool, are employed. The genetic operators have been specially designed to fit the examination timetabling problem. A simplified version of the simulated annealing strategy is designed to speed the convergence of the algorithm. A clonal mechanism is introduced to preserve population diversity. Extensive experiments carried out on 12 benchmark examination timetabling instances show that the proposed algorithm is able to produce promising results for the uncapacitated examination timetabling problem.

Solving a real-world university examination timetabling is an intractable problem. Examination timetabling problems (ETP) can be defined as a problem of assigning a set of final examinations schedules to a given number of timeslots and rooms subject to a set of constraints. These constraints are usually divided into hard and soft constraints. Hard constraints must be satisfied under any circumstances. Whilst, soft constraints are not only essential but also should be satisfied as much as possible. Due to the complexity of the problem, it is usually impossible to have solutions that satisfy all soft constraints. The quality of the timetable is measured based on soft constraints violation. A weighted penalty/value is attached to each violation of the soft constraint and the objective is to minimize the total penalty value of these violations. Although there have been a numerous approaches in solving the ETP, but this paper only focus on Genetic Algorithm (GA) method based on the fact that it is robust and can fits into complex problem space. This paper gives a summary of the GA method on ETP and it aims in helping other researcher to shorten their time in understanding the application of GA in the selected problem.

Hyper-Heuristics based on Reinforcement Learning, Balanced Heuristic Selection and Group Decision Acceptance

A selection hyper-heuristic is a high level search methodology which operates over a fixed set of low level heuristics. During the iterative search process, a heuristic is selected and applied to a candidate solution in hand, producing a new solution which is then accepted or rejected at each step. Selection hyper-heuristics have been increasingly, and successfully, applied to single-objective optimization problems, while work on multi-objective selection hyper-heuristics is limited. This work presents one of the initial studies on selection hyper-heuristics combining a choice function heuristic selection methodology with great deluge and late acceptance as non-deterministic move acceptance methods for multi-objective optimization. A well-known hypervolume metric is integrated into the move acceptance methods to enable the approaches to deal with multi-objective problems. The performance of the proposed hyper-heuristics is investigated on the Walking Fish Group test suite which is a common benchmark for multi-objective optimization. Additionally, they are applied to the vehicle crashworthiness design problem as a real-world multi-objective problem. The experimental results demonstrate the effectiveness of the non-deterministic move acceptance, particularly great deluge when used as a component of a choice function based selection hyper-heuristic.

In general, uncapacitated exam timetabling is conducted manually, which can be time-consuming. Many studies aim to automate and optimize uncapacitated exam timetabling. However, pinpointing the most efficient algorithm is challenging since most studies assert that their algorithms surpass previous ones. To identify the optimal algorithm, this research evaluates the performance of four algorithms: Hill climbing (HC), simulated annealing (SA), great deluge (GD), and tabu search (TS) in addressing the exam timetabling problem. The Kempe chain operator’s influence on optimization solutions is also examined. A simple random method is employed to select the low-level heuristic (LLH). The Carter (Toronto) dataset served as the test material, with each algorithm undergoing 200,000 iterations for comparison. The results indicate that the TS algorithm is superior, providing the best solution in 13 instances. The use of a tabu list enhanced the search process’s efficiency by preventing redundant modifications. The Kempe chain LLH exhibited a tendency towards achieving better solutions.

There is a growing body of work in the field of hyper-heuristics. Hyper-heuristics are high level search methodologies that operate on the space of heuristics to solve hard computational problems. A frequently used hyper-heuristic framework mixes a predefined set of low level heuristics during the search process. While most of the work on such selection hyper-heuristics in the literature are empirical, we analyse the runtime of hyper-heuristics rigorously. Our initial analysis shows that mixing heuristics could lead to exponentially faster search than individual (deterministically chosen) heuristics on chosen problems. Both mixing of variation operators and mixing of acceptance criteria are investigated on some selected problems. It is shown that mixing operators is only efficient with the right mixing distribution (parameter setting). Additionally, some of the existing adaptation mechanisms for mixing operators are also evaluated.

Selection hyper-heuristics are methods that are typically used to solve computationally hard optimisation problems (see [1]). A selection hyper-heuristic selects heuristics from a given set of low level heuristics, deciding which heuristic to apply at a given point during the optimisation process. The sequences of low level heuristic selections and objective function values that result from the application of a simple selection hyper-heuristic to the HyFlex problem set (see [3]) are used to construct an offline learning database. The intention is to select effective subsequences of heuristics from this database and use them as inputs to machine learning algorithms in order to improve optimisation.

This study presents an offline learning Simulated Annealing approach to generate a constructive hyper-heuristic evaluated through training and testing on a set of instances for solving the Job-Shop Scheduling problem. The generated hyperheuristic uses a range of state features to control a set of low-level constructive heuristics. A hyper-heuristic is represented in terms of a set of rules, where each rule contains a fixed set of values for the features in consideration and the low level heuristic to be invoked. At each constructive step, the 'closest' rule is selected and then the corresponding constructive low level heuristic is applied. Our distance metric is the Euclidean distance between the values within the rule and the state features characterising the partial schedule along with the remaining jobs to be scheduled for the partial solution. In this paper, we study a set of features computed with various well-known metrics and different feature transformation methods for improving the characterization of the problem instances and solutions to Job-Shop Scheduling as a part of our approach. Eight different scenarios are evaluated on a set of randomly generated problem instances. Each scenario represents a distinct approach combining a different feature transformation applied during the training and testing phases. The empirical results show that transformations can improve the spread of feature values and the choice of the transformation methods is influential on the performance of the overall approach. A particular choice generates a slightly better performance when compared to the standard approach, which uses the original features at all times, indicating the potential of the proposed approach for the future studies.

University examination timetabling problem is a complicated, multi-constraint combinatorial optimization problem. The current grouping genetic algorithm could get a better optimistic solution. Although the block encoding technique was applied in, the probabilities of crossover and mutation operation were still based on simple genetic algorithm. So the universality of this algorithm is not very well because the same probabilities of crossover and mutation operation could obtain different effects in different cases of examination arrangements. To find a more general algorithm, an Improved Adaptive Genetic Algorithm (IAGA) for practical applications was presented in this paper. IAGA can reconstruct the probabilities of crossover and mutation operation according to different cases of examination arrangements. To make further improvement of its convergence speed and overcome premature problem, IAGA appropriately adjusts mutation operation by Population's Maturity in addition. Moreover, the Memory Operator was added into the Algorithm to ensure getting the global optimization solution. Finally, IAGA was tested in function optimization and Examination Timetabling problems, and the experiments show that the results are promoted very well.

An informed genetic algorithm for the examination timetabling problem

An examination timetabling problem at a large American university is presented. Although there are some important differences, the solution approach is based on the ITC 2007 winning solver which is integrated in the open source university timetabling system UniTime. In this work, nine real world benchmark data sets are made publicly available and the results on four of them are presented in this paper. A new approach to further decreasing the number of student conflicts by allowing some exams to be split into multiple examination periods is also studied.

Hyper-heuristics are a domain-independent problem solving approach where the main task is to select effective chains of problem-specific low-level heuristics on the fly for an unseen instance. This task can be seen as a reinforcement learning problem, however, the information available to the hyper-heuristic is very limited, usually leading to very limited state representations. In this work, for the first time we use the trajectory of solution changes for a larger set of features for reinforcement learning in the novel hyper-heuristic LAST-RL (Large-State Reinforcement Learning). Further, we introduce a probability distribution for the exploration case in our epsilon-greedy policy that is based on the idea of Iterated Local Search to increase the chance to sample good chains of low-level heuristics. The benefit of the collaboration of our novel components is shown on the academic benchmark of the Cross Domain Heuristic Challenge 2011 consisting of six different problem domains. Our approach can provide state-of-the-art results on this benchmark where it outperforms recent hyper-heuristics based on reinforcement learning, and also demonstrates high performance on a benchmark of complex real-life personnel scheduling domains.

Recent years have witnessed great success of ant based hyper heuristics applying to real world applications. Ant based hyper heuristics intend to explore the heuristic space by traversing the fully connected graph induced by low level heuristics (LLHs). However, existing ant based models treat LLH in an equivalent way, which may lead to imbalance between the intensification and the diversification of the search procedure. Following the definition of meta heuristics, we propose an Ant based Hyper heuristic with SpAce Reduction (AHSAR) to adapt the search over the heuristic space. AHSAR reduces the heuristic space by replacing the fully connected graph with a bipartite graph, which is induced by the Cartesian product of two LLH subsets. With the space reduction, AHSAR enforces consecutive execution of intensification and diversification LLHs. We apply AHSAR to the p-median problem, and experimental results demonstrate that our algorithm outperforms meta heuristics from which LLHs are extracted.