Large-Scale and Big Optimization Based on Hadoop

This paper presents a distributed server allocation model with preventive start-time optimization against a single server failure. The presented model preventively determines the assignment of servers to users under each failure pattern to minimize the largest maximum delay among all failure patterns. We formulate the proposed model as an integer linear programming (ILP) problem. We prove the NP-completeness of the considered problem. As the number of users and that of servers increase, the size of ILP problem increases; the computation time to solve the ILP problem becomes excessively large. We develop a heuristic approach that applies simulated annealing and the ILP approach in a hybrid manner to obtain the solution. Numerical results reveal that the developed heuristic approach reduces the computation time by 26% compared to the ILP approach while increasing the largest maximum delay by just 3.4% in average. It reduces the largest maximum delay compared with the start-time optimization model; it avoids the instability caused by the unnecessary disconnection permitted by the run-time optimization model.

As millimeter wave (mmWave) technology becomes more integrated into today's communication infrastructure, its application to mobile scenarios has attracted considerable interests. In particular, mmWave technology is one of the leading candidates in Vehicle-to-Everything (V2X) community to address the ever-rising data exchange demand from vehicles. However, the inherent directionality of the technology poses challenges to peer discovery and transmission scheduling in typical V2X scenarios, causing significant loss of network capacity. To mitigate this issue, we propose a novel sub-6 GHz V2X-assisted synchronous mmWave communication scheduler in this paper. The sub-6 GHz channel serves as control channel for mmWave scheduling. The scheduler carefully considers data importance and freshness in determining communication schedule, such that network utility across mmWave links is maximized. We formulate the scheduling problem into an integer linear programming (ILP) problem. However, ILP problem is NP-hard and has exponential computational complexity for the worst cases. We then develop a scheduling algorithm, maximum weight matching based iterative algorithm (MWMIA), that operates in polynomial time with performance guarantees for the NP-hard ILP, by leveraging structural properties of link conflicts. Comparative simulation studies verify the effectiveness of our scheduler against other solutions.

Ranking in quadratic integer programming problems

In elastic optical networks (EONs), a major obstacle to using the spectrum resources efficiently is the spectrum fragmentation. In the literature, several defragmentation approaches have been presented. For 1+1 path protection, conventional defragmentation approaches consider designated primary and backup paths. This exposes the spectrum to fragmentations induced by the primary lightpaths, which are not to be disturbed in order to achieve hitless defragmentation. This paper proposes a defragmentation scheme using path exchanging in 1+1 path protected EONs. We exchange the path function of the 1+1 protection with the primary toggling to the backup state, while the backup becomes the primary. This allows both lightpaths to be reallocated during the defragmentation process, while they work as backup, offering hitless defragmentation. Considering path exchanging, we define a static spectrum reallocation optimization problem that minimizes the spectrum fragmentation while limiting the number of path exchanging and reallocation operations. We then formulate the problem as an integer linear programming (ILP) problem. We prove that a decision version of the defined static reallocation problem is NP-complete. We present a spectrum defragmentation process for dynamic traffic, and introduce a heuristic algorithm for the case that the ILP problem is not tractable. The simulation results show that the proposed scheme outperforms the conventional one and improves the total admissible traffic up to 10%.

Recently, space-division multiplexing (SDM) has been incorporated with elastic optical networks (EONs) to enhance the fiber capability, which forms space-division multiplexing-based elastic optical networks (SDM-EONs). During transmission of optical signals through multi-core fibers, inter-core crosstalk (XT) and intra-core physical layer impairments (PLIs) arise, which deteriorate the signal quality. Existing models typically handle inter-core XT and intra-core PLIs separately and set a single XT threshold for each modulation format, which results in an unacceptable lightpath due to the degradation of signal quality or leads to inefficient spectrum utilization. This paper proposes a routing, modulation, spectrum, and core allocation (RMSCA) model for SDM-EONs to consider inter-core XT and intra-core PLIs jointly. For each modulation format, it sets different XT thresholds and transmission reaches according to inter-core XT and intra-core PLIs. An optimization problem is formulated as an integer linear programming (ILP) problem. We prove that the RMSCA decision problem is NP-complete. We introduce a heuristic algorithm when the ILP problem is not tractable. Numerical results demonstrate that, by setting several XT limits for each modulation format, the proposed model increases spectrum efficiency compared to a benchmark model based on the literature.

Network Function Virtualization (NFV) is a paradigm that virtualizes traditional network functions and instantiates Virtual Network Functions (VNFs) as software instances separate from hardware appliances. Service Chaining (SC), seen as one of the major NFV use cases, provides customized services to users by concatenating VNFs. A VNF placement model for SC that relaxes the visit order constraints of requested VNFs has been considered. Relaxing the VNF visit order constraints reduces the number of VNFs which need to be placed in the network. However, since the model does not permit any loop within an SC path, the efficiency of utilization of computation resources deteriorates in some topologies. This paper proposes a VNF placement model for SC which minimizes the cost for placing VNFs and utilizing link capacity while allowing both relaxation of VNF visit order constraints and configuration of SC paths including loops. The proposed model determines routes of requested SC paths, which can have loops, by introducing a logical layered network generated from an original physical network. This model is formulated as an Integer Linear Programming (ILP) problem. A heuristic algorithm is introduced for the case that the ILP problem is not tractable. Simulation results show that the proposed model provides SC paths with smaller cost compared to the conventional model.

This paper addresses the fault diagnosis problem in discrete event systems using backward conflict-free Petri nets, where two different transitions have no common output places. The notion of generalized markings is employed, where the number of tokens in a place can be negative. Based on the special structures of backward conflict-free Petri nets and the generalized marking corresponding to an observed transition sequence, an integer linear programming (ILP) problem is constructed where the numbers of unknown integer variables and constraints are independent of the length of the observed transition sequence. A diagnosis algorithm is proposed by associating two different objective functions with the ILP problem and solving it. The computational efficiency of the proposed algorithm depends on the net structure only, regardless of the length of an observed transition sequence.

An accelerated Benders decomposition approach for virtual base station formation in stochastic Cloud-RANs

Space-division multiplexing-based elastic optical networks (SDM-EONs) enhance the fiber capacity. Inter-core crosstalk (XT) and intra-core physical layer impairments (PLIs) occur in the multi-core fiber and degrade the optical signal. An existing model separately considers inter-core XT and intra-core PLIs, and sets a single XT threshold to each modulation format. This can lead to an unacceptable lightpath due to signal degradation or the occurrence of spectrum inefficiency. This paper proposes a routing, modulation, spectrum, and core allocation (RMSCA) model, which jointly considers inter-core XT and intra-core PLIs for SDM-EONs. The proposed model sets multiple XT thresholds for each modulation format based on inter-core XT and intra-core PLIs. We present an optimization problem and formulate it as an integer linear programming (ILP) problem. We introduce a heuristic algorithm for a network where the ILP problem is not tractable. Numerical results observe that the proposed model improves the spectrum efficiency by setting multiple XT thresholds to each modulation format.

Optimized solvers for the Boolean satisfiability (SAT) problem have many applications in areas such as hardware and software verification, FPGA routing, planning, and so forth. Further uses are complicated by the need to express "counting constraints" in conjunctive normal form (CNF). Expressing such constraints by pure CNF leads to more complex SAT instances. Alternatively, those constraints can be handled by integer linear programming (ILP), but generic ILP solvers may ignore the Boolean nature of 0-1 variables. Therefore, specialized 0-1 ILP solvers extend SAT solvers to handle these so-called "pseudo-Boolean" (PB) constraints. This work provides an update on the on-going competition between generic ILP techniques and specialized 0-1 ILP techniques. To make a fair comparison, we generalize recent ideas for fast SAT-solving to more general 0-1 ILP problems that may include counting constraints and optimization. This generalization is embodied in our PB constraint solver and optimizer PBS, which is compared with state-of-the-art CNF and generic ILP solvers. Another aspect of our comparison is the evaluation on 0-1 ILP benchmarks that originate in electronic design automation (EDA) but that cannot be directly solved by an SAT solver. Specifically, we solve instances of the max-SAT and max-ONEs optimization problems, which seek to maximize the number of satisfied clauses and the "true" values over all satisfying assignments, respectively. Those problems have straightforward applications to SAT-based routing and are additionally important due to reductions from max-cut, max-clique, and min vertex cover. Our experimental results show that specialized 0-1 techniques implemented in PBS tend to outperform generic ILP techniques on Boolean optimization problems, as well as on general EDA SAT problems.

Data-parallel applications generate a mix of coflows with and without deadlines. Deadline coflows are mission-critical and must be completed within deadlines, while the non-deadline coflows desire to be completed as soon as possible. Scheduling such mix-coflows is an important problem in modern datacenters. However, existing solutions only focus on one of the two types of coflows: they either solely concentrate on meeting the deadlines of deadline-aware coflows or reducing the coflow completion times (CCTs) of non-deadline coflows. In this article, we study the problem of optimizing deadline and non-deadline coflows simultaneously. To this end, we present a new optimization framework, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mixCoflow</i> , to schedule deadline coflows to minimize and balance their bandwidth footprint, such that non-deadline coflows can be scheduled as early as possible. Specifically, we develop the mathematical model and formulate the scheduling problem for deadline coflows as a lexicographical min-max integer linear programming (ILP) problem. Through rigorous theoretical analysis, this ILP problem has been proved to be equivalent to a linear programming (LP) problem that can be solved with standard LP solvers. By solving this LP, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mixCoflow</i> is able to balance the bandwidth footprint of deadline coflows while guaranteeing their deadlines. As a result, non-deadline coflows can be scheduled as soon as possible whenever they arrive. To demonstrate the effectiveness of our work, we have conducted extensive simulations based on a widely used Facebook data trace. The simulation results verify that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mixCoflow</i> can achieve significant improvement on the average CCT of non-deadline coflows, at no expense of increasing the deadline miss rates of deadline coflows, when compared to the state-of-art solutions.

Open Radio Access Network (O-RAN) is a novel architecture aiming to disaggregate the network components to reduce capital and operational costs and open the interfaces to ensure interoperability. In this work, we consider the problem of allocating computing resources to process the data of enhanced Mobile BroadBand (eMBB) users and Ultra-Reliable Low-Latency (URLLC) Users. Supposing the processing of users’ frames from different base stations is done in a shared O-Cloud, we model the computing resources allocation problem as an Integer Linear Programming (ILP) problem that aims at fairly allocating computing resources to eMBB and URLLC users and optimizing the QoS of URLLC users without neglecting eMBB users. Due to the high complexity of solving an ILP problem, we model the problem using Reinforcement Learning (RL). Our results demonstrate the ability of our RL-based solution to perform close to the ILP solver while having much lower computational complexity. For a different number of Open Radio Units (O-RUs), the objective value of the RL agent does not deviate from the ILP objective by more than 6%.

With the emergence of increasingly computing-intensive and delay-sensitive tasks, the processing of computing tasks on cloud servers cannot meet the current needs any longer. The emergence of mobile edge computing (MEC) technology and the popularity of 5G applications can solve these demands. Offloading tasks to the MEC server reduces the energy consumption of local devices, and also has a lower latency than offloading to the cloud server. In this paper, an MEC–edge cloud server collaborative system model with energy harvesting technology is designed to minimize the processing delay of computing tasks by allocating computing resources. We propose an optimal integer linear programming (OILP) algorithm with two steps. Firstly, we propose a Lyapunov stability optimization algorithm based on task priority. With the constraints of local mobile device power stability, the divide-and-conquer idea is used for solving the target values of the processing tasks locally, and the MEC and edge cloud servers separately. Therefore, the objective problem is transformed into an integer linear programming problem, and then an integer linear programming algorithm based on CPU utilization optimization is proposed to obtain a resource allocation scheme. Simulation results show that the proposed OILP algorithm can further reduce the delay, improve the CPU’s utilization of the MEC server, and reduce the number of the tasks that cannot be processed under the condition of the energy stability of the local device.

Metaheuristic and exact methods are one of the most common tools to solve Mixed-Integer Optimization Problems (MIPs). Most of these problems are NP-hard problems, being intractable to obtain optimal solutions in a reasonable time when the size of the problem is huge. In this paper, a hybrid parallel optimization algorithm for matheuristics is studied. In this algorithm, exact and metaheuristic methods work together to solve a Mixed Integer Linear Programming (MILP) problem which is divided into two different subproblems, one of which is linear (and easier to solve by exact methods) and the other discrete (and is solved using metaheuristic methods). Even so, solving this problem has a high computational cost. The algorithm proposed follows an efficient decomposition which is based on the nature of the decision variables (continuous versus discrete). Because of the high cost of the algorithm, as this kind of problem belongs to NP-hard problems, parallelism techniques have been incorporated at different levels to reduce the computing cost. The matheuristic has been optimized both at the level of the problem division and internally. This configuration offers the opportunity to improve the computational time and the fitness function. The paper also focuses on the performance of different optimization software packages working in parallel. In particular, a comparison of two well-known optimization software packages (CPLEX and GUROBI) is performed when they work executing several simultaneous instances, solving various problems at the same time. Thus, this paper proposes and studies a two-level parallel algorithm based on message-passing (MPI) and shared memory (Open MP) schemes where the two subproblems are considered and where the linear problem is solved by using and studying optimization software packages (CPLEX and GUROBI). Experiments have also been carried out to ascertain the performance of the application using different programming paradigms (shared memory and distributed memory).

Diagnosability of faults in discrete event systems modeled with Petri nets can be assessed either via graph-based techniques (also called diagnoser, verifier/twin-plant based techniques), or via the solution of optimization problems. The approaches that belong to the former class are based on the analysis of the net reachability or coverability graphs (or of a more compact version of them). The latter approach exploits the mathematical representation of the net itself to specify and solve optimization problems, which are usually expressed as integer linear programming (ILP) problems. In this paper we exploit the railway Petri net model originally proposed in [16], and extended in [14] to be used as a benchmark for diagnosability analysis, to assess the efficiency of the approach based on the solution of ILP problems proposed in [3]. In order to show the effectiveness of the proposed technique, a comparison with a graph-based approach for analyzing diagnosability is also presented.