Concise representations of regular languages by degree and probabilistic finite automata

Many difficult open problems in theoretical computer science center around nondeterminism. We study the fundamental problem of converting a given deterministic finite automaton (DFA) to a minimal nondeterministic finite automaton (NFA). Despite extensive work on finite automata, this fundamental problem has remained open. Recently, in [Ji91] we studied this problem and showed that this (and related) problems are computationally hard. Here we study the restriction of this problem to the case when the input DFA is over a one-letter alphabet. Even in this restricted case the problem is computationally hard even though our evidence of hardness is different from (and is weaker than) the standard ones such as NP-hardness. Let A → B denote the problem of converting a finite automaton of type A to a minimal finite automaton of type B. Our main result is that DFA → NFA, when the input is a unary cyclic DFA (a DFA whose graph is a simple cycle), is in NP but not in P unless NP ⊑ DTIME(nO(log n)). Our work was also motivated by the problem of finding structurally simple ‘normal forms’ of NFA's over a unary alphabet. We present some normal forms for minimal NFA's over a unary alphabet and present an application to lower bounds on the size complexity of an NFA. In fact, the normal form result is used in a nontrivial manner to show the NP membership result stated above.

Regular Expressions (RegExes) are widely used in various applications to identify strings of text. Their flexibility, however, increases the complexity of the detection system and often limits the detection speed as well as the total number of RegExes that can be detected using limited resources. The two classical detection methods, Deterministic Finite Automaton (DFA) and Non-Deterministic Finite Automaton (NFA), have the potential problems of prohibitively large memory requirements and a large number of concurrent operations, respectively. Although recent schemes addressing these problems to improve DFA and NFA are promising, they are inherently limited by their scalability, since they follow the state transition model in DFA and NFA, where the state transitions occur per each character of the input. We recently proposed a scalable RegEx detection system called Lookahead Finite Automata (LaFA) to solve these problems with three novel ideas: 1. Provide specialized and optimized detection modules to increase resource utilizations. 2. Systematically reordering the RegEx detection sequence to reduce number of concurrent operations. 3. Sharing states among automata for different RegExes to reduce resource requirements. In this paper, we propose an efficient hardware architecture and prototype design implementation based on LaFA. Our proof-of-concept prototype design is built on a fraction of a single commodity Field Programmable Gate Array (FPGA) chip and can accommodate up to twenty-five thousand (25k) RegExes. Using only 7% of the logic area and 25% of the memory on a Xilinx Virtex-4 FX100, the prototype design can achieve 2-Gbps (gigabits-per-second) detection throughput with only one detection engine. We estimate that 34-Gbps detection throughput can be achieved if the entire resources of a state-of-the-art FPGA chip are used to implement multiple detection engines.

[Formula: see text] is a binary coded unary regular language, if there exists a unary regular language [Formula: see text] such that [Formula: see text] is in [Formula: see text] if and only if the binary representation of [Formula: see text] is in [Formula: see text]. If a unary language [Formula: see text] is accepted by a minimal deterministic finite automaton (DFA) [Formula: see text] with [Formula: see text] states, then its binary coded version [Formula: see text] is regular and can be accepted by a DFA [Formula: see text] using at most [Formula: see text] states, but at least [Formula: see text] states. There are witness languages matching these upper and lower bounds exactly, for each [Formula: see text]. More precisely, if [Formula: see text] uses [Formula: see text] states in the initial segment and [Formula: see text] states in the loop, where [Formula: see text] is odd and [Formula: see text], then the minimal [Formula: see text] for [Formula: see text] consists of a preamble with at most [Formula: see text] but at least [Formula: see text] states, except for [Formula: see text] with no preamble, and a kernel with at most [Formula: see text] but at least [Formula: see text] states. Also these lower bounds are matched exactly by witness languages, for each [Formula: see text]. If the length of the loop is fixed, the size of the preamble is bounded by [Formula: see text]. The conversion in the opposite way is not always granted: there are binary regular languages the unary versions of which are not even context free. The conversion of a unary nondeterministic finite automaton (NFA) to a binary NFA uses [Formula: see text] states and introduces a binary version of Chrobak normal form.

Proposes a natural subclass of regular languages, called alphabetic pattern constraints (APC), which is effectively closed under permutation rewriting, i.e. under iterative application of rules of the form ab/spl rarr/ba. It is well-known that regular languages do not have this closure property in general. Our result can be applied for example to regular model checking, for verifying properties of parametrized linear networks of regular processes and for modeling and verifying properties of asynchronous distributed systems. We also consider the complexity of testing membership in APC, and show that the question is complete for PSPACE when the input is an NFA (nondeterministic finite automaton) and complete for NLOGSPACE when it is a DFA (deterministic finite automaton). Moreover, we show that both the inclusion problem and the question of closure under permutation rewriting are PSPACE-complete when we restrict ourselves to the APC class.

The finite automata are mathematical models for finite state systems. More general finite automaton is the nondeterministic finite automaton (NFA) that cannot be directly used. It is usually transformed to the deterministic finite automaton (DFA) that then runs in time O(n), where n is the size of the input text. We present two main approaches to practical implementation of DFA considering CPU cache. The first approach (represented by Table Driven and Hard Coded implementations) is suitable forautomata being run very frequently, typically having cycles. The other approach is suitable for a collection of automata from which various automata are retrieved and then run. This second kind of automata are expected to be cycle-free.

Many network security applications in today's networks are based on deep packet inspection, checking not only the header portion but also the payload portion of a packet. For example, traffic monitoring, layer-7 filtering, and network intrusion detection all require an accurate analysis of packet content in search for predefined patterns to identify specific classes of applications, viruses, attack signatures, etc. Regular expressions are often used to represent such patterns. They are implemented using finite automata, which take the payload of a packet as an input string. However, existing approaches, both non-deterministic finite automata (NFA) and deterministic finite automata (DFA), do not deal with compressed traffic, which becomes more and more popular in HTTP applications. In this paper, we propose an efficient algorithm for regular expression matching to implement deep packet inspection on compressed traffic. Based on the observations of DFA, we design a scheme to skip most of the matching process in the compressed parts of traffic. To the best of our knowledge, this is the first effort to design an efficient regular expression matching on compressed traffic. We evaluate our algorithm using rule sets provided by Snort, a popular open-source intrusion detection system. The evaluation results show that our approach can reduce the number of state access in the DFA significantly.

It is known that a nondeterministic finite automaton (NFA) with n states and branching k can be simulated by a deterministic finite automaton with multiple initial states (MDFA) having k ·n states. We give a lower bound \(\frac{k}{1 + \log k} \cdot n\) for the size blow-up of this conversion. We consider also upper and lower bounds for the number of states an MDFA needs to simulate a given NFA of finite tree width.

Chapter 1 - Formal Requirements and Finite Automata Overview

Many network security applications such as Intrusion Detection System (IDS), Firewall and Data Loss Prevention System (DLPS) are based on deep packet inspection, in this packets header as well as payload of the packets are checked with predefined attack signature to identify whether it contains malicious traffic or not. To perform this checking different pattern matching methods are used by NIDS. The most popular method to implement pattern matching is to use of Finite Automata (FA). Generally, regular expressions are used to represent most of the attack signatures defined by NIDS. They are implemented using finite automata, which takes the payload of packet as input string. However, existing approaches of Finite Automata (FA), both deterministic finite automata (DFA) and non-deterministic finite automata (NFA) for pattern matching are having their own advantages and some drawbacks. The DFA based pattern matching methods are fast enough but require more memory. However, NFA based pattern matching methods are comparatively takes less memory but the speed of matching is very slow, to overcome these drawbacks of finite automata there are many approaches have been proposed. This paper discuses comparative study of some Finite Automata (FA) based techniques for pattern matching in network intrusion detection system (NIDS).

Generalized acceptance, succinctness and supernondeterministic finite automata

The equivalence of non-deterministic finite automata (NFA ), deterministic finite automata (DFA), regular expressions (RE), and parallel finite automata (PFA) with augmented regular expressions (PRE) is used to relate parallel finite automata to the shuffle operation on regular languages. Our goal is to capitalize on the equivalence of regular expressions and augmented regular expressions to increase parallelization efficiency. We use the existing automated transformation algorithms for the various finite automata listed above, as well as one of our own, to inform a transformation from augmented regular expressions to minimized deterministic finite automata (DFAmin). We then discuss the serialization of augmented regular expressions, and present a brief summary of potential applications, related research, and future directions

We present a unified view to sequential algorithms for many pattern matching problems, using a finite automaton built from the pattern which uses the text as input. We show the limitations of deterministic finite automata (DFA) and the advantages of using a bitwise simulation of non-deterministic finite automata (NFA). This approach gives very fast practical algorithms which have good complexity for small patterns on a RAM machine with word length O(log n), where n is the size of the text. For generalized string matching the time complexity is O(mn/log n) which for small patterns is linear. For approximate string matching we show that the two main known approaches to the problem are variations of the NFA simulation. For this case we present a different simulation technique which gives a running time of O(n) independently of the maximum number of errors allowed, k, for small patterns. This algorithm improves the best bit-wise or comparison based algorithms of running time O(kn) and can be used as a basic block for algorithms with good average case behavior. We also formalize previous bit-wise simulation of general NFAs achieving O(mn log log n/log n) time.

In multi-agent systems, to improve task completion efficiency, tasks need to be forwarded among agents to reach as many agents as possible. Existing studies suffer from limitations such as over-reliance on global information, low efficiency in state synchronization, and poor practicality. To address these issues, this paper proposes a task forwarding model inspired by game theory. The model considers agent distribution, communication environment, and the agents' own willingness, and conducts research on a 2D square lattice network based on a public goods game with differentiated contributions. Furthermore, the concept of finite state machines (FSM) is introduced, with the improved deterministic finite automaton (DFA) algorithm and non-deterministic finite automaton (NFA) algorithm incorporated sequentially. Results show that under the same conditions, integrating the aforementioned algorithms increases the proportion of agents adopting forwarding strategies and accelerates game convergence, while the model also responds rapidly to dynamic external environments. Additionally, this paper explores how agent density in the model affects the evolutionary speed of the game. Findings indicate that in this model, higher agent density does not necessarily result in faster game evolution; on the contrary, an opposite pattern may emerge under certain environmental conditions.

We introduce E-equivalence, which is a straightforward generalization of almost-equivalence. While almost-equivalence asks for ordinary equivalence up to a finite number of exceptions, in E-equivalence these exceptions or errors must belong to a (regular) set E. The computational complexity of minimization problems and their variants w.r.t. almost- and E-equivalence are studied. Roughly speaking, whenever nondeterministic finite automata (NFAs) are involved, most minimization problems, and their equivalence problems they are based on, become PSPACE-complete, while for deterministic finite automata (DFAs) the situation is more subtle. For instance, hyper-minimizing DFAs is NL-complete, but E-minimizing DFA s is NP-complete, even for finite E. The obtained results nicely fit to the known ones on ordinary minimization for finite automata. Moreover, since hyper-minimal and E-minimal automata are not necessarily unique (up to isomorphism as for minimal DFAs), we consider the problem of counting the number of these minimal automata. It turns out that counting hyper-minimal DFAs can be done in FP, while counting E-minimal DFA s is #P-hard, and belongs to the counting class #·coNP.

The aim of the paper is to concoct a novel algorithm for the metamorphosis of parallel regular expressions to e-free non- deterministic finite automata. For a given parallel regular expression r, let m be the number of symbols that occur in r and let C denote the number of concatenation operators in r. In the worst case, 2 m+1 states are required for the construction of the non-deterministic finite automaton using the novel algorithm. In the earlier existing approaches, th e number of states of the non-deterministic finite automaton in the worst case is equal to 2 2jrj−3C.