Classical Language Equations and the Substitution Property

Chapter 1 - Formal Requirements and Finite Automata Overview

Regular Expressions (RegExes) are widely used in various applications to identify strings of text. Their flexibility, however, increases the complexity of the detection system and often limits the detection speed as well as the total number of RegExes that can be detected using limited resources. The two classical detection methods, Deterministic Finite Automaton (DFA) and Non-Deterministic Finite Automaton (NFA), have the potential problems of prohibitively large memory requirements and a large number of concurrent operations, respectively. Although recent schemes addressing these problems to improve DFA and NFA are promising, they are inherently limited by their scalability, since they follow the state transition model in DFA and NFA, where the state transitions occur per each character of the input. We recently proposed a scalable RegEx detection system called Lookahead Finite Automata (LaFA) to solve these problems with three novel ideas: 1. Provide specialized and optimized detection modules to increase resource utilizations. 2. Systematically reordering the RegEx detection sequence to reduce number of concurrent operations. 3. Sharing states among automata for different RegExes to reduce resource requirements. In this paper, we propose an efficient hardware architecture and prototype design implementation based on LaFA. Our proof-of-concept prototype design is built on a fraction of a single commodity Field Programmable Gate Array (FPGA) chip and can accommodate up to twenty-five thousand (25k) RegExes. Using only 7% of the logic area and 25% of the memory on a Xilinx Virtex-4 FX100, the prototype design can achieve 2-Gbps (gigabits-per-second) detection throughput with only one detection engine. We estimate that 34-Gbps detection throughput can be achieved if the entire resources of a state-of-the-art FPGA chip are used to implement multiple detection engines.

We give two new algorithms for constructing small nondeterministic finite automata (NFA) from regular expressions. The first constructs NFAs with ?-moves (?NFA) which are smaller than all the other ?NFAs obtained by similar constructions. Their size is at most 3/2 |?|+ 5/2, where ? is the regular expression. The second constructs NFAs. It uses ?-elimination in the ?NFAs we just introduced and builds a quotient of the well-known position automaton. Our NFA is always smaller and faster to compute than the position automaton. It uses optimally the information from the positions of a regular expression.

Minimization of nondeterministic finite automata (NFA) is a hard problem (PSPACE-complete). Bisimulations are then an attractive alternative for reducing the size of NFAs, as even bisimilarity (the largest bisimulation) is almost linear using the Paige and Tarjan algorithm. NFAs obtained from regular expressions (REs) can have the number of states linear with respect to the size of the REs and conversion methods from REs to equivalent NFAs can produce NFAs without or with transitions labelled with the empty word (e-NFA). The standard conversion without e-transitions is the position automaton, \(\mathcal{A}_{pos}\). Other conversions, such as partial derivative automata (\(\mathcal{A}_{pd}\)) or follow automata (\(\mathcal{A}_{f}\)), were proven to be quotients of the position automata (by some bisimulations). Recent experimental results suggested that for REs in (normalized) star normal form the position bisimilarity almost coincide with the \(\mathcal{A}_{pd}\) automaton. Our goal is to have a better characterization of \(\mathcal{A}_{pd}\) automata and their relation with the bisimilarity of the position automata. In this paper, we consider \(\mathcal{A}_{pd}\) automata for regular expressions without Kleene star and establish under which conditions they are isomorphic to the bisimilarity of \(\mathcal{A}_{pos}\).

Many difficult open problems in theoretical computer science center around nondeterminism. We study the fundamental problem of converting a given deterministic finite automaton (DFA) to a minimal nondeterministic finite automaton (NFA). Despite extensive work on finite automata, this fundamental problem has remained open. Recently, in [Ji91] we studied this problem and showed that this (and related) problems are computationally hard. Here we study the restriction of this problem to the case when the input DFA is over a one-letter alphabet. Even in this restricted case the problem is computationally hard even though our evidence of hardness is different from (and is weaker than) the standard ones such as NP-hardness. Let A → B denote the problem of converting a finite automaton of type A to a minimal finite automaton of type B. Our main result is that DFA → NFA, when the input is a unary cyclic DFA (a DFA whose graph is a simple cycle), is in NP but not in P unless NP ⊑ DTIME(nO(log n)). Our work was also motivated by the problem of finding structurally simple ‘normal forms’ of NFA's over a unary alphabet. We present some normal forms for minimal NFA's over a unary alphabet and present an application to lower bounds on the size complexity of an NFA. In fact, the normal form result is used in a nontrivial manner to show the NP membership result stated above.

Many network security applications such as Intrusion Detection System (IDS), Firewall and Data Loss Prevention System (DLPS) are based on deep packet inspection, in this packets header as well as payload of the packets are checked with predefined attack signature to identify whether it contains malicious traffic or not. To perform this checking different pattern matching methods are used by NIDS. The most popular method to implement pattern matching is to use of Finite Automata (FA). Generally, regular expressions are used to represent most of the attack signatures defined by NIDS. They are implemented using finite automata, which takes the payload of packet as input string. However, existing approaches of Finite Automata (FA), both deterministic finite automata (DFA) and non-deterministic finite automata (NFA) for pattern matching are having their own advantages and some drawbacks. The DFA based pattern matching methods are fast enough but require more memory. However, NFA based pattern matching methods are comparatively takes less memory but the speed of matching is very slow, to overcome these drawbacks of finite automata there are many approaches have been proposed. This paper discuses comparative study of some Finite Automata (FA) based techniques for pattern matching in network intrusion detection system (NIDS).

Many network security applications in today's networks are based on deep packet inspection, checking not only the header portion but also the payload portion of a packet. For example, traffic monitoring, layer-7 filtering, and network intrusion detection all require an accurate analysis of packet content in search for predefined patterns to identify specific classes of applications, viruses, attack signatures, etc. Regular expressions are often used to represent such patterns. They are implemented using finite automata, which take the payload of a packet as an input string. However, existing approaches, both non-deterministic finite automata (NFA) and deterministic finite automata (DFA), do not deal with compressed traffic, which becomes more and more popular in HTTP applications. In this paper, we propose an efficient algorithm for regular expression matching to implement deep packet inspection on compressed traffic. Based on the observations of DFA, we design a scheme to skip most of the matching process in the compressed parts of traffic. To the best of our knowledge, this is the first effort to design an efficient regular expression matching on compressed traffic. We evaluate our algorithm using rule sets provided by Snort, a popular open-source intrusion detection system. The evaluation results show that our approach can reduce the number of state access in the DFA significantly.

Meyer and Fischer b][MF] proved that nondeterministic finite automata (NFA) can be exponentially more concise than deterministic finite automata (DFA) in their representations of regular languages. Several variants of that basic finite state machine model are now being used to analyze parallelism and to build real-time software systems [HL+]. Even though these variants can sometimes represent regular languages in a more concise manner than NFA, the underlying models fundamentally differ from NFA in how they operate. Degree automata [W] (DA), however, differ from NFA only in their acceptance criteria and accept only regular languages. We show here that DA are also exponentially more concise than NFA on some sequences of regular languages. We also show that the conciseness of probabilistic automata [R] with isolated cutpoints can be unbounded over DA and, concurrently, i.e., over the same sequence of languages, those DA can be exponentially more concise than NFA.

The aim of the paper is to concoct a novel algorithm for the metamorphosis of parallel regular expressions to e-free non- deterministic finite automata. For a given parallel regular expression r, let m be the number of symbols that occur in r and let C denote the number of concatenation operators in r. In the worst case, 2 m+1 states are required for the construction of the non-deterministic finite automaton using the novel algorithm. In the earlier existing approaches, th e number of states of the non-deterministic finite automaton in the worst case is equal to 2 2jrj−3C.

Learning TOC – begins is a web application which is useful for Learning TOC (Theory of Computation). It covers hypothesis of the basic points with cases and it likewise has Exercise segment in which client can check different speculations for all intents and purposes. It likewise creates drawing of different cases. So the client can learn it adequately and additionally quick. Client can build FA of the string without anyone else and print or fare it for his task work. It has the office of Test to check his score and readiness work. So it is extremely valuable for client as exam arrangement. This Web Application is valuable for the educators and understudies and additionally different clients which are has a place with the Computer Science field. The fundamental reason for this web application is to pick up everything outwardly and graphically. It covers the listed topics given below- Regular Expression, Finite Automata, Context Free Grammar, Push Down Automata, Turing Machine, Exercise of the topics, Mock test.

Generalized acceptance, succinctness and supernondeterministic finite automata

The equivalence of non-deterministic finite automata (NFA ), deterministic finite automata (DFA), regular expressions (RE), and parallel finite automata (PFA) with augmented regular expressions (PRE) is used to relate parallel finite automata to the shuffle operation on regular languages. Our goal is to capitalize on the equivalence of regular expressions and augmented regular expressions to increase parallelization efficiency. We use the existing automated transformation algorithms for the various finite automata listed above, as well as one of our own, to inform a transformation from augmented regular expressions to minimized deterministic finite automata (DFAmin). We then discuss the serialization of augmented regular expressions, and present a brief summary of potential applications, related research, and future directions

From the Publisher: This text makes in-depth explorations of a broad range of theoretical topics in computer science. It plunges into the applications of the abstract concepts in order to confront and address the skepticism of readers, and instill in them an appreciation for the usefulness of theory. A two-part presentation integrates logic and formal language—both with applications. Chapter topics cover mathematical preliminaries, propositional logic, proving things: why and how, predicate logic, proving with predicates, program verification, logic programming, language models for computer science, language models, finite automata and their languages, regular expressions, Lex: a tool for building lexical scanners, context-free grammars, pushdown automata and parsing, and turing machines. For future computer scientists.

The number of states of a deterministic finite automaton, which is equivalent to a nondeterministic finite automaton is bounded by 2n, where n is the number of states of the nondeterministic finite automaton. This bound is very pesimistic. Some subclasses of finite automata are shown, for which the complexity of determinization is far lower. On the base of pioneering idea of homogenous finite automata, two classes of finite automata are defined and the complexity of their determinization are shown. These classes are called generalized homogenous finite automata and semihomogenous finite automata.

The finite automata are mathematical models for finite state systems. More general finite automaton is the nondeterministic finite automaton (NFA) that cannot be directly used. It is usually transformed to the deterministic finite automaton (DFA) that then runs in time O(n), where n is the size of the input text. We present two main approaches to practical implementation of DFA considering CPU cache. The first approach (represented by Table Driven and Hard Coded implementations) is suitable forautomata being run very frequently, typically having cycles. The other approach is suitable for a collection of automata from which various automata are retrieved and then run. This second kind of automata are expected to be cycle-free.