Editorial: Special Issue on Computability, Complexity and Randomness

Computability is one of the fundamental notions of mathematics and computer science, trying to capture the effective content of mathematics and its applications. Computability Theory explores the frontiers and limits of effectiveness and algorithmic methods. It has its origins in Godel's Incompleteness Theorems and the formalization of computability by Turing and others, which later led to the emergence of computer science as we know it today. Computability Theory is strongly connected to other areas of mathematics and theoretical computer science. The core of this theory is the analysis of relative computability and the induced degrees of unsolvability; its applications are mainly to Kolmogorov complexity and randomness as well as mathematical logic, analysis and algebra. Current research in computability theory stresses these applications and focuses on algorithmic randomness, computable analysis, computable model theory, and reverse mathematics (proof theory). Recent advances in these research directions have revealed some deep interactions not only among these areas but also with the core parts of computability theory. The goal of this Dagstuhl Seminar is to bring together researchers from all parts of computability theory and related areas in order to discuss advances in the individual areas and the interactions among those.

1. UNIVERSALITYThe theory of Kolmogorov complexity is based on thediscovery, by Alan Turing in 1936, of the universal Turingmachine. After proposing the Turing machine as anexplanation of the notion of a computing machine, Turingfound that there exists one Turing machine which cansimulate any other Turing machine.Complexity, according to Kolmogorov, can be measuredby the length of the shortest program for a universal Turingmachine that correctly reproduces the observed data. It hasbeen shown that, although there are many universal Turingmachines(andthereforemanypossible ‘shortest’ programs),the corresponding complexities differ by at most an additiveconstant.The main thrust of the theory of Kolmogorov complexityis its ‘universality’; it strives to construct universal learningmethods based on universal coding methods. Thisapproach was originated by Solomonoff and made moreappealing to mathematicians by Kolmogorov. Typicallythese universal methods will be computable only in someweak sense. In applications, therefore, we can only hopeto approximate Kolmogorov complexity and related notions(such as randomnessdeficiency and algorithmic informationmentioned below). This special issue contains both materialon non-computable aspects of Kolmogorov complexity andmaterial on many fascinating applications based on differentways of approximating Kolmogorov complexity.2. BEGINNINGSAs we have already mentioned, the two main originators ofthe theory of Kolmogorovcomplexity were Ray Solomonoff(born 1926) and Andrei Nikolaevich Kolmogorov (1903–1987). The motivations behind their work were completelydifferent; Solomonoff was interested in inductive inferenceand artificial intelligence and Kolmogorov was interestedin the foundations of probability theory and, also, ofinformation theory. They arrived, nevertheless, at the samemathematical notion, which is now known as Kolmogorovcomplexity.In 1964 Solomonoff published his model of inductiveinference. He argued that any inference problem can bepresentedas a problemof extrapolatinga verylongsequenceof binary symbols; ‘given a very long sequence, representedby T, what is the probability that it will be followed by a

Hardness of computing the Kolmogorov complexity of a given string is closely tied to a security proof of hitting set generators, and thus understanding hardness of Kolmogorov complexity is one of the central questions in complexity theory. In this paper, we develop new proof techniques for showing hardness of computing Kolmogorov complexity under surprisingly efficient reductions, which were previously conjectured to be impossible. It is known that the set R K of Kolmogorov-random strings is PSPACE-hard under polynomial-time Turing reductions, i.e., PSPACE ⊂ P R K , and that NEXP ⊂ NP R K , which was conjectured to be tight by Allender (CiE 2012). We prove that EXP NP ⊂ P R K , which simultaneously improves these hardness results and refutes the conjecture of Allender under the plausible assumption that EXP NP ≠ NEXP. At the core of our results is a new security proof of a pseudorandom generator via a black-box uniform reduction, which overcomes an impossibility result of Gutfreund and Vadhan (RANDOM/APPROX 2008). Our proof techniques have further consequences, including: 1. Applying our proof techniques to the case of resource-bounded Kolmogorov complexity, we obtain NP-hardness of the problem MINcKT SAT of computing conditional polynomial-time-bounded SAT-oracle Kolmogorov complexity under polynomial-time deterministic reductions. In contrast, the Minimum SAT-Oracle Circuit Size Problem, which is a version of sublinear-time-bounded Kolmogorov complexity, cannot be NP-hard under polynomial-time deterministic reductions without resolving EXP ≠ ZPP. Our hardness result is the first result that overcomes the non-NP-hardness results of MCSP. We also prove DistNP-hardness of MINKT SAT , which is a partial converse of the approach of Hirahara (FOCS 2018) for proving the equivalence between worst-case and average-case complexity of NP. 2. We prove S 2 p -hardness of Kolmogorov complexity under quasi-polynomial-time nonadaptive reductions. This is the first result that overcomes a P/poly barrier result of Allender, Buhrman, Friedman, and Loff (MFCS 2012). We also establish a firm link between non-trivial satisfiability algorithms and immunity of random strings, and obtain the following unconditional lower bounds. 1. It has been a long-standing open question whether the set of subexponential-time-bounded Kolmogorov-random strings is decidable in P. We resolve this open question, by showing that the set of super-polynomial-time-bounded Kolmogorov-random strings is P-immune, which is a much stronger lower bound than an average-case lower bound. 2. The set of Levin’s Kolmogorov-random strings is (P-uniform ACC)-immune.

Parallel and distributed computing has been the basis to many emerging areas, such as smart networks, cloud computing, big data analysis, and blockchain technology. Without the development of parallel and distributed computing technologies and various types of systems, it is not possible to meet the requirement on efficiency, accuracy, scalability, and reliability for various critical applications that support our modern economy and society. While parallel and distributed computing played a vital role in modern science, engineering, biology, medicine, pharmacy, astronomy, geology, and archaeology, its application has also been extended to business, finance, economics, management, government, and defense, covering all aspects of our modern society and life. Furthermore, parallel and distributed computing has emerged in recent advances of many hotspot research directions including artificial intelligence, machine learning, Internet of Things, bioinformatics, digital medicine, cybersecurity, and social computing, resulting in numerous ground-breading discoveries that are changing our society and life.

This thesis is dedicated to studying the theory of entropy and its relation to the Kolmogorov complexity. Originating in physics, the notion of entropy was introduced to mathematics by C. E. Shannon as a way of measuring the rate at which information is coming from a data source. There are, however, a few different ways of telling how much information there is: an alternative approach to quantifying the amount of information is the Kolmogorov complexity, which was proposed by A. N. Kolmogorov. The Shannon entropy is the key ingredient in the definition of the Kolmogorov-Sinai entropy of a measure-preserving systems. Roughly speaking, the Kolmogorov-Sinai entropy is the expected amount of information in `Shannon sense' that one obtains per unit of time by observing the evolution of a measure-preserving system. In topological dynamics, the topological entropy takes place of the Kolmogorov-Sinai entropy. For metrizable systems, the topological entropy measures the exponential growth rate of the number of distinguishable partial orbits of length n as n tends to infinity. Originally defined for Z-actions, the 'classical' theories of entropy were later extended to actions of amenable groups. We provide a necessary background on amenable groups, topological/measurepreserving dynamics and the entropy theory in Chapters 1, 2, 3 and 5. The main focus of this thesis is extending the following results. First of all, a common generalization of the topological and the Kolmogorov-Sinai entropy theories for Z-systems was suggested by G. Palm. We provide an abstract generalization of the work of Palm for actions of discrete amenable groups in the language of measurement functors in Chapter 6. Secondly, we investigate the connection of entropy and Kolmogorov complexity. Originally, the equality between the topological entropy and a certain quantity measuring maximal asymptotic Kolmogorov complexity of the trajectories was established by A. A. Brudno for subshifts over Z. Later, he proved the equality of the Kolmogorov-Sinai entropy and the asymptotic Kolmogorov complexity of almost every trajectory for ergodic subshifts over Z. We provide a generalization of these results as follows. Firstly, in Chapter 4 we give a background on computability and Kolmogorov complexity and, further, introduce computable Folner monotilings, which are central in our extensions of Brudno's results. We treat the 'first' and the 'second' theorems of Brudno in Chapter 7. The first theorem is generalized for subshifts over computable groups admitting computable Folner monotilings, while the second theorem is proved under the assertion of regularity of the monotiling, which we introduce in Chapter 7 as well.

There are many ways to measure the complexity of a given object, but there are two measures of particular importance in the theory of computing: One is Kolmogorov complexity, which measures the amount of information necessary to describe an object. Another is computational complexity, which measures the computational resources necessary to recognize (or produce) an object. The relation between these two complexity measures has been studied since the 1960s. More recently, the more generalized notion of resource bounded Kolmogorov complexity and its relation to computational complexity have received much attention. Now many interesting and deep observations on this topic have been established. This book consists of four survey papers concerning these recent studies on resource bounded Kolmogorov complexity and computational complexity. It also contains one paper surveying several types of Kolmogorov complexity measures. The papers are based on invited talks given at the AAAI Spring Symposium on Minimal-Length Encoding in 1990. The book is the only collection of survey papers on this subject and provides fundamental information for researchers in the field.

Morphological computation proposes the idea that in a physical system, certain computational processes can be off-loaded to the body. However, the concept has still eluded serious theoretical quantification attempts, unlike traditional computational theory. This perspective examines the notion of morphological computation from the well established theories of traditional computation and computational complexity, drawing parallels between the two, to understand the differences and similarities. Further, we look at the quantification efforts of morphological computation and attempt to link it to the unexplored field of control complexity. We argue that the development of complexity theory for control problems is necessary to study and utilize the concept of morphological computation, if it is possible.

Welcome to the special issue on selected papers from the ELECO-2013 Conference!, the 8th International Conference on Electrical and Electronics Engineering, held 28–30 November 2013, in Bursa-Turkey. ELECO conferences go back to 1986, when it was initially organized as national ELMEKSEM Electromechanical Conference by the Chamber of Electrical Engineers (EMO) Bursa Branch and upon that success repeated in 1988 and 1993 and 1997. The Bursa Branch has further gone on to organize a series of Electrical and Electronics Engineering Conferences in Bursa. ELECO conferences are organized at international level in odd numbered years and as national conferences in even numbered years, such as ELECO’2013 is the eight international conference which participants coming from various countries, presenting papers from the wide spectrum of electrical and electronics engineering. ELECO 2013 is jointly organized by Uludag University, Bursa; Istanbul Technical University (ITU), Istanbul and the Chamber of Turkish Electrical Engineers (EMO) Bursa Branch. The IEEE Region 8, IEEE Circuits and Systems (CAS) Turkey section and The Scientific and Technological Research Council of Turkey are co-sponsors of ELECO 2013. Taking the broad scope of the technical program, it is probably the largest electrical and electronics engineering conference ever held in Turkey. The scope of the conference covers topics ranging over electric power systems, electrical machines and drives, power electronics, high voltage techniques, electrical materials, electronics, circuits and systems, signal processing, electromagnetics, antennas and propagation, microwave theory, communication systems, mechatronics, control theory, control applications, automation systems, robotics and intelligent control systems. Two hundred and thirty five papers have been submitted for presentation and 138 papers have been accepted. If we look at the contributions with respect to geographical point of view, it is seen that ELECO 2013 contributions are coming from 17 different countries. Selected papers from previous international ELECO Conferences were invited to be published in extended form in special ELECO issues of international journals, under peer review. Three special ELECO issues appeared in the journal Analog Integrated Circuits and Signal Processing containing selected papers from ELECO’2001, ELECO’2007 and ELECO’2011. Furthermore, four special ELECO issues appeared in the journal TJEECS: Turkish Journal of Electrical Engineering and Computer Sciences where selected papers from ELECO’2003, ELECO’2005, ELECO’2007, and ELECO’2009 were published in extended form, respectively. This special issue on selected papers from ELECO’2013 contains extended versions of 12 papers presented at the conference, chosen among 19 recipients of the call for the papers for the special issue. All papers had to undergo the usual ALOG review process before they were finally accepted for this special issue. One of these papers is selected from invited talks given by world-known scientists during the conference. Six papers are selected from the papers presented in Special Session on Advances and New Design Possibilities for Analog Signal Processing. The other five are chosen from regular submitted manuscripts. H. H. Kuntman (&) S. Ozoguz O. Cicekoglu Istanbul Technical University, Istanbul, Turkey e-mail: kuntman@itu.edu.tr

The theory of computational complexity is based on the tradeoff between two computational resources, memory space and computer time. This paper investigates the physical counterparts of these resources. Memory space is the number of bits of information required by a computer over the course of a computation. Physically, information is intimately connected with entropy: entropy can be thought of as random bits of information. The physical analogue of the amount of memory space required for a computation, Ф- SPACE, is the difference between the maximum entropy Smax of the computing system and its actual entropy, S.Ф-SPACE = S - Smax is the thermodynamic depth of the computation. Ф-SPACE is the maximum number of bits of memory space available for a physical system to perform computation. In conventional computational complexity theory, computer time is identified as the number of elementary logical operations (AND, OR, NOT, COPY) required to perform the computation. The physical analogue of time computational complexity is simply the time required for the physical computing system to perform the computation. In order to obtain a number that measures the number of physical operations that can be performed by the system over time t, we define Ф-TIME to be proportional to the computation time multiplied by the energy E of the computing system over the course of the computation: Ф-TIME = 2Et/πħ. Here, the proportionality factor 2/πħ is chosen to match the time it takes a computing system with energy E to flip a bit in time t. Ф-TIME is thus the number of bit flips that can take place in an information processing system. In addition, we propose a novel measure of physical complexity that combines these measures. The free energy F = E - TS of a system is the amount of energy that is available to do useful work - as opposed to the energy bound up in the thermal fluctuations of heat. We define the free complexity to be the free energy × time product, 2Ft/πħ required by a physical system to perform a computation. Physical spatial and temporal complexity provide upper bounds on the amount of computation that can be performed by any physical system. We discuss the implications of physical complexity for the computational complexity hierarchy, in particular, we show that in the presence of errors in the computation, Ф-SPACE is proportional Ф-TIME. We apply the resulting theory of physical computational complexity to analyze the computational power of black holes: exactly half of the energy of a black hole is available for physical computation.

This paper is an introduction to the computational complexity theory. I believe that the standard courses in complexity theory make some things much more important than they really are, while things which I find extremely interesting are marginalized. Thus, it can be seen as my subjective view on what is important and interesting in computational complexity, and what is not. We state the basic facts from computational complexity theory using the blackboard computations (one-dimensional cellular automata) as the standard computation model; we argue the benefits of such approach. This article is somewhat based on my series of talks from Warsztaty Logiczne 2014 in Białka Tatrzańska, although these talks covered more topics, such as NL=coNL, and links between BPP and the polynomial hierarchy.

This special issue of the VLSI DESIGN: an International Journal of Custom-Chip Design, Simulation and Testing is devoted to spectral techniques, new types of decision trees and diagrams, their applications, efficient ways of their calculation and mutual relations between spectral techniques and classical decision diagrams. Manipulations and calculations with discrete functions are fundamental tasks in many areas of Computer Science and Engineering. Many problems in digital system design, simulation and testing can be expressed as a sequence of operations on discrete functions. The performance of Computer-Aided Design systems used in solving various problems in this area strongly depends on the efficiency of representations of discrete functions. Decision Diagrams (DDs) have proved very convenient data structure for discrete function representations, permitting manipulations and calculations with large functions efficiently in terms of time and space. In many applications, as for example those involving large matrices, conventional algorithms are significantly improved by using DDs. In logic design, such applications relate to basic problems of the design, verification, simulation and testing logical networks. Besides Boolean functions, some other types of discrete representations, such as multi-valued functions, cube sets, EXOR-based spectral and probabilistic formulas have to be dealt with. Hence the numbers of different decision trees and diagrams have been introduced for these various applications and representations. The spectral or polynomial representation of Boolean function is obtained by expressing it in terms of an orthogonal basis. The most popular basis is formed by discrete Walsh functions. Note that the spectral and the polynomial representation of a Boolean function are equivalent. The discrete orthogonal functions are referred to as the basis functions when the spectral or abstract harmonic terminology is used and as monomials when the polynomial terminology is used. Such polynomial representations have extensive applications in coding theory and cryptography as well as design of nonlinear filters. Spectral techniques based on Walsh, Haar, Arithmetic and Reed–Muller transforms have been used in digital logic design for more than 30 years. These techniques have been used for Boolean function classification, disjoint decomposition, identification of symmetries, parallel and serial linear decomposition, spectral translation synthesis (extraction of linear preand post-filters), multiplexer synthesis, threshold logic synthesis, state assignment and testing, and evaluation of logic complexity. The renewed interest in applications of spectral methods in design of VLSI digital circuits is caused by their excellent design for testability properties and in their efficiency in Boolean mapping problems. Furthermore, the practical applicability of these techniques was greatly improved with the development of efficient methods that allow calculating and operating on spectra of Boolean functions directly from reduced representations of such functions in the form of arrays of cubes or decision diagrams. Following from these recent developments, techniques have been presented for efficient spectral translation to decompose a circuit into a cascade of two sub circuits: linear block composed of EXOR gates fed by the primary input of the overall circuit. Such linear decomposition drastically simplifies the synthesis task. The choice of suitable linear transformation is based on a complexity measure assigned to each Boolean function that is heuristically related to the complexity of the final circuit implementation. To avoid the experimental costs of computing the Walsh transform, Binary Decision Diagrams (BDDs) based techniques are used. Recently the idea of linear transformations is applied to decision diagrams. By combining powerful spectral linear techniques with variable reordering techniques, it is possible to synthesize large Boolean functions with standard Computer-Aided Design tools that fail otherwise. The areas addressed by this special issue span spectral transforms, their applications and efficient calculation methods through decision diagrams. New spectral decision diagrams are also emerging that support larger data structures allowing more complex discrete representations to be implemented that was previously impossible.

We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and where they are fundamentally different. We discuss and relat...

Image distortion analysis is a fundamental issue in many image processing problems, including compression, restoration, recognition, classification, and retrieval. Traditional image distortion evaluation approaches tend to be heuristic and are often limited to specific application environment. In this work, we investigate the problem of image distortion measurement based on the theory of Kolmogorov complexity, which has rarely been studied in the context of image processing. This work is motivated by the normalized information distance (NID) measure that has been shown to be a valid and universal distance metric applicable to similarity measurement of any two objects (Li et al. in IEEE Trans Inf Theory 50:3250–3264, 2004). Similar to Kolmogorov complexity, NID is non-computable. A useful practical solution is to approximate it using normalized compression distance (NCD) (Li et al. in IEEE Trans Inf Theory 50:3250–3264, 2004), which has led to impressive results in many applications such as construction of phylogeny trees using DNA sequences (Li et al. in IEEE Trans Inf Theory 50:3250–3264, 2004). In our earlier work, we showed that direct use of NCD on image processing problems is difficult and proposed a normalized conditional compression distance (NCCD) measure (Nikvand and Wang, 2010), which has significantly wider applicability than existing image similarity/distortion measures. To assess the distortions between two images, we first transform them into the wavelet transform domain. Assuming stationarity and good decorrelation of wavelet coefficients beyond local regions and across wavelet subbands, the Kolmogorov complexity may be approximated using Shannon entropy (Cover et al. in Elements of information theory. Wiley-Interscience, New York, 1991). Inspired by Sheikh and Bovik (IEEE Trans Image Process 15(2):430–444, 2006), we adopt a Gaussian scale mixture model for clusters of neighboring wavelet coefficients and a Gaussian channel model for the noise distortions in the human visual system. Combining these assumptions with the NID framework, we derive a novel normalized perceptual information distance measure, where maximal likelihood estimation and least square regression are employed for parameter fitting. We validate the proposed distortion measure using three large-scale, publicly available, and subject-rated image databases, which include a wide range of practical image distortion types and levels. Our results demonstrate the good prediction power of the proposed method for perceptual image distortions.

In this paper we study subsequences of random numbers. In Kamae (1973), selection functions that depend only on coordinates are studied, and their necessary and sufficient condition for the selected sequences to be normal numbers is given. In van Lambalgen (1987), an algorithmic analogy to the theorem is conjectured in terms of algorithmic randomness and Kolmogorov complexity. In this paper, we show different algorithmic analogies to the theorem.

Boolean function is one of the most fundamental computation models in theoretical computer science. The two most common representations of Boolean functions are Fourier transform and real polynomial form. Applying analytic tools under these representations to the study Boolean functions has led to fruitful research in many areas such as complexity theory, learning theory, inapproximability, pseudorandomness, metric embedding, property testing, threshold phenomena, social choice, etc. In this thesis, we focus on \emph{sparse representations} of Boolean function in both Fourier transform and polynomial form, and obtain the following new results. A classical result of Rothschild and van Lint asserts that if every non-zero Fourier coefficient of a Boolean function $f$ over $\mathbb{F}_2^{n}$ has the same absolute value, namely $|\hat{f}(\alpha)|=1/2^k$ for every $\alpha$ in the Fourier support of $f$, then $f$ must be the indicator function of some affine subspace of dimension $n-k$. Here we slightly generalize their result, and show that Boolean functions whose Fourier coefficients take values in the set $\{-2/2^k, -1/2^k, 0, 1/2^k, 2/2^k\}$ are indicator functions of two disjoint affine subspaces of dimension $n-k$ or four disjoint affine subspaces of dimension $n-k-1$. Our main technical tools are results from additive combinatorics which offer tight bounds on the affine span size of a subset of $\mathbb{F}_2^{n}$ when the doubling constant of the subset is small. For polynomial representation of Boolean function, we study the distribution of the number of non-zero coefficients of \emph{random} Boolean functions. For a random Boolean function $f:\{0,1\}^n\rightarrow \{0,1\}$, i.e., a function whose value at each point on the Boolean cube is chosen independently and uniformly at random from $\{0,1\}$, in real polynomial representation, we give several bounds and concentration results about the distribution of the sparsity of $f$.