Chapter 1 - SQL Is Declarative, Not Procedural

Scala is a new programming language bringing together object-oriented and functional programming. Its defining features are uniformity and extensibility. Scala offers great flexibility for programmers, allowing them to grow the language through libraries. Oftentimes what seems like a language feature is in fact implemented in a library, effectively giving programmers the power of language designers. The downside of this flexibility is that familiar looking code may hide unexpected performance costs. It is important for Scala compilers to bring down this cost as much as possible. We identify several areas of impact for Scala performance: higher-order functions and closures, and generic containers used with primitive types. We present two complementary approaches for improving performance in these areas: optimizations and specialization. Compiler optimization can bring down the cost through a combination of aggressive inlining of higher-order functions, an extended version of copy-propagation and dead-code elimination. Both anonymous functions and boxing can be eliminated by this approach. We show on a number of benchmarks that these language features can be up to 5 times faster when properly optimized, on current day JVMs. We propose a new approach to compiling parametric polymorphism for performance at primitive types. We mix a homogeneous translation scheme with user-directed specialization for primitive types. Type parameters may be annotated to require specialization of code depending on them. We propose definition-site specialization for primitive types, achieving separate compilation and no boxing when both the definition and call site are specialized. Specialized classes are compatible with unspecialized code, and specialization agnostic code can work with specialized instances, meaning that specialization is opportunistic. We present a formalism of a small subset of Scala with specialization and prove that specialization preserves types. We implemented this translation in the Scala compiler and report on improvements on a set of benchmarks, showing that specialization can make programs more than two times faster.

Today, relational databases are widely used in information systems. SQL (structured query language) is taught extensively in universities and professional schools across the globe as a programming language for its data management and accesses. Previously, we have studied a web-based programming learning assistant system (PLAS) to help novice students learn popular programming languages by themselves through solving various types of exercises. For SQL programming, we have implemented the grammar-concept understanding problem (GUP) and the comment insertion problem (CIP) for its initial studies. In this paper, we propose an SQL Query Description Problem (SDP) as a new exercise type for describing the SQL query to a specified request in a MySQL database system. To reduce teachers’ preparation workloads, we integrate a generative AI-assisted SQL query generator to automatically generate a new SDP instance with a given dataset. An SDP instance consists of a table, a set of questions and corresponding queries. Answer correctness is determined by enhanced string matching against an answer module that includes multiple semantically equivalent canonical queries. For evaluation, we generated 11 SDP instances on basic topics using the generator, where we found that Gemini 3.0 Pro exhibited higher pedagogical consistency compared to ChatGPT-5.0, achieving perfect scores in Sensibleness, Topicality, and Readiness metrics. Then, we assigned the generated instances to 32 undergraduate students at the Indonesian Institute of Business and Technology (INSTIKI). The results showed an average correct answer rate of 95.2% and a mean SUS score of 78, which demonstrates strong initial student performance and system acceptance.

Biodiversity data exchange depends on established standards, such as Darwin Core, Audiovisual Core, Taxon Concept Schema, etc. Standards provide terms with defined semantic meaning and structures for data exchange. Standards simplify interchange of biodiversity data among government agencies, researchers, and engineers. A notable challenge during such data transfers is the complexity involved in processing data for various uses. Most data exchanges are performed via Comma Separated Value (CSV), Extensible Markup Language (XML), or JavaScript Object Notation (JSON), and need to be parsed and imported into a database before being queryable. For example, to explore the data in a Darwin Core Archive zipped file, it is necessary to decompress it, parse XML-based metadata about the content of the file, parse and read Ecological Metadata Language (EML) to extract provenance metadata, and correctly open the text-delimited files that use a wide variety of character encodings, delimiters, enclosures, and escape characters. All of it requires non-trivial data management and programming skills from users. We propose a paradigm shift toward using queryable SQLite-based files for more straightforward data interchange. This approach would reduce friction in data processing by directly using Structured Query Language (SQL). SQLite is an open source, high performance, lightweight database already installed on most computers (SQLite 2024). It integrates a database into a single file, facilitating straightforward data exchange and compression. Its robust engine is able to manage terabytes of data. The SQLite developers are committed to maintaining backward compatibility of both binary and SQL text file versions until 2050 (SQLite 2024). Connectivity to SQLite databases is supported by all popular programming languages. Furthermore, the United States Library of Congress endorses SQLite alongside XML and JSON for data archives, attesting to its long-term reliability (United States Library of Congress 2024). We at the Species File Group are experimenting with using SQLite to create a universal data converter, in which an SQLite database serves as an intermediate data storage format. This provides several useful advantages: 1. Universal data converter By learning a standard SQL schema for biodiversity data, users are able to efficiently write importers and exporters to/from the intermediate schema for a variety of other biodiversity data-exchange formats. Any programming language can be used to develop the conversion scripts. Import and export are completely decoupled from each other, allowing mixing and matching of converters in a flexible way. For example, we aim to use SQLite as an intermediary archive to convert data between Darwin Core Archives, Catalogue of Life Data Package (ColDP), Global Names Verifier, and TaxonWorks (Fig. 1). 2. Semantic versioning of the archive Like any other data format, the intermediate data exchange SQLite schema will evolve over time. Using Git tags and branches, it is straightforward to follow a semantic versioning paradigm (Semantic Versioning 2024). For example, in "v1.2.5" the numbers respectively indicate major, minor, and patch versions. Major versions include substantial changes that might not be backwards compatible, while minor versions are backwards compatible and patch versions indicate tiny bug fixes. 3. Data querying With SQLite-based archives being relational databases, they can immediately be queried without any additional parsing and importing. This allows the archives to be explored with ease by anyone familiar with SQL. The Carpentries also provide online learning resources and courses for researchers to learn SQLite (Martinez and Poisot 2017). 4. Biodiversity application development As SQLite-based archives are relational databases, and the intermediate data exchange schema is standardized and stable, a database file can be used directly by a variety of applications. For example, it would be possible to develop a web application frontend that utilizes the SQLite archive as a backend. It could be possible to develop an ecosystem of biodiversity informatics applications based on SQLite archives. We believe that adopting SQLite for biodiversity data exchange can reduce operational friction, enhance data accessibility, and promise a streamlined and universally compatible data management framework.

I will discuss the design of Ct, an API for nested data parallel programming in C++. Ct uses meta-programming and functional language ideas to essentially embed a pure functional programming language in impure and unsafe languages, like C++. I will discuss the evolution of the design into functional programming ideas, how this was received in the corporate world, and how we plan to proliferate the technology in the next year.Ct is a deterministic parallel programming model integrating the nested data parallelism ideas of Blelloch and bulk synchronous processing ideas of Valiant. That is, data races are not possible in Ct. Moreover, performance in Ct is relatively predictable. At its inception, Ct was conceived as a simple library implementation behind C++ template magic. However, performance issues quickly forced us to consider some form of compilation. Using template programming was highly undesirable for this purpose as it would have been difficult and overly specific to C++ idiosyncrasies. Moreover, once compilation for performance was considered, we began to consider a language semantics that would enable powerful optimizations like calculational fusion, synchronization barrier elimination, and so on. The end result of this deliberation is an API that exposes a value-oriented, purely functional vector processing language. Additional benefits of this approach are numerous, including the important ability to co-exist within legacy threading programming models (because of the data isolation inherent in the model). We will show how the model applies to a wide range of important (at least by cycle count) applications. Ct targets both shipping multi-core architectures from Intel as well as future announced architectures.The corporate reception to this approach has (pleasantly) surprised us. In the desktop and high-performance computing space, where C, C++, Java, and Fortran are the only programming models people talk about, we have made serious inroads into advocating advanced programming language technologies. The desperate need for productive, scalable, and safe programming languages for multi-core architectures has provided an opening for functional, type-safe languages. We will discuss the struggles of multi-core manufacturers (i.e. Intel) and their software vendors that have created this opening.For Intel, Ct heralds its first serious effort to champion a technology that borrows functional programming technologies from the research community. Though it is a compromise that accommodates the pure in the impure and safe in the unsafe, this is an important opportunity to demonstrate the power of functional programming to the unconverted. We plan to share the technology selectively with partners and collaborators, and will have a fully functional and parallelizing implementation by year's end. At CUFP, we will be prepared to discuss our long term plans in detail.

Object-oriented programming languages are suitable for describing real-world objects, functional programming languages for algebraic values. In this paper we propose an object-oriented functional programming language, called TOFL, which combines many desired properties of the two paradigms. In particular, TOFL unifies object classes, inheritance, (method) redefinitions and late binding as in object-oriented languages, algebraic data types, higher-order functions, type classes and type inference (without type reconstruction in this paper) as in functional languages. We translate TOFL into a stratified and explicitly typed λ-calculus T with overloaded functions, where redefinitions and late binding become late binding of overloaded functions. The operational semantics of T gives a semantics and a simple prototyping implementation of TOFL.

Higher-order functions are essential for generic programming. While they are naturally supported in functional programming languages, there are no higher-order functions in C++. There, various function datatypes exist that simulate the effect of higher-order programming in different ways - as classes designed in the imperative, object-oriented, or meta-programming spirit. With the recent addition of concepts, another alternative for supporting parameterization by functions has yet become available. No guidelines exist, however, whether, or when, to prefer a concept for functions over any of the existing function datatypes; nor have the function datatypes themselves been compared. We provide an empirical study that assesses performance, expressivity, and convenience. The study shows that the function concept mechanism is faster and at least as expressive as the best function datatype, but, due to the principal difference between concept- and type-based approaches, also less convenient to use.

This thesis proposes a novel way of performing type checking, whose results are incremental, depending on the provided local information. This new way of type checking is called co-contextual, where all context information of expressions, methods, classes, etc., is removed. Instead, we introduce corresponding structures using requirements. Standard type systems are translated to the co-contextual ones systematically using dualism as technique. Type systems play an important role to prevent execution errors from occurring during runtime. They are used to check programs statically for potential errors. Programs are type checked against a given set of rules. Depending on these rules programs are well-typed or not. The set of these rules is called typing rules. Each type rule associates types to the constructs of a program given a certain context. There can be different forms of contexts, depending on the features of the typed programming language. Functional languages use a typing context of variables and their types; object-oriented (OO) languages use additional class tables. Class tables are used for example to ensure that method and class declarations are well-typed. Type checking is performed top-down. While traversing the syntax tree of a program, typing contexts are extended with information on the expressions and their types. In case of OO, class tables are extended with clauses from class declarations, including the class members, i.e., fields, methods, or constructors. Contexts are passed through the nodes of the syntax tree in order to coordinate type checking between them. Therefore, while traversing the syntax tree top-down, the type checker creates dependencies between otherwise independent subexpressions. This way, it inhibits incrementalization and parallelization of type checking. That is, a change to a node of the syntax tree would require to redo the type check of the whole syntax tree. In this thesis a novel formulation of type systems is proposed, in order to remove dependencies between subexpressions. We propose a co-contextual formulation of typing rules that depends only on the local program constructs, e.g., expressions, methods, classes. The co-contextual typing rules have as conclusion a type and sets of requirements. That is, contexts and class tables are replaced by the dual concept of context and class table requirements. In addition, operations on contexts and class tables are replaced by new dual operations on requirements. The co-contextual type checker traverses a syntax tree bottom-up and merges context requirements of independently checked subexpressions. We describe a method for systematically constructing a co-contextual formulation of type rules from a regular context-based formulation and we show how co-contextual type rules give rise to incremental type checking. We derive co-contextual type checkers for functional and OO languages. As a representative of functional languages we consider PCF and extensions of it: records, parametric polymorphism, structural subtyping and let-polymorphism. Also, we investigate featherweight java (FJ) as the basis of OO languages and extensions of it: method overloading and generics. We build a1 co-contextual type checker for FJ enabling key features of OO languages: subtype polymorphism, nominal typing and implementation inheritance. The dualism between the co-contextual and contextual type systems preserves the correctness of the contextual calculus. That is, we prove the correctness of the co-contextual calculus via the equivalence between contextual type rules and their co-contextual formulations. We implemented an incremental type checker for PCF along with a performance evaluation showing that co-contextual type checking has performance comparable to standard context-based type checking, and incrementalization can improve performance significantly. Regarding FJ, we implemented a co-contextual type checker with incrementalization and compared its performance against javac on a number of realistic programs. Our performance evaluation shows significant speedups for the co-contextual type checker with incrementalization in comparison to javac.

This paper describes the implementation of recursive algorithms in unique factorization domains, namely multivariate polynomial greatest common divisors (gcd) and factorization into irreducible parts in the Java computer algebra library (JAS). The implementation of gcds, resultants and factorization is part of the essential building blocks for any computation in algebraic geometry, in particular in automated deduction in geometry. There are various implementations of these algorithms in procedural programming languages. Our aim is an implementation in a modern object oriented programming language with generic data types, as it is provided by Java programming language. We exemplify that the type design and implementation of JAS is suitable for the implementation of several greatest common divisor algorithms and factorization of multivariate polynomials. Due to the design we can employ this package in very general settings not commonly seen in other computer algebra systems. As for example, in the coefficient arithmetic for advanced Grobner basis computations like in polynomial rings over rational function fields or (finite, commutative) regular rings. The new package provides factory methods for the selection of one of the several implementations for non experts. Further we introduce a parallel proxy for gcd implementations which runs different implementations concurrently.

The relational database model is the main focus of the database for the previous time. It provides robust data storage and structure that supports transaction characteristics and data retrieval with structured query language- SQL. The emergence of web technologies and clustering technology in large servers called for the need to store unstructured data and move away from the pattern of tables and static fields and not topic to transaction conditions, especially in distributed systems. Instead, it used CAP theory. New data models are emergence including: Document Data model, Key-Value model, Column-Family model, and Graph model, And new programming languages that deal with these models. An asset inventory system is a model of systems that are not topic to a fixed structure. Each location in the organization has fairly different assets and assets in the same location have different specifications that are difficult to organize in the form of tables and columns. For this reason using of NOSQL CouchDB database system was adopted and the use of MAP-REDUCE technology to organize data display and Cloudant Query in Mango-view method to retrieve data from the inventory database.

Chapter 9 - Software engineering

A Web-Based Perioperative Dashboard as a Platform for Anesthesia Informatics Innovation.

This thesis presents a refinement calculus for transforming highly abstract specifications into programs written in a functional programming language. Refinement calculi allow the derivation of a program from a formal specification by a sequence of correctness-preserving transformation steps. Most refinement calculi target imperative programming languages. Functional programming languages are often more expressive than imperative programming languages. This expressive power leads to a reduced gap between the concepts used to express a problem (as a specification) and the concepts used to express its solution (as a program). Thus, a refinement calculus which targets a functional programming language can lead to a simpler development process which produces a final product more quickly. We build a refinement calculus for functional programs by adding specification constructs to a functional language and examining transformations over the resulting language.This thesis complements other work in the area in two major ways. Firstly, we investigate the addition of truly-nondeterministic, rather than underdetermined, choice constructs to a functional language. These constructs allow specifications which are more abstract and which admit more implementations. Secondly, most refinement calculi for functional programs add erratic choice constructs to a functional language. We investigate the addition of both demonic and angelic nondeterminism to a functional language. Demonic and erratic choice are similar: given a number of alternatives, they choose any one. In contrast, given a number of alternatives angelic choice always makes the correct choice, if one exists. Angelic choice is difficult to reason about, but allows the concise specification of powerful parallel constructs.

Dataflow execution models are used to build highly scalable parallel systems. A programming model that targets parallel dataflow execution must answer the following question: How can parallelism between two dependent nodes in a dataflow graph be exploited? This is difficult when the dataflow language or programming model is implemented by a monad, as is common in the functional community, since expressing dependence between nodes by a monadic bind suggests sequential execution. Even in monadic constructs that explicitly separate state from computation, problems arise due to the need to reason about opaquely defined state. Specifically, when abstractions of the chosen programming model do not enable adequate reasoning about state, it is difficult to detect parallelism between composed stateful computations. In this paper, we propose a programming model that enables the composition of stateful computations and still exposes opportunities for parallelization. We also introduce smap, a higher-order function that can exploit parallelism in stateful computations. We present an implementation of our programming model and smap in Haskell and show that basic concepts from functional reactive programming can be built on top of our programming model with little effort. We compare these implementations to a state-of-the-art approach using monad-par and LVars to expose parallelism explicitly and reach the same level of performance, showing that our programming model successfully extracts parallelism that is present in an algorithm. Further evaluation shows that smap is expressive enough to implement parallel reductions and our programming model resolves short-comings of the stream-based programming model for current state-of-the-art big data processing systems.

Many problems need some sort of dynamic data structures. There are two approaches to adding these to a programming language. The first is to hide the pointers from the programmer, they are created and destroyed by the code generated by the compiler. C. A. R. Hoare suggested such semantics in [19] in a precursor to Pascal. The disadvantage of this approach is that data-structures that need pointers that pointed back into itself cannot be easily programmed - the advantages are numerous! The second approach is to have explicit pointers, as can be found in nearly all procedural programming languages. The difficulty with this second approach is that pointer management has to be done by the programmer, and if not done correctly the misuse of pointers can lead to dead storage (also called storage leakage, storage that cannot be accessed from the program) and dangling pointers (pointers that point to storage that, in some sense, does not exist). This chapter deals with both approaches to dynamic data structures.

In this thesis, we explore techniques for the development and verification of programs in a high-level, expressive, and safe programming language. Our programs can express problems over unbounded domains and over recursive and mutable data structures. We present an implementation language flexible enough to build interesting and useful systems. We mostly maintain a core shared language for the specifications and the implementation, with only a few extensions specific to expressing the specifications. Extensions of the core shared language include imperative features with state and side effects, which help when implementing efficient systems. Our language is a subset of the Scala programming language. Once verified, programs can be compiled and executed using the existing Scala tools. We present algorithms for verifying programs written in this language. We take a layer-based approach, where we reduce, at each step, the program to an equivalent program in a simpler language. We first purify functions by transforming away mutations into explicit return types in the functions' signatures. This step rewrites all mutations of data structures into cloning operations. We then translate local state into a purely functional code, hence eliminating all traces of imperative programming. The final language is a functional subset of Scala, on which we apply verification. We integrate our pipeline of translations into Leon, a verifier for Scala. We verify the core functional language by using an algorithm already developed inside Leon. The program is encoded into equivalent first-order logic formulas over a combination of theories and recursive functions. The formulas are eventually discharged to an external SMT solver. We extend this core language and the solving algorithm with support for both infinite-precision integers and bit-vectors. The algorithm takes into account the semantics gap between the two domains, and the programmer is ultimately responsible to use the proper type to represent the data. We build a reusable interface for SMT-LIB that enables us to swap solvers transparently in order to validate the formulas emitted by Leon. We experiment with writing solvers in Scala; they could offer both a better and safer integration with the rest of the system. We evaluate the cost of using a higher-order language to implement such solvers, traditionally written in C/C++. Finally, we experiment with the system by building fully working and verified applications. We rely on the intersection of many features including higher-order functions, mutable data structures, recursive functions, and nondeterministic environment dependencies, to build concise and verified applications.