Dynamic Software Update from Development to Production.

Live programming is an activity in which the programmer edits code while observing the result of the program. It has been exercised mainly for pedagogical and artistic purposes, where outputs of a program are not straightforwardly imagined. While most live programming environments so far target programs that explicitly generate visual or acoustic outputs, we believe that live programming is also useful for data structure programming, where the programmer often has a hard time to grasp a behavior of programs. However, it is not clear what features a live programming environment should provide for such kind of programs. In this paper, we present a design of live programming environment for data structure programming, identify the problems of synchronization and mental map preservation, and propose solutions based on a calling-context sensitive identification technique. We implemented a live programming environment called Kanon, and tested with 13 programmers.

Several studies provide the questions developers ask during software evolution tasks, providing foundations for subsequent work. Nevertheless, none of them focus on Live Programming environments that gain in popularity as they are perceived to have a positive effect on programming tasks. Studying the impact of a Live Programming environment on software development activities is thus the goal of this study. In a partial replication of the study by Sillito et al., we conducted 17 software evolution sessions in a Live Programming environment and report 1,161 developer questions asked during these sessions. We contrast our results with the results by Sillito et al., focusing on the question occurrences, question complexity and what information participants used to gain a required knowledge. We report eight new questions and observe that the Live Programming facilities do have an impact on the way developers ask questions about source code and use tools to gain corresponding knowledge.

Programming language definitions assign formal meaning to complete programs. Programmers, however, spend a substantial amount of time interacting with incomplete programs - programs with holes, type inconsistencies and binding inconsistencies - using tools like program editors and live programming environments (which interleave editing and evaluation). Semanticists have done comparatively little to formally characterize (1) the static and dynamic semantics of incomplete programs; (2) the actions available to programmers as they edit and inspect incomplete programs; and (3) the behavior of editor services that suggest likely edit actions to the programmer based on semantic information extracted from the incomplete program being edited, and from programs that the system has encountered in the past. This paper serves as a vision statement for a research program that seeks to develop these missing semantic foundations. Our hope is that these contributions, which will take the form of a series of simple formal calculi equipped with a tractable metatheory, will guide the design of a variety of current and future interactive programming tools, much as various lambda calculi have guided modern language designs. Our own research will apply these principles in the design of Hazel, an experimental live lab notebook programming environment designed for data science tasks. We plan to co-design the Hazel language with the editor so that we can explore concepts such as edit-time semantic conflict resolution mechanisms and mechanisms that allow library providers to install library-specific editor services.

The promise of live programming is to shorten or even break the infamous edit-compile-run cycle, providing live feedback on a program's envisioned behavior while it is being written. Several live programming languages and environments exist, from venerable examples (Smalltalk, LISP) to more recent efforts like Ruby. In most cases either the IDE comes as an afterthought, after the language is designed, or novel languages are made to fit into existing IDEs. We pursue a middle ground by co-evolving both a language and its IDE: we are developing a novel live programming language, called Moon, from scratch, and are concurrently building its IDE. We illustrate our efforts so far and discuss our overall vision.

Dynamic Software Updating (DSU) consists in updating running programs on the fly without any downtime. This feature is interesting in critical applications that must run continuously. Because updates may lead to safety errors and security breaches, the question of their correctness is raised. Formal methods are a rigorous means to ensure the high level of safety requested by such applications. The detection of points to perform safe updates is a critical issue in DSU. Indeed, an hazardous update point leads the updated system to erroneous and unexpected behavior. We present in this paper a mechanism to detect safe update points in DSU for Java Card applications. The mechanism is then formally verified using model checking against correctness properties: deadlock free, activeness safety and DSU-liveness.

Preserving instance state during refactorings in live environments

The correct implementation of network policies (e.g., routing, NAT, VPNs, load balancing, and IDS/IPS) for underlying network functions is critical, as it determines the security, availability and performance of a production network. However, it is notoriously known that making sure network policies are correctly implemented is challenging, even for basic reachability policies. This becomes more challenging in cloud environments featured with SDN-enabled NFV, where multiple tenants are hosted with richer in-network services in the form of chained, virtualized network functions with dynamic, customized network policies. To address this problem, existing approaches have been proposed to model network behaviors, generate synthetic network traffic, and verify network policies. However, these solutions face a fundamental challenge in SDN-enabled NFV --- lack of capturing dynamics of the production system. For example, virtual network functions (running in virtual machines) can be arbitrarily composed to realize service chaining on the fly; the chained network functions create more complex unpredictable network policies. Further, the on-demand cloud service model compounds this complexity with dynamic loads and varying network function requirements. One (seemingly straightforward) solution may be to extend existing network models to capture dynamic system behaviors, and thus generate test traffic with broader coverage. However, despite the possibility of doing so, such model-based approaches will easily result in state-space explosion, which will take extensive time for completing a simple network verification task even for a small network. On the other hand, focusing on a subset of intended polices may reduce the state space, but could fail to catch some critical sources of violations in practice --- in most cases, it is even hard (or impossible) to know the polices without really operating network functions in a production environment (e.g., with improvised changes in NFV configurations/policies). Ideally, conducting network verification in a live production environment --- complementary to model-based verification approaches --- is attractive, as production traffic captures the most exact, realistic dynamic state of the system that model-based verification tools cannot provide. However, doing so brings potential risks of impacting or even damaging a live production environment, as mis-configured inline test network functions could wrongly manipulate network traffic --- numerous network outrages are actually caused by (tiny) mis-configurations of a live production system. It becomes more problematic in multi-tenant cloud, as such misbehaviors could impact other tenants. In this paper, we present ShadeNF, a novel online verification platform for testing in-cloud network functions in a production-like environment, without disrupting the live production system. ShadeNF enables such a production-like test environment (i.e., the shadow system) with an exact clone of the production network functions (to be tested), which captures the dynamic state and vulnerabilities of the live production system. Further, ShadeNF delicately steers live production traffic to the shadow system as the test traffic, which captures the dynamic state of the production workloads. The actual verification is operated in a completely isolated environment with desired resources (e.g., CPU, memory and storage), thus not interfering with the production system. We make three key contributions in ShadeNF: First, ShadeNF introduces a new live, consistent snapshot approach to clone chained, dependent network functions by leveraging SDN virtual switches. This approach both preserves a consistent snapshot and reduces performance overhead with no modifications to VMs software and legacy network flows. Second, to capture the dynamics of the production workloads, ShadeNF creates a new traffic forwarding plane, which selectively, unidirectionally steers the production traffic to the test system with new programmable forwarding pipes. These forwarding pipes also enable the autochaining of arbitrary network functions. Last, to explore broader test coverage, ShadeNF advances existing model-based approaches by taking patterns of real production traffic into consideration --- ShadeNF populates synthetic test traffic with realistic traffic patterns that are captured and provided automatically. We have implemented a ShadeNF platform prototype upon OpenStack. Our evaluation in a real cloud testbed shows that: (1) ShadeNF captures the dynamics of a production system without affecting the production system; (2) ShadeNF can effectively detect a variety of policy violations.

Learning how to program can be quite difficult. Beginners face two obstacles to learning to program: developing a mental model of the steps of computation and learning the syntax of a programming language. Because programmers are prone to make mistakes that introduce errors into their code, it is important that they understand what each line of code is doing in their program. I propose that a system be built in which writing code and running code happen simultaneously. This system shows the results of code to the user immediately after it is typed; this type of system is called a live programming environment. The environment's features allow the programmer to highlight individual lines of code to see their result. The programmer can walk through loops and explore the details of a function call. The programmer can unit test a function or class. The environment displays variables declared in the code and their values and allows the programmer to create a visualization for class objects. The environment appears to be beneficial for beginners based on the results of two surveys with a total of 27 participants. The results of this thesis show that live programming environments should be developed to help the beginner learn to program.

Live programming environments enable programmers to edit a running program\nand obtain immediate feedback on each individual change. The liveness quality\nis valued by programmers to help work in small steps and continuously add or\ncorrect small functionality while maintaining the impression of a direct\nconnection between each edit and its manifestation at run-time. Such immediacy\nmay conflict with the desire to perform a combined set of intermediate steps,\nsuch as a refactoring, without immediately taking effect after each individual\nedit. This becomes important when an incomplete sequence of small-scale changes\ncan easily break the running program. State-of-the-art solutions focus on\nretroactive recovery mechanisms, such as debugging or version control. In\ncontrast, we propose a proactive approach: Multiple individual changes to the\nprogram are collected in an Edit Transaction, which can be made effective if\ndeemed complete. Upon activation, the combined steps become visible together.\nEdit Transactions are capable of dynamic scoping, allowing a set of changes to\nbe tested in isolation before being extended to the running application. This\nenables a live programming workflow with full control over change granularity,\nimmediate feedback on tests, delayed effect on the running application, and\ncoarse-grained undos. We present an implementation of Edit Transactions along\nwith Edit-Transaction-aware tools in Squeak/Smalltalk. We asses this\nimplementation by conducting a case study with and without the new tool\nsupport, comparing programming activities, errors, and detours for implementing\nnew functionality in a running simulation. We conclude that workflows using\nEdit Transactions have the potential to increase confidence in a change, reduce\npotential for run-time errors, and eventually make live programming more\npredictable and engaging.\n

It is widely desired that running software systems can be updated on the fly in response to the changes in the environment they are situated in and in the requirements they must satisfy. Systematic support for dynamic software update must make it safe, efficient and easy to use. This chapter overviews related work and our efforts on dynamic software update at three different levels of granularity: code-level update of Java programs, component-level update of distributed systems, and process-level instance migration of workflow processes.

Operators that, over time, produce successive approximations to the correct output are called progressive operators. These operators, and measures for evaluating them, are defined. Desirable properties for progressive operators are identified. Methods for their design are suggested. Examples of progressive operators for image processing are presented. >

Nuggets 1 is a European project researching the use of MXF files on IT networks, allowing remote control of cameras in a live production environment. The project partners (Thomson Grass Valley, ZDF, Snell & Wilcox, T-Systems, Enertec, INESC Porto) are creating a local area network/ wide-area network (LAW/WAN)-based workflow that will be demonstrated in 2004. This paper will outline many of the issues discovered in the process of moving from a traditional tape/SDI point-to-point environment to a server/file interchange environment. Details of the proposed network functionality will be given in order to generate audience discussion of the requirements of a file interchange-based workflow.

Recommendation systems usually create static models from historical data. Due to concept drift and changes in the environment, such models are doomed to become stale, which causes their performance to degrade. In live production environments, models are therefore typically retrained at fixed time-intervals. Of course, every retraining comes at a significant computational cost, making very frequent model updates unrealistic in practice. In some cases, the cost is worth it, but in other cases an update could be redundant and the cost an unnecessary loss. The research question then consists of finding an acceptable update schedule for your recommendation system, given a limited budget. This work provides a pragmatic analysis of model staleness for a variety of collaborative filtering algorithms in news and retail domains, where concept drift is a known impediment. We highlight that the rate at which models become stale is highly dependent on the environment they perform in and that this property can be derived from data. These findings are corroborated by empirical observations from four large-scale online experiments. Instead of retraining at regular intervals, we propose an adaptive scheduling method that aims to maximise the accuracy of the recommendations within a fixed resource budget. Offline experiments show that our proposed approach improves recommendation performance while keeping the cost constant. Our findings can guide practitioners to spend their available resources more efficiently.

Automatic Multiple Sensor Data Acquisition System in a Real-time Production Environment

Inference Enterprise Modeling (IEM) is a methodology developed to address test and evaluation limitations that insider threat detection enterprises face due to a lack of ground truth and/or missing data. IEM uses a collection of statistical, data processing, analysis, and machine learning techniques to estimate and forecast the performance of these enterprises. As part of developing the IEM method, models satisfying various detection system evaluation requirements were created. In this work, we extend IEM as a digital twin generation technique by representing modeled processes as executable UML Activity Diagrams and tracing solution processes to problem requirements using ontologies. Using the proposed framework, we can rapidly prototype a digital twin of a detection system that can also be imported and executed in systems engineering simulation software tools such as Cameo Enterprise Architecture Simulation Toolkit. Cyber security and threat detection is a continuous process that requires regular maintenance and testing throughout its lifecycle, but there often exists access issues for sensitive and private data and proprietary detection model details to perform adequate test and evaluation activities in the live production environment. To solve this issue, organizations can use a digital twin technique to create a real-time virtual counterpart of the physical system. We describe a method for creating digital twins of live and/or hypothetical insider threat detection enterprises for the purpose of performing test and evaluation activities on continuous monitoring systems that are sensitive to disruptions. In this work, we use UML Activity Diagrams to leverage the integrated simulation capabilities of Model-Based Systems Engineering (MBSE).