Collaborative Design in DARPA’s FANG I Challenge

The term ‘morphing aircraft’ describes a broad range of air vehicles and vehicle components that adapt to planned and unplanned multipoint mission requirements. Adaptation or morphing requires changing system features including vehicle ‘states,’ such as vehicle shape, during in-flight operation. The term morphing can be applied to almost any activity in which in-flight vehicle features are changed. As such, morphing has become a buzzword loosely applied to a wide variety of activities, some of which are disconnected from air vehicle morphing development. This has led to three myths: 1) morphing shape change is too expensive, 2) morphing aircraft must weigh more than nonmorphing aircraft, and 3) morphing requires exotic materials and complex systems. This paper attempts to dispel these myths by reviewing early morphing aircraft history to identify inventions and innovations that led to both successes and failures. The review also discusses recent government-sponsored activities in the United States: in particular, morphing systems development sponsored by the Defense Advanced Research Projects Agency viewed from the author’s perspective as a former Defense Advanced Research Projects Agency Program Manager. The review concludes with identification of possible avenues for future morphing aircraft evolution and morphing device development.

Renewed interest in centimeter-scale flight vehicles for short-range urban surveillance and sampling missions led to the Defense Advanced Research Projects Agency’s Nano Air Vehicle (NAV) program (2006-2008). This effort explored technology for fixed wing, rotary wing, or flapping wing vehicles of 7.5 centimeter length and 10 grams total mass that can fly for 20 minutes with a range of 1 kilometer. Under this program the authors participated in a Lockheed Martin-led development of a unique self-propelled monowing rotorcraft, Samarai, inspired by a maple seed [1]. Following the end of the NAV program, continued research and development led by Lockheed Martin has resulting in successful demonstration of flight by Samarai vehicles at multiple scales. 7

The network intrusions highly depend on exploiting the victim's vulnerabilities by performing multiple intrusive actions. However, the security experts need to inspect a considerable number of events to generate the knowledge to figure out the intrusive behavior with heavy costs. In this paper, we propose a data-driven analysis framework for event analysis for network intrusion detection systems (NIDSs) by data mining approaches without experts heavily involved. The proposed system enhances the intrusion detection capability of NIDSs by reducing the false alarms and correlating the causal relations in different attack steps. We implement the proposed framework named as IDEAs, evaluate it by the DARPA'99 benchmark and deploy it successfully in a real world organization. It could easily plug in the state-of-the-art open source IDS architecture (e.g. snort) for being helpful to identify attacks and reduce false alarms. This paper presents IDEAs's mechanism, implementation, and experiment evaluations that regard the DARPA'99 benchmark and real world data collected from ChungHwa Telecommunication Laboratories.

End user involvement and input into the design and evaluation of information systems has been recognized as being a critical success factor in the adoption of information systems. Nowhere is this need more critical than in the design of health information systems. Consistent with evidence from the general software engineering literature, the degree of user input into design of complex systems has been identified as one of the most important factors in the success or failure of complex information systems. The participatory approach goes beyond user-centered design and co-operative design approaches to include end users as more active participants in design ideas and decision making. Proponents of participatory approaches argue for greater end user participation in both design and evaluative processes. Evidence regarding the effectiveness of increased user involvement in design is explored in this contribution in the context of health IT. The contribution will discuss several approaches to including users in design and evaluation. Challenges in IT evaluation during participatory design will be described and explored along with several case studies.

Mass and energy rate (ER) data have been collected for a wide variety of (complex) systems from the biological, cultural, and cosmological realms. They range from the cytochrome oxidase protein (10-22 kg and 6x10-19 W) to the observable universe (1.5x1053 kg and 1048 W) and, thus, span 75 mass and 66 ER orders of magnitude. Many of these systems are relevant for the big history (BH) narrative, i.e., the development of complexity over “big time” from the Big Bang up to the human society on Earth of today. The purpose of this paper is not per se to describe their history though, but to explore a master plot of ER vs. mass. Notably, the development of systems over big time has followed a rather tortuous path criss-crossing over this ER vs. mass master plot. The true mass of the system as a whole is used (for example trees including the non-living wood, living organisms including their intrinsic water, and social systems including the built constructs), because these inactive parts are essential for the performance of the system and facilitate its ER. A double logarithmic master plot of all ER vs. mass data shows clusters of data points. To some extent, this provides quantitative support for the distinction between the (sub-)realms, which is based on a qualitative description of their material structure and energy processing. In the master plot, small systems with low mass and ER converge into larger systems with larger mass and ER, which is typically accompanied by a decrease of the energy rate density (ERD = ER/mass). Correlation of ER with mass for various groups of systems demonstrates both sub- and supra-linear scaling with the power law β constant varying between 0.5 and 4.0, showing that the mechanisms of self-organisation are quite different for the corresponding system groups. The combination of convergence and scaling with β always larger than zero explains why the ER & mass data points fall in a diagonal band with a width of 17 orders of magnitude.
 ER and mass have changed over wide ranges during the evolution of groups of systems, suggesting that evolution can be viewed as a process of systems exploring a larger ER vs. mass area until they run into ER and/or mass limitations. Indeed, there is a diagonal ER vs. mass limit for stable systems in all realms, corresponding to an ERD value of around 105 W/kg. Systems with ER & mass combinations above this limit, such as bombs, super-novae and cosmological transients, are unstable and “explosive”. This raises the interesting question of whether such an ERD maximum puts a limit on the development of complexity over big time. It seems that the low, right side of the master plot is empty. However, it is argued here that it is full of systems with low ER, such as dormant, living organisms, technological systems with their power adjusted or even switched off, as well as cooling, cosmological objects. Such systems are typically considered of less interest in a BH context, but they are viewed here as simple, complex systems which are out of equilibrium with matter, energy and information stored in their structure. While ERD appears to increase with the ‘advancement’ of systems over big time [5,51,52], there are quite some confounding factors regarding the efficacy of ERD as a metric for complexity in BH. For example, ERD decreases during the lifetime of a human and the human society (the mass of human-made constructs has grown faster than the global energy consumption), as well as during the evolution of living organisms and stars, whereas complexity is considered to increase. High ERD values of system parts may be illustrative for the complexity of the larger system, but are not representative for ERD of the system itself. Machines with an increased efficiency of energy conversion have a lower ERD, but could be considered more complex. The smallest and largest ERD values observed for the various realms appear to correlate with activity level and reciprocally with size, which do not per se reflect complexity. It is hoped that the raw data collected and the major trends observed in this paper will offer new insights into various aspects of the evolution of the universe over big time, and serve as an important resource for other related studies.

McDermott Technology, Inc. (MTI) and JX Crystals have developed an innovative design for a compact, 500-watt, thermophotovoltaic (TPV) power supply using diesel fuel. Under a contract with the Defense Advanced Research Projects Agency (DARPA) and managed by the U.S. Army Communications-Electronics Command (CECOM), this design is being reduced to hardware. Prototypes of the two main subsystems, the power converter assembly (PCA) and the burner/emitter/recuperator (BER), have been designed, fabricated, and tested. The PCA uses low-band-gap gallium antimonide (GaSb) photovoltaic (PV) cells for high efficiency and power density. The prototype PCA will be air cooled for system simplicity and portability. However, initial testing was performed on a water-cooled PCA. The BER uses a thermal vaporizer to produce a stable, high-intensity, low-emissions combustion zone inside an impervious emitter. A thermally integrated recuperator is utilized to boost system efficiency by transferring the unused energy in the exhaust stream to the incoming fuel and combustion air. This paper describes the design, testing and performance of the first-generation PCA and BER along with model predictions used for design and evaluation.

It is very challenging to design complex machines and systems that operate in very difficult remote locations, under largely unknown or uncertain conditions. Specifications for such systems must be extremely detailed and extensive, with input from professionals who have designed such systems before, and who gained considerable experience from their operations. Since much of the operating environment is not known in advance, cognitive informatics and computing should play a critical role in such design and operation. This paper describes such a complex system, the T-Sat1 nanosatellite, including its characteristics, its mission, subsystems, as well as the development of specifications, protocols for verification, testing, launch, early operating procedures, and concepts for nominal operations. Particular attention is given to the formation and maintenance of a team of teams, with a multitude of their interactions. The design teams must focus on the satellite subsystems, assembly, integration and testing. The teams of advisors (from academia, aerospace and other industries, business, military, government, and other organizations such as the radio community) must focus on optimal assistance provided to the corresponding design teams.

Design of complex systems in the presence of large uncertainties: A statistical approach

The crowdsourcing literature has shown that domain experts are not always the best solvers for complex system design problems. Novices and specialists in adjacent domains can, under certain conditions, provide novel solutions at lower costs. Additionally, the best types of solvers for different sub-problems are dependent on the architecture of complex systems. The assignment of solvers based on the architecture, referred to as Solver-Aware System Architecting (SASA), expands traditional system architecting practices by considering solver characteristics and contractual incentive mechanisms in the design process and aims to improve complex system design and innovation by leveraging the strengths of domain experts, crowds, and specialists for different parts of the problem. Given the complexity of system design problems and the variety of solvers available, it is desirable to have heuristics to guide solver assignment to different problems. Developing effective heuristics for solver assignments in complex system design is challenging due to a large number of possible combinations of problem-solver pairs. To address this challenge, this paper presents a computational approach using a multi-armed bandit (MAB) formulation to generate heuristics for solver assignment. The approach is demonstrated using a simple and idealized problem of golf, which has characteristics similar to design problems, including how the problem is decomposed into sub-problems, and solved by different solvers. The results show that the proposed approach is effective in deriving a rich set of heuristics for the golf problem, and can be extended in the future to more complex systems design problems.

A close look at many existing systems reveals their two- or multi-layer nature, where a number of coexisting networks interact and depend on each other. For instance, in the Internet, any application-level graph (such as a peer-to-peer network) is mapped on the underlying IP network that, in turn, is mapped on a mesh of optical fibers. This layered view sheds new light on the tolerance to errors and attacks of many complex systems. What is observed at a single layer does not necessarily reflect well the state of the entire system. On the contrary, a tiny, seemingly harmless disruption of one layer, may destroy a substantial or essential part of another layer, thus making the whole system useless in practice. In this thesis we consider such two-layer systems. We model them by two graphs at two different layers, where the upper-layer (or logical) graph is mapped onto the lower-layer (physical) graph. Our main goals are the following. First, we study the robustness to failures of existing large-scale two-layer systems. This brings us some valuable insights into the problem, e.g., by identifying common weak points in such systems. Fortunately, these two-layer problems can often be effectively alleviated by a careful system design. Therefore, our second major goal is to propose new designs that increase the robustness of two-layer systems. This thesis is organized in three main parts, where we focus on different examples and aspects of the two-layer system. In the first part, we turn our attention to the existing large-scale two-layer systems, such as peer-to-peer networks, railway networks and the human brain. Our main goal is to study the vulnerability of these systems to random errors and targeted attacks. Our simulations show that (i) two-layer systems are much more vulnerable to errors and attacks than they appear from a single layer perspective, and (ii) attacks are much more harmful than errors, especially when the logical topology is heterogeneous. These results hold across all studied systems. A natural next step consists in improving the failure robustness of two-layer systems. In particular, in the second part of this thesis, we consider the IP/WDM optical networks, where an IP backbone network is mapped on a mesh of optical fibers. The problem lies in designing a survivable mapping, such that no single physical failure disconnects the logical topology. This is an NP-complete problem. We introduce a new concept of piecewise survivability, which makes the problem much easier in practice. This leads us to an efficient and scalable algorithm called SMART, which finds a survivable mapping much faster (often by orders of magnitude) than the other approaches proposed to date. Moreover, the formal analysis of SMART allows us to prove that a given survivable mapping does or does not exist. Finally, this approach helps us to find vulnerable areas in the system, and to effectively reinforce them, e.g., by adding new links. In the third part of this thesis, we shift our attention one layer higher, to the application-over-IP setting. In particular, we consider the design of Application-Level Multicast (ALM) for interactive applications, where a single source sends a delay-constrained data stream to a number of destinations. Interactive ALM should (i) respect stringent delay requirements, and (ii) proactively protect the system against overlay node failures and against (iii) the packet losses at the IP layer. We propose a two-layer-aware approach to this problem. First, we prove that the average packet loss rate observed at the destinations can be effectively approximated by a purely topological metric that, in turn, drops with the amount of IP-level and overlay-level path diversity available in the system. Therefore, we propose a framework that accommodates and generalizes various techniques to increase the path diversity in the system. Within this framework we optimize the structure of ALM. As a result, we reduce the effective loss rate of real Internet topologies by typically 30%-70%, compared to the state of the art. Finally, in addition to the three main parts of the thesis, we also present a set of results inspired by the study of ALM systems, but not directly related to the 'two-layer' paradigm (and thus moved to the Appendix). In particular, we consider a transmission of a delay-sensitive data stream from a single source to a single destination, where the data packets are protected by a Forward Error Correction (FEC) code and sent over multiple paths. We show that the performance of such a scheme can often be further improved. Our key observation is that the propagation times on the available paths often significantly differ, typically by 10-100ms. We propose to exploit these differences by appropriate packet scheduling, which results in a two- to five-fold improvement (reduction) in the effective loss rate.

One of today's major challenges is to integrate the different disciplines involved in the design, operation and maintenance of complex systems. Examples are transportation systems (cars, trains, airplanes…), energy production plants (renewable energy, thermal and nuclear power…), transportation and distribution networks (energy transmission and distribution, water distribution, gas transportation, road and railway networks) etc. Company executives, project managers, operation and design engineers need to grasp the opportunities coming from these complex, integrated systems while avoiding risks, to the extent possible. Decisions must be taken in uncertain environments: then, uncertainty needs to be handled, for effective and rational decision-making. The one-week PhD school aims at providing an advanced training on concepts, methods and tools for assessing and managing risks and opportunities in complex systems. The courses offered cover concepts and methods for the evaluation, management and control of technical risks, as well as the uncertainties in the evaluations for confident decision-support. The presented approaches are notably based on Systems Analysis, Systems Engineering, Applied Mathematics and Computer Science, The objective is to provide the participants with adequate tools for tackling the problem with scientific rigor. The acquired concepts, methods and tools constitute an essential part of the skills that researchers, engineers and managers must have. In the end, the courses provide the knowledge and competence needed to architect and operate complex systems, i.e. to make them efficient and reliable in operation, and resilient to major hazardous events. One key objective of the PhD school is to engage multi-disciplinary and multi-cultural teams of students in the definition, design and development of a scientific research process in the areas of interest for the disciplines of risk assessment and uncertainty analysis. To this aim, “mixed” teams of participants with different backgrounds and Institutions of origin are formed at the beginning of the PhD school, and charged with the definition and statement of a research problem to be addressed. The procedures and frames of work for the development of original solutions to the problems defined by the various teams are discussed in self-managed sessions at the end of each day. Engagement by industrial partners in the monitoring of the projects is offered. At the end of the week, the teams present the definitions of the problems and illustrate the research work that they intend to carry out for their original solutions. The teams have, then, to continue the work throughout the following semester, with the objective of producing scientific papers presenting the findings of the research. Once positively evaluated by the responsibles of the PhD school, the papers can be submitted to the appropriate international communities of peers for publication on international journals or presentation at international conferences. The benefits of the projects are expected to be an independent and direct exposure to: research problem definition and solution; multi-national, multi-cultural, multi-disciplinary, long-distance research work management.

The Importance of Models in Making Complex Systems Comprehensible

Complex systems are often a derivative of the current trend that dictates increased functionality and performance. Mobile phones that provide internet access and music player, fighter aircraft with fly-by-wire for flight control, and energy grid that also monitors usage and recommends corrective actions are few of the examples of emerging complex systems. There are a large number of causes that could lead to increased complexity of a system, such as (1) increase in number of parts, (2) increase in number of interactions, and (3) integration of multiple technologies. Increase in complexity makes these systems significantly challenging in terms of design and optimization, resulting in large time and resource investment. Therefore, it is typical for complex systems to use non-mature technologies that are expected to mature along the way. However, such infusion introduces significant levels of uncertainty, which historically have led to explosion in development cost. In this paper, we assess the impact of uncertainty on a complex system. Through numerical simulations, we demonstrate that increasing complexity does not necessarily result in vulnerability of a system to uncertainties. These results challenge the conviction held by complex system design community, which assumes complexity and uncertainty have a direct correlation. Our results indicate that selection of system architecture is critical for system robustness. This should provide motivation for further research in the complex system arena.

Engineers are faced with two major challenges when carrying out the thermal-fluid design of a complex system consisting of many interacting components such as the PBMR power plant. The first challenge is to predict the performance of all the individual thermal-fluid components. The second challenge is to predict the performance of the integrated plant consisting of all its sub-systems. The complexity associated with the thermal-fluid design of complex systems requires the use of a variety of analysis techniques and simulation tools. These range from simple one-dimensional models that do not capture all the significant physical phenomena, to large-scale three-dimensional CFD codes that, for practical reasons, can not simulate the entire plant as a single integrated model. In the systems CFD approach a network code serves as the framework to link the models of the various components together and to control the solution. The models of the components can be of varying degrees of complexity. These can range from simple lumped models to complex fully three-dimensional CFD models. This paper gives a brief overview of the systems CFD (SCFD) approach and an overview of the SCFD model of the pebble bed nuclear reactor that was developed.

The classical engineering fields have evolved standards and techniques for developing complex systems. For example, both mechanical and electrical engineers have a wide variety of standard components, with defined capabilities, that they can draw upon (e.g., gears, transistors) in the design of complex systems. On the other hand, software engineering has struggled with the basic idea of reusability. Software engineering approaches, such as the use of components [1] that promote the concept of information hiding [2] and the introduction of structured programming languages [3], offer a roadmap to an improved software reuse. Unfortunately, their adoption by robotics researchers has been slow, impeded by the tradition of individual research groups crafting independent and incompatible solutions to common problems.