PYREF: Refactoring Detection in Python Projects

Python has emerged as one of the most popular programming languages, extensively utilized in domains such as machine learning, data analysis, and web applications. Python’s dynamic nature and extensive usage make it an attractive candidate for dynamic program analysis. However, unlike for other popular languages, there currently is no comprehensive benchmark suite of executable Python projects, which hinders the development of dynamic analyses. This work addresses this gap by presenting DyPyBench, the first benchmark of Python projects that is large-scale, diverse, ready-to-run (i.e., with fully configured and prepared test suites), and ready-to-analyze (by integrating with the DynaPyt dynamic analysis framework). The benchmark encompasses 50 popular open-source projects from various application domains, with a total of 681k lines of Python code, and 30k test cases. DyPyBench enables various applications in testing and dynamic analysis, of which we explore three in this work: (i) Gathering dynamic call graphs and empirically comparing them to statically computed call graphs, which exposes and quantifies limitations of existing call graph construction techniques for Python. (ii) Using DyPyBench to build a training data set for LExecutor, a neural model that learns to predict values that otherwise would be missing at runtime. (iii) Using dynamically gathered execution traces to mine API usage specifications, which establishes a baseline for future work on specification mining for Python. We envision DyPyBench to provide a basis for other dynamic analyses and for studying the runtime behavior of Python code.

Python is a popular typical dynamic programming language. In Python, dynamic typing is one of the most critical dynamic features. The lack of type information is likely to hinder the maintenance of Python projects. However, existing work has seldom focused on studying the impact of Python dynamic typing on project maintenance. This paper focuses on the two most common practices of Python dynamic typing, i.e. inconsistent-type assignments (ITA) and inconsistent variable types (IVT). Two approaches are proposed to identify ITA and IVT, i.e. identifying ITA by analyzing Abstract Syntax Trees and comparing identifiers types and identifying IVT by constructing a type dependency graph. In empirical experiments, we first locate the usage of ITA and IVT in 10 open-source Python projects. Then, we investigate the relations between the occurrence of ITA and IVT and the results of maintenance tasks. The study results show that projects are more prone to change as the number of dynamic typing identifiers increases. There is a weak connection between change-proneness and variable dynamic typing. There is a high probability that maintenance time and the acceptance of commits decrease as dynamic typing identifiers increase in projects. These results implicate that dynamic and static variables should be divided while developing new programming languages. Dynamic typing identifiers may not be the direct root causes for most software bugs. The categories of these bugs are worth exploring.

Refactoring as a process is aimed at improving the quality of a software system while preserving its external behavior. In practice, refactoring comes in the form of many specific and diverse refactoring operations, which have different scopes and thus a different potential impact on both the production and the test code. We present a large-scale quantitative study complemented by a qualitative analysis involving 615,196 test cases to understand how and to what extent different refactoring operations impact a system's test suites. Our findings show that while the vast majority of refactoring operations do not or very seldom induce test breaks, some specific refactoring types (e.g., “RENAME Attribute” and “RENAME Class”) have a higher chance of breaking test suites. Meanwhile, “ADD Parameter” and “CHANGE Return Type” refactoring operations often require additional lines of changes to fix the test suite they break. While some modern IDEs provide features to automatically apply these two types of refactoring operations, they are not always able to avoid test breaks, thus demanding extra human efforts.

Software cost estimation is a process of planning, risk analysis, and decision making for project management in software development. Cost of project development encompasses a software project’s effort and development time. One popular model of software cost estimation is constructive cost model (COCOMO) model, which is a mathematical model proposed by Boehm, used for estimate the software effort and development time. The objective of this paper is to improve the basic COCOMO model’s coefficients for modern programming languages like Python, R, C++, etc. Many techniques were presented in the past for effort and time estimation using machine learning. But all these techniques were trained and tested for older programming languages. In order to improve the accuracy of COCOMO for modern programming languages, six Python projects have been considered and genetic algorithm (GA) is applied in these projects to define new values for basic COCOMO coefficients and the development time is calculated for Python projects. The time estimated using GA coefficients is compared with the original COCOMO and actual time. Using mean magnitude relative error, the error from the original COCOMO time is 54.49% and error from GA time is 21.23%.

The latest rankings of computer language popularity once again list C as the most popular programming language [1]. As a computer scientist that has written a lot of C code over the years, I must admit that makes me smile. While I don't write code much anymore, I like writing code, and I like writing it in C. Apparently so do a lot of other people. However, C was also the most popular programming language 25 years ago which is one indication that, in the field of software development, not much has changed. Are software developers unwilling to accept new paradigms or are the new paradigms proposed to-date simply unacceptable? This talk discusses what has and hasn't changed in the area of software development, how change (or the lack of it) relates to application security, and concludes with some thoughts on possible directions for the future.

Due to wide adoption of object-oriented programming in software development, there is always a requirement to produce well-designed software systems, so that the overall software maintenance cost is reduced and reusability of the component is increased. But, due to prolonged maintenance activities, the internal structure of software system deteriorates. In this situation, restructuring is a widely used solution to improve the overall internal structure of the system without changing its external behavior. As, it is known that, one technique to perform restructuring is to use refactoring on the existing source code to alter its internal structure without modifying its external functionality. However, the refactoring solely depends on our ability to identify various code smells present in the system. Refactoring aims at improving cohesion and reducing coupling in the software system. So, in this paper, a restructuring approach based on refactoring is proposed through improvement in cohesion. This paper focuses on improving the cohesion of different classes of object-oriented software using a newly proposed similarity metric based on Frequent Usage Patterns (FUP). The proposed similarity metric measure the relatedness among member functions of the classes. The metric makes use of FUPs used by member functions. The FUP consists of unordered sequences of member variables accessed by member function in performing its task. The usage pattern includes both direct and indirect usages based on sub-function calls within a member function. Based on the values of the similarity metric, we performed hierarchical agglomerative clustering using complete linkage strategy to cluster member functions. Finally, based on the clusters obtained, the source code of the software is refactored using proposed refactoring algorithm. The applicability of our proposed approach is tested using two java projects related to different domains of real life. The result obtained encourages the applicability of proposed approach in the restructuring of a software system.patterns, refactoring, hierarchical clustering, maintainability. usage patterns, refactoring, hierarchical clustering, maintainability.

Java has emerged as a popular programming language and platform for Web applications. JavaBeans defines the software component model in the Java programming language that is used for creating reusable, coarse‐grained components. Beans are used in numerous complementary technologies including Enterprise JavaBeans, Java Abstract Windowing Toolkit, Java Database Connectivity, JavaMail, and Java Management Extensions to create software components. Beans provide a hybrid of object‐oriented and loosely coupled architectural styles, where the interaction between components is in the form of both events and method calls. This chapter focuses on the support for component technology in Java, specifically in the form of JavaBeans and its technology variants that fulfil the needs of Web application development. The chapter also discusses the role of JavaBeans and its technology variants in architecture‐based software development.

ADA is a popular programming language that was named after Lady Ada Lovelace (1815-1852) and recommended by Department of Defense, USA, for development of large scale safety-critical software systems. In this Fast Abstract, ADA is reinterpreted as Autonomous, Dependable and Affordable. It is argued that future software systems should be ADA. A few affordable techniques are briefly described to highlight how to make software systems ADA.

Since its inception in 1996, aspect-oriented programming (AOP) has been believed to reduce the effort required to maintain software systems by replacing cross-cutting code with aspects. However, little convincing empirical evidence exists to support this claim, while several studies suggest that AOP brings new obstacles to maintainability. This paper discusses two controlled experiments conducted to evaluate the impact of AspectJ (the most mature and popular aspect-oriented programming language) versus Java on software evolvability. We consider evolvability as the ease with which a software system can be updated to fulfill new requirements. Since a minor language was compared to the mainstream, the experiments were designed so as to anticipate that the participants were much more experienced in one of the treatments. The first experiment was performed on 35 student subjects who were asked to comprehend either Java or AspectJ implementation of the same system, and perform the corresponding comprehension tasks. Participants of both groups achieved a high rate of correct answers without a statistically significant difference between the groups. Nevertheless, the Java group significantly outperformed the AspectJ group with respect to the average completion time. In the second experiment, 24 student subjects were asked to implement (in a non-invasive way) two extension scenarios to the system that they had already known. Each subject evolved either the Java version using Java or the AspectJ version using AspectJ. We found out that a typical AspectJ programmer needs significantly fewer atomic changes to implement the change scenarios than a typical Java programmer, but we did not observe a significant difference in completion time. The overall result indicates that AspectJ has a different effect on two sub-characteristics of the evolvability: understandability and changeability. While AspectJ decreases the former, it improves one aspect of the latter.

The recent successful application of Datalog in a number of advanced projects, has renewed the interest in Datalog-based systems for developing real-world applications. Indeed, in the last few years Datalog has been successifully applied in many different areas of computer science, including: Artificial Intelligence, Data Extraction, Information Integration and Knowledge Management. Interestingly, besides the scientific applications, Datalog-based systems were also applied for developing some industrial systems. Nonetheless, in order to boost the adoption of Datalog-based technologies in the scientific community and especially in industry, it is important to provide effective programming tools, which support the activities of researchers and implementors and simplify users' interactions with Datalog systems. Indeed, development frameworks and tools provide indispensable means for assisting and simplifying application development. For this reason, the most popular programming languages and also commercial off-of- the-shelf software products (e.g., DBMSs) are always complemented by Integrated Development Environments (IDE) and Software Development Kits (SDK).

The ACM/ICPC is recognized as the largest and highest level international collegiate programming contest in the world, and it is one of the most important ways to train and select top students in computer science worldwide. Currently, C++ is the most popular and practical computer advanced programming language in the world. It is an important professional main course and one of the main tools for teaching programming and computer software development. It has become a programming course for universities and colleges. With further deepening of education reform, the paperless examination has become an important sign of the college examination reform, and its advantages have become more and more obvious. Also, the paperless examination has gradually become a trend of various examinations. Although there have been some paperless examination systems, but these systems usually can only automatically score four types of objective questions, including true and false questions, fill-in-the-blank questions, single-choice questions and multiple-choice questions. These paperless examination systems are still unable to conduct fair and impartial automatic scoring for programming questions which are however important subjective questions. In response to this problem, based on the ACM/ICPC platform, we propose a new design method for the C++ paperless examination system, which realizes the automatic scoring of C++ examinations. The system can greatly improve the flexibility, fairness, impartiality, credibility and work efficiency of the examination. Our design system can be an important platform for deepening the teaching reform of colleges and universities, promoting the innovation of C++ teaching, and improving the efficiency of college examinations. Therefore, it has a certain promotion and demonstration value.

Regular expressions (regexes) are a popular and powerful means of automatically manipulating text. Regexes are also an understudied denial of service vector (ReDoS). If a regex has super-linear worst-case complexity, an attacker may be able to trigger this complexity, exhausting the victim’s CPU resources and causing denial of service. Existing research has shown how to detect these superlinear regexes, and practitioners have identified super-linear regex anti-pattern heuristics that may lead to such complexity. In this paper, we empirically study three major aspects of ReDoS that have hitherto been unexplored: the incidence of super-linear regexes in practice, how they can be prevented, and how they can be repaired. In the ecosystems of two of the most popular programming languages — JavaScript and Python – we detected thousands of super-linear regexes affecting over 10,000 modules across diverse application domains. We also found that the conventional wisdom for super-linear regex anti-patterns has few false negatives but many false positives; these anti-patterns appear to be necessary, but not sufficient, signals of super-linear behavior. Finally, we found that when faced with a super-linear regex, developers favor revising it over truncating input or developing a custom parser, regardless of whether they had been shown examples of all three fix strategies. These findings motivate further research into ReDoS, since many modules are vulnerable to it and existing mechanisms to avoid it are insufficient. We believe that ReDoS vulnerabilities are a larger threat in practice than might have been guessed.

Python Powered Intelligence And ML is designed to be your essential companion in your journey through the world of Artificial Intelligence and Python programming. We understand the importance of building a solid foundation in AI concepts, as well as mastering the tools and techniques needed to implement AI solutions effectively. What You’ll Find Inside: Foundation of Artificial Intelligence: In Chapter 1, we lay the groundwork for your AI education, providing a strong understanding of the fundamentals. Knowledge Presentation: Chapter 2 delves into how knowledge is represented in AI systems, a crucial element for creating intelligent machines. Informed / Heuristic Search Strategies: Chapter 3 explores strategies for problem-solving and decision-making, crucial in the AI domain. Natural Language Processing: In Chapter 4, we dive into the world of language understanding and processing, a key area of AI. Soft Computing: Chapter 5 introduces the concept of soft computing, which enables AI systems to work with uncertainty and imprecision. Neural Networks: Chapter 6 covers neural networks, a fundamental technology in modern AI, inspired by the human brain. Fuzzy Systems: Chapter 7 is all about fuzzy logic, which allows AI systems to deal with vagueness and uncertainty. History of Genetic Algorithms: Chapter 8 takes you through the fascinating history and principles of genetic algorithms. Regression: In Chapter 9, we explain regression techniques for predictive modeling, a crucial tool in AI. Python Programming: The second part of Python Powered Intelligence And ML focuses on Python, one of the most versatile and popular programming languages. Numpy: Chapter 11 introduces the powerful library for numerical computing in Python. Pandas: In Chapter 12, we explore Pandas, a tool for data manipulation and analysis. Matplotlib: Chapter 13 introduces you to data visualization using Matplotlib. Regression: We revisit regression in Chapter 14, providing more insights and applications. Our aim is to empower you with the knowledge and skills to excel in Artificial Intelligence and Python programming. Whether you are a beginner or an experienced programmer, this book is your go-to resource for AI and Python. We hope you enjoy this journey with us, and may it inspire you to explore the exciting world of Artificial Intelligence and Python programming. Happy learning!

Water-Drive Gas Reservoirs: Influence of Pulse-Testing on the Indetermination Pulse-Testing on the Indetermination Range of Reserve Estimates For water-drive gas reservoirs, the accuracy of reserve estimates that are based on past history depends upon the practical possibility of applying to the reservoir, for long periods of time, pulsed rates of large amplitude. Introduction Estimating the reserves of a water-drive gas reservoir from its past production history leads to results that are intrinsically affected by some indetermination. The indetermination range, which can be greatly widened by errors in past performance data, becomes especially important when the reservoir has been produced at a fairly constant rate. produced at a fairly constant rate. Following a suggestion made by Muskat, this paper investigates the possibility of narrowing the paper investigates the possibility of narrowing the reserves indetermination range by depleting a gas reservoir, at the very beginning of its life, with a "squarewave" production rate program. The effects of frequency, shape, amplitude and number of cycles of the square-wave production program upon the range of indetermination have been program upon the range of indetermination have been investigated with reference to a hypothetical gas reservoir. Practical limitations were given due consideration. Theoretical Considerations A gas reservoir-aquifer ensemble is a nonlinear system whose canonical equations are known and whose internal structure (gas reserves value, aquifer geometry and spatial distribution of aquifer characteristics, h, k) can be ascertained through direct measurements only within large limits of error. This is especially true for the aquifer, for which the sources of information are usually limited to some peripheral wells that produce water. The reservoir-aquifer system is therefore a "black box". It is known that for a black box the principle of indetermination holds: accordingly, the number of different internal structures that can account for the same external behavior (i.e., the same response to an external stimulus) is infinite. As a consequence, it is impossible to ascertain the internal structure of a reservoir-aquifer system (its initial gas reserves value in particular) from the reservoir past performance. The fact that a unique solution of the problem cannot be found is only of theoretical interest. From a practical point of view, it is important to evaluate the practical point of view, it is important to evaluate the range of indetermination encompassing the initial gas reserves value as determined from the reservoir past performance. It is known that this range can be very performance. It is known that this range can be very large when the past performance data are affected by experimental errors, even of modest size. It must be pointed out that the reservoir-aquifer systems are subjected to a special constraint. Contrary to other physical systems that the information theory deals with, the reservoir-aquifer systems cannot be subjected more than once to a predetermined type of external stimulus, as it is impracticable to restore them to initial conditions; for example, their frequency-response characteristics cannot be determined. A theoretical analysis of the problem shows that a square-wave stimulus is the most suitable one to determine the transfer function (that is, the external behavior) of a system subjected to the kind of constraint noted in the preceding paragraph. JPT P. 1521

The object-oriented theory has been widely used in software design and development and Java programming language is a popular object-oriented programming language for computer science related majors in higher education. With the huge demand for intelligent manufacturing employees in the work market, more and more universities have listed Java programming language in their curriculum for non-CS undergraduates. However, learning object-oriented programming languages, such as Java, is tough even for CS students. This paper presents how to improve non-CS undergraduate students’ learning outcomes of java language by adopting industry standard professional integrated development tool Eclipse and interactive visual educational tool BlueJ. Our Java programming language course is an alternative 8-week course including 16 h lecture and 16 h lab. Forty-seven junior students from mechanical engineering major were divided into two groups, 23 in the control group (CG) and 24 in the experimental group (EG). Students in both groups used Eclipse to write and run programs during the course while BlueJ was adopted to demonstrate the object-oriented concepts during the fifth and sixth weeks only for students in the EG. A pre-test, which was taken by students from both groups, shows no statistical differences in programming knowledge between the two groups. Results from the post-test at the end of the sixth week show students in EG achieved better performance than students in CG. Results of the survey for the EG reveal that students thought BlueJ was helpful for mastering the concepts of object-oriented programming.KeywordsJava programmingLearning outputHigher educationInteractiveVisualBlueJEclipse