While standardization has empowered the software industry to substantially scale software development and to provide affordable software to a broad market, it often does not address smaller market segments, nor the needs and wishes of individual customers. Software product lines reconcile mass production and standardization with mass customization in software engineering. Ideally, based on a set of reusable parts, a software manufacturer can generate a software product based on the requirements of its customer. The concept of features is central to achieving this level of automation, because features bridge the gap between the requirements the customer has and the functionality a product provides. Thus features are a central concept in all phases of product-line development. The authors take a developer’s viewpoint, focus on the development, maintenance, and implementation of product-line variability, and especially concentrate on automated product derivation based on a user’s feature selection. The book consists of three parts. Part I provides a general introduction to feature-oriented software product lines, describing the product-line approach and introducing the product-line development process with its two elements of domain and application engineering. The pivotal part II covers a wide variety of implementation techniques including design patterns, frameworks, components, feature-oriented programming, and aspect-oriented programming, as well as tool-based approaches including preprocessors, build systems, version-control systems, and virtual separation of concerns. Finally, part III is devoted to advanced topics related to feature-oriented product lines like refactoring, feature interaction, and analysis tools specific to product lines. In addition, an appendix lists various helpful tools for software product-line development, along with a description of how they relate to the topics covered in this book. To tie the book together, the authors use two running examples that are well documented in the product-line literature: data management for embedded systems, and variations of graph data structures. They start every chapter by explicitly stating the respective learning goals and finish it with a set of exercises; additional teaching material is also available online. All these features make the book ideally suited for teaching – both for academic classes and for professionals interested in self-study.

This is the first book organized around code clone analysis. To cover the broad studies of code clone analysis, this book selects past research results that are important to the progress of the field and updates them with new results and future directions. The first chapter provides an introduction for readers who are inexperienced in the foundation of code clone analysis, defines clones and related terms, and discusses the classification of clones. The chapters that follow are categorized into three main parts to present 1) major tools for code clone analysis, 2) fundamental topics such as evaluation benchmarks, clone visualization, code clone searches, and code similarities, and 3) applications to actual problems. Each chapter includes a valuable reference list that will help readers to achieve a comprehensive understanding of this diverse field and to catch up with the latest research results. Code clone analysis relies heavily on computer science theories such as pattern matching algorithms, computer language, and software metrics. Consequently, code clone analysis can be applied to a variety of real-world tasks in software development and maintenance such as bug finding and program refactoring. This book will also be useful in designing an effective curriculum that combines theory and application of code clone analysis in university software engineering courses.

Redundancies in program source code - software clones - are a common phenomenon. Although it is often claimed that software clones decrease the maintainability of software systems and need to be managed, research in the last couple of years showed that not all clones can be considered harmful. A sophisticated assessment of the relevance of software clones and a cost-benefit analysis of clone management is needed to gain a better understanding of cloning and whether it is truly a harmful phenomenon. This thesis introduces techniques to model, analyze, and evaluate versatile aspects of software clone evolution within the history of a system. We present a mapping of non-identical clones across multiple versions of a system, that avoids possible ambiguities of previous approaches. Though processing more data to determine the context of each clone to avoid an ambiguous mapping, the approach is shown to be efficient and applicable to large systems for a retrospective analysis of software clone evolution. The approach has been used in several studies to gain insights into the phenomenon of cloning in open-source as well as industrial software systems. Our results show that non-identical clones require more attention regarding clone management compared to identical clones as they are the dominating clone type for the main share of our subject systems. Using the evolution model to investigate costs and benefits of refactorings that remove clones, we conclude that clone removals could not reduce maintenance costs for most systems under study.

The Art and Science of Analyzing Software Data provides valuable information on analysis techniques often used to derive insight from software data. This book shares best practices in the field generated by leading data scientists, collected from their experience training software engineering students and practitioners to master data science. The book covers topics such as the analysis of security data, code reviews, app stores, log files, and user telemetry, among others. It covers a wide variety of techniques such as co-change analysis, text analysis, topic analysis, and concept analysis, as well as advanced topics such as release planning and generation of source code comments. It includes stories from the trenches from expert data scientists illustrating how to apply data analysis in industry and open source, present results to stakeholders, and drive decisions. - Presents best practices, hints, and tips to analyze data and apply tools in data science projects - Presents research methods and case studies that have emerged over the past few years to further understanding of software data - Shares stories from the trenches of successful data science initiatives in industry

Duplicated passages of source code - code clones - are a common property of software systems. While clones are beneficial in some situations, their presence causes various problems for software maintenance. Most of these problems are strongly related to change and include, for example, the need to propagate changes across duplicated code fragments and the risk of inconsistent changes to clones that are meant to evolve identically. Hence, we need a sophisticated analysis of clone evolution to better understand, assess, and manage duplication in practice. This thesis introduces Clone Evolution Graphs as a technique to model clone relations and their evolution within the history of a system. We present our incremental algorithm for efficient and automated extraction of Clone Evolution Graphs from a system's history. The approach is shown to scale even for large systems with long histories making it applicable to retroactive analysis ofclone evolution as well as live tracking of clones during software maintenance.We have used Clone Evolution Graphs in several studies to analyze versatile aspects of clone evolution in open-source as well as industrial systems. Our results show that the characteristics of clone evolution are quite different between systems, highlighting the need for a sophisticated technique like Clone Evolution Graphs to track clones and analyze their evolution on a per-system basis. We have also shown that Clone Evolution Graphs are well-suited to analyze the change behavior of individual clones and can be used to identify problematic clones within a system. In general, the results of our studies provide new insights into how clones evolve, how they are changed, and how they are removed.

Effective detection of code clones is important for software maintenance. Code clones introduce difficulty in software maintenance and lead to bug propagation. Detection of duplicated bugs within a piece of software is challenging, especially when duplications are semantic in nature, where textually two pieces of code are different although they perform the same task. Similar issues can also be observed in malware detection or more precisely, obfuscated code detection. In this dissertation, we first conduct a comprehensive study on state-of-the-art clone detection tools and report an empirical comparative analysis of different methods. Next, we propose a new hybrid clone detection technique. It is a two-step process. First, it uses a coarse grained technique to analyze clones effectively to improve precision. Subsequently, it uses a fine-grained detector to obtain additional information about the clones and to improve detection accuracy of Type-I, Type-II and Type-III clones. The task of clone detection is more challenging when clones are semantically similar in nature, but have no textual resemblance to each other. We present a novel machine learn- ing framework for automated detection of all four types of clones using features extracted from Abstract Syntax Trees (ASTs) and Program Dependency Graphs (PDGs), from pairs of code blocks. Majority of publicly available clone data sets are incomplete in nature and lack la- beled samples of Type-IV. It makes difficult for any machine learning framework using such datasets to be useful. In our third contribution, we propose a new scheme for labeling semantic code clones or Type-IV clones. We introduce a new dataset of clone references, which is a set of correct Type-IV clones. This contribution can help researchers evaluate techniques that detect cloned code of Type-IV. Code obfuscation is a technique to alter the original content of the code to confound reverse engineering. Obfuscated code detection is challenging due to the availability of code obfuscation tools. We observe a resemblance between semantic clones and obfuscated code. We apply our clone detection scheme to detect obfuscated code. We propose a framework that can detect both code clones and obfuscated code as our final contribution. Our results are far superior in comparison to state-of-the-art obfuscated code detection methods.