Home »
Archives
Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.
Software Evolution for Machine Learning
Refactorings and Technical Debt in Machine Learning Systems
Introduction
Machine Learning (ML), including Deep Learning (DL), systems, i.e., those with ML capabilities, are pervasive in today’s data-driven society. Such systems are complex; they are comprised of ML models and many subsystems that support learning processes. As with other complex systems, ML systems are prone to classic technical debt issues, especially when such systems are long-lived, but they also exhibit debt specific to these systems. Unfortunately, there is a gap in knowledge about how ML systems actually evolve and are maintained. In this project, we fill this gap by studying refactorings, i.e., source-to-source semantics-preserving program transformations, performed in real-world, open-source software, and the technical debt issues they alleviate. We analyzed 26 projects, consisting of 4.2 MLOC, along with 327 manually examined code patches. The results indicate that developers refactor these systems for a variety of reasons, both specific and tangential to ML, some refactorings correspond to established technical debt categories, while others do not, and code duplication is a major cross-cutting theme that particularly involved ML configuration and model code, which was also the most refactored. We also introduce 14 and 7 new ML-specific refactorings and technical debt categories, respectively, and propose several recommendations, best practices, and anti-patterns. The results can potentially assist practitioners, tool developers, and educators in facilitating long-term ML system usefulness.
Researchers
Publications
Below is a list of publications. My and my research students‘ names are boldfaced, undergraduate students are italicized, and female students are underlined:
Yiming Tang, Raffi Khatchadourian, Mehdi Bagherzadeh, Rhia Singh, Ajani Stewart, and Anita Raja. An empirical study of refactorings and technical debt in Machine Learning systems. In International Conference on Software Engineering, ICSE ’21, pages 238–250. IEEE/ACM, IEEE, May 2021. (138/615; 22% acceptance rate). [ bib | DOI | data | slides | http ]
Presentations
Safe Automated Refactoring for Intelligent Parallelization of Java 8 Streams
Introduction
The Java 8 Stream API sets forth a promising new programming model that incorporates functional-like, MapReduce-style features into a mainstream programming language. However, using streams efficiently may involve subtle considerations. For example, although streams enable developers to run their code in parallel with little alteration, it is often not obvious if such code runs more efficiently this way. In fact, under certain conditions, running stream code in parallel can be less efficient than running it sequentially. Moreover, it can be unclear if running sequential stream code in parallel is safe and interference-free due to possible lambda expression side-effects.
This project involves an automated refactoring approach that assists developers in writing optimal stream client code in a semantics-preserving fashion. The approach, based on a novel data ordering and typestate analysis, consists of refactorings that include preconditions and transformations for automatically determining when it is safe and possibly advantageous to convert a sequential stream to parallel and improve upon already parallel streams. The approach is implemented as a plug-in to the popular Eclipse IDE utilizing both WALA and SAFE.
Researchers
Publications
My and my research students‘ names appear in boldface. Undergraduate students appear in italics.
Raffi Khatchadourian, Yiming Tang, and Mehdi Bagherzadeh. Safe automated refactoring for intelligent parallelization of Java 8 streams. Science of Computer Programming, 195:102476, 2020. [ bib | DOI | http ]
Raffi Khatchadourian, Yiming Tang, Mehdi Bagherzadeh, and Syed Ahmed. Safe automated refactoring for intelligent parallelization of Java 8 streams. Technical Report 544, City University of New York (CUNY) Hunter College, 695 Park Ave, New York, NY 10065 United States, July 2019. [ bib | http ]
Raffi Khatchadourian, Yiming Tang, Mehdi Bagherzadeh, and Syed Ahmed. Safe automated refactoring for intelligent parallelization of Java 8 streams. In International Conference on Software Engineering, ICSE ’19, pages 619–630, Piscataway, NJ, USA, May 2019. ACM/IEEE, IEEE Press. (109/529; 20.6% acceptance rate). [ bib | DOI | tool | slides | http ]
Raffi Khatchadourian, Yiming Tang, Mehdi Bagherzadeh, and Syed Ahmed. A tool for optimizing Java 8 stream software via automated refactoring. In International Working Conference on Source Code Analysis and Manipulation, SCAM ’18. IEEE, September 2018. (9/17; 53% acceptance rate). Best paper award. [ bib | tool | slides | http ]
Yiming Tang, Raffi Khatchadourian, Mehdi Bagherzadeh, and Syed Ahmed. Towards safe refactoring for intelligent parallelization of Java 8 streams. In International Conference on Software Engineering: Companion Proceeedings, ICSE ’18, pages 206–207, New York, NY, USA, May 2018. ACM/IEEE, ACM. [ bib | DOI | poster | http ]
Research Prototype
Our research prototype may be found on GitHub.
Supporting Repositories
- edu.cuny.hunter.streamrefactoring.annotations: Annotation library to manually specify program entry points.
Study Data
Subject Descriptions
| Subject | Version | Description |
|---|---|---|
| htm.java | 0.6.13 | Hierarchical Temporal Memory implementation in Java – an official Community-Driven Java port of the Numenta Platform for Intelligent Computing (NuPIC). |
| JacpFX | 2.1 | An API to create Rich Clients in MVC style with JavaFX, Spring (or other DI frameworks) and an Actor like component approach. |
| java-design-patterns | 1.12.0 (bc94d0f) | Design patterns implemented in Java. |
| jdk8-experiments | 3c7d717 | An personal repository for experimentation with the new JDK 8 features. |
| jetty.project | 9.4.8 | Eclipse Jetty – Web Container & Clients – supports HTTP/2, HTTP/1.1, HTTP/1.0, websocket, servlets, and more. |
| jOOQ | 3.10.4 | jOOQ is the best way to write SQL in Java. |
| koral | Scientific publishing with HTML. | |
| monads | 0.5 | Basic monads for Java. |
| retrolambda | 2.5.3 | Backport of Java 8’s lambda expressions to Java 7, 6 and 5. |
| streamql | 0.5.8 | A query tool for java.util.stream.Stream. |
| ThreeTen Extra | 1.3.2 | Provide additional date-time classes that complement those in JDK 8. |
Subject Source Code
The set of open-source Java projects packaged as Eclipse projects used for assessing our refactoring can be downloaded as an archive. Please refer to the included README.md file for building instructions and the LICENSE.md file for licensing information.
Raw Results
The data set consisting of the raw results produced during our study can be found on Zenodo.
Scripts used to produce the results can be found in our prototype implementation repository. htm_java.sh and jmh_profile.sh are the scripts used to execute the performance evaluation of htm.java and java-design-patterns, respectively. They assume that separate git branches are used to store the original source code and the refactored source code; these can be created using the above-mentioned patch files and git. mvn_check.sh, which also assumes the same git structure, is used to verify that the same test results exist before and after the refactoring. It can be easily adapted for projects that use Gradle.
Presentations
Automatic Migration of Legacy Java Method Implementations to Interfaces
Java 8 is one of the largest upgrades to the popular language and framework in over a decade. It offers several new, key features that can help make programs easier to read, write, and maintain, especially in regards to collections. These features include Lambda Expressions, the Stream API, and enhanced interfaces, many of which help bridge the gap between functional and imperative programming paradigms and allow for succinct concurrency implementations. We investigate several open issues related to automatically migrating (refactoring) legacy Java software to use enhanced interfaces correctly, efficiently, and as completely as possible. Our goal is to help developers to maximally understand and adopt this new feature thus improving their software.
Automated Refactoring of Legacy Java Software to Enumerated Types
Introduction
Modern Java languages introduce several new features that offer significant improvements over older Java technology. In this project, we consider the new enum construct, which provides language support for enumerated types. Prior to recent Java languages, programmers needed to employ various patterns (e.g., the weak enum pattern) to compensate for the absence of enumerated types in Java. Unfortunately, these compensation patterns lack several highly-desirable properties of the enum construct, most notably, type safety. We present a novel fully-automated approach for transforming legacy Java code to use the new enumeration construct. This semantics-preserving approach increases type safety, produces code that is easier to comprehend, removes unnecessary complexity, and eliminates brittleness problems due to separate compilation. At the core of the proposed approach is an interprocedural type inference algorithm that tracks the flow of enumerated values. The algorithm was implemented as an open-source, publicly available Eclipse plug-in and evaluated experimentally on 17 large Java benchmarks. Our results indicate that analysis cost is practical and the algorithm can successfully refactor a substantial number of fields to enumerated types. This work is a significant step towards providing automated tool support for migrating legacy Java software to modern Java technologies.
Researchers
Publications
My and my research students‘ names appear in boldface. Undergraduate students appear in italics.
Raffi Khatchadourian. Automated refactoring of legacy Java software to enumerated types. Automated Software Engineering, 24(4):757–787, December 2017. [ bib | DOI | http ]
Raffi Khatchadourian and Benjamin Muskalla. Enumeration refactoring: A tool for automatically converting Java constants to enumerated types. In International Conference on Automated Software Engineering, ASE ’10, pages 181–182, New York, NY, USA, September 2010. IEEE/ACM. (18/45; 40% acceptance rate). [ bib | DOI | tool | slides | http ]
Raffi Khatchadourian, Jason Sawin, and Atanas Rountev. Automated refactoring of legacy Java software to enumerated types. In International Conference on Software Maintenance, ICSM 2007, pages 224–233. IEEE, October 2007. (46/214; 21% acceptance rate). [ bib | DOI | slides | http ]
Raffi Khatchadourian, Jason Sawin, and Atanas Rountev. Automated refactoring of legacy Java software to enumerated types. Technical Report OSU-CISRC-4/07-TR26, Ohio State University, April 2007. [ bib | .pdf ]
Research Prototype
Our research prototype may be found on GitHub.
Presentations
Porting the NetBeans Lambda Expression Refactoring to Eclipse
Introduction
Java 8 is one of the largest upgrades to the popular language and framework in over a decade. There are several new key features of Java 8 that can help make programs easier to read, write, and maintain. Java 8 comes with many features, especially related to collection libraries. These include such new features as Lambda Expressions, the Stream API, enhanced interfaces, and more.
While JDT Core/UI has incorporated many Java 8 quick fixes and refactorings, there are still many features left to be done. For example, NetBeans has a refactoring that converts loops to Lambda expressions. This project is for exploring the porting of the mechanism in NetBeans to Eclipse. These may be manifested as refactoring and/or “quick fixes.”
This project is open source. Please see the implementation page for download links.
People
Md. Arefin
Raffi Khatchadourian
Publications
CUNY researchers appear in bold. Undergraduate students appear in italics.
Md. Arefin and Raffi Khatchadourian. Porting the netbeans Java 8 enhanced for loop lambda expression refactoring to Eclipse. In Companion Proceedings of the 2015 ACM SIGPLAN International Conference on Systems, Programming, Languages and Applications: Software for Humanity, SPLASH Companion 2015, pages 58–59, New York, NY, USA, October 2015. ACM. [ bib | DOI | poster | http ]
Research Prototype
Our research prototype may be found on GitHub.
