Introduction
This page is intended for students who are interested in program analysis and transformation research and who would like more information on the topic. This page may be used as a primer to getting involved in this research area.
There are essentially two ways to analyze programs. In each way, the input is the program text, e.g., a Java source code file (.java
). If the analysis studies the program without executing it, that is called static analysis. This is very much related to compiler technology. A compiler, e.g., javac
, is a program that receives a program text as input. It does not run the program; instead, it analyzes it and produces either an executable file or a file containing an intermediate representation., e.g., a bytecode file (.class). You can think of this activity as a translation from a higher-level language into a lower-level one. A great book to read about compiler technology is Compilers: Principles, Techniques, and Tools.
If, on the other hand, the analysis runs the program and studies what the program performs given various inputs (or test cases), that is called dynamic analysis. Dynamic analysis can either study the program’s results or a trace of the program, which may include the sequence of method calls, field accesses, etc. Dynamic analyses are also beneficial in finding information about a program and, often, can be more accurate and faster than static analyses. However, dynamic analysis is dependent on the test suite, i.e., the input to the studied program, whereas static analyses are independent of such input. Thus, there is a trade-off between the two kinds of analyses and, oftentimes, can be used in conjunction.
Since much of my research pertains to program transformation, of which the source code is required as input, I mainly deal with static analysis. Unlike a compiler, however, a general program transformation may not always transform programs between language levels. In fact, such a transformation may transform a program from a high-level language to a program in the same language. In this case, the program’s meaning (or semantics) remains the same, but the text has changed. This is called refactoring.
Refactoring is performed for different reasons, but the main being to improve a program’s structure or migrate the program to use new language features. Refactoring: Improving the Design of Existing is a great book that contains many common refactorings, i.e., ways to improve existing software. Refactoring to Design Patterns talks about refactorings involving existing software by introducing design patterns or “best practices” for software development (a great book related to patterns is Design Patterns: Elements of Reusable Object-Oriented Software.
Note that while the above resources describe program transformation, the steps portrayed are manual. Automated refactorings, one of the main focuses of my research, are programs that perform the refactorings automatically (or sometimes semi-automatically, occasionally needing help from the developer). Many automated refactorings have not only the program text as input but also additional input. For example, a rename refactoring requires the original program, the entity to be renamed, and what the new name should be. Moreover, many automated refactorings are manifested as plug-ins into integrated development environments (IDEs), like Eclipse and NetBeans.
Resources
I have listed some additional resources below. Please also refer to my publications and software projects for explanations and examples. An RSS feed is also available.
Overview
Articles |
Books |
Courses |
Internal |
Mailing lists and forums |
Organizations |
Presentations |
References |
Reports |
Student Resources |
Tools |
Tutorials |
Websites |
Details
- Abstract Syntax Tree How to do program (static) analysis and manipulation in Eclipse.
- Abstract syntax tree – Wikipedia
- Building python from source on Ubuntu How to build an older version of Python from source on Ubuntu.
- Eclipse Corner Article: Abstract Syntax Tree This page is deprecated and may contain some information that is no longer relevant or accurate.
- Eclipse Corner Article: Unleashing the Power of Refactoring
- Eclipse Google Java Style Guide How to use the Google Java style guide with Eclipse IDE.
- Eclipse Refactoring A good discussion of refactoring infrastructure in Eclipse and its architecture.
- Getting the most out of threads in Slack When to use threads vs. when to post in a channel in Slack.
- How to create a Minimal, Reproducible Example An article from Stack Overflow's Help Center on how to create an MWE.
- How to fix Eclipse IDE flickering issue on Debian
- How to use Java Eclipse Abstract Syntax Tree in a project outside of Eclipse? An article on how to traverse and rewrite an AST using the Eclipse JDT libraries outside of Eclipse and without using the visitor pattern. Can be still helpful within an Eclipse plugin and with using the visitor pattern (when at a particular node).
- Stop Gradle builds after one failing test Halt a gradle build after the first failing test.
- The AST type in Eclipse
- The Language Toolkit: An API for Automated Refactorings in Eclipse-based IDEs
- Understanding Maven SNAPSHOT dependencies A SNAPSHOT version is a version of the project/dependency that has not been released.
- Using Maven Artifacts in PDE/RCP and Tycho Builds
- Visitor pattern – Wikipedia In object-oriented programming and software engineering, the visitor design pattern is a way of separating an algorithm from an object structure on which it operates.
- What Makes Good Research in Software Engineering? A nice description of software engineering research, including processes and methodologies.
- Apache Maven Cookbook Recipes to successfully build and automate development life cycle tasks following Apache Maven conventions and best practices.
- Compilers: Principles, Techniques, & Tools A computer science textbook by Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman about compiler construction for programming languages.
- Core Java SE 9 for the Impatient A book by my favorite Java author.
- Design Patterns: Elements of Reusable Object-Oriented Software Capturing a wealth of experience about the design of object-oriented software, four top-notch designers present a catalog of simple and succinct solutions to commonly occurring design problems.
- Eclipse 4 Plug-in Development
- Effective Java by Joshua Bloch The Definitive Guide to Java Platform Best Practices–Updated for Java 7, 8, and 9.
- Logic in Computer Science An excellent book on an introduction to verification and validation of programs.
- Mastering Eclipse Plug-in Development
- Object-Oriented Software Construction The definitive reference on the most important development in software technology for the last 20 years: object-orientation.
- Refactoring to Patterns Introduces the theory and practice of pattern-directed refactorings: sequences of low-level refactorings that allow designers to safely move designs to, towards, or away from pattern implementations.
- Refactoring: Improving the Design of Existing Code The de facto book on software refactoring.
- The Java Developer’s Guide to Eclipse
- The Professor Is In The definitive career guide for grad students, adjuncts, post-docs and anyone else eager to get tenure or turn their Ph.D. into their ideal job.
- Writing Logically, Thinking Critically A small textbook on critique thinking and logical arguments.
- COMP 621 Program Analysis and Transformations (Fall 2016) This course provides an introduction to the tools and techniques are used in optimizing compilers.
- CSE 5239: Compile-Time Program Analysis and Transformations, Autumn 2014
- Designing Code Analyses for Large-scale Software Systems Static analysis course by Eric Bodden at Paderborn University
- DIT635: Software Quality and Testing (Spring 2020) Good introduction to dataflow analysis.
- Static Analysis: Introduction part 1 A course focused on software security but includes coverage of general static analysis techniques.
- Code of Conduct and Confidentiality Agreement Guidelines for our lab's researchers.
- CUNY Graduate Center Writing Center Assists current and past graduate students in the cultivating writing skills and habits through individual consultations, workshops, and other programming,
- CUNY PONDER Asana Workspace Asana workspace for our research group
- CUNY PONDER Calendar Internal group calendar.
- CUNY PONDER GitHub Organization GitHub organization for our research group.
- CUNY PONDER Google Group Our internal Google group for email and sharing Google services.
- CUNY PONDER Research Virtual Machine Virtual machine of the CUNY PONDER group used for experimentation.
- CUNY PONDER Slack Instant messaging workspace for our research group.
- CUNY PONDER upcoming papers plan Upcoming planned research papers for our group.
- CUNY PONDER Wiki Organizational wiki pages.
- Early research initiative Discover opportunities for funded research in archives, museums, field sites, special collections, laboratories, and research centers.
- Formatting files CUNY PONDER common source code formatting files.
- Funding for current doctoral students Financial awards, scholarships, grants, and employment opportunities for current doctoral students at the CUNY Graduate Center.
- Git branching policy Internal policy on using Git branches.
- Guidelines for preparing master’s theses in Arts & Sciences Hunter's A&S' guide for MA students preparing a thesis.
- Link you Google Account with another email address Instructions on how to link your Google account with your (school) email address.
- Software Evolution and Machine Learning Google Group Project-specific Google group.
- Stack Overflow for CUNY PONDER Stack overflow for our research group
- Boa language and infrastructure user forum Mailing list and forum for the Boa software repository mining tool and infrastructure.
- Computer Science Research and Practice A slack workspace for folks interested in Computer Science research and development.
- CUNY PONDER Google Group Our internal Google group for email and sharing Google services.
- CUNY PONDER Slack Instant messaging workspace for our research group.
- Eclipse Java Development Tools (JDT) Forum Questions and technical discussions about how to use the Java development tools (JDT)
- Eclipse jdt-core-dev mailing list Eclipse JDT Core developers list.
- Eclipse jdt-dev mailing list Eclipse JDT general developers list.
- Eclipse jdt-ui-dev mailing list Eclipse JDT UI developers list.
- ECOOP Info Mailing List A european-based software engineering mailing list
- jython-dev mailing list Discussion of Jython's development.
- jython-users mailing list Discussion of and questions about using Jython.
- NEPLS Mailing list for the New England Programming Languages and Systems Symposium Series.
- NJPLS New Jersey Programming Language and Systems Seminar.
- PLTea An initiative to bring together the members of the programming languages community for informal virtual discussions.
- pydev-code Mailing list for PyDev Eclipse developers.
- SEWORLD Mailing List A mailing list provided as a service to the Software Engineering community.
- Software Evolution and Machine Learning Google Group Project-specific Google group.
- Stack Overflow page for the Eclipse JDT Question and answers about the Eclipse JDT.
- tech-debt mailing list Technical Debt research mailing list
- WALA mailing list Mailing list for the T.J. Watson Libraries for Analysis (WALA).
- New England Programming Languages and Systems Symposium Series (NEPLS) A regional venue to discuss current research in programming languages and systems.
- New Jersey Programming Languages and Systems Seminar Series (NJPLS) An informal forum that promotes interaction among programming languages and systems researchers in the New Jersey area.
- A Brief Introduction to Static Analysis A nice overview of static analysis with real-world application.
- Introduction to Static Analysis Another nice introduction to static analysis.
- Software Engineering at Google with Titus Winters The secret to software engineering at Google.
- Static Analysis for Java in Eclipse Possibly outdated (but conceptually relevant) slides on program (static) analysis in Eclipse.
- Unleash the Power of Refactorings in your Products Talk at EclipseCon 2007
- Cohen’s kappa – Wikipedia A statistic used to measure inter-rater reliability for qualitative (categorical) items.
- CUNY PONDER upcoming papers plan Upcoming planned research papers for our group.
- Installing Linux on Windows 10 Instructions for installing a linux subsystem on a windows platform.
- Introduction to Machine Learning for Java Developers Video from the NYJavaSIG on Nov 7, 2019
- Machine Learning Glossary General machine learning terms, plus terms specific to TensorFlow.
- Migrate to TensorFlow 2 Learn how to migrate your TensorFlow code from TensorFlow 1.x to TensorFlow 2.
- Proof templates
- Proposing a new meeting time in Outlook If you receive a meeting request for a time you're not available, you can decline the request or propose a new meeting time. If you are the meeting coordinator, you may receive new time proposals from the people you invited.
- Stack Overflow for CUNY PONDER Stack overflow for our research group
- Working in the Linux Command-Line A Short List of the Best and Greatest Commands
- Debug a Maven build in Eclipse Attach Eclipse's debugger remotely to debug a Maven build.
- Eclipse Java development tools (JDT) Overview of the Eclipse Java Development Tools (JDT) components.
- Eclipse JDT Plug-in Developer Guide A really excellent guide on how to create JDT plug-ins in Eclipse.
- Eclipse Platform Java development user guide Using Eclipse to develop Java programs.
- Eclipse source code for the eclipse.jdt.core component GitHub repository for the eclipse.jdt.core component of the Eclipse IDE.
- Eclipse source code for the eclipse.jdt.ui component GitHub repository for the eclipse.jdt.ui component of the Eclipse IDE
- Frequently used internal APIs in JDT UI A list of helper classes frequently used for internal development of the JDT UI.
- JDT Plug-in Developer Guide A guide to developing Eclipse Java Development Tools (JDT) plug-ins
- On the Job: The Eclipse 3.0 Jobs API A guide to concurrency and progress monitoring in Eclipse.
- PyDev Developers’ Guide How to develop atop of the PyDev plug-in for Eclipse
- The Official Eclipse FAQs Frequently asked questions about Eclipse development.
- An Illustrated Guide to Git on Windows
- Git branching policy Internal policy on using Git branches.
- Git documentation Documentation and reference on the popular Git versioning system.
- Git for Computer Scientists A quick introduction to Git internals.
- Git Magic Rough instructions for particular Git effects.
- Git ready Learn git one commit at a time.
- Resolving a merge conflict using the command line How to resolve merge conflicts using the command line and a text editor.
- Setting your commit email address in Git and GitHub Instructions for how to change your email to your professional email address.
- try.github.com Git quickstart.
- Creating a pull request Create a pull request to propose and collaborate on changes to a repository. These changes are proposed in a branch, which ensures that the default branch only contains finished and approved work.
- Documenting your project with wikis You can use a wiki to share detailed, long-form information about your project.
- Getting permanent links to files on GitHub When viewing a file on GitHub, you can press the "y" key to update the URL to a permalink to the exact version of the file you see.
- Reviewing proposed changes in a pull request In a pull request, you can review and discuss commits, changed files, and the differences (or "diff") between the files in the base and compare branches.
- Syncing a fork Sync a fork of a repository to keep it up-to-date with the upstream repository on GitHub.
- Maven Publish Plugin Publish Gradle build artifacts to an Apache Maven repository.
- A (Not So) Short Introduction to LATEX2ε If you want to learn how to write your documents with LaTeX, this introduction is for you.
- Biblatex Cheat Sheet A BibLATEX/Biber ‘cheat sheet’
- Creating an annotated bibliography How to create an annotated bibliography in LaTeX.
- LaTeX General Help A web based LaTeX help, based on George D. Greenwade's LaTeX help for VMS machines.
- LaTeX Introduction A thorough introduction to LaTeX by Robert Dyer.
- Notes for using minted.sty on arXiv arXiv cannot process using the --shell-escape option, as this is disabled in arXiv's system for security reasons.
- Overleaf introduction to LaTeX A free online introduction to LaTeX from the online editor, Overleaf.
- Debug a Maven build in Eclipse Attach Eclipse's debugger remotely to debug a Maven build.
- How to give a great research talk Simple guidelines from Simon Peyton Jones of Microsoft Research for giving research talks.
- WALA API Documentation API documentation for the T.J. Watson Library for Analysis (WALA).
- WALA Pointer Analysis Guide WALA provides a framework for flow-insensitive Andersen-style pointer analysis. You can customize the pointer analysis context-sensitivity policy in various ways.
- WALA Starter Kit A basic Gradle configuration to get started with WALA.
- Annotated bibliography samples Sample annotations from annotated bibliographies, each with a different research project.
- Empirical Studies: Qualitative vs. Quantitative An explanation of the differences between qualitative and quantitative empirical research
- Guide to writing a scientific research report (IMRaD) A guide to writing IMRaD papers from GMU.
- How to write a great research paper This talk from Microsoft Research offers seven simple, concrete suggestions for how to improve your research papers.
- How to write a great research proposal This page is an attempt to collect together a number of suggestions about what makes a good proposal.
- How to write a paper critique
- How to write a technical paper This document describes several simple, concrete ways to improve your writing, by avoiding some common mistakes.
- Mastering Markdown Markdown guide from GitHub.
- Patterns for writing good rebuttals
- The Devil's Guide to Incremental Research
- What is Empirical Research? An introduction to empirical research from USU Libraries
- Writing Logically, Thinking Critically A small textbook on critique thinking and logical arguments.
- Report about creating refactorings in Eclipse Interesting report that discusses the architecture of Eclipse refactoring plugins.
- 3 “I”s of PhD Research, and the Oft-Forgotten 3rd “I” Talk by Abhik Roychoudhury on succeeding in a PhD in research.
- CUNY Graduate Center Writing Center Assists current and past graduate students in the cultivating writing skills and habits through individual consultations, workshops, and other programming,
- Digital Networking and Community This talk gives hints on the benefits and risks of science on social media.
- Dissertation Proposal Tips Tips for writing a dissertation and defending a thesis.
- Early research initiative Discover opportunities for funded research in archives, museums, field sites, special collections, laboratories, and research centers.
- Financial Resources Student financial resources, including emergency grants and financial literacy.
- Funding for current doctoral students Financial awards, scholarships, grants, and employment opportunities for current doctoral students at the CUNY Graduate Center.
- GitHub Education Student Developer Pack The best developer tools, free for students.
- Guidelines for preparing master’s theses in Arts & Sciences Hunter's A&S' guide for MA students preparing a thesis.
- How to review Computer Science papers A guide for reviewing CS papers, published in XRDS: Crossroads, The ACM Magazine for Students.
- How to Write Up a Ph.D. Dissertation Turn your research (once it's done) into a readable multi-chapter document.
- LaTeX Homework Templates Some templates for writing your homework in latex.
- Library remote access Accessing library material from home. You may need these instructions to access protected papers.
- Planning a Research Project Talk by Abhik Roychoudhury on research strategy for graduate students.
- The Teach@CUNY Handbook Practical guidance to both beginning and experienced instructors across CUNY and beyond. Section II: "Strategies” may be particularly useful for graduate students teaching their first course.
- How to Review a Technical Paper This article assists the reader in the efficient preparation of constructive reviews.
- How to review Computer Science papers A guide for reviewing CS papers, published in XRDS: Crossroads, The ACM Magazine for Students.
- Analysis Tools A curated list of static and dynamic analysis tools for all programming languages, config files, build tools, and more.
- CUNY PONDER Research Virtual Machine Virtual machine of the CUNY PONDER group used for experimentation.
- Display #Anchors Displays anchors for all content in the current web page without breaking the layout.
- gitcproc Tool for analyzing git log messages and diffs.
- google-java-format Reformats Java source code to comply with Google Java Style.
- Link you Google Account with another email address Instructions on how to link your Google account with your (school) email address.
- Online Kappa Calculator Can be used to calculate kappa--a chance-adjusted measure of agreement--for any number of cases, categories, or raters
- Using zoom.us on Linux Set the QT_SCALE_FACTOR environmental variable to 2.
- WALA T.J. Watson Libraries for Analysis.
- Asana Chrome extension: quick-add tasks • Asana Quickly add tasks to Asana from any web page in Chrome.
- CUNY PONDER Asana Workspace Asana workspace for our research group
- Google Drive Asana: Attach files to project tasks • Asana The Google Drive file chooser is built into the Asana task pane so you can easily attach My Drive files directly to tasks.
- Eclipse Eclipse is a powerful and extendable Integrated Development Environment (IDE) for many languages, including Java.
- Eclipse Buildship Eclipse Plug-ins for Gradle.
- Eclipse Environment Variables Plug-in Allows you to add environment variables to eclipse.
- Eclipse for RCP and RAP Developers Eclipse download for plug-in developers.
- Eclipse Installer An installer for Eclipse. Be sure to pick the RCP and RAP developer package.
- EGit Git and GitHub plugin for Eclipse
- Formatting files CUNY PONDER common source code formatting files.
- M2Eclipse Tight integration for Apache Maven into the Eclipse IDE.
- org.eclipse.jdt.astview – AST View A view to visualize the AST (abstract syntax tree) of a Java file open in the editor.
- org.eclipse.jdt.jeview – JavaElement View A view to visualize the Java elements in Eclipse's Java model.
- PyDev A Python IDE for Eclipse, which may be used in Python, Jython and IronPython development.
- Using Git in Eclipse Use a personal access token instead of your normal password.
- EGit Git and GitHub plugin for Eclipse
- Git Large File Storage (LFS) Replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub.com or GitHub Enterprise.
- GitKraken GitKraken is a Git GUI client for Windows, Mac, and Linux.
- Gitless An experimental version control system built on top of Git.
- MeGit Standalone Git GUI based on EGit
- Sourcetree Sourcetree simplifies how you interact with your Git repositories so you can focus on coding.
- Using Git in Eclipse Use a personal access token instead of your normal password.
- GitHub CLI Command-line interface to GitHub.
- miner-utils Scripts to help call the GitHub API.
- Eclipse Buildship Eclipse Plug-ins for Gradle.
- Stop Gradle builds after one failing test Halt a gradle build after the first failing test.
- arXiv LaTeX Cleaner Easily clean the LaTeX code of your paper to submit to arXiv
- JabRef Free reference manager that works with Bibtex and Latex.
- JabRef Browser Extensions This official browser extension automatically identifies and extracts bibliographic information on websites and sends them to JabRef with one click.
- LaTeX Homework Templates Some templates for writing your homework in latex.
- LaTeX proof package Write derivations in LaTeX.
- syntastic: Syntax checking hacks for Vim Catch LaTeX errors in Vim
- TablesGenerator.com Create LaTeX tables online.
- TeX “comment” package A LaTeX package to exclude large portions of text, possibly based on "versions."
- Google Drive for Slack If you use Google Drive, you can install the Google Drive app to share and access files right from your workspace.
- LaTeX for Slack A simple /slash-command integration for Slack that enables you to share LaTeX snippets in Slack channels with others.
- Slack + Asana: Automated updates to tasks Get instant notifications in Slack when changes are made to tasks in Asana.
- Slack downloads With the Slack app, your team is never more than a click away.
- Grammarly Free Online Writing Assistant
- A Simple Makefile Tutorial A starter's guide to quickly and easily create your own makefiles for small to medium-sized projects.
- Concurrency in Java Introduces Java's basic concurrency support and summarizes some of the high-level APIs in the java.util.concurrent packages.
- Generating source code JARs for Maven projects How to use the “maven-source” plugin to package your source code and deploy along with your project.
- Passwordless SSH using public-private key pairs How to generate private/public key pairs to use ssh (and scp) without a password.
- PLAST-Lab YouTube Channel Various tutorials from the Programming Languages and Advanced Software Technologies at the University of Athens
- UNIX/Linux Tutorial for Beginners A beginners guide to the Unix and Linux operating system. Eight simple tutorials which cover the basics of UNIX/Linux commands.
- Creating Your Own Plugin in Eclipse How to create a new plug-in to Eclipse.
- Eclipse Java Development Tools (JDT) Fundamentals JDT tutorial from EclipseCon 2010.
- Eclipse JDT – Abstract Syntax Tree (AST) and the Java Model Usage of the Abstract Syntax Tree (AST) and the Java model in Eclipse.
- Extending the Eclipse IDE A plug-in development tutorial.
- Using Java libraries (from Maven Central) for OSGi, Eclipse Plug-in and Eclipse RCP development How to use Java libraries for Eclipse plug-in and OSGi development.
- Using Maven within the Eclipse IDE Describes the usage of Maven within the Eclipse IDE for building Java applications.
- Git Tutorial Git tutorial from W3schools. Has a quiz.
- gittutorial Explains how to import a new project into Git, make changes to it, and share changes with other developers.
- Understanding Git Conceptually A tutorial on the Git version control system.
- Getting Started with WALA This page steps through the process to download, build, and run a core subset of WALA for standard Java analysis. Later we will describe other available components, in addition to this core subset.
- Static and Dynamic Program Analysis Using WALA PLDI 2010 Tutorial on WALA
- CUNY PONDER GitHub Organization GitHub organization for our research group.
- CUNY PONDER Wiki Organizational wiki pages.
- JDT UI Component Development Resources