Introduction
This page is intended for students who are interested in program analysis and transformation research and who would like more information on the topic. This page may be used as a primer to getting involved in this research area.
There are essentially two ways to analyze programs. In each way, the input is the program text, e.g., a Java source code file (.java
). If the analysis studies the program without executing it, that is called static analysis. This is very much related to compiler technology. A compiler, e.g., javac
, is a program that receives a program text as input. It does not run the program; instead, it analyzes it and produces either an executable file or a file containing an intermediate representation., e.g., a bytecode file (.class). You can think of this activity as a translation from a higher-level language into a lower-level one. A great book to read about compiler technology is Compilers: Principles, Techniques, and Tools.
If, on the other hand, the analysis runs the program and studies what the program performs given various inputs (or test cases), that is called dynamic analysis. Dynamic analysis can either study the program’s results or a trace of the program, which may include the sequence of method calls, field accesses, etc. Dynamic analyses are also beneficial in finding information about a program and, often, can be more accurate and faster than static analyses. However, dynamic analysis is dependent on the test suite, i.e., the input to the studied program, whereas static analyses are independent of such input. Thus, there is a trade-off between the two kinds of analyses and, oftentimes, can be used in conjunction.
Since much of my research pertains to program transformation, of which the source code is required as input, I mainly deal with static analysis. Unlike a compiler, however, a general program transformation may not always transform programs between language levels. In fact, such a transformation may transform a program from a high-level language to a program in the same language. In this case, the program’s meaning (or semantics) remains the same, but the text has changed. This is called refactoring.
Refactoring is performed for different reasons, but the main being to improve a program’s structure or migrate the program to use new language features. Refactoring: Improving the Design of Existing is a great book that contains many common refactorings, i.e., ways to improve existing software. Refactoring to Design Patterns talks about refactorings involving existing software by introducing design patterns or “best practices” for software development (a great book related to patterns is Design Patterns: Elements of Reusable Object-Oriented Software.
Note that while the above resources describe program transformation, the steps portrayed are manual. Automated refactorings, one of the main focuses of my research, are programs that perform the refactorings automatically (or sometimes semi-automatically, occasionally needing help from the developer). Many automated refactorings have not only the program text as input but also additional input. For example, a rename refactoring requires the original program, the entity to be renamed, and what the new name should be. Moreover, many automated refactorings are manifested as plug-ins into integrated development environments (IDEs), like Eclipse and NetBeans.
Resources
I have listed some additional resources below. Please also refer to my publications and software projects for explanations and examples. An RSS feed is also available.
Overview
Articles |
Books |
Courses |
Internal |
Mailing lists and forums |
Organizations |
Presentations |
References |
Reports |
Student Resources |
Tools |
Tutorials |
Websites |
Details
- Abstract Syntax Tree How to do program (static) analysis and manipulation in Eclipse.
- Abstract syntax tree – Wikipedia
- Eclipse Corner Article: Abstract Syntax Tree This page is deprecated and may contain some information that is no longer relevant or accurate.
- Eclipse Corner Article: Unleashing the Power of Refactoring
- Eclipse Refactoring A good discussion of refactoring infrastructure in Eclipse and its architecture.
- Getting the most out of threads in Slack When to use threads vs. when to post in a channel in Slack.
- How to create a Minimal, Reproducible Example An article from Stack Overflow's Help Center on how to create an MWE.
- How to use Java Eclipse Abstract Syntax Tree in a project outside of Eclipse? An article on how to traverse and rewrite an AST using the Eclipse JDT libraries outside of Eclipse and without using the visitor pattern. Can be still helpful within an Eclipse plugin and with using the visitor pattern (when at a particular node).
- The AST type in Eclipse
- The Language Toolkit: An API for Automated Refactorings in Eclipse-based IDEs
- Visitor pattern – Wikipedia In object-oriented programming and software engineering, the visitor design pattern is a way of separating an algorithm from an object structure on which it operates.
- What Makes Good Research in Software Engineering? A nice description of software engineering research, including processes and methodologies.
- Apache Maven Cookbook Over 90 hands-on recipes to successfully Build and automate development life cycle tasks following Maven conventions and best practices.
- Compilers: Principles, Techniques, & Tools A computer science textbook by Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman about compiler construction for programming languages.
- Core Java SE 9 for the Impatient A book by my favorite Java author.
- Design Patterns: Elements of Reusable Object-Oriented Software Capturing a wealth of experience about the design of object-oriented software, four top-notch designers present a catalog of simple and succinct solutions to commonly occurring design problems.
- Eclipse 4 Plug-in Development
- Effective Java by Joshua Bloch The Definitive Guide to Java Platform Best Practices–Updated for Java 7, 8, and 9.
- Logic in Computer Science An excellent book on an introduction to verification and validation of programs.
- Mastering Eclipse Plug-in Development
- Refactoring to Patterns Introduces the theory and practice of pattern-directed refactorings: sequences of low-level refactorings that allow designers to safely move designs to, towards, or away from pattern implementations.
- Refactoring: Improving the Design of Existing Code The de facto book on software refactoring.
- The Java Developer’s Guide to Eclipse
- The Professor Is In The definitive career guide for grad students, adjuncts, post-docs and anyone else eager to get tenure or turn their Ph.D. into their ideal job.
- Writing Logically, Thinking Critically A small textbook on critique thinking and logical arguments.
- COMP 621 Program Analysis and Transformations (Fall 2016) This course provides an introduction to the tools and techniques are used in optimizing compilers.
- CSE 5239: Compile-Time Program Analysis and Transformations, Autumn 2014
- Designing Code Analyses for Large-scale Software Systems Static analysis course by Eric Bodden at Paderborn University
- DIT635: Software Quality and Testing (Spring 2020) Good introduction to dataflow analysis.
- Static Analysis: Introduction part 1 A course focused on software security but includes coverage of general static analysis techniques.
- Code of Conduct and Confidentiality Agreement Guidelines for our lab's researchers.
- CUNY PONDER Asana Workspace Asana workspace for our research group
- CUNY PONDER Calendar Internal group calendar.
- CUNY PONDER GitHub Organization GitHub organization for our research group.
- CUNY PONDER Google Group Our internal Google group for email and sharing Google services.
- CUNY PONDER Research Virtual Machine Virtual machine of the CUNY PONDER group used for experimentation.
- CUNY PONDER Slack Instant messaging workspace for our research group.
- CUNY PONDER upcoming papers plan Upcoming planned research papers for our group
- CUNY PONDER Wiki Organizational wiki pages.
- Early research initiative Discover opportunities for funded research in archives, museums, field sites, special collections, laboratories, and research centers.
- Formatting files CUNY PONDER common source code formatting files.
- Funding for current doctoral students Financial awards, scholarships, grants, and employment opportunities for current doctoral students at the CUNY Graduate Center.
- Git branching policy Internal policy on using Git branches.
- Guidelines for Preparing Master’s Theses in Arts & Sciences Hunter's A&S' guide for MA students preparing a thesis
- Link you Google Account with another email address Instructions on how to link your Google account with your (school) email address.
- Software Evolution and Machine Learning Google Group Project-specific Google group.
- Stack Overflow for CUNY PONDER Stack overflow for our research group
- Boa Language and Infrastructure User Forum
- Computer Science Research and Practice A slack workspace for folks interested in computer science research and development.
- CUNY PONDER Google Group Our internal Google group for email and sharing Google services.
- CUNY PONDER Slack Instant messaging workspace for our research group.
- Eclipse Java Development Tools (JDT) Forum Questions and technical discussions about how to use the Java development tools (JDT)
- Eclipse jdt-core-dev mailing list Eclipse JDT Core developers list.
- Eclipse jdt-dev mailing list Eclipse JDT general developers list.
- Eclipse jdt-ui-dev mailing list Eclipse JDT UI developers list.
- ECOOP Info Mailing List A european-based software engineering mailing list
- NEPLS Mailing list for the New England Programming Languages and Systems Symposium Series.
- NJPLS New Jersey Programming Language and Systems Seminar
- PLTea An initiative to bring together the members of the programming languages community for informal virtual discussions.
- pydev-code Mailing list for PyDev Eclipse developers.
- SEWORLD Mailing List A mailing list provided as a service to the Software Engineering community.
- Software Evolution and Machine Learning Google Group Project-specific Google group.
- Stack Overflow page for the Eclipse JDT Question and answers about the Eclipse JDT.
- tech-debt mailing list Technical Debt research mailing list
- WALA mailing list Mailing list for the T.J. Watson Libraries for Analysis (WALA).
- New England Programming Languages and Systems Symposium Series (NEPLS) A regional venue to discuss current research in programming languages and systems.
- A Brief Introduction to Static Analysis A nice overview of static analysis with real-world application.
- Introduction to Static Analysis Another nice introduction to static analysis.
- Static Analysis for Java in Eclipse Possibly outdated (but conceptually relevant) slides on program (static) analysis in Eclipse.
- Unleash the Power of Refactorings in your Products Talk at EclipseCon 2007
- Cohen’s kappa – Wikipedia A statistic used to measure inter-rater reliability for qualitative (categorical) items.
- CUNY PONDER upcoming papers plan Upcoming planned research papers for our group
- Installing Linux on Windows 10 Instructions for installing a linux subsystem on a windows platform.
- Introduction to Machine Learning for Java Developers Video from the NYJavaSIG on Nov 7, 2019
- Proof templates
- Stack Overflow for CUNY PONDER Stack overflow for our research group
- Working in the Linux Command-Line A Short List of the Best and Greatest Commands
- Eclipse Java development tools (JDT) Overview of the Eclipse Java Development Tools (JDT) components.
- Eclipse JDT Plug-in Developer Guide A really excellent guide on how to create JDT plug-ins in Eclipse.
- Eclipse Platform Java development user guide Using Eclipse to develop Java programs.
- Eclipse source code for the eclipse.jdt.core component GitHub repository for the eclipse.jdt.core component of the Eclipse IDE.
- Eclipse source code for the eclipse.jdt.ui component GitHub repository for the eclipse.jdt.ui component of the Eclipse IDE
- Frequently used internal APIs in JDT UI A list of helper classes frequently used for internal development of the JDT UI.
- JDT Plug-in Developer Guide A guide to developing Eclipse Java Development Tools (JDT) plug-ins
- On the Job: The Eclipse 3.0 Jobs API A guide to concurrency and progress monitoring in Eclipse.
- PyDev Developers’ Guide How to develop atop of the PyDev plug-in for Eclipse
- The Official Eclipse FAQs Frequently asked questions about Eclipse development.
- An Illustrated Guide to Git on Windows
- Git branching policy Internal policy on using Git branches.
- Git documentation Documentation and reference on the popular Git versioning system.
- Git for Computer Scientists A quick introduction to Git internals.
- Git Magic Rough instructions for particular Git effects.
- Git ready Learn git one commit at a time.
- Resolving a merge conflict using the command line How to resolve merge conflicts using the command line and a text editor.
- Setting your commit email address in Git and GitHub Instructions for how to change your email to your professional email address.
- try.github.com Git quickstart.
- Creating a pull request Create a pull request to propose and collaborate on changes to a repository. These changes are proposed in a branch, which ensures that the default branch only contains finished and approved work.
- Reviewing proposed changes in a pull request In a pull request, you can review and discuss commits, changed files, and the differences (or "diff") between the files in the base and compare branches.
- Syncing a fork Sync a fork of a repository to keep it up-to-date with the upstream repository on GitHub.
- A (Not So) Short Introduction to LATEX2ε If you want to learn how to write your documents with LaTeX, this introduction is for you.
- Biblatex Cheat Sheet A BibLATEX/Biber ‘cheat sheet’
- Creating an annotated bibliography How to create an annotated bibliography in LaTeX.
- LaTeX General Help A web based LaTeX help, based on George D. Greenwade's LaTeX help for VMS machines.
- LaTeX Introduction A thorough introduction to LaTeX by Robert Dyer.
- Notes for using minted.sty on arXiv arXiv cannot process using the --shell-escape option, as this is disabled in arXiv's system for security reasons.
- Overleaf introduction to LaTeX A free online introduction to LaTeX from the online editor, Overleaf.
- TeX in CEU A short introduction to the scientific typesetting languages TeX and LaTeX, with some additional resources.
- How to give a great research talk Simple guidelines from Simon Peyton Jones of Microsoft Research for giving research talks.
- WALA API Documentation API documentation for the T.J. Watson Library for Analysis (WALA).
- WALA Pointer Analysis Guide WALA provides a framework for flow-insensitive Andersen-style pointer analysis. You can customize the pointer analysis context-sensitivity policy in various ways.
- WALA Starter Kit A basic Gradle configuration to get started with WALA.
- Annotated bibliography samples Sample annotations from annotated bibliographies, each with a different research project.
- Empirical Studies: Qualitative vs. Quantitative An explanation of the differences between qualitative and quantitative empirical research
- Guide to writing a scientific research report (IMRaD) A guide to writing IMRaD papers from GMU.
- How to write a great research paper This talk from Microsoft Research offers seven simple, concrete suggestions for how to improve your research papers.
- How to write a great research proposal This page is an attempt to collect together a number of suggestions about what makes a good proposal.
- How to write a paper critique
- How to write a technical paper This document describes several simple, concrete ways to improve your writing, by avoiding some common mistakes.
- Mastering Markdown Markdown guide from GitHub
- Patterns for writing good rebuttals
- The Devil's Guide to Incremental Research
- What is Empirical Research? An introduction to empirical research from USU Libraries
- Writing Logically, Thinking Critically A small textbook on critique thinking and logical arguments.
- Report about creating refactorings in Eclipse Interesting report that discusses the architecture of Eclipse refactoring plugins.
- 3 “I”s of PhD Research, and the Oft-Forgotten 3rd “I” Talk by Abhik Roychoudhury on succeeding in a PhD in research.
- Digital Networking and Community This talk gives hints on the benefits and risks of science on social media.
- Dissertation Proposal Tips Tips for writing a dissertation and defending a thesis.
- Early research initiative Discover opportunities for funded research in archives, museums, field sites, special collections, laboratories, and research centers.
- Financial Resources Student financial resources, including emergency grants and financial literacy.
- Funding for current doctoral students Financial awards, scholarships, grants, and employment opportunities for current doctoral students at the CUNY Graduate Center.
- GitHub Education Student Developer Pack The best developer tools, free for students.
- Guidelines for Preparing Master’s Theses in Arts & Sciences Hunter's A&S' guide for MA students preparing a thesis
- How to review Computer Science papers A guide for reviewing CS papers, published in XRDS: Crossroads, The ACM Magazine for Students.
- How to Write Up a Ph.D. Dissertation Turn your research (once it's done) into a readable multi-chapter document.
- LaTeX Homework Templates Some templates for writing your homework in latex.
- Library remote access Accessing library material from home. You may need these instructions to access protected papers.
- Planning a Research Project Talk by Abhik Roychoudhury on research strategy for graduate students.
- The Teach@CUNY Handbook Practical guidance to both beginning and experienced instructors across CUNY and beyond. Section II: "Strategies” may be particularly useful for graduate students teaching their first course.
- Analysis Tools A curated list of static and dynamic analysis tools for all programming languages, config files, build tools, and more.
- CUNY PONDER Research Virtual Machine Virtual machine of the CUNY PONDER group used for experimentation.
- gitcproc Tool for analyzing git log messages and diffs.
- Link you Google Account with another email address Instructions on how to link your Google account with your (school) email address.
- Online Kappa Calculator Can be used to calculate kappa--a chance-adjusted measure of agreement--for any number of cases, categories, or raters
- Using zoom.us on Linux Set the QT_SCALE_FACTOR environmental variable to 2.
- WALA T.J. Watson Libraries for Analysis.
- Asana Chrome extension: quick-add tasks • Asana Quickly add tasks to Asana from any web page in Chrome.
- CUNY PONDER Asana Workspace Asana workspace for our research group
- Google Drive Asana: Attach files to project tasks • Asana The Google Drive file chooser is built into the Asana task pane so you can easily attach My Drive files directly to tasks.
- Eclipse Eclipse is a powerful and extendable Integrated Development Environment (IDE) for many languages, including Java.
- Eclipse Buildship Eclipse Plug-ins for Gradle.
- Eclipse Environment Variables Plug-in Allows you to add environment variables to eclipse.
- Eclipse for RCP and RAP Developers Eclipse download for plug-in developers.
- Eclipse Installer An installer for Eclipse. Be sure to pick the RCP and RAP developer package.
- EGit Git and GitHub plugin for Eclipse
- Formatting files CUNY PONDER common source code formatting files.
- M2Eclipse Tight integration for Apache Maven into the Eclipse IDE.
- org.eclipse.jdt.astview – AST View A view to visualize the AST (abstract syntax tree) of a Java file open in the editor.
- org.eclipse.jdt.jeview – JavaElement View A view to visualize the Java elements in Eclipse's Java model.
- PyDev A Python IDE for Eclipse, which may be used in Python, Jython and IronPython development.
- Using Git in Eclipse Use a personal access token instead of your normal password.
- EGit Git and GitHub plugin for Eclipse
- GitKraken GitKraken is a Git GUI client for Windows, Mac, and Linux.
- Gitless An experimental version control system built on top of Git.
- Sourcetree Sourcetree simplifies how you interact with your Git repositories so you can focus on coding.
- Using Git in Eclipse Use a personal access token instead of your normal password.
- GitHub CLI Command-line interface to GitHub.
- miner-utils Scripts to help call the GitHub API
- arXiv LaTeX Cleaner Easily clean the LaTeX code of your paper to submit to arXiv
- JabRef Free reference manager that works with Bibtex and Latex.
- JabRef Browser Extensions This official browser extension automatically identifies and extracts bibliographic information on websites and sends them to JabRef with one click.
- LaTeX Homework Templates Some templates for writing your homework in latex.
- LaTeX proof package Write derivations in LaTeX.
- syntastic: Syntax checking hacks for Vim Catch LaTeX errors in Vim
- TeX “comment” package A LaTeX package to exclude large portions of text, possibly based on "versions."
- Google Drive for Slack If you use Google Drive, you can install the Google Drive app to share and access files right from your workspace.
- LaTeX for Slack A simple /slash-command integration for Slack that enables you to share LaTeX snippets in Slack channels with others.
- Slack + Asana: Automated updates to tasks Get instant notifications in Slack when changes are made to tasks in Asana.
- Grammarly Free Online Writing Assistant
- A Simple Makefile Tutorial A starter's guide to quickly and easily create your own makefiles for small to medium-sized projects.
- PLAST-Lab YouTube Channel Various tutorials from the Programming Languages and Advanced Software Technologies at the University of Athens
- UNIX/Linux Tutorial for Beginners
- Creating Your Own Plugin in Eclipse How to create a new plug-in to Eclipse.
- Eclipse Java Development Tools (JDT) Fundamentals JDT tutorial from EclipseCon 2010.
- Eclipse JDT – Abstract Syntax Tree (AST) and the Java Model Usage of the Abstract Syntax Tree (AST) and the Java model in Eclipse.
- Extending the Eclipse IDE A plug-in development tutorial.
- Git Tutorial Git tutorial from W3schools. Has a quiz.
- gittutorial This tutorial explains how to import a new project into Git, make changes to it, and share changes with other developers.
- Understanding Git Conceptually A tutorial on the Git version control system.
- Getting Started with WALA This page steps through the process to download, build, and run a core subset of WALA for standard Java analysis. Later we will describe other available components, in addition to this core subset.
- Static and Dynamic Program Analysis Using WALA PLDI 2010 Tutorial on WALA
- CUNY PONDER GitHub Organization GitHub organization for our research group.
- CUNY PONDER Wiki Organizational wiki pages.
- JDT UI Component Development Resources