User Interfaces for Live Programming
Live programming eliminates the gulf between code and execution. To design live programming systems, we first need to understand what we are developing. Once we gain domain-specific knowledge of the target applications, we create user interfaces to edit the running application, such as scrubbing sliders, color palettes, and timeline interfaces. The key to the fluid programming experience of live programming is the combination of deep understanding of the target applications and adoption of interactive user interfaces.
In the keynote talk at LIVE '17, I discussed the liveness of programming systems in light of user interface design. I introduced user interfaces for programming and discussed the importance of integrating graphical representations into programming environments. I extended the scope of "liveness" by describing two kinds of interactions, physical and developer-user, and provided insights on designing future live programming systems.
Live Programming systems provide concrete information about how the program behaves when it is executed. With help of this concrete information, programmers can easily iterate on editing and testing the program behavior.
Please note that the origin of Live Programming is not new as you can see early examples in visual programming and object-oriented programming environments. Then, why is it attracting much attention these days? In short, I consider it is because of its focus on the user experience.
From now on, I will discuss three perspectives on user interface design of Live Programming systems.
The following projects implement or discuss Live Programming experiences.
ODEN seamlessly supports the edit and experiment repetition in deep learning application development by allowing the user to construct the neural network (NN) with the live visualization and transits into experimentation to instantly train and test the NN architecture.
"Guided optimization" provides programmers a set of valid optimization options and interactive feedback about their current choices, which enables them to comprehend and efficiently optimize the image processing code without the time-consuming process of trial-and-error in traditional text editors.
This project aims at introducing the basics of user interface design
DeployGround is a framework for developing web-based tutorials that seamlessly connects the API playground and deployment target for kick-starting the application development.
Live programming eliminates the gulf between code and execution. User interface design plays the key role in providing live programming experience.
With appropriate user interface design, live programming can potentially benefit end-users, be used for applications whose computation takes a long time, and mean much more than merely providing real-time information of the running program.
The programming-with-examples (PwE) workflow lets developers create interactive applications with the help of example data. It takes a general programming environment and adds dedicated user interfaces for visualizing and managing the data.
This is particularly useful in developing data-intensive applications such as physical computing, image processing, video authoring, machine learning, and others that require intensive parameter tuning.
Reactile is an exploratory research project that investigates and proposes a new approach to programming Swarm UI by leveraging direct physical manipulation.
UGVs are variables generated by the end-users' requests and implemented (or discarded) by the programmers. The use of UGV helps both sending feature requests and their actual implementation.
Live Tuning is the subset of Live Programming interaction that only involves changes in constant values through parameter tuning interfaces. It expands Live Programming benefits to non-programmers.
A parametric design tool for physical computing devices for both interaction designers and end-users.
An integrated design environment for kinetic typography; desktop app revamped as a web service in 2015.
An IDE for example-centric programming of image processing applications, enabling fluid transition between abstract text-based coding and concrete direct manipulation.
An IDE for visual applications that shows interactive screencasts for replaying program executions and editing visual parameters.
The TouchDevelop IDE enables live programming for user interface programming by cleanly separating the rendering and non-rendering aspects of a UI program, allowing the display to be refreshed on a code change without restarting the program.
An IDE that eases the iterative development of interactive camera-based programs with support for monitoring, recording, reviewing, and reprocess temporal data
A "pressure-sensitive" programming environment with visual feedbacks of the built-in 2D physical simulator.