User Interfaces for Live Programming

10/24/2017
Keynote Talk at LIVE '17

The page provides summary for the keynote talk at LIVE '17 workshop and links to relevant sub-projects including Live Tuning and User-Generated Variables.

翻訳が必要

概要

Live programming has attracted much attention for eliminating the gulf between code and execution. To design live programming systems, we first need to understand what we are developing. Once we gain domain-specific knowledge of the target applications, we create user interfaces to edit the running application, such as scrubbing sliders, color palettes, and timeline interfaces. The key to the fluid programming experience of live programming is the combination of deep understanding of the target applications and adoption of interactive user interfaces.

In the keynote talk at LIVE '16, I discussed the liveness of programming systems in light of user interface design. I introduced user interfaces for programming and discussed the importance of integrating graphical representations into programming environments. I extended the scope of "liveness" by describing two kinds of interactions, physical and developer-user, and provided insights on designing future live programming systems.

In particular, regarding the user interfaces for enhancing developer-user interactions, there are two sub-projects called Live Tuning and User-Generated Variables.

発表内容

LIVE 2017 発表スライド

What is Live Programming?

Live Programming refers to a PX that provides continuous feedback to the programmer about the program being developed.

Live Programming systems provide concrete information about how the program behaves when it is executed. With help of this concrete information, programmers can easily iterate on editing and testing the program behavior.

Please note that the origin of Live Programming is not new as you can see early examples in visual programming and object-oriented programming environments. Then, why is it attracting much attention these days? In short, I consider it is because of its focus on the user experience.

Live Programming requires decent user interfaces for exploring the problem space.

As such, building decent Live Programming systems requires deep understanding on the target application domains; e.g. avoid sudden changes in the program behavior, keep the program and its output relevant, and allow continuously exploring the problem space.

From now on, I will discuss three perspectives on user interface design of Live Programming systems.

Programming with End-users

As Live Programming systems provide interactive and intuitive user interfaces for debugging and editing programs, some of the user interfaces can be exposed to the end-users so that they can customize the programs as they wish. [Live Tuning]

When the provided user interfaces cannot satisfy their needs, they can still submit requests to programmers with rich contextual information provided by the Live Programming systems. [User-Generated Variables]

Programming this Material World

Live Programming often targets simple applications such as drawing graphics. In contrast, "software is eating the world" and there are interesting applications such as physical computing and virtual reality.

Their actual "framerate" can be very slow; e.g. printing device enclosures and producing scent. Emulating, or sometimes even pretending, is needed to provide the continuous feedback. Making full use of five senses in programming environments would be crucial in the near future.

Programming with Time Travel

Live Programming puts focus on providing live feedback to the programmer, but it does not always mean providing realtime information of the running program.

The timeline interface and record and replay features together provide time traveling experience and play an important role to find critical timings in the history. Once the timing of interest is found, editing the code and program input would update the program output, allowing the programmer to explore future"s."

As a summary, Live Programming is a very interdisciplinary topic and involves all of Programming Language design, Software Engineering, and designing Human-Computer Interaction.

LIVE, PX workshops and SIGPX are such an attempt to attract people from all of these research fields.

関連プロジェクト

The following projects are highly relevant to the topic.

DeployGround
APIチュートリアルに開発環境の機能を組み込み、サンプルプログラムの開発からデプロイまで支援するフレームワークです。
2018VL/HCC '18Live IDE
Reactile
Swarm UIのアプリケーション開発において、直接Swarmを操作することでSwarmの振る舞いをプログラミングできる手法(direct physical manipulation)を提案した研究です。
2017-2018CHI '18共著Physical PwE Live IDE
User-Generated Variables
UGVは、アプリケーションに対する機能要望を変数宣言のかたちに制限することで、ユーザからの要望が明確になり、プログラマも実装の可否を判断しやすくなるインタラクションデザインです。
2017PX '17Live IDE
Live Tuning
Live Tuningは、Live Programmingから定数値をインタラクティブに変更できるインタラクションデザインだけを抜き出したものです。プログラマでなくともプログラムの振る舞いをカスタマイズできるようにする技術です。
2016LIVE '16Live IDE
f3.js
IoT機器の筐体設計とファームウェア開発を同時に行える設計ツールです。専門知識不要で機器をカスタマイズできます。
TextAlive
音楽に同期して歌詞がアニメーションする動画を制作でき、演出をJavaScriptで拡張できる統合制作環境です。
2014-2018CHI '15Creativity Music PwE Live IDE Design
VisionSketch
処理したい映像上でスケッチすることで画像処理パイプラインを構築できるビジュアルなプログラミング環境です。既存アルゴリズムに満足できなければ、文字ベースのプログラミングにスムーズに移行できます。
2013-2014GI '14PwE Live IDE NUI
CapStudio
ゲームなどの視覚的なアプリケーション開発において、プログラム実行画面の録画映像を表示し、さらに参照している画像リソースなどを編集して動的に確認できる統合開発環境です。
2012-2014CHI EA '14共著PwE Live IDE
It's Alive! (TouchDevelop)
TouchDevelopはユーザインタフェースのLive Programmingを可能にするWebベースの統合開発環境です。Webアプリを起動したまま、そのユーザインタフェースを定義するソースコードを変更できます。
2012-2013PLDI '13Live IDE
DejaVu
Kinectなどのカメラ入力を用いたインタラクティブなアプリケーション開発を支援する統合開発環境です。カメラ入力を自動録画し、プログラム実行結果を何度も再生したり、プログラムを修正して実行結果を更新したりできます。
2012UIST '12PwE Live IDE NUI
Pressing
プログラムの実行結果がソースコード入力時のキーの打鍵圧に影響を受ける統合開発環境です。メソッド呼び出しを強く打鍵すると物理シミュレーション上でボールが速く動いたりします。
2009インタラクション2010デモLive IDE NUI

更新履歴

2/15/2018
Replace "Further reading" with the list of relevant projects. Please visit each project page for further readings.
1/21/2018
Upload the initial prototype version.