Integrated Support for Example-Centric Programming of Image Processing Applications

GI '14
ACM PLDI '13 Student Research Competition Finalist
Keywords: PwE, Live, IDE, NUI
Source code is available on GitHub! It works on both Windows and Mac OS X.


We propose an integrated development environment (IDE) called "VisionSketch", which supports example-centric programming for easily building image processing pipelines.

With VisionSketch, a programmer is first asked to select the input video. Then, he can start building the pipeline with a visual programming language that provides immediate graphical feedback for algorithms applied to the video. He can also use a text-based editor to create or edit the implementation of each algorithm.

During the development, the pipeline is always ready for execution with a video player-like interface enabling rapid iterative prototyping. In a preliminary user study, VisionSketch was positively received by five programmers, who had prior experience of writing text-based image processing programs and could successfully build interesting applications.

Presentation materials

GI 2014 Demo Video

GI 2014 Slides

VisionSketch IDE

Overview of the VisionSketch IDE (GI '14)
Overview (original version)
Prototype of the VisionSketch IDE (PLDI '13)
Example use case #1: counting grinds
An example image processing pipeline


Cite this!2014GI '14

VisionSketch: Integrated Support for Example-centric Programming of Image Processing Applications

Jun Kato, Takeo Igarashi
Proceedings of the 2014 Graphics Interface Conference, pp.115-122
Copy to the clipboard

Visionsketch: Gesture-based Language for End-user Computer Vision Programming

2013Student Research Competition at PLDI '13ACM PLDI '13 Student Research Competition Finalist
Jun Kato
The 34th ACM SIGPLAN Conference on Programming Language Design and Implementation