DejaVu

2012

UIST '12

For those interested in this IDE: Unfortunately, there's no short term plan for the distribution. You can still use Kinect Studio that partially provides similar features (recording and replaying data stream) though without tight integration with the IDE.

The increasing popularity of interactive camera-based programs highlights the inadequacies of conventional IDEs in developing these programs given their distinctive attributes and workflows. We present DejaVu, an IDE enhancement that eases the development of these programs by enabling programmers to visually and continuously monitor program data in consistency with the frame-based pipeline of computer-vision programs; and to easily record, review, and reprocess temporal data to iteratively improve the processing of non-reproducible camera input. DejaVu was positively received by three experienced programmers of interactive camera-based programs in our preliminary user trial.

This research was done during my internship at Microsoft Research Asia HCI Group. My mentors were Xiang Cao and Sean McDirmid.

YouTube

Download

We added two interlinked components to a general purpose development environment.

Visualizing various variable contents by drag-and-drop
Continuously showing camera input, variable contents, window output
Sketching capability

Auto-recording, replaying (input streams) & refreshing (variables) to handle non-reproducible input
Contents synchronized with canvas

Cite this!2012UIST '12

Jun Kato, Sean McDirmid, Xiang Cao

Proceedings of the 25th Annual ACM symposium on User Interface Software and Technology, pp.189-196

Paper PDF Poster PDF Slides PDF Resolve DOI

BibTeX:

@inproceedings{uist2012-kato-dejavu,
  title = {DejaVu: Integrated Support for Developing Interactive Camera-Based Programs},
  author = {Kato, Jun and McDirmid, Sean and Cao, Xiang},
  year = {2012},
  month = {oct},
  booktitle = {Proceedings of the 25th Annual ACM symposium on User Interface Software and Technology},
  location = {Cambridge, Massachusetts, USA},
  publisher = {ACM},
  address = {New York, NY, USA},
  series = {UIST '12},
  pages = {189--196},
  doi = {10.1145/2380116.2380142},
  isbn = {978-1-4503-1580-7},
  url = {http://doi.acm.org/10.1145/2380116.2380142},
  numpages = {8},
  acmid = {2380142},
  keywords = {computer vision, development environment},
  projectsite = {https://junkato.jp/dejavu},
  pdf = {https://junkato.jp/publications/uist2012-kato-dejavu.pdf},
  poster = {https://junkato.jp/publications/uist2012-kato-dejavu-poster.pdf},
  slides = {https://junkato.jp/publications/uist2012-kato-dejavu-slides.pdf},
  project = {dejavu}
}

Copy to the clipboard

DejaVu
Integrated Support for Developing Interactive Camera-Based Programs

Abstract

Presentation materials

UIST 2012 Demo Video

UIST 2012 Slides

DejaVu IDE

Canvas

Timeline

Publications

DejaVu: Integrated Support for Developing Interactive Camera-Based Programs

DejaVuIntegrated Support for Developing Interactive Camera-Based Programs