Projects

This page shows the list of research projects. See Design for the design-oriented list of my projects.

Research projects

The following research-oriented projects have been conducted under a certain scientific context and most of them have been published at archival venues.

Lyric App Framework

We propose a "lyric app," a novel media format of interactive lyric-driven visual art. We developed a framework to support its development, collected 52 examples in the wild, and gained insights into the future of creative culture and the intersection of music and programming.

Creativity Support Environments

To realize a creative society in which people want to create content with the help of AI technologies, a sustainable ecosystem in which content inspires the next creation serves a critical role.

I shed light on the creative culture in such an ecosystem, build a "creativity support environment (CSE)" that supports the content creation and distribution process, and aim to establish Human-Computer Interaction (HCI) technologies that engineer, nurture, and maintain the creative culture.

2022-2023CHI EA '23Creativity
ODEN

ODEN seamlessly supports the edit and experiment repetition in deep learning application development by allowing the user to construct the neural network (NN) with the live visualization and transits into experimentation to instantly train and test the NN architecture.

2022IUI '22Co-authoredLive IDE
Guided Optimization for Image Processing Pipelines

"Guided optimization" provides programmers a set of valid optimization options and interactive feedback about their current choices, which enables them to comprehend and efficiently optimize the image processing code without the time-consuming process of trial-and-error in traditional text editors.

2021VL/HCC '21Co-authoredLive IDE
Griffith

Griffith is a web-based authoring tool for Japanese animation (anime) storyboards.

2019-2023SAS '21Creativity
Programming as Communication

Programming should be a more social activity than what it is right now. A programming environment does not need to be only for programmers. It should instead be designed for users with diverse technical backgrounds.

Such inclusive design enables the users to communicate with each other through programming-related activities, delivers benefits of programming to all of them, and would give empowerment to them.

2019-2023Convivial Computing Salon '20Communication
DeployGround

DeployGround is a framework for developing web-based tutorials that seamlessly connects the API playground and deployment target for kick-starting the application development.

2018VL/HCC '18Live IDE
Society, Science, and Technology

I have been involved in various activities with a broad interest in science and technology. In the past, I constructed a website for promoting science and technology as the leader of the Tachibana seminar. After that, I have been continuously interested in science communication.

Since I started my research career, I have constantly been working on implementing the research results into society, writing essays on the future opened up by scientific research, and commenting on settings in science fiction films.

2018-2023(Work in progress)
User Interfaces for Live Programming

Live programming eliminates the gulf between code and execution. User interface design plays the key role in providing live programming experience.

With appropriate user interface design, live programming can potentially benefit end-users, be used for applications whose computation takes a long time, and mean much more than merely providing real-time information of the running program.

-2023LIVE '17Live UI
Programming with Examples

The programming-with-examples (PwE) workflow lets developers create interactive applications with the help of example data. It takes a general programming environment and adds dedicated user interfaces for visualizing and managing the data.

This is particularly useful in developing data-intensive applications such as physical computing, image processing, video authoring, machine learning, and others that require intensive parameter tuning.

2016Special Issue on 21st User InterfacesPwE Live IDE UI
Reactile

Reactile is an exploratory research project that investigates and proposes a new approach to programming Swarm UI by leveraging direct physical manipulation.

2017-2018CHI '18Co-authoredPhysical PwE Live IDE
Songle Sync

The Songle Sync platform allows controlling numerous devices in synchrony with the songs with its SDK.

2017-2023MM '18Creativity Music IDE Design
User-Generated Variables

UGVs are variables generated by the end-users' requests and implemented (or discarded) by the programmers. The use of UGV helps both sending feature requests and their actual implementation.

Live Tuning

Live Tuning is the subset of Live Programming interaction that only involves changes in constant values through parameter tuning interfaces. It expands Live Programming benefits to non-programmers.

2016LIVE '16Communication Live IDE
f3.js

A parametric design tool for physical computing devices for both interaction designers and end-users.

TextAlive

An integrated design environment for kinetic typography; desktop app revamped as a web service in 2015.

CrossSong

A logic puzzle for "musical thinking," deepening your understanding on music while enjoying the puzzle.

2015-2017SMC '15Co-authoredMusic
Sharedo

A to-do list interface for sharing tasks between human and agents including robots and software personal assistants.

2011-2014HAI '14Robot
VisionSketch

An IDE for example-centric programming of image processing applications, enabling fluid transition between abstract text-based coding and concrete direct manipulation.

2013-2014GI '14PwE Live IDE NUI
CapStudio

An IDE for visual applications that shows interactive screencasts for replaying program executions and editing visual parameters.

2012-2014CHI EA '14Co-authoredPwE Live IDE
It's Alive! (TouchDevelop)

The TouchDevelop IDE enables live programming for user interface programming by cleanly separating the rendering and non-rendering aspects of a UI program, allowing the display to be refreshed on a code change without restarting the program.

2012-2013PLDI '13Live IDE
Picode

An IDE that shows inline photos representing posture data in source code. It manages the data binding between the photos and posture data and show photos to aid source code comprehension and editing.

2011-2013CHI '13PwE Communication IDE Robot
OpenPool

An open-source project that augments a standard billiard table with the cutting-edge computing technologies to provide the "super-billiard" experience.

2012-2013IEEE GCCE '13Toolkit NUI
DejaVu

An IDE that eases the iterative development of interactive camera-based programs with support for monitoring, recording, reviewing, and reprocess temporal data

2012UIST '12PwE Live IDE NUI
Phybots

A toolkit that allows researchers and interaction designers to rapidly prototype applications with locomotive robotic things.

2009-2012DIS '12Toolkit Physical Robot
Surfboard

A technique to detect simple gestures of "surfing" (moving a hand horizontally) on a standard keyboard by analyzing recorded sounds in real-time with a microphone attached close to the keyboard.

2010UIST '10 AdjunctNUI
Pressing

A "pressure-sensitive" programming environment with visual feedbacks of the built-in 2D physical simulator.

2009インタラクション2010デモLive IDE NUI
CRISTAL

A table-top interface that provides a streaming video view of the living room, so users can walk up to it, see the layout of the room and interact with the TV, lighting, cleaning robot, and so on.

2009SIGGRAPH '09 E-TechCo-authoredRobot NUI
Multi-touch Interface for Controlling Multiple Mobile Robots

A multi-touch interface with a top-down view from a ceiling-mounted camera for simultaneously controlling multiple mobile robots.

2008-2009CHI EA '09Robot NUI

Design and/or engineering projects

The following design and engineering projects have been conducted as part of my hobby or probono activities.

Magical Mirai 2018 live lyrics animation

Once again, authored lyrics animation for a live performance by Hatsune Miku!

2018Design
SNOW MIKU LIVE! 2018 lyrics animation

Authored lyrics animation for a live performance by a renowned virtual idol singer - Hatsune Miku!

2018Design
"junkato.jp" Website

This portfolio website is built with various OSS and itself is open-sourced on GitHub.

2018-2023Design
Tenju Works web design

Helped renovation of the Tenju Works website.

2017Design
OngaCREST service flyer and poster design

Designed flyers and posters of the four web services developed by the OngaCREST project.

2016Design
Home hacking

Have been hacking my home to automate various tasks.

2011-2023NUI
RemotiTunes

A Java-based tool for listening to your home iTunes Library from anywhere.

2011
IntelliViewer

A user interface for helping navigation of source code in IDEs. For each row of the keyboard, available namespaces, classes, member variables and methods are shown on the keytops.

2010IDE NUI
CD jacket design

Designed multiple CD jackets.

2007-2008Design
Chippie

An extendible PHP framework for content management of news website.

2006-2008Design Toolkit
arX MusicPlayer

Yet another music player for Windows with a built-in interpreter of a domain-specific language to change "skins."

2002-2007
arX Launcher

Yet another program launcher for Windows written with <300 physical loc.

2004