This page shows updates to the website as well as publication records in their chronological order.



Served as an Associate Chair for the EIST subcommittee and the Student Research Competition Co-chair at ACM CHI 2021. Congratulations to the winners (see the bottom of the Awards page)!
Gave an invited talk titled "Creativity Support for People with Diverse Technical Background" and presented a poster on the ongoing project for Japanese animation storyboards from Arch Research at ISID 2021, which got one of the Best Poster Awards!


Invited Talks

Mar. 2021
ISID 2021, "Creativity Support for People with Diverse Technical Background"



Released TextAlive App API, a JavaScript library for building "lyric apps" (cf. lyric videos) that show lyrics synchronized with the music playback. Songle Sync (ACM Multimedia 2018) was used to celebrate Hatsune Miku's birthday with over 10,000 participants. Thank you for your birthday messages!
Appeared in the Microsoft Research article celebrating the 20th year of MSRA Fellowship, titled "A Story That Has Lasted Twenty Years, A Fellowship That Has Changed a Group of People".
Presented Rethinking Programming "Environment" in Convivial Computing Salon 2020, as part of my Programming as Communication project. The recording and slides of my talk are already available.
My affiliation, AIST, and OTON GLASS, Inc., a Japanese company utilizing personal fabrication technologies to create customized smart glasses, have started a collaborative research project. Please refer to "Programming as Communication" for more details.
There have been many committee roles these days. I will provide an invited talk on "Programming for X" at RIEC International Symposium on Human-Computer Interaction which will be held as part of the preparation for CHI 2021!

Invited Talks

Jan. 2020
RIEC International Symposium on HCI, "Programming for X: Programming Experience Research for Convivial Computing"


Rethinking Programming "Environment": Technical and Social Environment Design toward Convivial Computing

2020Convivial Computing Salon '20Programming as Communication
Jun Kato, Keisuke Shimakage
Companion Proceedings of the 4th International Conference on the Art, Science, and Engineering of Programming, pp.95-103

Live Programming Environment for Deep Learning with Instant and Editable Neural Network Visualization

2020PLATEAU '19
Chunqi Zhao, Tsukasa Fukusato, Jun Kato, Takeo Igarashi (Eds. Sarah Chasins, Elena L. Glassman, Joshua Sunshine)
10th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2019), pp.7:1-7:5



It's been a while ... Songle Sync (ACM Multimedia 2018) and TextAlive (ACM CHI 2015) were both used at Hatsune Miku "Magical Mirai" 2019. Thank you for your birthday messages presented with Songle Sync and hope you enjoyed the live performance and disc jockey performance using TextAlive!
Attended the Shonan meeting on exploratory and live programming and organized the Sixth Workshop of SIGPX at a venue of The University of Tokyo. The slides at SIGPX are available.



Presented Songle Sync in the "Experience" session at ACM Multimedia 2018. Slides are available for download in the project page.
Presented DeployGround in the "APIs and Use of Programming Languages" session at IEEE VL/HCC 2018. Slides are available for download in the project page.
Started a new part-time technical advisor position at Arch Inc., a Japanese animation production company. Please contact me if you want to pursue research on "Japanimation!"
My first-authored papers about Songle Sync and DeployGround are accepted to ACM Multimedia 2018 and IEEE VL/HCC 2018, respectively. Looking forward to attending the conferences!
At ACM CHI 2018, Ryo presented our work on the Reactile project. Its demo video and slides are available.
Organized the Forth Workshop of SIGPX at The University of Tokyo.
With TextAlive, I created lyrics animation videos used in SNOW MIKU LIVE! 2018, a live performance by a renowned virtual idol singer - Hatsune Miku!
A co-authored paper of the Reactile project discussing how to program Swarm UI has been accepted to ACM CHI 2018.
Renewed the website. See how it's made—source code available! Beside UI changes, newly-added content includes the follow-up page for the LIVE keynote talk discussing user interfaces for Live Programming, the f3.js project page, the TextAlive project page, and the design portfolio.


Apr. 2018
IEEE PacificVis 2018 Honorable Mention Poster Award for the co-authored TextTimeline poster


Songle Sync: A Large-Scale Web-based Platform for Controlling Various Devices in Synchronization with Music

2018MM '18Songle Sync
Jun Kato, Masa Ogata, Takahiro Inoue, Masataka Goto
Proceedings of the 26th ACM International Conference on Multimedia, pp.1697-1705

DeployGround: A Framework for Streamlined Programming from API playgrounds to Application Deployment

2018VL/HCC '18DeployGround
Jun Kato, Masataka Goto
Proceedings of the 2018 IEEE Symposium on Visual Languages and Human-Centric Computing, pp.259-263

Placing Music in Space: A Study on Music Appreciation with Spatial Mapping

2018DIS '18 Companion
Shoki Miyagawa, Yuki Koyama, Jun Kato, Masataka Goto, Shigeo Morishima
Proceedings of the 2018 ACM Conference Companion Publication on Designing Interactive Systems, pp.39-43

Reactile: Programming Swarm User Interfaces Through Direct Physical Manipulation

2018CHI '18Reactile
Ryo Suzuki, Jun Kato, Mark D. Gross, Tom Yeh
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp.199:1-199:13



Provided a keynote talk at SPLASH LIVE 2017. I'm writing a follow-up article about the talk, but for now, please refer to the talk abstract and slide deck.
Will serve as a keynote speaker at SPLASH LIVE 2017. See you in Vancouver!
Visited Pittsburgh for ACM UIST 2017 PC meeting, gave a talk at CMU HCII, dropped by Seattle and visited UW and Microsoft Research.
Presented f3.js at ACM DIS 2017. Slides are available for download in the publications page.
Presented User-Generated Variables at <Programming> PX workshop. It's always good to get to know new people!
Publicly demonstrated Robocam (f3js-logs) for the first time at ACM/IEEE HRI 2017, which is an accompanying tool and web-based interface to f3.js that helps recording event logs of IoT applications and authoring their demo videos.

Invited Talks


F3.Js: A Parametric Design Tool for Physical Computing Devices for Both Interaction Designers and End-users

2017DIS '17f3.js
Jun Kato, Masataka Goto
Proceedings of the 2017 Conference on Designing Interactive Systems, pp.1099-1110

The Future of Programming and Data Sciences

Jun Kato (Eds. Xiaojun Bi, Otmar Hilliges, Takeo Igarashi, Antti Oulasvirta)
Computational Interactivity (Dagstuhl Seminar 17232), pp.56-56

A Singing Instrument for Real-Time Vocal-Part Arrangement of Music and Audio Signals

2017SMC '17
Yuta Ojima, Tomoyasu Nakano, Satoru Fukayama, Jun Kato, Masataka Goto, Katsutoshi Itoyama, Kazuyoshi Yoshii
Proceedings of the 14th Sound and Music Computing Conference, pp.559-563

User-Generated Variables: Streamlined Interaction Design for Feature Requests and Implementations

Jun Kato, Masataka Goto
Companion to the First International Conference on the Art, Science and Engineering of Programming, pp.28:1-28:7

The CrossSong Puzzle: Developing a Logic Puzzle for Musical Thinking

Jordan B.L. Smith, Jun Kato, Satoru Fukayama, Graham Percival, Masataka Goto
Journal of New Music Research, pp.1-16

A Robotic Framework for Video Recording and Authoring

2017HRI '17 Companionf3.js
Jun Kato, Masataka Goto
Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, pp.41-41



Attended the First ACM SIGCHI Asian Symposium and was named as a board member for Japan ACM SIGCHI Chapter.
Served as a Student Volunteer Chair for ACM UIST 2016. Thanks all for coming to Tokyo!
Created a web page for the new ECOOP 2016 LIVE workshop paper that introduces "Live Tuning".
Created a web page for the new IEEE Computer journal article that introduces "Programming with Examples".
Visited VPRI, Y Combinator Research HARC, and several other places around Bay Area. Organized Japanese HCI Symposium at ACM CHI 2016.
Organized the First Workshop of SIGPX. English translation of the press release that introduces the TextAlive web service is now available.

Invited Talks

Apr. 2016
Viewpoints Research Institute and Y Combinator Research HARC, "Programming with {Examples | People}"


Live Tuning: Expanding Live Programming Benefits to Non-Programmers

2016LIVE '16Live Tuning
Jun Kato, Masataka Goto
Proceedings of the Second Workshop on Live Programming Systems

Programming with Examples to Develop Data-Intensive User Interfaces

2016IEEE Computer Special Issue on 21st User InterfacesProgramming with Examples
Jun Kato, Takeo Igarashi, Masataka Goto
Computer 49(7), pp.34-42

PlaylistPlayer: An Interface Using Multiple Criteria to Change the Playback Order of a Music Playlist

2016IUI '16
Tomoyasu Nakano, Jun Kato, Masahiro Hamasaki, Masataka Goto
Proceedings of the 21st International Conference on Intelligent User Interfaces, pp.186-190



Received the Best Paper Award at SMC 2015 for co-authoring a paper about an interactive music puzzle named CrossSong - please enjoy :)
Publicly demonstrated f3.js for the first time at ACM UIST 2015, which is an IDE to create IoT applications purely by code.
Presented TextAlive Online (ver.βjs) at ICLC 2015 and visited friends around UK!
Presented TextAlive in the "Programming Environments" session and organized Japanese HCI Symposium at ACM CHI 2015.
In preparation for the CHI presentation, TextAlive is finally released as ver.βjs! Make videos within your browser.
Served as an organizing committee member for a special event named IPSJ-ONE. It gathered 19 top-notch researchers in Information Processing (computer science, information science, that sort of thing) from all over Japan, each of them giving 5-minutes talk introducing their work with their passion for research.
My paper about TextAlive, an IDE to create Kinetic Typography videos with live programming, is accepted to ACM CHI 2015. It wins a silver medal! See you in Korea :)


Jul. 2015
SMC 2015 Best Paper for the co-authored CrossSong paper

Invited Talks


Form Follows Function(): An IDE to Create Laser-cut Interfaces and Microcontroller Programs from Single Code Base

2015UIST '15 Adjunctf3.js
Jun Kato, Masataka Goto
Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology, pp.43-44

CrossSong Puzzle: Generating and Unscrambling Music Mash-ups with Real-time Interactivity

2015SMC '15SMC '15 Best Paper AwardCrossSong
Jordan Smith, Graham Parcival, Jun Kato, Masataka Goto, Satoru Fukayama
Proceedings of the 12nd Sound and Music Computing Conference, pp.61-67

TextAlive Online: Live Programming of Kinetic Typography Videos with Online Music

2015ICLC '15TextAlive
Jun Kato, Tomoyasu Nakano, Masataka Goto
Proceedings of the First International Conference on Live Coding, pp.199-205

TextAlive: Integrated Design Environment for Kinetic Typography

2015CHI '15ACM CHI '15 Best Paper Honorable Mention AwardTextAlive
Jun Kato, Tomoyasu Nakano, Masataka Goto
Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp.3403-3412



Chaired "Novel Interaction Techniques" session and presented Sharedo which was nominated for the best paper at HAI 2014.
My paper about Sharedo that discusses sharing tasks between human and agents is accepted to HAI 2014. See you in Japan this October!
Visited the University of Toronto DGP and gave a talk. I'm also visiting the University of Manitoba HCI Lab in a few days and will give another talk. Both of these talks are about "Live Programming" in the real world, which is based on my Ph.D dissertation and discusses the direction of future work in my current position.
Serving as a program committee member for ACM UIST 2014 is exciting but difficult. Apart from it, I'm getting settled in the new environment and have just started to prepare for attending Graphics Interface 2014. I'm going to present VisionSketch in the "Video and Collaboration" session on 5/8.
Received a Ph.D degree in Computer Science and started to work as a researcher in AIST, a research organization funded by the Japanese government in which over 2000 researchers work hard to contribute to both industry and science. My primary task won't change - to conduct interesting research, write good paper, and do some other stuff to make world better place with help of information technology :)
Upgraded Bootstrap framework from v2 to v3 for better browsing experience. What a modern flat design!
Defended my dissertation! Details will come later. My paper about VisionSketch is accepted to Graphics Interface 2014.


Invited Talks

Jun. 2014
University of Toronto DGP seminar, "Toward Live Programming in the Real World"


Sharedo: To-do List Interface for Human-agent Task Sharing

2014HAI '14HAI '14 Best Paper NomineeSharedo
Jun Kato, Daisuke Sakamoto, Takeo Igarashi, Masataka Goto
Proceedings of the Second International Conference on Human-Agent Interaction, pp.345-351

VisionSketch: Integrated Support for Example-centric Programming of Image Processing Applications

2014GI '14VisionSketch
Jun Kato, Takeo Igarashi
Proceedings of the 2014 Graphics Interface Conference, pp.115-122

CapStudio: An Interactive Screencast for Visual Application Development

2014CHI EA '14CapStudio
Koumei Fukahori, Daisuke Sakamoto, Jun Kato, Takeo Igarashi
Extended Abstracts on Human Factors in Computing Systems, pp.1453-1458

Integrated Graphical Representations for Development of Programs with Real-world Input and Output

Jun Kato



Visited St. Andrews to for ACM UIST 2013, gave a talk at University of Washington dub, met friends of the Microsoft campus, and finished the Adobe internship... I'm back in Japan!
Started internship at Adobe Creative Technologies Lab in Seattle.
I am going to attend ACM UIST 2013 ... expecting fruitful discussion at Doctoral Symposium. My paper describing the development process of OpenPool is accepted to present at IEEE GCCE 2013.
Got some sort of recognition at ACM conferences, ACM CHI and ACM PLDI, detailed in Awards section. Will work for Adobe Research Seattle this summer.
My sole-authored poster about VisionSketch is accepted to present at Student Research Competition of ACM PLDI 2013.
My colleagues are going to present OpenPool at SXSW 2013. I've worked on the project as one of the main programmers. Stay tuned for its open-source distribution!
Collaboration with PL people turned out to be a full paper at ACM PLDI 2013. It's about live programming on TouchDevelop Web App!


Invited Talks

Oct. 2013
University of Washington dub seminar, "Integrated Graphical Representations for Programming with Real-world Input and Output", Video


Integrated Visual Representations for Programming with Real-world Input and Output

2013UIST '15 Adjunct (Doctoral Symposium)
Jun Kato
Adjunct Proceedings of the 26th Annual ACM symposium on User Interface Software and Technology, pp.57-60

OpenPool: Community-based Prototyping of Digitally-augmented Billiard Table

2013IEEE GCCE '13OpenPool
Jun Kato, Takashi Nakashima, Hideki Takeoka, Kazunori Ogasawara, Kazuma Murao, Toshinari Shimokawa, Masaaki Sugimoto
Proceedings of the 2nd IEEE Global Conference on Consumer Electronics, pp.175-176

Visionsketch: Gesture-based Language for End-user Computer Vision Programming

2013Student Research Competition at PLDI '13ACM PLDI '13 Student Research Competition FinalistVisionSketch
Jun Kato
The 34th ACM SIGPLAN Conference on Programming Language Design and Implementation

It's Alive! Continuous Feedback in UI Programming

Sebastian Burckhardt, Manuel Fahndrich, Peli de Halleux, Sean McDirmid, Michal Moskal, Nikolai Tillmann, Jun Kato
Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp.95-104

Picode: Inline Photos Representing Posture Data in Source Code

2013CHI '13ACM CHI '13 Best Paper Honorable Mention AwardPicode
Jun Kato, Daisuke Sakamoto, Takeo Igarashi
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp.3097-3100



My short paper about Picode is conditionally accepted to ACM CHI 2013. See you in Paris!
Attended 21 CCC in Tianjin, China and received this year's Microsoft Research Asia Fellowship.
Finished my talk at ACM UIST 2012 in the "Toolkits" session on 10/9. Will serve as a Publicity Co-chair at ACM UIST 2013!
Back in Japan again. The future plan includes a visit to Beijing in this month and Boston and New York in October.
My paper about an IDE named DejaVu is accepted to ACM UIST 2012. The details will come later.
Started to work with TouchDevelop team as a research intern in RiSE Group, Microsoft Research Redmond.
Back in Japan... In June, I will start an internship in Redmond.
My paper about Phybots is accepted to ACM DIS 2012. I will present the paper in the "Absolutely Fab" session on 6/13.
Renewed the website.
Started an internship at HCI Group of Microsoft Research Asia.



DejaVu: Integrated Support for Developing Interactive Camera-Based Programs

2012UIST '12DejaVu
Jun Kato, Sean McDirmid, Xiang Cao
Proceedings of the 25th Annual ACM symposium on User Interface Software and Technology, pp.189-196

Phybots: A Toolkit for Making Robotic Things

2012DIS '12Phybots
Jun Kato, Daisuke Sakamoto, Takeo Igarashi
Proceedings of the 9th Conference on Designing Interactive Systems, pp.248-257

2011 and before


Apr. 2011 - Mar. 2012
Microsoft Research 7th Collaborative Research (CORE7) Program Principal Investigator
Aug. 2009
ACM SIGGRAPH '09 Laval Virtual Award (Best of Emerging Technologies)

Invited Talks


Surfboard: Keyboard with Microphone as a Low-cost Interactive Surface

2010UIST '10 AdjunctSurfboard
Jun Kato, Daisuke Sakamoto, Takeo Igarashi
Adjunct Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology, pp.387-388

A Toolkit for Easy Development of Mobile Robot Applications with Visual Markers and a Ceiling Camera

2009UIST '09 DemoPhybots
Daisuke Sakamoto, Jun Kato, Masahiko Inami, Takeo Igarashi
The 22nd Annual ACM Symposium on User Interface Software and Technology

CRISTAL, Control of Remotely Interfaced Systems using Touch-based Actions in Living Spaces

2009SIGGRAPH '09 E-TechLaval Virtual Award (SIGGRAPH '09 Best of E-Tech)CRISTAL
Thomas Seifried, Christian Rendl, Florian Perteneder, Jakob Leitner, Michael Haller, Daisuke Sakamoto, Jun Kato, Masahiko Inami, Stacey D. Scott
ACM SIGGRAPH 2009 Emerging Technologies

Multi-touch Interface for Controlling Multiple Mobile Robots

2009CHI EA '09ACM CHI '09 Student Research Competition 1st PlaceMulti-touch Interface for Controlling Multiple Mobile Robots
Jun Kato, Daisuke Sakamoto, Masahiko Inami, Takeo Igarashi
Extended Abstracts on Human Factors in Computing Systems, pp.3443-3448