Top page
TextAlive
Griffith
Songle Sync
f3.jsTimeline
This page shows updates to the website as well as publication records in their chronological order.
2024
Updates
May 13, 2024
Presented Griffith: A Storyboarding Tool Designed with Japanese Animation Professionals in the "Design Tools B" session from 14:00 Mon, May 13 at ACM CHI 2024.
April 12, 2024
Started my sabbatical-like year as a Visiting Scientist at Université Paris-Saclay. I belong to Ex-Situ and conduct research on methods and tools for creativity support.
Invited Talks
August 23, 2024
Mizuho India Japan Study Centre at IIM Bangalore, Meijin-Samvad webinar series, "Impact & Synergy of Manga, Anime, and Video Games: Japanese Soft Power’s Impact on Global Markets - Positive? Negative? Balanced?" panelist
Lectures
June 19, 2024, June 26, 2024
The University of Tokyo Non-Research Tips for Information Science Researchers Guest Lectures "Research community" and "DevOps for research"
Publications
Griffith: A Storyboarding Tool Designed with Japanese Animation Professionals
Proceedings of the CHI Conference on Human Factors in Computing Systems
DanceUnisoner: A Parametric, Visual, and Interactive Simulation Interface for Choreographic Composition of Group Dance
2024
IEICE Transactions on Information and Systems E107.D(3), pp.386-399
2023
Updates
December 13, 2023
Concluded the "AIST Creative HCI Seminar series" with the sixth edition, focusing on "HCI research in the wild", inviting Jennifer Jacobs and Koya Narumi.
October 23, 2023
Served as Co-chair of LIVE 2023 and had a great time — thanks to everyone who was involved in the workshop.
September 28, 2023
Added a summary of the Japanese tutorial course titled "Introduction to Live Programming: Try It, Make It, Know It" held as part of The 40th JSSST Annual Conference to the page "User Interfaces for Live Programming".
August 20, 2023
My resume has been outdated for a while, and I finally managed to update it.
June 23, 2023
The Summer 2023 issue of ACM XRDS (Crossroads) is dedicated to "Exploring the Horizon of Computation for Creativity", to which I contributed an article. It discusses the role of computer scientists in helping creative and cultural activities "in the wild." For more details, please refer to my blog article.
June 20, 2023
Presented Lights, Animation, Interaction! – Synchronizing Music with Computer-controlled Visuals in Live Performances in the "Sound and Motion" session of SAS34. In this talk, I discussed how the "animating lights" contribute to the stage performances from my experience helping creators author lyric videos projected on the stages.
May 8, 2023
Was featured in People of ACM while attending ACM CHI 2023 — such excellent timing!
April 16, 2023
Will organize Special Interest Group on Creativity and Cultures in Computing from 16:35 Mon, Apr 24 and present Lyric App Framework: A Web-based Framework for Developing Interactive Lyric-driven Musical Applications (Honorable Mention Award) in the "Creative Applications" session from 9:00 Thu, Apr 27 at ACM CHI 2023.
March 31, 2023
Started a seminar series titled "AIST Creative HCI Seminar" and held its first edition on 3/7/2023. It focused on "creativity support" and invited Jonas Frich and Zhicong Lu. The second edition will be held online on 4/14/2023 with focus on "AI for HCI" and invite Amy Pavel and Xingyu Bruce Liu. We are calling for participation!
February 24, 2023
Received 2023 Japan ACM SIGCHI Chapter Distinguished Young Researcher Award and gave the award talk titled "The HCI researcher as toolsmith".
February 4, 2023
LIVE 2022 in Auckland (NZ) was a big success, thank you! One paper and one SIG proposal were conditionally accepted to ACM CHI 2023, while one paper was accepted to SAS Conference 2023. I'm resuming (physical) travels this year, looking forward to seeing you somewhere.
Awards
April 23, 2023
February 18, 2023
Invited Talks
February 18, 2023
Japan ACM SIGCHI Chapter International Symposium on HCI, "The HCI researcher as toolsmith," award talk for 2023 Japan ACM SIGCHI Chapter Distinguished Young Researcher Award, YouTube
February 8, 2023
Mozilla MIECO weekly meeting, "A Toolsmith for Programmers, Musicians, and Anime Creators"
Publications
On the Relationship between HCI Researchers and Creators---Or How I Became a Toolsmith
XRDS 29(4), pp.26-31
Lights, Animation, Interaction! – Synchronizing Music with Computer-controlled Visuals in Live Performances
Society for Animation Studies 34th Annual Conference: The Animated Environment
Lyric App Framework: A Web-based Framework for Developing Interactive Lyric-driven Musical Applications
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Special Interest Group on Creativity and Cultures in Computing
Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems
2022
Updates
September 30, 2022
Is serving as a Co-chair of LIVE 2022 (co-located with SPLASH 2022) and an Associate Chair of the EIST subcommittee at ACM CHI 2023.
July 1, 2022
Participated in the Dagstuhl Seminar #22231 on "Theories of Programming" with amazing fellow researchers and had great discussions last month -- will need more time to digest what I got.
June 30, 2022
Presented Past, Present, and Future of "Toolsmiths" in Japanese Animation in the "Machine Learning and AI in animation" session of SAS 2022. This is a follow-up work to the SIGGRAPH Asia 2021 panel session and last year's presentation at SAS 2021.
March 15, 2022
Gave a poster presentation titled Griffith: Prototype of A Web-based Tool for Authoring Japanese Anime Storyboards at ISID 2022 and received one of the Best Poster Awards.
February 16, 2022
Participated in the art exhibition titled Design Alternative Realities and discussed "Democratization of the Process and Outcome of Computer Science Research."
Awards
March 15, 2022
Invited Talks
December 7, 2022
Publications
Past, Present, and Future of “Toolsmiths” in Japanese Animation
Society for Animation Studies 2022 Conference: Animation Unlocked
Griffith: Prototype of A Web-based Tool for Authoring Japanese Anime Storyboards
ODEN: Live Programming for Neural Network Architecture Editing
2022IUI '22
27th International Conference on Intelligent User Interfaces, pp.392-404
2021
Updates
December 7, 2021
Will lead the featured session titled R&D for Anime Production: State-of-the-Art and Future Prospects and also serve as the Real-Time Live! Chair at ACM SIGGRAPH Asia 2021.
September 4, 2021
Participated in the German anime and manga convention Connichi and presented in the panel session "How Academics Contribute to Anime's Future" -- glad to be in Germany (virtually), my country of birth!
June 17, 2021
Presented Past, Present, and Future of Storyboarding in Japanese Animation in the Anime session of SAS 2021. This is the first full-length oral presentation as Arch Research!
May 19, 2021
Served as an Associate Chair for the EIST subcommittee and the Student Research Competition Co-chair at ACM CHI 2021. Congratulations to the winners (see the bottom of the Awards page)!
March 16, 2021
Gave an invited talk titled "Creativity Support for People with Diverse Technical Background" and presented a poster on the ongoing project for Japanese animation storyboards from Arch Research at ISID 2021, which got one of the Best Poster Awards!
Awards
October 13, 2021
IEEE VL/HCC '21 Best Short Paper Award for the co-authored Roly-poly short paper
March 17, 2021
ISID '21 Best Poster Award
Invited Talks
September 4, 2021
Connichi invited panel, "How Academics Contribute to Anime's Future" panelist
March 15, 2021
ISID 2021, "Creativity Support for People with Diverse Technical Background"
Publications
Guided Optimization for Image Processing Pipelines
2021VL/HCC '21IEEE VL/HCC '21 Best Short Paper Award
Proceedings of the 2021 IEEE Symposium on Visual Languages and Human-Centric Computing
Past, Present, and Future of Storyboarding in Japanese Animation
Society for Animation Studies 2021 Conference: Animated Energies
Research on Anime Storyboards for Individual and Collaborative Creativity
2020
Updates
September 20, 2020
Released TextAlive App API, a JavaScript library for building "lyric apps" (cf. lyric videos) that show lyrics synchronized with the music playback. Songle Sync (ACM Multimedia 2018) was used to celebrate Hatsune Miku's birthday with over 10,000 participants. Thank you for your birthday messages!
May 31, 2020
Appeared in the Microsoft Research article celebrating the 20th year of MSRA Fellowship, titled "A Story That Has Lasted Twenty Years, A Fellowship That Has Changed a Group of People".
May 15, 2020
Added a preprint of the paper on Rethinking Programming "Environment" in the Programming as Communication project page.
May 9, 2020
Presented Rethinking Programming "Environment" in Convivial Computing Salon 2020, as part of my Programming as Communication project. The recording and slides of my talk are already available.
April 1, 2020
My affiliation, AIST, and OTON GLASS, Inc., a Japanese company utilizing personal fabrication technologies to create customized smart glasses, have started a collaborative research project. Please refer to "Programming as Communication" for more details.
January 16, 2020
There have been many committee roles these days. I will provide an invited talk on "Programming for X" at RIEC International Symposium on Human-Computer Interaction which will be held as part of the preparation for CHI 2021!
Invited Talks
January 24, 2020
RIEC International Symposium on HCI, "Programming for X: Programming Experience Research for Convivial Computing"
Publications
Lyric Video Analysis Using Text Detection and Tracking
2020DAS '20
Document Analysis Systems, pp.426-440
Rethinking Programming "Environment": Technical and Social Environment Design toward Convivial Computing
Companion Proceedings of the 4th International Conference on the Art, Science, and Engineering of Programming, pp.95-103
Live Programming Environment for Deep Learning with Instant and Editable Neural Network Visualization
2020PLATEAU '19
10th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2019), pp.7:1-7:5
2019
Updates
December 5, 2019
It's been a while ... Songle Sync (ACM Multimedia 2018) and TextAlive (ACM CHI 2015) were both used at Hatsune Miku "Magical Mirai" 2019. Thank you for your birthday messages presented with Songle Sync and hope you enjoyed the live performance and disc jockey performance using TextAlive!
March 2, 2019
Attended the Shonan meeting on exploratory and live programming and organized the Sixth Workshop of SIGPX at a venue of The University of Tokyo. The slides at SIGPX are available.
2018
Updates
October 25, 2018
Presented Songle Sync in the "Experience" session at ACM Multimedia 2018. Slides are available for download in the project page.
October 7, 2018
Presented DeployGround in the "APIs and Use of Programming Languages" session at IEEE VL/HCC 2018. Slides are available for download in the project page.
August 20, 2018
First-authored projects Songle Sync (ACM Multimedia 2018) and TextAlive (ACM CHI 2015) will both be used at Hatsune Miku "Magical Mirai" 2018. Celebrate her birthday with Songle Sync or enjoy the live performance along with kinetic typography videos created with TextAlive!
July 9, 2018
Started a new part-time technical advisor position at Arch Inc., a Japanese animation production company. Please contact me if you want to pursue research on "Japanimation!"
July 7, 2018
My first-authored papers about Songle Sync and DeployGround are accepted to ACM Multimedia 2018 and IEEE VL/HCC 2018, respectively. Looking forward to attending the conferences!
April 28, 2018
At ACM CHI 2018, Ryo presented our work on the Reactile project. Its demo video and slides are available.
March 7, 2018
Organized the Forth Workshop of SIGPX at The University of Tokyo.
February 18, 2018
With TextAlive, I created lyrics animation videos used in SNOW MIKU LIVE! 2018, a live performance by a renowned virtual idol singer - Hatsune Miku!
January 30, 2018
A co-authored paper of the Reactile project discussing how to program Swarm UI has been accepted to ACM CHI 2018.
January 20, 2018
Renewed the website. See how it's made—source code available! Beside UI changes, newly-added content includes the follow-up page for the LIVE keynote talk discussing user interfaces for Live Programming, the f3.js project page, the TextAlive project page, and the design portfolio.
Awards
April 12, 2018
IEEE PacificVis '18 Honorable Mention Poster Award for the co-authored TextTimeline poster
Publications
Songle Sync: A Large-Scale Web-based Platform for Controlling Various Devices in Synchronization with Music
Proceedings of the 26th ACM International Conference on Multimedia, pp.1697-1705
DeployGround: A Framework for Streamlined Programming from API playgrounds to Application Deployment
Proceedings of the 2018 IEEE Symposium on Visual Languages and Human-Centric Computing, pp.259-263
Placing Music in Space: A Study on Music Appreciation with Spatial Mapping
2018DIS '18 Companion
Proceedings of the 2018 ACM Conference Companion Publication on Designing Interactive Systems, pp.39-43
Reactile: Programming Swarm User Interfaces Through Direct Physical Manipulation
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp.199:1-199:13
2017
Updates
November 8, 2017
Provided a keynote talk at SPLASH LIVE 2017. I'm writing a follow-up article about the talk, but for now, please refer to the talk abstract and slide deck.
June 29, 2017
Visited Pittsburgh for ACM UIST 2017 PC meeting, gave a talk at CMU HCII, dropped by Seattle and visited UW and Microsoft Research.
June 14, 2017
April 4, 2017
Presented User-Generated Variables at <Programming> PX workshop. It's always good to get to know new people!
March 27, 2017
Publicly demonstrated Robocam (f3js-logs) for the first time at ACM/IEEE HRI 2017, which is an accompanying tool and web-based interface to f3.js that helps recording event logs of IoT applications and authoring their demo videos.
Invited Talks
October 24, 2017
June 1, 2017
CMU HCII, Special Human-Computer Interaction Talk, "Delivering Benefits of Programming to Everyone through Programming Experience Research", Video
June 1, 2017
Dagstuhl Seminar 17232 (Computational Interactivity), "The Future of Programming and Data Sciences"
Publications
User Interfaces for Live Programming
F3.Js: A Parametric Design Tool for Physical Computing Devices for Both Interaction Designers and End-users
Proceedings of the 2017 Conference on Designing Interactive Systems, pp.1099-1110
The Future of Programming and Data Sciences
2017
Computational Interactivity (Dagstuhl Seminar 17232), pp.56-56
A Singing Instrument for Real-Time Vocal-Part Arrangement of Music and Audio Signals
2017SMC '17
Proceedings of the 14th Sound and Music Computing Conference, pp.559-563
User-Generated Variables: Streamlined Interaction Design for Feature Requests and Implementations
Companion to the First International Conference on the Art, Science and Engineering of Programming, pp.28:1-28:7
The CrossSong Puzzle: Developing a Logic Puzzle for Musical Thinking
2017CrossSong
Journal of New Music Research, pp.1-16
A Robotic Framework for Video Recording and Authoring
Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, pp.41-41
2016
Updates
December 18, 2016
Attended the First ACM SIGCHI Asian Symposium and was named as a board member for Japan ACM SIGCHI Chapter.
October 19, 2016
Served as a Student Volunteer Chair for ACM UIST 2016. Thanks all for coming to Tokyo!
August 7, 2016
Organized the Second Workshop of SIGPX.
July 16, 2016
Created a web page for the new ECOOP 2016 LIVE workshop paper that introduces "Live Tuning".
July 15, 2016
Created a web page for the new IEEE Computer journal article that introduces "Programming with Examples".
May 23, 2016
Visited VPRI, Y Combinator Research HARC, and several other places around Bay Area. Organized Japanese HCI Symposium at ACM CHI 2016.
February 27, 2016
Organized the First Workshop of SIGPX. English translation of the press release that introduces the TextAlive web service is now available.
Invited Talks
April 1, 2016
Viewpoints Research Institute and Y Combinator Research HARC, "Programming with {Examples | People}"
Publications
Live Tuning: Expanding Live Programming Benefits to Non-Programmers
Proceedings of the Second Workshop on Live Programming Systems
Programming with Examples to Develop Data-Intensive User Interfaces
IEEE Computer 49(7), pp.34-42
PlaylistPlayer: An Interface Using Multiple Criteria to Change the Playback Order of a Music Playlist
2016IUI '16
Proceedings of the 21st International Conference on Intelligent User Interfaces, pp.186-190
2015
Updates
November 20, 2015
Publicly demonstrated f3.js for the first time at ACM UIST 2015, which is an IDE to create IoT applications purely by code.
September 23, 2015
My Ph.D dissertation finally got online on the university's repository. Jun Kato, "Integrated Graphical Representations for Development of Programs with Real-world Input and Output", Mar. 2014.
July 29, 2015
Presented TextAlive Online (ver.βjs) at ICLC 2015 and visited friends around UK!
May 14, 2015
Presented TextAlive in the "Programming Environments" session and organized Japanese HCI Symposium at ACM CHI 2015.
April 11, 2015
In preparation for the CHI presentation, TextAlive is finally released as ver.βjs! Make videos within your browser.
March 20, 2015
Served as an organizing committee member for a special event named IPSJ-ONE. It gathered 19 top-notch researchers in Information Processing (computer science, information science, that sort of thing) from all over Japan, each of them giving 5-minutes talk introducing their work with their passion for research.
March 2, 2015
My paper about TextAlive, an IDE to create Kinetic Typography videos with live programming, is accepted to ACM CHI 2015. It wins a silver medal! See you in Korea :)
Awards
July 30, 2015
SMC '15 Best Paper for the co-authored CrossSong paper
April 23, 2015
Invited Talks
Publications
Form Follows Function(): An IDE to Create Laser-cut Interfaces and Microcontroller Programs from Single Code Base
Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology, pp.43-44
CrossSong Puzzle: Generating and Unscrambling Music Mash-ups with Real-time Interactivity
Proceedings of the 12nd Sound and Music Computing Conference, pp.61-67
TextAlive Online: Live Programming of Kinetic Typography Videos with Online Music
Proceedings of the First International Conference on Live Coding, pp.199-205
TextAlive: Integrated Design Environment for Kinetic Typography
Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp.3403-3412
2014
Updates
December 6, 2014
June 19, 2014
Visited the University of Toronto DGP and gave a talk. I'm also visiting the University of Manitoba HCI Lab in a few days and will give another talk. Both of these talks are about "Live Programming" in the real world, which is based on my Ph.D dissertation and discusses the direction of future work in my current position.
April 25, 2014
Serving as a program committee member for ACM UIST 2014 is exciting but difficult. Apart from it, I'm getting settled in the new environment and have just started to prepare for attending Graphics Interface 2014. I'm going to present VisionSketch in the "Video and Collaboration" session on 5/8.
April 1, 2014
Received a Ph.D degree in Computer Science and started to work as a researcher in AIST, a research organization funded by the Japanese government in which over 2000 researchers work hard to contribute to both industry and science. My primary task won't change - to conduct interesting research, write good paper, and do some other stuff to make world better place with help of information technology :)
February 25, 2014
Upgraded Bootstrap framework from v2 to v3 for better browsing experience. What a modern flat design!
February 20, 2014
Defended my dissertation! Details will come later. My paper about VisionSketch is accepted to Graphics Interface 2014.
Awards
October 31, 2014
Invited Talks
June 1, 2014
June 1, 2014
University of Toronto DGP seminar, "Toward Live Programming in the Real World"
Publications
VisionSketch: Integrated Support for Example-centric Programming of Image Processing Applications
Proceedings of the 2014 Graphics Interface Conference, pp.115-122
CapStudio: An Interactive Screencast for Visual Application Development
Extended Abstracts on Human Factors in Computing Systems, pp.1453-1458
Integrated Graphical Representations for Development of Programs with Real-world Input and Output
2014
2013
Updates
November 19, 2013
Visited St. Andrews to for ACM UIST 2013, gave a talk at University of Washington dub, met friends of the Microsoft campus, and finished the Adobe internship... I'm back in Japan!
August 12, 2013
Started internship at Adobe Creative Technologies Lab in Seattle.
August 4, 2013
I am going to attend ACM UIST 2013 ... expecting fruitful discussion at Doctoral Symposium. My paper describing the development process of OpenPool is accepted to present at IEEE GCCE 2013.
June 24, 2013
Got some sort of recognition at ACM conferences, ACM CHI and ACM PLDI, detailed in Awards section. Will work for Adobe Research Seattle this summer.
April 16, 2013
My sole-authored poster about VisionSketch is accepted to present at Student Research Competition of ACM PLDI 2013.
March 2, 2013
Collaboration with PL people turned out to be a full paper at ACM PLDI 2013. It's about live programming on TouchDevelop Web App!
Awards
June 17, 2013
April 27, 2013
Invited Talks
October 1, 2013
University of Washington dub seminar, "Integrated Graphical Representations for Programming with Real-world Input and Output", Video
Publications
Integrated Visual Representations for Programming with Real-world Input and Output
2013UIST '15 Adjunct (Doctoral Symposium)
Adjunct Proceedings of the 26th Annual ACM symposium on User Interface Software and Technology, pp.57-60
OpenPool: Community-based Prototyping of Digitally-augmented Billiard Table
Proceedings of the 2nd IEEE Global Conference on Consumer Electronics, pp.175-176
Visionsketch: Gesture-based Language for End-user Computer Vision Programming
2013Student Research Competition at PLDI '13ACM PLDI '13 Student Research Competition FinalistVisionSketch
The 34th ACM SIGPLAN Conference on Programming Language Design and Implementation
It's Alive! Continuous Feedback in UI Programming
Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp.95-104
Picode: Inline Photos Representing Posture Data in Source Code
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp.3097-3100
2012
Updates
December 13, 2012
My short paper about Picode is conditionally accepted to ACM CHI 2013. See you in Paris!
October 28, 2012
Attended 21 CCC in Tianjin, China and received this year's Microsoft Research Asia Fellowship.
October 15, 2012
Finished my talk at ACM UIST 2012 in the "Toolkits" session on 10/9. Will serve as a Publicity Co-chair at ACM UIST 2013!
September 18, 2012
Back in Japan again. The future plan includes a visit to Beijing in this month and Boston and New York in October.
July 21, 2012
My paper about an IDE named DejaVu is accepted to ACM UIST 2012. The details will come later.
June 21, 2012
Started to work with TouchDevelop team as a research intern in RiSE Group, Microsoft Research Redmond.
May 5, 2012
Back in Japan... In June, I will start an internship in Redmond.
March 23, 2012
My paper about Phybots is accepted to ACM DIS 2012. I will present the paper in the "Absolutely Fab" session on 6/13.
March 19, 2012
Renewed the website.
January 9, 2012
Started an internship at HCI Group of Microsoft Research Asia.
Awards
Publications
DejaVu: Integrated Support for Developing Interactive Camera-Based Programs
Proceedings of the 25th Annual ACM symposium on User Interface Software and Technology, pp.189-196
Phybots: A Toolkit for Making Robotic Things
Proceedings of the 9th Conference on Designing Interactive Systems, pp.248-257
2011 and before
Awards
April 2011 - March 2012
Microsoft Research 7th Collaborative Research (CORE7) Program Principal Investigator
2010
August 2009
ACM SIGGRAPH '09 Laval Virtual Award (Best of Emerging Technologies)
April 2009
ACM CHI '09 Student Research Competition 1st Place
Invited Talks
May 1, 2011
Publications
Surfboard: Keyboard with Microphone as a Low-cost Interactive Surface
Adjunct Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology, pp.387-388
A Toolkit for Easy Development of Mobile Robot Applications with Visual Markers and a Ceiling Camera
The 22nd Annual ACM Symposium on User Interface Software and Technology
CRISTAL, Control of Remotely Interfaced Systems using Touch-based Actions in Living Spaces
ACM SIGGRAPH 2009 Emerging Technologies
Multi-touch Interface for Controlling Multiple Mobile Robots
2009CHI EA '09ACM CHI '09 Student Research Competition 1st PlaceMulti-touch Interface for Controlling Multiple Mobile Robots
Extended Abstracts on Human Factors in Computing Systems, pp.3443-3448