Tokyo Lab Pushes Mind-Wobbling Possibilities For Projection Mapping

October 20, 2016 by Dave Haynes


The big problem with projection mapping – apart from high costs and weather – was for quite some the technical challenge of aligning the projector or blended projectors precisely to a target projection surface.

It got harder if the surface wasn’t flat.

Software has fixed a lot of that, particularly projection-mapping software coming from companies like Christie.

But what the stuff now on the market can do is nothing compared to this rather mind-blowing proof concept technology, which comes out of Tokyo. This is dynamic projection mapping that conforms and rapidly adapts to moving shapes, and can change visuals on the fly, based on what’s in front of the projector.

This is the summary of a project from a lab at the University of Tokyo, called: Dynamic projection mapping onto deforming non-rigid surface using a high-speed projector

The video demo is the best way to get a sense of this:

Here’s the written summary for the propeller-head crowd:

Projection mapping attracts attentions as an emerging technology to extend the real world. However, almost realized examples have been limited to static or quasi-static environments. This research aims at overcoming this limitation and realizes dynamic projection mapping in which dynamically-changing real-world and virtual visual information are completely merged in the level of human visual perception. This high-speed dynamic projection mapping requires a high-speed projector enabling high-frame-rate and low-latency projection. In order to meet this demand, we have developed a high-speed projector “DynaFlash” that can project 8-bit images up to 1,000fps with 3ms delay.

In particular, as a challenging target for the dynamic projection mapping, we focus on a non-rigid surface. Sensing of non-rigid surface deformation is difficult to be achieved at high speed because it has high degrees-of-freedom and involves self-occlusions as well as external occlusions. Our newly proposed method overcomes this limitation. Our method can obtain the deformation robustly at 1,000 fps by using an originally proposed marker “Deformable Dot Cluster Marker”, even when the target causes large deformation and occlusions.

Using these base technologies including DynaFlash and Deformable Dot Cluster Marker, we realize a new dynamic projection mapping onto deforming non-rigid surface. In this demonstration, by drawing the marker on the target with IR ink, we allow the marker to be invisible to human and enable robust sensing independently of the projected images. In our technology, both the projection and sensing are operated at a speed of 1,000 fps. Therefore, it is possible to keep the projection consistent with the deformation and extend the real world as if the projected image is printed or existed as an original (digital) texture on the target. Especially, focusing on new paradigms in the field of user interface and fashion, we have demonstrated dynamic projection mapping onto a deformed sheet of paper and T-shirt. Also we show that projection to multiple targets can be controlled flexibly by using multiple markers.

This could be pretty interesting, when and if commercialized, in specialty retail settings and museums, and all kinds of things I’m smart enough to dream up.

Hat Tip to Jared Ficklin for flagging this:

Leave a comment