Proto, Amazon Web Services Collaborate On Text To Hologram(ish) Displays For Retail

July 11, 2024 by Dave Haynes

Are there many retailers who will spend $50K+ USD for a giant LCD that shows products floating around on a transparent screen?

I don’t think so, but some people in the cloud computing side of online retailing behemoth Amazon see something I just maybe don’t.

Amazon Web Services and LA-based Proto have collaborated on what is described as “a cutting-edge text-to-hologram pipeline that empowers users to generate dynamic, immersive content for Proto’s state-of-the-art holographic displays with just a few lines of text.”

In short, various components of the AWS technology stack are used to generate product videos from AI-based prompts, and the output is shown on Proto’s transparent LCDs, which are marketed as holograms but are more accurately hologram-ish. It’s just a flat image that appears to float, because the display interior is a big white case and shadows have been designed into the Stable Diffusion AI-generated images that are then made into a rendered video.

The description of all this, on the AWS site, borders on breathless. Example: In today’s digital landscape, where innovation is the driving force behind technological advancements, we find ourselves on the cusp of a paradigm shift.

If I was 16 again, you’d see me eyes rolling up in mock disgust.

But wait, there’s more …

This revolutionary technology is poised to redefine creative expression, communication, and visualization across a wide range of industries. Imagine artists effortlessly sculpting holographic masterpieces with the mere stroke of a keyboard, designers bringing their concepts to life in breathtaking three-dimensional form, and entrepreneurs captivating audiences with immersive presentations that transcend the boundaries of traditional mediums.

To be clear, this use of cloud computing and AI to generate images is impressive and interesting on its own, and the AWS side of this goes into DEEP detail on how it was done, and the output, in an AWS blog post.

This text-to-hologram solution is built on a robust and scalable serverless architecture powered by Amazon Web Services (AWS). At the core of this solution lies an AWS Step Functions state machine, orchestrating a seamless workflow that harnesses the power of cutting-edge generative AI technologies.

The creation pipeline begins with Amazon Cognito, a secure user authentication and authorization service, ensuring that only authorized users can access and interact with the text-to-hologram pipeline. User input is then passed through REST API hosted by Amazon API Gateway, a fully managed service that acts as the entry point for our application. Once the user’s text input is received, this solution uses AWS Lambda to trigger an AWS Step Functions state machine.

The state machine uses AWS Lambda states to synchronously orchestrate the various steps require to generate holographic visuals from text. The state machine begins by leveraging the Amazon Titan Image Generator model, a state-of-the-art AI model trained on a vast corpus of images and available through Amazon Bedrock, to create stunning visual representations based on the user’s textual input.

This appears to be the prompt used: A dozen brown leather handbags floating in air against a white studio backdrop.

Next, the generated image is sent to Stability AI’s Stable Video Diffusion model using the Stability AI Developer Platform API. This model takes the static image and transforms it into a captivating, dynamic video, bringing the user’s vision to life with mesmerizing motion and detail.

There is a motion gif of this on the original blog post here. But 8 MB and it would not animate on my site.

Because this solution was built with AWS serverless services, data handled by this solution remains encrypted, whether at rest or in transit. By default, Amazon S3 and Amazon DynamoDB employ Amazon managed keys for encrypting data at rest. Optionally, if your organization has specific requirements around data encryption, you can supply AWS Key Management Service (KMS) customer managed keys. AWS Step Functions uses Transport Layer Security (TLS) for communication between integrated services. As a result, when data is in transit between AWS Lambda states of the state machine, it remains encrypted and secure.

Now that an image has been generated from text input and a video has been generated from the image, the last step is to convert the video to holographic content using Proto’s technology. In the final step of this solution the video is uploaded to the Proto Hologram Content Management System, where the content becomes viewable and spatial on a Proto holographic display.

Not convinced the video is converted, because this thing is not holographic. But perhaps submitted video files get run through Proto’s software. This part of the prompt – floating in air against a white studio backdrop – does the shadow job normally done by a Proto light and camera capture set-up.

Transparent LCD has been around for many years, and manufacturers have struggled to find scalable applications. It can be great, done well, but also very expensive and not something many retailers will willingly invest in. Proto started out as Portl and was the first company to develop a solution that paired custom transparent LCD enclosures with amplified edge lighting, with clever lighting and chromakey video capture. Several follow-on firms, perhaps “inspired” by Proto, have since popped up with competing products.

I like this tech for certain applications, especially remote appearances that allow someone to address a crowd and interact in real-time, while in a studio across the city or across the country. Remote teaching and tele-medicine are also interesting, though also expensive. The transparent LCD and video capture tech has also been marketed as a way to show floating, rotating NFTs (why???) and floating, rotating product images (again, why?).

There is probably a bit of special sauce from Proto’s end, but really the display and player are just end-points and this could run on any screen.

The other puzzling aspect of this is the use of image generation, as in pretend products based on data in an AI large language model. Image and video generation using AI is amazing, but retailers sell finished goods, not generated ideas of goods.

The floating handbags thing is, of course, just a conceptual representation of what’s possible. This whole thing would perhaps be more compelling and persuasive if it showed a pragmatic real-world application with real products, branding and calls to action.

There will be some money-is-no-barrier store owner in Dubai, Qatar, Riyadh or maybe Las Vegas who would invest in this sort of thing for the whizbang, wow factor effect,  but I don’t think you’re going to see these at any kind of scale.

In many respects, what’s genuinely interesting here is how far (and quickly) AI-based image and video generation is advancing. Creative people like designers are always going to be valuable for coming up with the effective ideas and concept visuals, but increasingly, the machines can very quickly do the laborious work to produce static and motion files.