Brad Koerner On How The Display And Experience Future Is Already Here, But Unevenly Distributed

Brad Koerner is a Harvard-trained architect who has spent decades looking at how technology affects and defines built environments. He has a specific interest in technologies like lighting and digital displays.

An American based now in beautiful Amsterdam, Koerner works with both end-users and technology companies. By his own admission, he’s obsessed by the question of how digital and interactive technologies are starting to disrupt centuries-old thinking about architectural design.

We met recently at Integrated Systems Europe, where he did a well-received talk on his ideas and observations. He later sent me the presentation deck, and it was pretty clear I needed to get him on this podcast.

In our chat, we get into a whole bunch of things – but focus quite a bit on the terms immersive and experiential … what they mean and how they get applied.

Subscribe to this podcast: iTunes * Google Play * RSS

TRANSCRIPT

Brad, thank you for joining me from Amsterdam. Can you give me a background on what you do and what Koerner Design is all about?

Brad Koerner: Yeah, thanks, Dave, for having me. It’s really an honor. So Koerner Design is my own design firm, and I focus on the future of the built environment, iSPAN, architectural lighting, digital signage, and circular economy product design.

What would be a typical engagement? If there is such a thing as typical. 

Brad Koerner: A typical engagement for me is working with lighting design companies to create sustainable products. I’ve been engaged with a few digital signage and marketing firms looking at trends in digital media. I’m also working with DC Power folks, thinking about sort of infrastructure-level improvements that help lighting and digital signage.

So a company would come to you saying, we are thinking about doing this, but we don’t have our heads wrapped around how it would all come together? 

Brad Koerner: Yeah I speak a lot. I talk about the future of the built environment through a variety of different channels, and a lot of people find inspiration in the pieces that I do. For example, I just spoke at Integrated Systems Europe on immersive digital environments, and an earlier presentation I gave was called “Every surface is a screen, now what?” 

The year before that I presented at Integrated Systems Europe, also on DC Power Systems. These videos go out there and they get people really inspired. They start to see these industries in new ways. They look at their problems with a fresh mind, and they really want to engage them in an innovation process, right? A proper design-driven innovation process. So I help them do a future envisioning session: what are the trends, what are the options, what do they have? Then we turn that into a sort of proper wishlist of product concepts or new business concepts, and then we drive it into the roadmap where it’s scoped and prioritized, and they focus on that. 

I also then take it all the way out and help ’em with product marketing and marketing communications for those new launches.

So they would come to you because you’re not selling them anything other than your insight and expertise as opposed to trying to angle them toward how they’re gonna use a fine-pitch LED wall? 

Brad Koerner: Correct. I’m agnostic when it comes to all the technologies and equipment. 

Here’s a video of Brad speaking at ISE …

You talk in your presentations a lot about immersive digital experiences and I’m very curious about how you define immersive because I just wrote the other day about a company that described a billboard along a roadway as immersive, and I thought, boy, that’s really stretching to call that immersive, but maybe I’m wrong.

Brad Koerner: I think it’s helpful for your audience to understand by background. I’m an architect. I have two degrees in architecture, and when I was young, I always wanted to be a Disney Imagineer as a kid, and that’s what drove me into architecture, and then as a side interest, I took up theater lighting and stage set design.

So I really think of immersive digital experiences from that sort of the architectural point of view where you are in physical places, you are surrounded by six surfaces and in today’s world, all of those can become digital, they can become luminous, they can become a portal to the internet or to the digital world in some form or another. I’ve said this because I cross over between architectural lighting and digital signage a lot in my work. 

Every pixel is a light source and every light source is a pixel in these modern building projects. And a lot of people still don’t quite understand that concept yet. An immersive digital experience is becoming how you design an architectural space, and I think particularly a lot of architects and interior designers are really trailing behind the technology. They look at signage as a thing that’s applied after the fact almost like a typical signage project, non-digital signage. They don’t yet understand how to take everything they’ve been taught about architecture placemaking, creating thresholds, creating progression, creating a sense of space or wonder, efficiency or economy for working environments, or branded retail experience. They don’t know how to take what they’re so good at and apply digital to it and mix digital into that and use digital to create something really engaging placemaking. That’s what I mean by immersive digital experiences. 

You say they don’t know how, but is it the case that they do want to?

Brad Koerner: Some for sure, some absolutely not. I saw Michael Schneider from Gensler speak at the Integrated Systems Europe show a few years ago, and Gensler has a whole group now that’s called the Digital Experience Design Group, and this is exactly what they’re focused on.

Gensler hired the Head of Imagineering at Disney.

Brad Koerner: Bob Weiss, right? So they get it. For every Gensler, that’s out there, There are a lot of architects that think of digital experience design as well, “Don’t put a TV on my wall that’s gonna show a Coke ad”, right? And they don’t get it right. They still think of architecture as concrete and steel and glass and like Le Corbusier’s famous quote, “It’s the magnificent play of forms bathed in light”, and I’ve inverted that many times and I’ve spoken and I said you know what happens when the forms themselves emit light and they become digital, how are you gonna design that? How do you design the element of time? 

And with the element of time, you get this sort of very active storytelling capacity within architectural placemaking. So it’s no longer enough for you to design a wall and it just sits there forever. You have to think about how that wall will change over time, right? These sorts of cycles of time, whether it’s days, weeks, seasons, hours, minutes, or whatever that is, that wall can change dynamically. So why will you change it? How will you use that for placemaking and creating engaging experiences? I don’t think most traditionally educated architects and interior designers can really get their heads around that yet. Even lighting designers have this sort of classic preset scene notion when it comes to controls. They’re struggling with getting their heads around digital media and that live data stream, live media, and sort of interactivity. 

But you seem to be suggesting that this is a matter of time as opposed to maybe it’ll happen because I keep writing and talking about how that time is coming fairly quickly when architects and people who design physical spaces are thinking about LED and projection and other technologies as design materials, as design considerations. 

Brad Koerner: Yeah. I think it’s inevitable. The best science fiction has shown this for decades now. It’s shown this amazing potential world we can live in, both the positive and the dystopian use of it like Children of Men. I just spoke in Integrated Systems Europe and I started my presentation by saying, “The future is now!” You look at Blade Runner, you look at Minority Report, you look at Star Trek, and all of those things that everybody still thinks of as like out there decades away in the future, now in fact, is decades behind us, right? And people haven’t admitted to where we are, right?

The future is already here. It’s just unevenly distributed and digital signage is definitely a world where that is super true, right? You go to the trade shows and a few years ago Sony had an 8k native-resolution digital wall that was eight meters wide and four meters tall, and it was hyperrealistic. That technology exists, but then you go to clients out there and you know they can’t afford at any budget, anything, or they simply won’t choose to do that, and I think it’s inevitable. These architects that are afraid of it, I think what happens is somebody will put a digital sign in their space whether they like it or not for other reasons, and the worst-case scenario is it does become an ad, right? And that’s not what they want in their space. So they better get their head around it and integrate it actively into their design concepts and really look at the poetics of it. How can they use simple things like beautiful motion graphics and beautiful textures?

Just like an interior designer would make a material sample board, a swatch board, they need to think of the digital media like that. What is the sort of swatches of digital media that they’re presenting to their clients when they’re designing these grand lobbies or offices or retailers or whatever?

I wonder though, with Gensler, they are an extremely well-established company with huge clients and everything else, and they work with Fortune 100s, fortune 500s, giant airports, and everything else. But there’s a whole bunch of designers that are working with like a regional insurance company or something like that, and they’re just saying, we get what you’re saying, but our customers aren’t gonna spend that money. They want a defined ROI. They don’t want something that’s just artistic and ethereal and vague in terms of what this does.

Brad Koerner: I think you’re talking about a couple of things, right? So first off, there’s just cheap, right? You’ll always have customers that can never be cheap enough, right? But you have to segment the market, right? There are always customers at the high end of the range that wants the newest, the coolest, the hottest things at the beginning of the cycle. I joke that it’s the sort of corporate lobby art budget crowd that always seems to have the money to do those sorts of fanciful things.

But the technology keeps plummeting in price, right? A lot of this technology was indeed available even 20 years ago, but it was at such a price point no one could afford it unless you’re like U2 going on a concert tour with a LED screen with the width of a football field. They could afford it but no one else. 

Or Comcast and their lobby because they were a cable company before streaming! 

Brad Koerner: Yeah, the Comcast lobby, right? What is that already 15 years ago, right?  It’s like I said, the future is here. It’s just unevenly distributed. So the price points just keep coming down until they become more and more common.

Could you have imagined even a decade ago that every little restaurant and coffee shop, and donut shop would have digital menu boards? It’s amazing how fast that swept through the market, and right now we have these sorts of virtual production spaces, right? I think it was, what, just three years ago, the Mandalorian showed sort of the first instance of that, and there was that movie First Man Before, I think was the first that used an LED screen in camera on film. Now it’s everywhere, right? Every studio around the world is installing these virtual production facilities within a year. 

The accelerating rate of technological innovation is a term that’s thrown around, and I don’t think people understand what accelerating rate means. AI image generation six months ago exploded onto the scene, and now everyone is using it every designer is thinking about how it’s gonna disrupt them, and every content producer is thinking about how they can suddenly reduce the cost of their content generation using this sort of AI image generation, or increase their margin.

That was just six months ago, so I think with the technology becoming so cheap, it’s low cost to visualize the concepts. It’s such a low cost to design, commission, and program them. The hardware is continually plummeting in costs, so you to open up new opportunities, right? The menu boards in little mom-and-pop restaurants. There will always be the high end of the market going down into the middle end of the market, and they will use these, right? And they will have very smart design teams that come up with real ROI stories for why these things work, and it becomes fanciful and sci-fi today or yesterday, tomorrow just becomes normal and accepted. People don’t even think about it anymore. The bottom end of the market will always be cheap. There’ll always be people who can never save enough money or be stingy enough. That’s in every market, right? Lighting, construction, you name it. It’s always like that.  

You’re suggesting in your presentation that the digital and physical worlds are fusing in that with physical spaces being portals to a virtual world. I’m curious about what you mean by that, and maybe you can give me a couple of examples of how that’s actually playing out.

Brad Koerner: Let me go back to when I was in school. I have a Master’s in Architecture from Harvard, and when I was there, I did a thesis titled ‘Active Object Surfaces and Zones’ I looked at using physical interactive controls for retail displays and lighting, and this was in 1999. So I was a bit ahead of the scene on that one. 

But in the early 200s, I believed that physical spaces would become the best interface to the internet which is, I know, a wild concept for many now. But you have to remember back then we were still using 20-inch Sony Trinitron screens were like the hot technology, and people were still using three-and-a-half-inch floppy discs and dial-up modems but the internet showed so much promise and there were a lot of designers doing really amazing websites and that was very spatial, right? And even just the notion of hypertext itself is very spatial. 

So I kept imagining that physical spaces and using your body as the control and creating progression and threshold and a lot of the sort of architectural principles that you see in the internet experience could be combined. But then, in 2007, Steve Jobs launched the iPhone, and little black mirrors hijacked our internet experience, right?

Now though, I think people are over that, and we’re saturated with personal devices and little black mirrors everywhere, the retailers are finally waking up to say, Hey, we need that digital in our physical experience, and so are the hospitality providers and healthcare providers. And they’re starting to think, wait for a second, now we can tie all this digital data o tour spaces, right? And we can take all these great media that we have on our little black mirrors, and we can put it into our physical spaces. We can create these great experiences, and we can complete this cycle of gathering data from the real world, using it to drive great media content creation, live and interactivity and use it to drive behavior back in the real world, right? And it completes that virtuous cycle, and that’s what I mean when I say architecture becomes a portal to the virtual world. A portal you can go back and forth between, right? The digital might come from into the space, and the spatial actions might drive digital data, right?

Can you give me some examples of where you’ve seen this applied and you think it really works because I’ve walked into some spaces that retail spaces and other spaces that are called immersive and experiential and thought to myself, being an old fart, cranky and everything that that’s nice, but I don’t see the point of this and I sometimes struggle with how they’re gonna see a return out of this? 

Brad Koerner: Yeah, I haven’t seen many. Long story short: I think you just have this great divide where you have, for example, a lot of startups doing smart buildings, right? And they’re deploying all these sensors, and they’re gathering up all this data, but then they don’t return that data back to the spaces. The data does very little to act on the physical space. Then you have all this great media content that’s out there and you’ll throw up media content on these screens, and it’s not tied to anything that’s happening in the space, right?

So it has no recognition of if somebody’s even looking at it or not, much more if that person is gazing at it or wanting to engage it. There’s been a lot of crazy stuff. There’s indoor GPS positioning using lighting systems and apps. That was a flop. People have tried to tie app experiences into the real world. Not a lot of that has any real success story. You see a lot of these sorts of art-driven installations where I call it the be in Me and My Shadow problem. You can put a stereo vision camera system in space and track people exactly, but then, all they do is show the person’s presence on some huge digital wall, and it’s like me in my shadow, and there’s no other point to it, so you have to think about why you need interaction in a space, right? I say for lighting and digital media, you can deliver the right light or the right content at the right place at the right time. You can use it to create really memorable human experiences, or you can use it to drive action, right? And those are areas that are not well explored yet, right? You don’t have a lot of good designers out there connecting all of those systems together to create genuinely good experiences. 

I actually worked with a startup called Digi Valet that makes a hotel room control system for luxury hotel rooms. So they make an app that sits on an iPad, but the other half of their system is this black box that interfaces with every physical control system in a modern hotel room like the thermostat, the blinds, the lighting, the media, everything that’s Bluetooth, the Bluetooth controlled faucet on the bathtub, the Bluetooth coffee maker, the Bluetooth perfume/scent sprayer, and all that stuff.

And it was great because they asked me to help them. This had a lot of customers, these hotel chains wanted to develop a brand of digital media and lighting experiences as part of this iPad app, right? And it was a fascinating way to think about it. So you’re in this hotel room, and you click, I want to watch a movie. It immediately says on your iPad, okay, can we set the cinema lighting? Yes. Can we lower the blinds? Yes. Would you like us to order you champagne and popcorn? Yes. 

It totally changes the way you think of the room, right? You don’t have lighting control pads and blinds, and you don’t have to find the remote control for the TV. It’s all about having this really smart butler that just knows what to do when you want to watch a movie. 

So if you’re a frequent flier or whatever, you travel between different Marriotts, and you use your loyalty card, and it just sets it up in your room. So you don’t even do anything; that’s your configuration. 

Brad Koerner: That’s the next level, right? That’s future beyond that when you can add in the CRM systems on top of that so it remembers your preferences. Then the next level beyond that is there’s almost this genie-like ability where they begin to understand your desires so well that they can start  to add magic to your experience that you are not even expecting or the hotel can’t do it at scale, right? I just think that’s fascinating, like how could you take those principles of experience design and apply them into high-end retail or high-end healthcare, or even just a commercial office environment, right? It’s a beautiful UX/UI experience in a space. We desperately need to see more intelligence and creativity around using digital in physical spaces. 

Yeah, I wanted to ask about the discipline that needs to be enforced at the start of these things. When I’ve done consulting in my dark past, I would try to ensure the first question out of my mouth that I would throw at the customer or a client was: why are you doing this? What do you want to see out of it? And so on. 

Is that the sort of thing that needs to be addressed super early so that it’s not just, “We’ve seen these big video walls and other lobbies, we want one too.” 

Brad Koerner: Usually, the first question I ask is, what’s your budget? But that doesn’t work too well.

Can you afford me? 

Brad Koerner: It’s both of those, right? It’s what’s your budget and why? I think that, first off, many of these companies have a lot more budgets if they want. They just don’t want to at first, they don’t understand what is possible, they don’t understand what it would cost, and they don’t understand the ROI on that investment. So it’s a real uphill battle, and that tail is as old as time, that’s an architect preaching an upgraded finish on the oak panels, or that’s a lighting designer preaching adding dimming into the system. It’s always like that in these construction projects, and you are right, about the why, you can have all this technology in the world, right? Anything you can dream, you can do, right? So technology is not the limiting factor. It’s imagination, right? Imagination is the limiting factor and thinking is almost like a movie director or the early stages of any media content where you have to think in storyboards, right? You have to think in moments of time. You have to think about their journey, what’s the user journey, and what’s the user experience, right?

If you’ve seen any of these big design firms, they map user journeys, right? Throughout the omnichannel retail experience, they create these huge flow charts that take up a whole wall. You have to think about that in physical places now. So if you’re walking into the shopping mall, do you put signage at the door’s threshold? Classically, in retail design, you don’t put anything really important at the threshold of the door because you need a sort of decompression zone where people charge into a space. Then they slow down, and then they look around, right? 

There’s just a lot of classic common sense design stuff that is not being employed in digital signage, particularly in any interactivity, right? You need these new combinations of skill sets that just don’t exist yet. You almost need to take a game designer with a world-class architect and make them work together and see what happens, right? You need to take a Hollywood storyboard artist and combine them with a technologist and make them work together and see what happens., and that’s what’s missing right now from all of this, and I think you have companies like Moment Factory and Gensler and some out there are on that bleeding edge that they are trying to do that. Here in Amsterdam, there’s Purple Storytelling, and there are lots of little groups that see the future that they struggle with, right?

I think they struggle to see, and get the clients to understand the potential. I think things like Unreal Engine and live rendering and that sort of starting with a game engine, which is so powerful with live rendering, is going to make visualizing these scenarios so much faster, so much more profound, instead of starting with a classic architectural sketch, and then you went to an architectural photorealistic rendering, but it didn’t move. Now architects are using things like Unreal Engine to make these animations, particularly in the luxury real estate marketing firm. Have you ever seen what some of these high-end luxury real estate developments are doing for their marketing? It’s unreal. It’s Hollywood-grade special effects from just 10 years ago, and they’re using it just to sell condos. 

You start to take the power of that, and you add it into very specific segments. So, retailers, have their very specific sort of customer flows, customer journeys, and ROI expectations, and hospitality operators have their very specific desires, healthcare facilities, have very different customer journeys. With Unreal Engine, you can now tie together these professions. It’s the first time in my career that I’ve seen this flow complete, that you can use architectural models in BIM in Unreal Engine, and you can show these scenarios. You can animate them, you can set up the interactivity, right? Cuz it’s a game engine at heart, and then you can use that for commissioning these systems. I think that will be the next step in all of this. 

But are people like architects and those who design physical spaces, are they conditioned and trained and understanding about the ROI needs of their clients? Is that something they’ve always had to address, or is this new because of this more mysterious ROI that you would see out of an immersive space? 

Brad Koerner: It’s a great question. I don’t think they are. I have two degrees in architecture. I was never trained to think of a business scenario. Again, it’s combining different skill sets, right? It’s almost like you need to combine an architect with an MBA and think about why, what’s the point? It’s a real challenge, right? Obviously, if you’re a high-end real estate developer and you’re doing luxury condos, you know that if you add marble to the lobby, you’re going to get a certain ROI. You might not have it calculated, but you understand your customers, and you understand it’s going to help with sales. You understand that it’s worth it, right? You can’t just put chipboard and cheap carpet in, you have gotta do the upgraded finishes, but you also know where not to spend the money, and you know where it’s not going to get return value to you.

And there’s an intuitive aspect to that you can never just set up in a spreadsheet, and $5,223.32 will be your ROI in 32 days. You’ll never get that precise, and that’s why you need a creative mind and a business mind, and they need to come together to figure these things out, but it will happen, right? If you create a great experience for a hospitality provider, right? They’ll know it. They’ll know it from the customer feedback, reviews, and qualitative comments on that, right? And eventually, that drives revenue for them. But those sort of attribution problems for ROI is vexing in every industry.

Marketing goes through this all the time, but it will happen more and more in physical placemaking with these systems, and I think it’s a skill. Again, people have to get good at this. It doesn’t exist now, and it’s tricky because it combines several skill sets that have never worked together in the past and you have to fuse them to sort these things. 

Yeah, I listened to a panel at Digital Signage Experience, and I believe it was somebody from Moment Factory who was saying that in terms of a return, they’re now starting to hear from the HR departments of companies who are saying that having an experiential aspect to their lobby and their overall space is incredibly important in terms of recruitment and retainment of employees these days that particularly in technology jobs where you may have several choices as to who you’re going to work for, what that space looks like and how you feel in it matters.

Brad Koerner: Yeah. It’s like in the commercial office section, right? I forget the exact numbers, but it’s $3 a square foot, $30 a square foot, and $300 a square foot, right? Three bucks are your cost of energy, and 300 is your cost of salary, right? So should you focus on saving a few pennies of energy, or should you focus on saving hundreds of dollars of efficiency for your employees and salaries? That’s just the concept that has to be employed everywhere. There’s this sort of scale of effect that is critical to ROI. Understanding that is often siloed, right? You get a salesperson running in with some smart building system. They’re talking about saving energy because we’ll turn all the lights off more. And they don’t understand that will create a lousy experience for the workers, right? And it will really damage the effectiveness of the workers and retention and all that, right? Same thing with digital signage, anything, right? If you put a big LED wall into a commercial office, will you just put a waterfall on it? Is that going to help make your employees happy? Maybe, maybe it’s as dumb as that. But could you do something more sophisticated with it? Could you recognize employee accomplishments live? Could you show employee performance live depending on what your business or industry is, do you give people a pat on the back instantaneously? There are so many scenarios that could be developed around these technologies when, again, when the surfaces you’re surrounded by become digital. You need to think about what they do, how they react to you, and how people react to those surfaces.? What is that cycle of action-reaction? 

It sounds like you’re saying there’s more to this stuff than eye candy.

Brad Koerner: Eye candy’s great. I’m not going to argue against eye candy. There’s a lot in this world that is just for eye candy’s sake, and that makes a big difference, right? This is a classic design. This is architecture, this is interior design, this is a brand design, and retail design. Some of it is just eye candy, and people know how to justify that, right? That’s a tale as old as time, right? It’s making a statement. It’s making a brand, culture, making, and experience. Why does Starbucks charge $8 for a coffee when they spend 50 cents on it?

Because they’ve invested heavily in how their stores look, they feel and smell and sound, and there’s just a lot of eye candy there, right? They consciously built all that so that they could charge that price premium. So yeah, it will just be eye candy for some of the digital stuff. I joke about the waterfalls, but can you beat the waterfall? In terms of your media content, it’s mesmerizing, right? It’s biomimetic, it makes you feel comfortable. I think humans have these deep-seated connections to natural effects. Maybe you just put a glorious force scene on your huge LED wall, and somehow the best thing you can show, right? I don’t know. It could be as dumb as that. You have to test it. 

I think the other thing people have to get savvy on is that you don’t just build it and walk away. You have to build and operate it, and these teams that are developing these concepts will have to work with the operators, whoever it is to tweak it, right? To look at, we’re going to make a whole bunch of assumptions, right? There are cycles of time, there’s media content, there’s interactivity, there are all these new things that people have to figure out. They can simulate it upfront. Nowadays, they can go into the virtual world during the construction project and get it mostly right or pretty close. But then, who will fine-tune that in the field over time or refresh it over time? Most people don’t even think of the media budget. How many people forget about, oh wait, you mean we need a media budget for all these screens we’ve built? They can’t even do that, and it’s a long way before you’re going to have clients actively spending the money to tweak this stuff and make sure it’s optimal over time. 

All right. Great conversation. I think we could have gone on for three hours, but gotta cut it off at some point. If people want to find out more about your company or perhaps bring you out to speak to their company or a conference, where do they find you online? 

Brad Koerner: They can find me on LinkedIn just Brad Koerner or KoernerDesign.com. 

All right. Thank you very much for spending some time with me.

Brad Koerner: Great. Thanks, Dave.