Todd Eckert: Mixed Reality and the Human Experience

CHARLIE MELCHER:

Hi, I’m Charlie Melcher, founder of the Future of StoryTelling. Welcome to the FoST Podcast. On today’s episode, I’m joined by artist and innovator, Todd Eckert, President and CEO of Tin Drum, a studio creating original work for mixed reality. Mixed reality is still unfamiliar to many, but Todd has spent years exploring its potential as a storytelling medium, blending the physical and digital worlds. His work includes The Life with Marina Abramovic, Kagami with Ryuichi Sakamoto, and most recently An Ark, a piece now at The Shed here in New York, which I was lucky enough to experience myself. In our conversation, we explore how An Ark uses mixed reality to forge a relationship between audience and performer, balancing the personal with the universal. Todd shares his philosophy on storytelling and why it’s worth wrestling with imperfect, emerging technologies in service of deeper human connection. Please join me in welcoming Todd Eckert to the FoST Podcast.

 

Todd, welcome to the Future of Storytelling Podcast. Such a delight to have you here.

TODD ECKERT:

I’m very, very happy to be here.

CHARLIE MELCHER:

So congratulations on An Ark. I was so honored to be at the premiere last night at The Shed. It was amazing.

TODD ECKERT:

Thank you. It’s an extraordinary thing to make something that takes years. And on that night that you’re opening it, you’re trying to make sense of the fact that it’s even happening.

CHARLIE MELCHER:

Well, it’s such a unique format. Could you start by telling us, for people who haven’t had the pleasure of experiencing it yet, what the experience is and how special it is with mixed reality?

TODD ECKERT:

Well, mixed reality is like any term in an up and coming medium. It’s a little bit difficult to define. The way I define it is that you have a device, transparent lenses that you see through, you see each other, you see the room. A dimensional recording of four actors is presented as part of the world. So it’s literally a seamless connection between the recording of, in this case, the play and the world around you. And it’s a scenario in which your agency has meaning, which is not particularly common in our current media. Todd, explain what

CHARLIE MELCHER:

The experience of an arc is for those who haven’t done it yet.

TODD ECKERT:

So you arrive on the second floor of the shed, the gallery level, and you go through a long haul. And at the end of it, you are met with an attendant. If you wear glasses, you’ll be asked to get your glasses checked so that we know what your prescription is, because you will be fitted with corrective lenses inside. And then everyone is asked to remove their shoes and place them in a cubby. And you go into a large gray, draped, red floored space. You sit down and there’s a device on each of the chairs and you put the device on and you see four chairs in front of you. And there are rituals and protocols and things that you go through. And then when the show starts, the four actors, in this case, Ian McKellen, Golda Rochevel, Lorenze Kenney and Rosie Sheehy come in one by one and they sit down and they are looking you right in the eyes.

So you can see the room, you can see each other, but the actors are speaking directly to you. And you go through a play in which every facet of life and death is presented to you essentially conversationally. It is a play, but it’s in a format that nobody has ever done before.

CHARLIE MELCHER:

I’m a huge believer in having agency in storytelling and also a huge believer in mixed reality. I’m so excited about this opportunity to layer the real world with the digital world. And by the way, I thought that was done beautifully in the piece. The intimacy with the actors, they felt really like they were in the room with me. You forget after a while that they’re not really there. I’m curious about this piece in this medium. And did you start with, I want to create a play in mixed reality? Did you start with a story and find that this was the right technology to convey it? How did that come about?

TODD ECKERT:

I didn’t feel like creating a scenario that reproduced what we understand in any kind of live theater proscenium arch or in the round or whatever, but I did want to see if it was possible to create a new kind of exchange between artists and audience. And ultimately, the idea of sitting in close proximity, literally human intimacy as a mechanism for a deeper artistic exchange was what we landed upon. And

CHARLIE MELCHER:

Certainly the eye contact that you make with the actors that they make with you, it’s so intense. It does feel so personal. You really feel like you’re seen, they’re talking to you. And I use this phrase a lot, but intimacy at scale. I felt that intimacy with those actors. So I do think that’s something very unique about the piece that you can’t get in a traditional theater situation.

TODD ECKERT:

Honestly, what’s funny about it, and I’m really glad that you felt that. The idea that you would present something like this that is so personal, but would do it in a room with a bunch of other people. I’ve had a lot of people come to me and say, “Well, why did you do it like that? ” And it’s actually very intentional because if you look at how the world works right now, the fact that we are so reliant for information of every variety on digital platforms that have no permanence at all. So in some ways, creating this relationship with the public that says, “All right, you’re having something incredibly personal, but that person that you see across the room or next to you or whatever, they are also having a personal reaction to the same material, which is meant on a sort of subliminal level to remind you that we as human beings actually have quite a bit more in common than we do as differences.” And this felt like something that we wanted to present without being didactic about it.

CHARLIE MELCHER:

I have to say, I wasn’t quite sure why I was in the room with all those other people because I didn’t really have any interaction with them. It’s interesting to think about, would it have been different for me if I had been alone in a room experiencing it, but I’m not sure if it really affected the power of the experience. And for me, that had to do with the eye contact with the actors and the very powerfully written piece. It had you obviously thinking about your own mortality intentionally, those characters and their words that they shared reminded me a little bit of Our Town, I guess, and a touch, being able to sort of look back on life in a way. So let me ask, where did the name of the piece come from? Where does an arc come from?

TODD ECKERT:

It’s literally, Simon had this idea that the biblical reference of we have survival from one catastrophe and coming out the other side, and these are the elements with which the world would be saved, and the world is you, and you are the survivor, at least in this story.

CHARLIE MELCHER:

Tell us a little bit about the story and the writing and how that came about. It was so strong.

TODD ECKERT:

Every part of the presentation has meaning. Our intention with this piece was to give it a kind of gravity and a dignity. You’re asked to take off your shoes because we come into this world and we leave this world without our shoes, and the four characters are barefoot. Going barefoot in January in the city was a little bit too much. So we said, “Okay, wear your socks. It’s fine.” So it’s the idea of a sacred space, but it’s non-secular. And that’s meant to say that human life, which is going to be described, debated, presented to you as both intimately yours, but also everybody’s. That life has incredible dignity unto itself. This is the great leveler. Everybody has the same value. How do you take these four characters with whom hopefully everybody will identify in some way or another and have them speak in both universalities, but also precision.

The actors all really felt as if they owned the individual rules and they don’t even have names in the script. They’re A, B, C, and D. But that holy crap moment, she’s talking about me, happens throughout it. That’s the whole point.

CHARLIE MELCHER:

No, there’s no question that it was both very specific and universal at the same time. And you at times related to it, and they said something that was as if they knew you. And at other times they were saying something that wasn’t you, but you realized was somehow everyone else or many other people. I mean, the other was, of course, just that sense that maybe these were ghosts, that they were there as abaritions. It says things we weren’t really seeing because we could see through them kind of, and they had a kind of wisdom to be able to look back at time as if they were speaking to us from perhaps the grave or the afterlife. Was that intentional or am I reading something in? Obviously we can all read whatever we want in, but …

TODD ECKERT:

No, it’s completely intentional. You nailed it because they are kind of ghosts, but they’re also angels, they’re kind of guardians. Pretty much every consideration of the afterlife or religion pays some attention to what happens to us after we die, and it’s terrifying. In some cultures, it’s liberating.

CHARLIE MELCHER:

I certainly felt like the combination of their being able to look at me in the eye and the ephemeral or — ethereal — ethereal is the word I’m looking for — nature of the characters who were there, but not there, were a beautiful use of mixed reality. Those components felt like something that I couldn’t get from a film, I couldn’t get from a traditional VR experience. Am I touching on things that resonate for you about your choice of doing this in mixed reality? Are there other things about the piece that you feel were unique to this medium?

TODD ECKERT:

Well, mixed reality is, I mean, let’s face it, it’s a pain in the ass. One device will do one thing really well, but other things poorly. Another device will do that first thing pretty poorly, but some other things better. Nothing does everything well. But if we look at film, I mean, film starts off and it has no color and it has no sound and it still transforms the way that we understand the world. Then it gets sound. Wow. Then it gets color. Get out of here. And truly, we would be completely different animals had the advent of film not happened. With mixed reality, I probably sound like a zealot sometimes because I am really interested in what it can do. And my company Tin Drum, we’ve described it as failing successfully for years, but our goals are really to provide experiences that are not possible in any other way because we think life is worth it.

And ultimately, it was never about the device in the same way that television was never about the box. Film was never about the projector.

CHARLIE MELCHER:

It’s such an interesting dance that takes place between the technology, the development of the hardware, the content creators creating something meaningful and worthwhile and of value. And then this education of the consumer to feel comfortable with what’s new and unique about this medium. We’re still a ways, I guess, from mixed reality sort of breaking through and having all three of those kind of grow up and mature together to the point where it becomes a mass consumer experience or product.

TODD ECKERT:

Dina, who was a particular motivator going through the process of making An Ark was Sergei Diaghilev. I don’t know if you are familiar with him. He kind of revived ballet as an art form. So he’s this Russian guy and he goes to Paris and he loves ballet, and this is early 20th century and he starts the ballet ruse. At that time, ballet was kind of stoic and stayed and predictable, and there was nobody making really great art out of it. And it was a kind of the law of diminishing returns was in full effect. There were old people that were going to it and then forget it. It’s just not going to happen anymore. But he was thrilled with the potential of what it could be. So he pays these young, insane, really talented people to completely revitalize and change the way ballets understood. And it was not because Diagheliv was jumping into ballet because it was so popular, it was because it was so financially viable.

He was doing it because he had this crazy passion for it and believed it could express something that nothing else could. And it turned out he was right. He changed whether we know it or not. He changed how we all hear the world. And when this project got really hard, which was often, I thought of this guy who didn’t give up and I’ve chosen not to either.

CHARLIE MELCHER:

We need people who are passionate and committed to make these things to find their form, to ultimately get mature enough to attract audiences. So seriously, applauding what you do and your stubbornness for it.

TODD ECKERT:

Well, I mean, thank you. But even I would ask you, storytelling, it almost, even by presenting it with using that word, it hearkens back to something that is as highfalutin or as primitive as you want it to be. That term is also elemental to the human spirit. So what makes you focus upon that thing as opposed to something more obviously futuristic?

CHARLIE MELCHER:

I am a believer that stories are the programming language for the human species. It’s literally how we kind of learn almost everything and understand who we are in our place in the world. So it is fundamental. It’s hardwired in us. I think that was through evolution. I think people who didn’t appreciate or listen to stories walked by the cave and got eaten by the bear and the ones who heard the story that was like, “Hey, there’s a scary bear.” And that they didn’t go. They listened and they had kids. And so we got programmed to listen to stories and realize that they have important information. They have literally survival information encoded in them. So I think it’s hard to separate who we are from storytelling. And again, that’s because we hear these stories from our childhood, they’re encoded in our genes and they’re how we remember things.

I mean, there’s plenty of science backing that up. Stories are what we remember, facts and figures come and go very easily. Very few people are actually swayed by data. They’re swayed by the power of a good story.

TODD ECKERT:

But I’ve never thought about it as an operating system, but I think you’re right. In fact, I have a friend who is a very important rapper from Africa, but he believes that the operating system that connects us, that we are eternal and that our bodies are passed on in the same way that our data goes from one phone to another phone to another phone to another phone. But the internal OS of what we understand, that goes along with us forever.

CHARLIE MELCHER:

I also am just so fascinated by people who are trying to use new technologies to tell stories in new ways. I think that the fundamental things that move us are part of those archetypal elements of stories, the characters, the narratives, the villains and the heroes and all of that, but they can be manifest in very different ways depending on the medium that we’re telling them in. And different media can allow for us to experience it in different ways. And certainly one of the things I talk about in my recent book is just that we went from an age when stories were embodied and social and participatory, where we had real agency to a time when stories became fixed and linear and flat and frankly sort of antisocial and disembodied. So I think that the kinds of stories that mixed reality, augmented reality, immersive entertainment in analog spaces like theater, immersive theater or immersive art, all of this is a movement to bringing us back into our stories, back into living our stories.

In fact, I refer to them as living stories. So I applaud when someone like yourself stays doggedly to invention of a new medium and particularly one where you’re talking about things like embodiment, we’re taking our shoes off, we’re free to look wherever we want, we’re experiencing it in a social setting. In fact, I would argue for you to think about even more agency as you move forward. But anyway, I’m so excited about the kind of work that you and others are doing because I believe that you are creating living stories that are much more powerful and organically human.

TODD ECKERT:

Well, and I mean, thank you. And the idea that it would be ultimately organic and hopefully transcend whatever the technical limitations are. I don’t know if you noticed last night, but when you walk in, there is an olfactory component. There’s a scent that was developed with this guy, Ogata, who’s a designer, and he owns tea houses and restaurants in Paris and Japan. And it wasn’t meant to hit you over the head. It was just meant to be this little, “Oh, wow, what was that kind of recognition?” And it’s through means other than something that’ll be spoken or something that is visual. It’s meant to kind of separate you from whatever you are processing from the day. Even something like the pile of the carpet was selected so that it would have a pleasing feeling underfoot. But as you were talking about, maybe the next one is more immersive — s o the next project that I’m doing does utilize the idea that a sense of place, almost like time should move as if it was a prop, and it needn’t be somebody coming up with a card saying, “Now you’re in the future, or now you’re in the past.” But if you take people who are incredibly good at creating a sense of place and of time, that that could literally just waft past you. And it’s a very ambitious project and I’m really excited about it.

CHARLIE MELCHER:

Can’t wait, Todd, that sounds amazing. I’m curious to know what you’ve wanted to do with mixed reality that you haven’t been able to yet, or what you think are some of the attributes of the medium that are still to be explored.

TODD ECKERT:

I would very much like to go from this limited stage into something that was larger and more fluid. Let’s say you want to do Nick Cave and the Bad Seeds. So because Nick Cave’s going to be climbing all over the place, you can either do him in his kind of quiet Leonard Cohen version, or you can do him as a truncated version of what he normally does. But I want it to be large enough that I can get the real Nick cave because he’s important enough that we should get him. So I think the scale of what we can do is the greatest limiting factor. When you think about scale, we’ve had a project, we’ve been developing it for a while, which is Frank Lloyd Wright’s Falling Water. And how do you create this really iconic house without it being VR? It’s really funny, when you do tests of physical things in this medium, you find that it is the irregularities that actually pronounce the reality of whatever it is that you’ve made.

And so we will need to have giant stores of hard drives to make this, but we will.

CHARLIE MELCHER:

Well, Todd, well, I really appreciate how you come at this from the soul of an artist and a storyteller. A lot of people with new technology come at it to show off the features of the tech and they miss the importance of creating things that resonate on a very deep and emotional level for people like stories do. And so you being somebody who clearly has that desire to help expand our understanding of the human condition and have people be moved, it’s so important and wonderful that you’ve chosen to devote your energies to these new trouble technologies. I mean, they’re amazing, but when they’re early and young, they’re hard to wrestle and solve and be just purely in service of what you want them to do. So I so appreciate your continuing to wrestle mixed reality to the service of artistic expression and storytelling.

TODD ECKERT:

Well, thanks a lot. It’s hard and it’s worth it. I was actually, I was thinking about it during my workout this morning. What’s the last story and any medium that you encountered that you loved? Because this is something that we trust more than anything else, recommendations of people that we admire. So I admire your relationship with stories and would therefore wish to ask you — because now that I’ve got this thing open, I can actually watch or read or do something. So I want to know what is the next thing that I’m watching or reading or doing.

CHARLIE MELCHER:

One of the areas that I’ve been exploring are out of a tradition of either escape rooms or video games, but they’re evolving from those expressions. So there’s a wonderful escape room in Amsterdam — it’s called The Alchemist from the Sherlocked Company, and it’s beautiful and totally immersive. And you’re living through a story in a delicately built jewel box that reveals layers as you move through it. And you have to do it with friends. You can’t do it alone. You have to have, I think, minimum three or four people to be able to play through. Another was a piece that I did with some friends in Belgium called The Demise of the Gricers, which was more of a — it was almost like a video game that you lived in. And it was extraordinary. You were dressed up in these train workers outfits and with knee pads and a special tech device and a helmet.

The story takes place in a train yard. So you are actually literally playing through these old trains and they have incredible lights and music. And it’s a little bit of a horror story, but not really, but enough to get your heart racing and so visceral, so real. Those are a couple of examples from the last six months that blew my mind and are unlike any of the kind of stories that I grew up with.

TODD ECKERT:

But that’s brilliant. That’s even better than I could have hoped for. So thank you.

CHARLIE MELCHER:

Well, I hope that many hundreds of thousands of people will go see An Ark and experience it. And I very much appreciate your joining us on the podcast today, Todd. It’s such a pleasure to spend time together.

TODD ECKERT:

Ah, it was so much fun.

CHARLIE MELCHER:

Once again, I’m Charlie Melcher, and this has been The FoST Podcast. Thank you to Todd Eckert for joining me today. If you enjoyed this episode, please check out my new book entitled, The Future of Storytelling: How Immersive Experiences Are Transforming Our World. It features nearly 50 of the most groundbreaking, immersive experiences and celebrates the companies and people who’ve created them. It also includes my insights into this emerging field. The book is available in stores and online wherever books are sold. The FoST Podcast is produced by Melcher Media in collaboration with our talented production partners, Charts & Leisure. I hope to see you again soon for another deep dive into the world of storytelling. Until then, please be safe, stay strong and story on.