Two Technology “Megatrends” That Will Change Health Care

“I invite you into my brain,” Paul Kruszewski, PhD, said to employers, benefits leaders, and consultants attending the Movement 2023 conference in Chicago. “We’re going to blast into the future.”

Dr. Kruszewski, Hinge Health’s vice president of deep technology, has spent his whole career living and breathing the what-comes-next realm of the tech space. He’s been at the intersection of artificial intelligence, computer graphics, and 3D simulation, and worked on everything from movie special effects to video games to military simulations to amusement park features.

And he’s “never been as excited and, more importantly, as inspired” as he is right now.

Why? Two of what he calls “megatrends”—digital twins and 3D avatars.

Watch the video of Dr. Kruszewski’s Movement 2023 presentation below, or read on for the full transcript, to find out why he believes these technologies are poised to make health care more personalized, more effective, and more fun.

And consider joining us to learn about the next chapter in cutting edge healthcare technology at Movement 2024.

Follow us on Facebook, Instagram, and LinkedIn for conference updates.

Human-centric technology

Who has heard of Chat GPT or generative AI? You might be surprised. I'm actually not going to talk that much about that today. And I'll tell you why.

I want to focus on what people are not focusing on, and that’s how we use these technologies in a human-centric way, because it's all about living happier and healthier lives.

I'm a serial entrepreneur. I've started and sold three AI companies. Two years ago, Dan Perez, CEO of Hinge Health, came to me and gave me his vision of how computer vision could help all of his members and impact their lives.

And it was a very powerful mission. Again, I've worked on video games and all kinds of cool, fun things, but this notion of helping people live healthier lives really resonated with me and the rest of our team.

And so we decided to join Hinge Health. And 18 months later, I'm focused on scouring the Earth for the deepest technology that is there and figuring out how we bring all this advanced technology into Hinge Health so that we can bring it out to you and really move the needle in your lives.

Today at Hinge Health, computer vision is really the jewel in the crown when it comes to innovation.

Dr. Kruszewski goes on to explain computer vision and its use at Hinge Health:

What's a digital twin and why does it matter?

A digital twin is simply a virtual copy of something in the physical world. 

Has anyone seen the movie “Apollo 13” with Tom Hanks? The movie came out in 1995, but the actual mission was in 1970.

Fifty-six hours into the Apollo 13 mission, 200,000 miles away from home, there was an oxygen tank explosion. This led to the famous Tom Hanks quote in the movie, “Houston, we have a problem.”

Why am I talking about this? Because NASA is the OG of digital twins.

The astronauts were trained on digital twins. In order to build the rockets and train the staff, NASA had to build a digital twin in the computer—in the 60s.

When the explosion happened, they reprogramed the digital twin on the PDP-11 back in Mission Control, proved what the solution was, then radioed it up to the astronauts. The astronauts did what they were told to do and the mission was saved.

So, I would say digital twins have literally been saving lives since 1970.

Now let's talk about what a use case for a digital twin would be in MSK (musculoskeletal care), our industry:

Why will the use of 3D models in health care be valuable?

It is all about ROI. At the end of the day, we have to make people healthier.

We're working on a very powerful computer vision system back in Montreal. If you came to the Movement conference last year, Dan Perez (Hinge Health’s co-founder and CEO) gave you a sneak peek of what this is.

A super cool technical innovation; we scientists love this. But again, how does this help? Because that's ultimately what this is about.

Well, just think about the use cases. What does this open up? In a certain sense, we've got a 360-degree mirror around you. Even if we're scanning you with one lens, we've got a 360-view.

I’ll give you an example. We all know squats are good for us, right? I am always doing my squats and feel like I’m doing them well.

But I think it would be really amazing if after a session of squats, the Hinge Health app said, “Hey, that’s awesome, Paul, but you’ve got to go a little [more].” And what if there was a video feed that came back and showed me from the other angle, highlighting [where I need to make adjustments].

By understanding the whole body, we're going to be able to treat the person in a more precise way and give more refined exercises to optimize the results.

But let's really go out there. Because this is supposed to be about the future:

Digital twins vs. 3D avatars

A 3D avatar is a digital persona of an intelligent entity in the digital world. It sounds like a digital twin, but digital twins are about digitizing the human so that we can run a simulation. Avatars are ways for us humans to interact with digital entities, if you will.

If you've ever used Alexa or Siri, you're already working with an avatar. You're not talking to anybody. You're actually talking to AI, but the AI is embodied by audio. That's an audio-only avatar.

With ChatGPT, you're typing into the command prompt and having a conversation. It feels like you're having a text conversation. The ChatGPT interface is a text-based avatar. It’s an interactive conversation.

Minecraft, Fortnite—any time you see a 3D character in a video game, that's a 3D avatar. When you're playing, you're controlling an avatar. But the vast majority of the 3D characters that you see on screen are controlled by AI. And that's what I want to talk about next.

In health care, I believe that avatars can do a lot of the frontline mechanical, repetitive work. And that gets back to the point about how we can enable technology to make health care cheaper.

Avatars should be believable. You should be able to converse with this 3D character, but not photorealistic. It should behave like a human, but clearly it's not a human. You have the visual cues to be comforted—to know that there is an AI system behind.

Because I don't know about you, but I really hate when I'm on a chat for customer service and I don’t know if it’s a bot. I’m OK if it is, but I need to know and don’t like to be faked.

A Hinge Health Use Case

Now, let’s imagine I’m an employee of a company that just started offering Hinge Health.

I’m in the lunchroom and see a poster that talks about the program.

Imagine if I were able to take out my phone, scan a QR code on the poster, and the browser opens up and connects me with a humanoid character. It’s pleasant, but I know that it’s not a real human being. Maybe it has green skin or something. Let’s call it Grace.

After asking my name, a conversation might go something like this:

Grace: How can I help you today, Paul.

Me: Well, I really like swimming, but I have shoulder mobility issues.

Grace: That’s too bad. Can you show me where it hurts?

[I point to the area of my shoulder and she mimics me.]

Grace: I see. Put your arm up like this. [She models the movement.]

[I put my arm up and stretch a little more.]

You can imagine this conversation going on for a couple of minutes. And then at the end she says, "OK, I understand. Check your email. There will be a link to download the Hinge Health app. It will have a playlist you can follow. Try that and we’ll talk soon."

That’s how I see avatars helping us. They will be these digital assistants. But it’s more than just voice—it’s a full 3D embodiment because we’re 3D creatures and we need to interact with AI in a 3D way.

Here, Dr. Kruszewski talks about the potential for avatars like Grace:

This avatar, which I codenamed Grace, she will work for you. She will know you. She sees you. She listens to you.

So, I see a world in the not-too-distant future where we are going to be living really great, healthy lives with an entire universe of healthcare avatars—not just the Hinge Health one, but those for other modalities. 

This transcript was lightly edited for clarity.