How iClone helped a Dutch studio transform a mo-capped dancer into a 3D character

Founded in 2016 and based in Arnhem, Gelderland, Het Nieuwe Kader is an innovative media studio with one of the largest green-screen film and photo studios in the Netherlands. With a passion for interactive games and apps, virtual production and motion capture, its dedicated team of over a dozen professionals delivers top-flight 3D and animation content to clients, including Karl Lagerfeld and Cara Delevingne.

One of its long-standing clients is Introdans, a prestigious Dutch dance company with a rich heritage of seamlessly blending classical ballet, modern dance and contemporary choreography. With a history spanning over five decades, the dance company recently decided to revitalise its website in a bid to recruit new dance members.

To help them accomplish this, Het Nieuwe Kader used motion capture technology on one of their accomplished dancers, Alberto Villanueva, infusing a contemporary vibe into their time-honoured tradition.

"I'd been pushing Intradans to look into motion capture for a long time," recalls owner and 3D/VFX artist Niels Bosch. "After all, dance is pretty much the coolest movement you can do as a human body. And since we have a very nice motion capture hardware setup and pipeline, we can make it look really good.

"Also, the new web platform was aiming to get children and younger people excited for dance," he adds. "So we thought: since young people don't like listening to adults, let's try convincing them with a 3D character!"

Read on to learn how they went about it and how the latest features in iClone helped smooth their workflow and keep the project on track.

Motion capture and import

The team invited Alberto into the studio and captured his movements using a motion-capture suit from Xsens and the Quantum Metagloves from Manus. During the session, they documented a variety of dances.

The shoot itself went without a hitch. "Alberto, he's the perfect dancer, the most complete dancer I've ever worked with," says Niels. "Whatever we threw at him, he did it beautifully, even with all the wires and sensors attached to him."

The team then shared the footage with the client. "We created video previews of all the material, all the data, and then Introdans made their selection of the sequences they wanted to use," explains Niels.

The next step was to use iClone to translate the scenes they'd shot onto their 3D character. Interestingly, the studio had only recently started using iClone, because the software hadn't previously been on their radar.

"But now that we've discovered iClone, I love it," Niels smiles. "It makes it so simple to retarget motion capture footage. Setting up a timeline and editing the Mo-cap data is easy. There are tonnes of easy-to-use libraries. And you can easily link everything together, such as when you have a face-capture mechanism."

To edit the animation, Niels collaborated with his colleague Tim de Haan, who referenced Reallusion's free iClone motion editing tutorials to supplement his industry expertise. And the latest features in iClone went a long way to optimising the pipeline, saving them a lot of time and effort on the projects.

Designing the character

The 3D character they were mapping the mo-cap data onto had previously been designed using Reallusion's Character Creator 4. And this character didn't actually look much like Alberto. This wasn't a mistake but by design.

"Introdans had requested that the character have a unisex appearance, so you couldn't tell if it's a boy or girl," Niels explains. That wasn't Alberto's look, but that didn't matter. He was the best dancer for the job, so that was important: everything else could be taken care of in the software.

"With a straightforward process, we seamlessly transferred the character to iClone," explains Niels. "Thanks to the integrated motion profiles, importing the Xsens data into iClone was a breeze. However, given the variance in body proportions between Alberto and our character, some animation refinement was necessary for iClone."

Arms, legs and feet

As the artists continued refining their character, the sophisticated features in iClone proved their stripes. By leveraging motion layers, the team could precisely position the character's arms and legs, eliminating any intersection with adjacent geometry. Applying the same principle, they also implemented posture correction to make movement more natural.

Foot contact was also seamlessly incorporated into the movements by accurately positioning the hips at the appropriate height, allowing the inverse kinematics to naturally bend the knees and firmly plant the feet on the floor.

To achieve a smooth rolling motion of the feet over the ground, Het Nieuwe Kader used the Curve Editor to identify and address any erratic keys. This footstep stabilisation helped to resolve any hovering motion.

Facial expressions

But while much of the effort was dedicated to cleaning up the dancer's body, it was important to pay attention to his face, too, because that's where you connect emotionally with the audience.

Here, Reallusion's Digital Soul pack proved to be an excellent resource, because of the wide variety of facial expressions available to use. Niels provides an example. "There's one where it's like he's focusing, but he's also looking pleased. It's very subtle. Because when you're dancing, you mostly put up a smile or look very focused, but a focused face is not very appealing. Conversely, if you always have a smile in 3D, it can look quite awkward."

"With this expression, though, you can tell by the face that tension is being put on the body, but the dancer still keeps the appealing expression. So that's where the Digital Soul pack was super-helpful and helped make our character look alive. I'd recommend the Digital Soul pack to anyone using iClone because it's amazing."

Finally, the team used the Unreal Live Link plugin for iClone, which made it simple to import the character into Unreal 5 with precision and view the animations in real-time. They also used Maya to create the dance stage and Substance Painter to craft realistic textures. The entire shoot, including preparation, spanned approximately half a day, while the overall production lasted around two weeks. The 3D character is now live on the Introdans site, here.

"We're known for taking on challenging projects, which no one else can do," says Niels. "So we're used to things getting difficult. But this was actually a fun project that was very straightforward, and iClone helped a lot in that regard."

You can try out iClone for free for 30 days and get access to 1,700 pre-built resources. Get started with iClone today. You can also reach out to Het Nieuwe Kader and Tpose.com for professional production and mo-cap services.