SpongeBob’s pal Patrick Star calls the game in real time
Since launching with the NFL Wild Card Game in 2021, Nickelodeon’s NFL-game coverage has evolved from a madcap slime-filled experiment to a multi-Emmy Award-winning must-see annual celebration of the sport. Although Sunday’s Broncos-Rams Christmas Day blowout didn’t provide much in terms of gridiron drama, CBS Sports and Nickelodeon’s “Nickmas” broadcast provided plenty of AR-fueled visuals aimed at drawing in the next generation of NFL fans.
After the game, Nickelodeon Executive Producer, NFL Slimetime, Shawn Robbins, who served as coordinating producer of the game, and Nickelodeon VP, Animation/Motion Graphics, Kurt Hartman sat down with SVG to discuss the third chapter in the annual tradition and how they were able to take this year’s broadcast to the next level — in terms of both technology and storytelling.
The Nickmas broadcast was produced side-by-side with the main CBS broadcast out of Game Creek Video Columbia, a B unit, and Edit 3 trucks. Robbins oversaw the production with producer Ken Mack and director Andy Freedman at the front bench. In addition to camera feeds from the main truck, Freedman had four unilateral cameras with AR functionality to incorporate into the broadcast.
The AR graphics were created in collaboration with Silver Spoon, leveraging the newly launched Unreal Engine 5 to create real-time AR animations and graphics. CBS Sports, Nickelodeon, and Silver Spoon used Pixotope virtual-production software to create the in-game elements and deployed an infrastructure comprising 12 primary/backup servers with a cavalcade of NVIDIA RTX A6000 GPUs to drive the system.
Using this setup, the production team integrated eye-popping virtual elements into the broadcast, including the now-iconic slime cannons and slime-dumping blimp, Christmas- and winter-themed characters on the field, a yeti dumping snow on the crowd, and a Hannukah-themed car race above the field.
In addition, SMT provided the optical tracking on three “up” cameras and Stype handled tracking on the low–end-zone camera — both in collaboration with Silver Spoon. Second Spectrum also helped with augmented slime trails on skill players for the second year in a row.
“I never like to say, ‘the first,’ but I would say that this is the biggest early use case of Unreal Engine 5 in a live–sporting-event capacity,” says Jason Cohen, VP, remote technical operations, CBS Sports. “Unreal Engine 5 has provided the next step in creating animated graphics in the augmented space for live sports. And there’s no better use case of augmented reality in a broadcast of live sports, in my opinion, than the Nickelodeon show. This [broadcast] has AR at its core because the entire point of the [presentation] is to create fun elements that can enhance storytelling and add entertainment at the same time.”
Throughout the game, a team of Nickelodeon animators in New York used Adobe After Effects to create quick-turnaround “Nickified” graphics incorporated into the live broadcast.
Perhaps most impressive was the live commentary courtesy of SpongeBob SquarePants’ best friend Patrick Star. Inside the Edit 3 truck, actor Bill Fagerbakke, who provides the voice of Patrick in the long-running cartoon series, wore a head harness with a cellphone motion-capture system to capture his facial movements. Silver Spoon then ran that mo-cap data through Unreal Engine 5 to create an avatar that was featured live in the broadcast as Patrick called the action and interacted with CBS Sports’ Nate Burleson, Noah Eagle, and Nick star Gabrielle Nevaeh Green in the booth.
“When you think about how Patrick was brought to life in real time, it’s pretty incredible,” says Cohen. “I don’t know of another time a real-time character or avatar has been brought into a show as a live commentator in an augmented reality, so we’re pretty proud of that,”
Playing key roles in pulling off the Nickmas production were, besides Cohen, Patty Power, EVP, operations and engineering; Mike Francis, Craig Stevens, and Evelyn Jackson from the CBS Sports engineering team; onsite tech managers Pete Kallander, Jon Muro, Beth Tuura, and Kevin Wang; onsite Production Managers Kristen Florian and Danny Kopilnick; and the CBS Sports broadcast-operations team led by Scott Davis and featuring Alyssa Blake, George Dimotheris, and Chris Rahner.
“It’s pretty amazing to see how far this has come,” Cohen adds. “The first [NFL game on Nickelodeon] was a novel approach and got high reviews, but you look back now and see how it was almost basic compared with what we’re doing now. It was revolutionary at the time, and it was fun to just get it off the ground, but, when I think about how far Sean and his creative team have taken this concept, it’s like 10 levels beyond where it started. My hope is that the audience appreciates how much it has evolved and morphed into something that’s not only a creative marvel but a true technological accomplishment as well.”
