Virtual environment merges LED and AR to create immersive production environment
After launching groundbreaking studio projects in recent years with its NASCAR virtual studio in Charlotte, NC; LED-fueled FOX NFL Sunday studio in L.A.; and AR-powered 2022 FIFA World Cup set in Doha, FOX Sports is changing the game once again with a new 360-degree immersive XR studio environment for its coverage of the Euro 24 and Copa America.
“With every project, we get better, and this project provides a genuine blank canvas for us to work with,” says Gary Hartley, EVP/creative director, FOX Sports. “Between the power of the technology and the creativity of the design, I think the sky’s the limit in terms of what we can do now.”
The two-story virtual studio based at FOX’s cutting-edge Stage A in L.A. boasts 5,130 sq. ft. of LED space and is anchored by extended-realty (XR) multicam technology driven by Unreal Engine, Vizrt, and Stype. Working with New Zealand-based architecture firm Architecture van Brandenburg, FOX Sports designed a virtual set for its coverage of the two tournaments; the set features a full 360-degree immersive environment live from Stage A.
“Whenever we are [evaluating new technologies], we often talk about UVD — unique visual difference — and this is the epitome of that,” says Zac Fields, SVP, graphic tech and innovation, FOX Sports. “Our studio here in L.A. is LED-based. Typically, we use AR and XR only as a set extension above the LED, but, with this 360-degree environment, we have the best of both worlds: we can leverage all the LED firepower from our state-of-the-art stage and facility here in L.A. but also extend the volume via AR in all directions.”
Match Made in Creative Heaven: Integrating the Set and the Animations
With EURO 24 and Copa America taking place across two continents, FOX opted to host its studio coverage for both tournaments from this central set on the FOX Studio Lot in Los Angeles. Inspired by beautiful palaces around the world, the set’s new technology allows the production to move through various areas with eye-catching imagery and graphics showcasing team colors, matchups, player stats, and more. The dynamic backgrounds will feature the cities where the matches are played, fully immersing audiences into the cities across Germany and the U.S. for the two tournaments.
“This project — more than any other that we’ve done so far with AR, VR, and XR — tightly integrates our animation and our set,” says Michael Dolan, SVP, Design, FOX Sports. “We’re literally painting the set with country colors and matchups and player graphics. The architecture is essentially the carrier for all the graphic design. As opposed to just floating [graphics] up in space, which is what we would typically do, we can now take over an entire room and drape it with [graphics] related to the topic.”
Inside the Tech: Sprawling Virtual World Driven by Unreal Engine
The floor area of interior volume is 56,840 sq. ft. (290 x 196 ft.); the central hall of the environment is 165 ft. long x 125 ft. wide x 65 ft. high.
“The [aspect] that takes this to another level is the enormous scale of it,” says Fields. “We built this out with specific segments in mind for the production team to use. We will have an area for formations, an area to come back for the halftime score, an area for a sit-down interviews — those were presented to the architect early on. It’s probably one of the reasons the location ended up being so enormous: they were essentially building rooms that were designated for specific [show segments].”
A whopping 23 Unreal Engines provide the stunning AR and XR visuals, and the show is controlled via Erizos Studio software. With 5,130 sq ft of LED wall and floor panels, the full LED size is 19 ft. 8 in. high x 46 ft. wide x 24 ft. 7 in. deep.
Stage A is outfitted with a 24-ft. Technocrane, and FOX deploys Stype camera-tracking systems and the StypeLand plugin for Unreal Engine to enable VR creation with its own chroma-keyer software.
Using Ghost Frame technology, FOX has provided the director up to six virtual cameras to produce the show within the 360º environment (the system is capable of up to six virtual cameras). FOX marries the tracking data from the camera with the data from the Unreal scene to allow the director to “fly around” from one set location to the next.
“The question became how we could get from one space to another so it feels like we’re truly utilizing the entire 56,000-sq.-ft. environment,” says Fields. “That’s where we started having conversations about using virtual cameras to merge what’s real and what’s [virtual]. The virtual cameras allow us to fly in and out of sets.
“There are multiple exterior shots that reflect time of day — daytime, dusk, nighttime, etc. — in this environment, representing Miami and Germany,” he continues. “This will allow us to go outside the set and show a logo bump while going to break or an interesting cityscape coming back from commercial. You can do that only with a virtual camera because you’re traveling a great distance within that [virtual space].”
Inside the control room, the production resembles a traditional operation with a standard front bench running the show. In addition, two set-content operators drive all the content, including environmental, data, and animations. In addition, two set-content producers work closely with these operators to create elements during the show.
“That choreography comes from all the experience we have doing it in Charlotte,” says Hartley. “I can’t emphasize enough the value of the experience of Charlotte in all of this, because we were able to learn a lot there without drastic consequences.”
Inside the Design: Creating a ‘Basilica of Soccer’
With the Euro 24 and Copa America tournaments taking place simultaneously this summer, Hartley say FOX was looking to create a “centralized palace for soccer.” During the 2023 FIFA Women’s World Cup, Fields was introduced to the Van Brandenberg firm, which was enlisted to create a virtual environment drawing on the design of iconic European cathedrals.
“When we started having conversations about doing a set for these two tournaments at the same time,” says Fields, “we knew we had a major technological advantage with our state-of-the-art stage and facility in L.A. That’s where the idea of architecting an entirely new environment in L.A. came from. We knew, if we could design an environment from scratch, we could essentially build in any functionality that we wanted.”
Within this gothic-architecture environment, Dolan and the design team were able to organically merge function with form by integrating the graphics package directly into the virtual sets. For example, FOX will regularly use a rose-window–themed portal to display everything from information and stats to decorative elements and team colors.
“The primary concept of the design is what we call ‘the basilica of soccer,’” says Dolan. “We play heavily off the rose window, which is frequently seen in European cathedrals. We created this very slick graphic system where we can show a country, a matchup, group stages, or anything else in that window. That rose window helped lay the foundation, and a lot of the graphic language was derived from that.”
The result is a fully virtualized set that includes zero practical elements. The base look and animation package are the same for both Euro 24 and Copa America — minus the tournament branding — so that FOX’s coverage can bleed from one segment into the next seamlessly.
“It has been a huge technical challenge to build this, and the results are pretty amazing,” says Hartley. “The most exciting thing is that what we go on the air with on day one will only be the tip of the iceberg. As we roll in new executions throughout the next month, it’s just going to get better and better, so that it will be utterly mind-blowing by the end.”
Just the Beginning: Looking Beyond FOX’s Summer of Soccer
While FOX’s attention is focused squarely on the next five weeks of international soccer, it’s hard not to look ahead to the possibilities this new virtual set presents for other live sports properties.
“Every project we’ve done up to this point has been a jump,” says Dolan, “but this feels like a leap — like we just cracked open Pandora’s box. I think where we get to college football or MLB or NFL is going to be an even a bigger jump. Now that we’ve done it, we’re already looking ahead to what else we can do. The sky’s the limit. Whatever we can imagine, we can now create.”
With years of experience and the full support of CEO/Executive Producer Eric Shanks and President of Production & Operations/Executive Producer Brad Zager, Hartley believes that FOX is currently ahead of the curve in virtual production compared with other broadcasters. However, he is quick to note, the only way to maintain that advantage is by continuing to innovate.
“I’ve always said that the barrier of entry to something like this is so profound that it’s hard for a network to commit to those kinds of dollars to get into it,” he says. “As a result of the commitment we made in Charlotte and with Stage A, right now we have a little bit of a lead [over other broadcasters]. I know it’s getting shorter, but having that lead allows us to do things like this where everything we’re doing on this project will leak into the NFL and into other [FOX properties] — not just XR but also AR. We have very big plans for Super Bowl as well.”
For now, though, all eyes remain on Euro 24 and Copa America.
“What’s exciting about this one is that this is a big studio show for a couple of very big events,” says Fields. “There are going to be millions of people watching. We’re doing this show on a set in a way that has never been done before, and there are going to be a lot of eyeballs that see it. Like any technology, it will just keep improving; it’s nowhere near being complete.”
