Vitreous — 360° Mixed Reality Vehicle Cockpit

Context

I joined Project Vitreous, a 360-degree mixed reality cockpit for Next Generation Combat Vehicles, built for the U.S. Army’s GVSC and ARL research programs. DCS Corporation asked me on as a contractor when the project needed one, and I ended up staying full time as the only DCS person on a six-person team that also included Jayse Hansen, known for HUD design work on films like Iron Man and Star Wars.

The problem we were solving: soldiers inside an armored vehicle lose situational awareness because they can’t see past the hull. We solved it with indirect vision, external cameras feeding a live 360-degree view into the cockpit, layered with an always-on HUD and command dashboard showing vehicle, mission, and crew status, tailored separately to driver, commander, and gunner.

The work got presented at the 2023 NDIA Ground Vehicle Systems Engineering and Technology Symposium and later published as a peer-reviewed paper (Hansberger, Wood, Conner, Hansen, Nix, Torres), and I’m a listed co-author. I came on to do modeling and materials, but since I was the only dedicated technical artist on the team, that scope grew fast into pipeline setup, engine work, and platform optimization.

My Work

Production Pipeline & Tooling

• Helped stand up the initial Unreal Engine project, establishing the folder structure and pipeline conventions the team worked from.
• Set up and administered Notion for technical documentation.
• Set up version control on Plastic SCM.

Modeling & Cockpit Art

• Modeled the digital double of the reference vehicle’s cockpit interior, built from real-world reference, including cleanup and lighting of the virtual cockpit.
• Modeled and sculpted physical cockpit hardware, including the seats and the wrist-worn cuff controller units.

Shaders & VFX

• Built the transition-effect shaders and VFX used throughout the cockpit UI.
• Built a custom TFLIR-style thermal post-process.
• Built a friend/foe identification shader using color and pattern logic to distinguish enemy and friendly units.
• Built a dynamic, box-clipped 3D/2D map material tied to the player’s position inside the vehicle.
• Built the in-vehicle map rotation system: a clip shader masks the map to a square or circular boundary that rotates live with the vehicle’s pitch, roll, and yaw, running entirely in-console.

Engine Migration & Optimization

• Led the project’s migration from UE4 to UE5, bringing in Nanite and Lumen for VR.
• Optimized a large open landscape environment to hit a 90 fps VR target.
• Converted the rendering pipeline from deferred to forward shading to support the Varjo XR-4 headset.
• Later joined the team porting the project to Apple Vision Pro.
• Ran a full optimization pass across LODs, HLODs, custom octahedral impostors, mipmap tuning, shadow optimization, and light actor optimization.

Interaction & Animation Programming

• Built an on-rails test build that triggered scripted events at set progress percentages.
• Manually ported Jayse Hansen’s After Effects HUD animations into Unreal, using Blueprint to drive playback and hand-keying 10 to 20 elements per HUD panel.

Conclusion

The Vitreous interface was validated in a usability study comparing it against two other display technologies; soldiers testing it showed better driving performance and reported the strongest sense of situational awareness of the group. What started as a modeling and materials role turned into ownership of the production pipeline, shader and VFX work, the UE4-to-UE5 migration, and VR/Apple Vision Pro optimization for a six-person defense R&D team. We were featured at SXSW 2024, where we demoed and presented our findings.