Dev turns 26-year-old PS2 into a working VTuber setup with full body tracking

An independent game developer has created VTuber software that renders a moving 3D avatar natively on a PlayStation 2, with plans to eventually add DIY full-body tracking.

VTubers generally use cameras and motion-tracking software to animate a digital avatar for videos and livestreams. Serenity, an indie developer publishing under the name Serenity Theater, is now attempting to bring that technology to Sony’s 26-year-old console.

In a video posted to X, Serenity demonstrated an early build called “Serenity VTuber Rig V1” running on a PS2 connected to a CRT television.

The demonstration shows a wireframe model rotating as live yaw, pitch, and roll values change onscreen. Serenity said the project was created with PS2SDK, a collection of open-source libraries used to develop applications for Sony’s console.

PS2 handles VTuber model without emulation

The current prototype is not yet a completely self-contained PS2 motion-capture setup.

Serenity explained that it connects to a Python script running on a PC. That script calculates the user’s positional data before sending it to the PS2 over a network every frame.

However, the avatar itself is being rendered by the PS2 rather than a PC or emulator. Serenity emphasized this distinction in the video, declaring: “No tricks! No emulation! No imitation!”

The software runs through the console’s Emotion Engine, the name given to the PS2’s main CPU. Sony says the processor was made up of eight separate units, each responsible for a different function.

For now, Serenity says the software simply displays a model that can be captured through programs such as OBS. When asked whether the avatar could appear inside a PS2 game, the developer replied: “Emphasis on ‘for now.’”

The project could eventually become more than a VTuber tool. Serenity said one of the original ideas was to use it to animate development videos about making PS2 games, with plans to also turn it into general animation software.

Serenity is also considering using the PS2’s EyeToy camera for a version of face tracking handled entirely by the console, although the developer described the accessory as “just an awful camera.”

Full-body tracking through DIY hardware is planned as well. Serenity also revealed that the software was built on a hardware abstraction layer, potentially allowing it to be ported to other consoles from the same generation.

When one commenter asked why the developer chose the PS2, Serenity simply responded: “It’s my goat.”

No release date has been announced, but Serenity said making the software user-friendly will be a priority after one user joked about becoming part of the first generation of “PS2bers.”