- Published on
Interactive Dance Tracker + Content Pipeline
- Authors
- Name
- Jorge Zozaya
Overview:
As a new era of devices rolls out, I’ve been asking: What do I want from Virtuality? How could AR or VR add value to my life? Is there a place where it makes sense? For me, the answer lies at the intersection of neuroplasticity and dancing—two topics at the top of my list in 2024.
Dancing is one of the most enjoyable, freeing experiences out there, but most live entertainment solutions require heavy headsets or multiple devices. None of them let you dance to your favorite music or connect to platforms like SoundCloud or YouTube.
So, I started exploring: Could we do something similar with technologies that have been lying around almost unnoticed? Can we create an immersive experience without strapping on a headset? What happens on the dance floor when things get wild? And, could wires lead to entanglement issues?
To answer these questions, I built this web app—a video game engine of sorts—that leverages off-the-shelf models accessible through your browser. All you need is a webcam. No headsets, no wires, no fuss.
Why It’s Cool:
- Privacy First: TensorFlow.js keeps everything in your browser. No data leaves your device. Period.
- Real-Time Moves: MoveNet tracks dancers with precision, turning their performance into shareable art.
- Stop-Motion Magic: I built a browser-based pipeline that stitches snapshots into stop-motion videos—no server needed.
- Interactive Drawing Tool: After the performance, users can unleash their creativity with a built-in drawing tool, adding a personal touch to their dance-generated assets.
- No Headsets, No Wires: Just turn on your webcam and dance to your favorite music.
The Tech Behind the Moves:
- Next.js: For a blazing-fast, full-stack experience.
- TensorFlow.js + MoveNet: For real-time pose estimation that’s as smooth as your dance moves.
What It Does:
- Tracks You in Real Time: Watch as MoveNet follows every twist, turn, and twirl.
- Creates Shareable Content: Automatically generates personalized videos and snapshots of your best moments.
- Interactive Drawing Tool: After the performance, users can draw and customize their dance-generated assets, making each creation truly unique.
- No Heavy Gear: No headsets, no wires—just your webcam and your moves.
- Keeps It Private: Everything happens in your browser—your data stays yours.
Why Shareable Content Rocks:
This app isn’t just a tool; it’s an experience. By giving users branded, personalized content, it:
- Boosts Engagement: Users share their videos and drawings, spreading the word organically.
- Builds Community: Dancers connect over their performances and creations, creating a vibrant, engaged audience.
- Makes Memories: Relive and personalize your dance moves anytime, anywhere.