NVIDIA MonsterHouse Runs Nemotron Halloween AI on One Workstation
NVIDIA’s MonsterHouse combines Nemotron speech, Bolt E4B, Magpie TTS, Audio2Face and vision models for a live Halloween garage display on one workstation.
Summary
On October 30, 2026, NVIDIA’s Rick Grandy detailed MonsterHouse, an interactive Halloween garage projection that hears trick or treaters, replies as a grumpy pirate ship, animates a fiery wooden mouth and softens after “please” or “thank you.” Nemotron 3 ASR transcribes speech, open Nemotron 3.5 Bolt E4B 4GB generates replies, Magpie TTS Zeroshot voices seven moods, and Audio2Face-3D animates an Autodesk Maya rig in Unreal Engine 5. Nemotron 3 Nano Omni describes costumes, while YOLO and ByteTrack count and track visitors.
Nemotron Voice Agent coordinates the off grid system on one workstation. Language and voice run on an RTX PRO 6000 Blackwell Max-Q, while graphics and Audio2Face-3D use an RTX PRO 4000 Blackwell. Grandy selected Bolt after testing more than 12 models, including Qwen3 30B-A3B and Qwen 3.6 35B-A3B, then shaped it with application state, prompts and a LoRA trained on over 5,000 reviewed multi-turn conversations. Median speech-to-audio latency is 1.70 seconds with Bolt Q4 and TTS, or 0.72 seconds for cached responses, including a 300 millisecond silence wait. A synthetic five-sheep video recognized pirate, vampire, witch, pumpkin and ghost costumes, but real nighttime visitors, animation and automatic candy delivery still need work. MonsterHouse code is not packaged, although every NVIDIA model is available and Voice Agent supports cloud, server and single-GPU deployments.
Positives
- 0.72-second median latency for cached responses lets MonsterHouse answer familiar interactions faster than live generation.
- More than 5,000 reviewed multi-turn conversations trained the LoRA controlling MonsterHouse’s personality and conversational flow.
- Seven vocal moods let Magpie TTS Zeroshot vary delivery from menacing to delighted.
- Nemotron 3 Nano Omni, YOLO and ByteTrack give the character costume context, visitor counts and tracking.
- Every NVIDIA model used by MonsterHouse is available, while Voice Agent supports cloud, server and single-GPU configurations.
Risks & concerns
- 1.70-second median live response latency includes a 300 millisecond silence wait and remains slower than cached audio.
- Real driveway groups, costumes and nighttime lighting have not completed performance testing.
- MonsterHouse code is not packaged for release, requiring builders to assemble their own implementation.
- Automatic candy delivery remains unfinished, preventing the system from independently running the full Halloween experience.
- Compute demands require separate RTX PRO 6000 and RTX PRO 4000 GPUs to protect animation frame rates.