Interactive 3D Avatar Setup in UE5
Budget / Salary$750–1,500
TypeFreelance project
LocationRemote
Posted1 hour ago
Project Title: Real-Time Interactive MetaHuman / Avatar Setup for UE5 (Cloud AI Voice Integrated)
1. Project Overview
We are building a real-time interactive 3D digital human (MetaHuman) inside Unreal Engine 5 that acts as a live, streaming avatar.
The system connects to a cloud-based AI backend (STT > Gemini LLM > ElevenLabs TTS over WebSockets).
Your job is to set up the avatar, lighting, shaders, and real-time audio-driven animation pipeline in Unreal Engine 5 to achieve maximum visual photorealism at 60 FPS.
2. Deliverables & Technical Scope
A. MetaHuman & Asset OptimizationCharacter Setup: Import, configure, and refine a MetaHuman (or custom high-fidelity 3D mesh mapped to the MetaHuman skeletal structure).
Level of Detail (LOD): Lock the main avatar to LOD 0 (Cinematic).Groom / Hair: Implement high-quality strand-based groom hair optimized to hold 60 FPS.Shaders & Skin: Fine-tune skin materials (Subsurface Scattering profiles, micro-roughness, specular maps, eye reflections, and teeth/tongue moisture shaders).
B. Real-Time Lighting & Cinematic SceneLighting Environment: Build a 3-point studio or natural lighting rig using Lumen with Hardware Ray Tracing.Camera Setup: Cinematic virtual camera (35mm / 50mm lens) with depth-of-field (DOF), motion blur tuned for live streaming, and auto-exposure locks.Performance Target: Scene must run at 4K / 60 FPS on a single NVIDIA RTX 4070 Ti Super / RTX 4080 (DLSS Quality enabled).
C. Real-Time Animation & Audio Sync (Critical)Audio-Driven Lip Sync: Set up the NVIDIA Audio2Face-3D UE5 Plugin (or Live Link / ARKit blendshape pipeline) to drive facial expressions directly from incoming audio streams.Idle & Micro-Expressions: Implement a Blueprint state machine for idle animations (breathing, eye blinks, subtle head sways, micro-gestures) so the avatar stays "alive" while waiting for or listening to the user.Audio Routing: Configure incoming streaming audio (PCM / WAV chunks from ElevenLabs) to drive facial animation weights and spatial audio playback simultaneously.
3. Required Skills & Experience
High proficiency in Unreal Engine 5 (5.4+), MetaHuman Creator, and Live Link protocols.Experience with NVIDIA Audio2Face-3D / ACE or ARKit blendshape mappings.Deep knowledge of real-time lighting (Lumen/Ray Tracing) and high-fidelity shader creation.Understanding of Blueprint / C++ networking for accepting external audio streams and animation trigger events.
1. Project Overview
We are building a real-time interactive 3D digital human (MetaHuman) inside Unreal Engine 5 that acts as a live, streaming avatar.
The system connects to a cloud-based AI backend (STT > Gemini LLM > ElevenLabs TTS over WebSockets).
Your job is to set up the avatar, lighting, shaders, and real-time audio-driven animation pipeline in Unreal Engine 5 to achieve maximum visual photorealism at 60 FPS.
2. Deliverables & Technical Scope
A. MetaHuman & Asset OptimizationCharacter Setup: Import, configure, and refine a MetaHuman (or custom high-fidelity 3D mesh mapped to the MetaHuman skeletal structure).
Level of Detail (LOD): Lock the main avatar to LOD 0 (Cinematic).Groom / Hair: Implement high-quality strand-based groom hair optimized to hold 60 FPS.Shaders & Skin: Fine-tune skin materials (Subsurface Scattering profiles, micro-roughness, specular maps, eye reflections, and teeth/tongue moisture shaders).
B. Real-Time Lighting & Cinematic SceneLighting Environment: Build a 3-point studio or natural lighting rig using Lumen with Hardware Ray Tracing.Camera Setup: Cinematic virtual camera (35mm / 50mm lens) with depth-of-field (DOF), motion blur tuned for live streaming, and auto-exposure locks.Performance Target: Scene must run at 4K / 60 FPS on a single NVIDIA RTX 4070 Ti Super / RTX 4080 (DLSS Quality enabled).
C. Real-Time Animation & Audio Sync (Critical)Audio-Driven Lip Sync: Set up the NVIDIA Audio2Face-3D UE5 Plugin (or Live Link / ARKit blendshape pipeline) to drive facial expressions directly from incoming audio streams.Idle & Micro-Expressions: Implement a Blueprint state machine for idle animations (breathing, eye blinks, subtle head sways, micro-gestures) so the avatar stays "alive" while waiting for or listening to the user.Audio Routing: Configure incoming streaming audio (PCM / WAV chunks from ElevenLabs) to drive facial animation weights and spatial audio playback simultaneously.
3. Required Skills & Experience
High proficiency in Unreal Engine 5 (5.4+), MetaHuman Creator, and Live Link protocols.Experience with NVIDIA Audio2Face-3D / ACE or ARKit blendshape mappings.Deep knowledge of real-time lighting (Lumen/Ray Tracing) and high-fidelity shader creation.Understanding of Blueprint / C++ networking for accepting external audio streams and animation trigger events.
Apply on Freelancer →
Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.