Social Conjurer, 2024
Multi-User Runtime Collaboration with AI in Building Virtual 3D Worlds
June 2024 - September 2024
Generative artificial intelligence has shown promise in prompting virtual worlds into existence, yet little attention has been given to understanding how this process unfolds as social interaction. We present Social Conjurer, a framework for AI-augmented 3D scene creation, where multiple users collaboratively build and modify virtual worlds in real-time. Through an expanded set of interactions, including social and tool-based engagements as well as spatial reasoning, our framework facilitates the creation of diverse virtual environments. Findings from a preliminary user study (N=12) provide insight into the user experience of this approach, how social contexts shape the prompting of spatial environments, and perspective on social applications of prompt-based 3D creation. In addition to highlighting the potential of AI-supported multi-user world creation and offering new pathways for AI-augmented creative processes in VR, we present a set of implications for designing human-centered interfaces that incorporate AI models into 3D content generation.
Read our pre-print on ArXiv.
This work was produced at Microsoft Research and belongs to the “Speaking the World into Existence” research initiative.
Technologies:
︎ Unity, Photon Fusion, LLMs
Team members: Amina Kobenova, Cyan DeVeaux, Samyak Parajuli, Andrzej Banburski-Fahey, Judith Amores Fernandez, Jaron Lanier.
Read our pre-print on ArXiv.
This work was produced at Microsoft Research and belongs to the “Speaking the World into Existence” research initiative.
Technologies:
︎ Unity, Photon Fusion, LLMs
Team members: Amina Kobenova, Cyan DeVeaux, Samyak Parajuli, Andrzej Banburski-Fahey, Judith Amores Fernandez, Jaron Lanier.