Have you played around with Sims-like plug-in objects, which include the knowledge of how to make the characters use themselves?
The important thing is that you can plug in new objects without reprogramming the people.
Sims objects (including characters) have a list of "advertisements" of possible interactions (usually shown as items on the user control pie menu, but also including invisible and special orchestration actions).
Each enabled action of every live object broadcasts its character-specific scores to each of the characters, adjusted to each character's personality, current stats, location, relationships, history, optionally tweaked by code.
Then to keep the sims from acting like perfectly optimized robots, they have a "behavioral dithering" that choses randomly between the top scoring few advertisements.
Here's a video of "Will Wright - Maxis - Interfacing to Microworlds - 1996-4-26" where he shows an pre-release version called "Dollhouse" and explains the design:
Jamie Doornbos gave a talk at GDC shortly after we released The Sims 1 in 2000, "Those Darned Sims: What Makes Them Tick?" in which he explains everything:
In The Sims 4 it's all been rewritten in Python, and has more fancy features, but it still uses the essential model for objects and advertisements.
The Sims 1 used a visual programming language called "SimAntics" to script the objects and characters, including functions that are run to score advertisements for each character.
But with LLMs you can write scoring functions and behavioral control in natural language!
Thanks for the comment! They can get pretty mad at each other relatively easily, frowning and battle crying, which is always fun to watch. When we turned on voice models (in the pipeline!!:)) their voices did as well
seriously, this embodied interaction angle seems like a much more humane way to understand AI behavior than just staring at walls of text. even if it occasionally feels like you're running a very advanced digital terrarium
