Last week, I posted about a hybrid approach I took to create low poly 3d assets for an RTS game I've been building. There was quite a bit of interest in a tutorial for it, so I created one and posted it on our blog.
Finally got around to posting a quick tutorial on the approach that's been working well for me. I don't believe LLMs or even diffusion models are ready to one-shot create game-ready assets. However, for those that are artistically challenged like me, a multi-step hybrid approach works really well as long as you are judging/iterating on the outputs at each step.
I have experimented with this type of modeling for a RTS game I'm building: https://playwarpact.com
Few things:
1. Opus 5 is just as good for this tbh and cheaper.
2. They don't generate optimized 3d models. They have high poly count for simple geometries.
3. A better approach I've utilized for game ready assets is to use the model to create low poly silhouettes of the 3d model and then bake textures that include a lot more details like windows, doors etc with tools like Meshy. Can post a tutorial if there's interest.
The models need some more RL to be able to do this autonomously.
> Opus 5 is just as good for this tbh and cheaper.
It depends on the fidelity, there's a threshold slightly over "low poly" that Opus cannot get over. Once you get into creating foliage l-systems, or physical mob animation, or a house with realistic appliances, Opus is under the convergence threshold for a world model no matter how much time you give it, it will flail around and say it's done even though it's nowhere close to plausible. Fable takes forever but knows how to zoom in and out on the abstractions.
> They don't generate optimized 3d models. They have high poly count for simple geometries.
By default LLMs will do the quick prototype thing and slap some primitives into a THREE.js scene. Which is probably correct because most users don't know or care. But if prompted Fable will connect the manifolds/sculpt with a mesher, instance out the pieces, etc. and make you an efficient art pipeline. You can make it optimized. Just not in 10 minutes.
> A better approach I've utilized for game ready assets is to use the model to create low poly silhouettes
You can also let Meshy make the geo and then fit it back to the world with PCA + BB's. Diffusion models are still 10x better at making physically plausible geometries than LLMs, plus they are faster all things considered. One technique I've found that works well is socketing; let the LLM generate the high level structure with blockouts for sockets, then slot in the higher poly objects that fit. Which is closer to the professional approach for world design.
I've tried several AI 3d generators that claim low-poly output, and they're all far off. None of them decide what belongs in the mesh vs. what should be
baked in with the texture, so the poly budget lands in the wrong places.
What I'm suggesting is if you want a certain semantic part of the scene to have a specific design or fidelity (say an interactive object, point of interest, character, or parts thereof), you can generate it via image and then use principal component analysis, bounding boxes, and a physics engine like Jolt to place and orient it programmatically into the right "socket" in your world so it aligns and joins correctly. The LLM can generate the high level spatial/semantic structure, and you can plug the assets into the right space. That works both at scene level and more locally (say a character's equipment they are wearing or holding).
This kind of decomposition helps to get the art composed the way you want it, though it won't help with a runtime poly budget.
But if you are intentionally doing low-poly style, it is doubtful you are (at least technologically) limited by having too many vertices on assets; you're limited by draws and submits and rendering architecture. You can push millions of animated polys with bells and whistles, postprocessing, physics etc, in the browser, at 90 FPS, on a macbook. You can have thousands of objects, but you have to cull/instance/share materials/atlas/etc, you can't just do `new THREE.Mesh` for each object -- which is what the LLMs will naively do by default unless you ask otherwise.
If resource size is a problem, meshes compress/quantize well with things like meshopt/DRACO.
I’m interested! I struggled to get goos detail on my Back to the Future Delorean 3D animation on my docs hero: https://contextify.sh/docs/
I posted this previously and people wanted more info so here is how I built it, with a deeper dive into some film scene recreation I worked on focusing on a scene from Apocalypto:
It is a 3D-ish model being animated by a procedural motion system, rather than a pre-rendered/model animation.
The car started from a low-poly DeLorean glTF. I had the model rebuild the geometry into inline Canvas code rather than just dropping a model into three.js.
The flight itself is procedural: position/rotation, hover, pitch, wobble, flames, camera, etc. are all driven by code and can be scrubbed/debugged.
That link has a link to an interactive page where you can adjust the car’s path and when it reaches 88mph if you want to see.
It is not mobile friendly atm, though.
I would like to apply textures to the people and scenery in the gauntlet scene. Just haven’t had the time or tokens, as getting motion and pathing and camera angle right seemed the mvp.
I created a skill just for the 3d models. Works with Unreal Engine too. It generates an image using stable diffusion, then it transforms the object in the image into a 3d object using Hunyuan3D-2. You can check it out here: https://github.com/LaurentiuGabriel/unreal-game-assets-creat...
I'd love to see a sample .fbx as a wireframe, because as far as I've tried that model and basically any other model under the roof that runs on less than 96GB VRAM and can spit out 3D, have had absolutely horrible outputs that are barely usable for anything else than lightweight prototype renders. I'd love to be proven wrong though, would help a ton with building higher fidelity prototypes of various things.
> I did you one better and showed you a working implementation involving lighting, physics, and all the stuff you were worried about!
Not showing wireframes is not "one better" than showing wireframes, when it's the quality of the meshes we're trying to get figure out, where lightning and anything else just distracts from actually being able to see it.
What I'm worried about, is this conversation continuing for even longer without any wireframes or actual .obj/.fbx appearing ever, and getting another non-answer why the quality of the mesh is no longer important or whatever, or for your particular art-style it doesn't matter, or ...
I just want to know concretely what quality people manage to get from it, because as far as I've tried various things, locally and remotely, the output (of the meshes) isn't very good.
Uh, I tried feeding 5.6 Sol High my 2D DXF drawings of my house floor layout & house exterior to generate 3D visualization within blender via MCP... and it was miserable. No understanding on how windows attach to walls, misaligned bearing walls between floors, gaps all around the places, chimney not continuous, windows don't line up and are in wrong places, house parts swapped. And still struggling when I explain those things - that it is a house and real-life architectural constraints should be taken into account. But I just had to keep prompting to fix this, that, redo this and that and then it breaks other things until I fed up.
Part of it is probably struggle to understand the DXF and mixing up "wall lines" with "dimension/helper lines" (I tried also feeding PDFs)
Anyways, if someone has an idea how to improve 3D modelling with AI or how to make understand 2D drawings to understand rooms/walls/square meters, etc - would be nice.
As you might have researched it, do you know if there's currently any AI model or arrangement of models that are able to produce truly game-ready 3D assets? Good meshes with appropriate poly count and good topology, decent UV maps and textures, etc. I've seen some products advertising that, but I'm not sure they are up to their promise, and I haven't had the bandwidth to research it in depth...
why would you force me to login to see what the game is, I have zero information and you're asking me for a password or my google information. hostile and creepy.
Sorry, the game is still in dev and recently opened up for early access. I am working on a marketing page for the game that would link to the gameplay.
That’s also going to be limited most likely. I don’t like governments deciding who can have the best products. I can’t shake the feeling there is money changing hands for getting on the access list as well with this administration.
Look, even if in this one instance Trump isn’t simply asking get his beak wet, that is the less believable scenario.
When “not cartoonishly corrupt” becomes hard to believe due to firmly established character and endless precedent, then it almost doesn’t matter if everything is on the level in this one matter.
The trust has been lost, undermining the ability to govern.
The fact that he’s still in power with firm support from everyone else that matters is all you need to know about how rotten to the core things are in your country.
We are building a command & conquer style RTS. If you're interested in taking a look at our very early demo of the game play, please check out:
https://warpact-rts-9221.web.app/
great question! we have logic to look for things like when certain network requests are completed, dom loaded, etc as well as a timeout so we are not waiting for ever. The LLM based on the screenshot can also decide to wait longer if the page hasn't fully loaded despite the checks we do.
They have browser automation, and a bunch of other agent tools to manage tasks, do things like PowerPoint slides, etc. I find chatgpt agent mode better for most tasks though.
I just did this test with our web QA agent - kodefreeze.com, it was able to test creating an account until it reached the screen that requires email confirmation.
Support for being able to receive email/custom actions is on our roadmap, but would love to see if getting this far would be valuable to you. The test was with the email=test@kodefreeze.com.
Finally got around to posting a quick tutorial on the approach that's been working well for me. I don't believe LLMs or even diffusion models are ready to one-shot create game-ready assets. However, for those that are artistically challenged like me, a multi-step hybrid approach works really well as long as you are judging/iterating on the outputs at each step.