Proposal: Biomes

[Basil42]

Hello,
I am planning to work on a procedural placement system on top of terrain 3D and would like to start a thread on it here, to work out specifics and scope to have something ready-ish to PR in a few weeks.

Core rationale

It would be useful to be able to procedurally place set dressing like vegetation and rocks, optionally gameplay element, without having to paint every corner of the world manually, while retaining the option to do so locally. I'm drawing heavy inspiration from this talk.
Of course it would take years to implement everything in this talk so I will be trying to narrow the scope to a few immediately usable and useful features.

Plan so far

Here is an outline of the minimal features I'd like to put together in the next 6 weeks or so.
In short, biomes will be collection of assets with density and spatial footprint values. The assets can then be placed by placing a volume associated with the biome, or painting the biome on a texture.

Defining biomes limits

I am hesitating between a few approaches:

• Biomes could be delimited by volumes, like in Unreal, probably the most straightforward. It is a bit fiddly for the user, and overlaps of different volumes with different biomes might be bug prone.
  -Biomes could be delimited by a biome map, of similar resolution that the height map, loaded in chunks in a similar fashion. It could be edited through painting, but using the data sounds a lot harder.
  -Biomes could be defined by the textures of the terrain, this is slightly more straightforward than the previous method and it makes some editing potentially easier. It might end up slightly less flexible than a dedicated map.

Placement logic

Initially, I'll simply implement the option to give each asset associated with the biome:

• A spatial footprint, used to space out objects.
• A density value.
• Density modifier associated with terrain data samples (to, for example, prevent grass from spawning on road dirt painted on the terrain).
• Some option for jittering the position to avoid objects lining up.

I will discretize the data using a similar dithering strategy than the Guerrilla talk. I'll try to have local stability out of the gate, to make future painting features easier to implement.

These are straightforward to use and do not require to build complex tooling and shader generation. It would be hard to tweak the output locally however. The most immediate upgrade is then the option to replace the density value by a density map that could be painted in directly using existing brush tool. I hope to do this upgrade during my initial work, but I might have to push it to later this year.

Later down the line I have a few ideas to build more complex placement logic with compute shaders. But I doubt it will happen in the coming weeks.

Use

Users would define a biome, stored in a config file, with a list of assets with settings as described previously and then either (depending how I end up doing it), drop a biome volume to populate the world with the assets, or paint a biome terrain layer (UX would be similar to the instancer).
This initial proposal does not require any major tooling that is not already implemented in some way.

Type of objects

The system would mostly output data to build multimeshes, (maybe through a slightly refactored instancer), but I'd like to give the option to instantiate Scene objects at lower frequencies as an easy way to place colliders and gameplay objects.

On the subject of multimesh, I think I should size them depending on the selected asset footprint, instead of doing MMIs that can span entire regions to get some mileage out of frustum culling.

Request and question

I'd love to read recommendations and opinions, on scope, implementation choices or things you think I should clear out before I dive in during the week, although some of it was already cleared out on Discord.

I'd like some precision on how future proof everything needs to be ? For example I might use a density map per asset directly for now, but later I'll probably need some more complex object (like the Layers described in the Guerrilla talk), which could break during upgrade without special precautions.

Have a nice week.

[TokisanGames]

Thanks for this write up. Here are my responses to your ideas:

to procedurally place set dressing like vegetation and rocks, optionally gameplay element, without having to paint every corner of the world manually, while retaining the option to do so locally.

I think this is a great, focused goal. I wouldn't worry about placing prefabbed scene objects with scripts in this phase.

Limits:

• volumes - An option, but less control
• textures of the terrain - We reuse textures everywhere. Various dirt and gravel textures are useful in all biomes. Green and brown debris are also used universally. These are sprinkled in and blended. Biomes will have different ratios of the same textures. You'd have to filter through finely detailed textures with pixel differences and at best would only be guessing. I don't think this is the way to go.
• biome map - We currently have paintable navigation. Paintable bits for biomes makes the most sense to me. See Add custom data painter #565 (comment) and the whole thread.

Placement:

A spatial footprint, used to space out objects.

Use our instancer and place the texture cards, then add one of the large rocks at the same strength value. You'll see we currently already adjust density based on AABB volume size.

Users would define a biome, stored in a config file,

I would not go with a config file. That will cut users by 90%. You should build biome selection in the UI. I suggest something like:

• Asset dock allows one to select multiple foliage types with checkboxes like UE, and will scatter them based on their individual density settings within the circle brush. A biome-lite option.
• The user can turn this selection into a group, a biome, that can be selected or not. Perhaps it's a new resource, Terrain3DBiome, that has it's own settings.
• Alternatively, or additionally, the asset dock has a biome tab that allows creating them and adding meshes and configuration to it.

The system would mostly output data to build multimeshes, (maybe through a slightly refactored instancer), but I'd like to give the option to instantiate Scene objects at lower frequencies as an easy way to place colliders and gameplay objects.

If going with MMIs, it should use our instancer. It can be improved to support what is needed. Collision for these scenes will be generated dynamically, which you could do.

As for placing prefab scenes w/ scripts instead of MMIs, it's an option. But I think this will be rarely used and low priority. Most people use an instancer for static rocks, debris, and foliage with animation done in shaders.

One thing for us to think about is what happens once the biome is placed on the ground? In the video, the whole system is designed for auto generation, with little manual control. It doesn't appear you can move a plant if you don't like where it is. This is a key factor for the environmental artists on my team. In fact, you should talk to environmental artists who will be using the tool. You don't want to design and code in a vacuum.

You could go with an entirely automated system where the MMIs you use are marked as managed, so that as the user paints biomes, the system regenerates those MMIs. Your system shouldn't touch existing MMIs that have been carefully hand placed. It might be a bit redundant in code and would be wasteful if the user wants to have both autogenerated and manual placement in the same area, which I forsee happening all the time. A person generates an area, likes it, but wants a few more trees or rocks. Now you have 2x the MMIs + LOD MMIs in the same area.

Alternatively, your system could drop continuous, dynamic regeneration and instead populate an area statically. It generates transforms which go into the existing system. This means all of those instances can be manually removed or added to. This is less flashy for demos, but I think much more usable for artists, reuses existing code, and more optimal using the same MMIs for auto and manual.

You could provide a button to manually regenerate the biome so non-artists could randomly iterate until they find things they like. But it should default to an option of regenerating within a brush radius so they don't accidentally regenerate areas they've already finished. A global regenerate option is fine.

Other key points:

• The system needs to be able to undo, which the current system already has built in.
• Currently instances move with the terrain as it is sculpted. That functionality should remain, which it will if generating within the current system.
• Artists cannot move instances, but they can remove them, and can readd the same type elsewhere.
• Some features we need:
  • Pick an instance on the ground to select it in the asset dock
  • From the asset dock, Highlight all instances on the ground of this type
  • Add a search bar in the asset dock to filter by name

re: multimesh, I think I should size them depending on the selected asset footprint, instead of doing MMIs that can span entire regions to get some mileage out of frustum culling.

Look at our instancer. We already parse transforms into a grid of many MMIs to take advantage of frustum, distance, and occlusion culling.

Using the PR that allows us to manipulate MMI buffers via a compute shader could be a great addition, but I'm not exactly sure what we would use it for right now. If used, we'd need a CPU fallback for non-vulkan systems.

I'd like some precision on how future proof everything needs to be ? For example I might use a density map per asset directly for now

Godot is the dominant open source engine, and Terrain3D is the dominant terrain. Whatever gets merged needs to be usable by people who just picked up the engine and pros. It needs to have sustainable performance that could be used in a commercial game. In Out of the Ashes we have over 50 assets in the instancer and are adding more. Terrain3D currently supports up to 65.5km x 65.5km. We couldn't have a density map of that size per asset stored in VRAM. But highly packed data streamed in that populates MMI transforms around the camera could work.

later I'll probably need some more complex object (like the Layers described in the Guerrilla talk), which could break during upgrade without special precautions.

If you make a fork just for your project you can do whatever and break whatever. But if you want people to use your work, it needs to be integrated into the system. It can move the system in a better direction but needs to be done in coordination with me and other maintainers. It needs to be designed and implemented in a clean way that someone else can understand, and fix or modify a year or two from now without you.

So here is one suggestion of what could be wrapped up in a complete project scope:

• Biome-lite - select meshes w/ checkboxes and scatter while manually painting - no density map
• Biome asset creation - Group mesh selections into a Terrain3DBiome. New biome tab on the asset dock, which can also bring up mesh selection on creation. I would do both. Terrain3DBiome has specific settings stored in Terrain3DAssets, while ground data stored in Terrain3DRegion.
• Biome painter
  • A start on a custom data painter with layers for biomes, and eventually layers navigation, and custom user data
  • Static placement into existing MMIs
  • A manual regeneration button for biome placement within a brush radius, and optionally the whole biome

Optional additions as time allows:

• Dynamically generated collision for instances as the camera moves around. This already happens for terrain and is one of the next features on the list
• Asset dock search
• Pick an instance on the ground to select it in the asset dock
• Highlight all instances of one type from the asset dock
• Dynamically generate (and destroy) MMIs as the camera moves. Currently it's all generated at scene start. But visible distances must be accounted for. Modifying the MMI buffers could be useful here. (lower priority as having thousands of MMI nodes isn't a big issue)
• Instance scenes instead of MMIs (lowest priority, no users requested it, fine if it isn't implemented)

You mentioned before you want to make heavy use of compute. But you can do most or all of this work without using compute at all and it would meet the stated goal above and be appreciated and used by many users. Perhaps you will use compute for density maps or scattering. I'm not clear there. To me, adding in compute and a CPU fallback is in danger of blowing out a 6-week window. Over time, there are areas we want to use compute in as well, such as creating a GPU workflow for terrain editing, and perhaps an instancer overhaul using compute to populate MMI buffers. But one step at a time.

Finally, UI work is hard, finicky, and time consuming to get operating smoothly. But it's necessary. Bad UIs won't get used. Plan time to work with it.

@Xtarsia Your thoughts?

[TokisanGames]

Also note this post: https://discord.com/channels/691957978680786944/1185492572127383654/1353729104133754922

[Basil42]

I can offer to collaborate on this, they seem already far along.

[TokisanGames]

It may not fit our ideal needs, which I think I've outlined above. And if it's in a separate repo, it won't get used as much as anything merged into this one. But if it does meet our needs, it certainly could be a collaborative effort. At the least, I just want to open the discussion.

[Xtarsia]

If building ontop of the instancer;

Then you have almost free-reign on how you're generating the xforms, and can pass the resulting xform list, per asset type directly to the current system (at the right entry point) and it'll handle the rest.

To keep editing hand placed, and generated seperate, then there would need to be 2 concurrent data types, that co-exist within each cell and have 2 sets of xform/color arrays which are appended when the mmi are generated. The current add xforms functions can have a flag that determines the target generated/hand-placed arrays, generated can replace, instead of append existing too.

This would allow hand crafted, and generated to share the same mmi. The remove tool could have an alt mode that removes from the generated arrays as well (if only doing static editor generation for now).

It could also be possible to have the editor show only generated, or only manual instances if the data is kept seperate.

Moving the nav data to a custom data map, and having data painting makes sense to me too. The extra map, would only need to be included in the editor shader, and not the main shader - so VRAM increase wouldnt occur from that at runtime. It also frees up 1 bit of the control map.

This could then be a case of painting the biome, then generating after the fact or, it could even generate on the fly as the biome is painted, based on the brush aabb size, just like update_transforms works now.

Also, hard agree on the UI point Cory made.