Option to use compressed color map #842

stan4dbunny · 2025-11-05T10:30:25Z

Fixes #379

I have implemented an option to use compressed color maps when the game is running (i.e when not editing). When the compress option is used and the scene is saved, the regions will all get a compressed color map. When the game is run (either launched from editor or when exported), the compressed color map will be used and the uncompressed color map will be unloaded.

I have tested that the compressed texture is used instead of the normal one when the game is running with the option enabled by analyzing the frame in Renderdoc. The compressed color map is also updated after the terrain has been sculpted/painted in the editor.

With the compressed color map, the color map size is 4x smaller.

I have also added some documentation in the existing style.

One thing that I noticed is that when there is a large terrain with a lot of regions, the saving can take a long time (several minutes) when the option has been clicked, but this should only happen once. Maybe this time could be reduced if the compression happend on the entire color map at once instead of for the tile in each region.

TokisanGames

Thanks for the PR. Here are some notes:

Your commits are coming on another account. You probably want to fix that before I merge. You can also squash your commits at the same time.

src/terrain_3d_region.h

src/terrain_3d_region.cpp

src/terrain_3d.h

src/terrain_3d_data.cpp

src/terrain_3d_region.cpp

src/terrain_3d_data.cpp

src/terrain_3d_region.cpp

src/terrain_3d_data.cpp

src/terrain_3d.cpp

src/terrain_3d.h

src/terrain_3d_region.cpp

project/addons/terrain_3d/src/editor_plugin.gd

project/addons/terrain_3d/src/export_plugin.gd

TokisanGames · 2025-11-11T19:51:25Z

#825 has been merged. From now on the principle is any setter function that is called with the same value is silently ignored. Logs, signals, and updates are not triggered unless the value is different. Once you rebase, you can use SET_IF_DIFF(_member_var, p_new_param); which will return if they are the same or set it if different, for most values including native types, pointers, and Godot Dictionaries and Arrays (compared by internal pointer). Occasionally a bit more complexity is warranted so a few setters accomplish the same idea with different means.

src/terrain_3d_region.cpp

project/addons/terrain_3d/src/export_plugin.gd

TokisanGames

Nice job so far with the code and the testing. Running the demo shows the right format change and VRAM reduction.

I have noticed one issue so far:

If compression mode is set and free_uncompressed_color_maps is false, it still frees the uncompressed colormap, when it shouldn't.

var region: Terrain3DRegion = $Terrain3D.data.get_region(Vector2i(0,0))
print("Color map: ", region.color_map)
print("Compressed color map: ", region.compressed_color_map)

color_compression = bptc
free_uncompressed_color_maps = false (same if true)
Color map: <Object#null>
Compressed color map: <Image#-9223371995985476204>

color_compression = none
free_uncompressed_color_maps = false (same if true)
Color map: <Image#-9223371996035807859>
Compressed color map: <Object#null>

You can also rebase and squash all of your commits.

src/terrain_3d.h

src/terrain_3d_region.h

src/terrain_3d.cpp

src/terrain_3d_region.cpp

stan4dbunny · 2025-12-02T09:53:10Z

Hello! I have now fixed the changes you requested. Let me know if anything should be changed.

Just wondering, but regarding squashing would it not be easier for you to squash the commits through the "squash and merge" GitHub option?

TokisanGames · 2025-12-03T11:23:56Z

Hello! I have now fixed the changes you requested. Let me know if anything should be changed.

Great, thank you.

Just wondering, but regarding squashing would it not be easier for you to squash the commits through the "squash and merge" GitHub option?

No, it needs to be squashed and rebased. Otherwise it will make a mess of the history. Github does one or other other, not both. And it doesn't rebase if there are conflicts, which is the case.

There are currently conflicts and 25 commits. Please squash your commits down to two: code changes in one, and docs in the other. Then rebase onto the latest main and resolve any conflicts. The docs explain how, and I can help on discord.

When I prepare it for merging I will tweak the code and docs, generate the rst files, and csharp bindings. It's helpful if I don't also have to squash, rebase, and fix conflicts myself.

…runtime. Option to free the uncompressed color map during runtime and export when the compression mode is set.

TokisanGames

Looking better, thank you. I have now taken over this PR. Comments are for your edification only.

Pending:

Review Terrain3DRegion:
- free_uncompressed_color_map still frees when disabled.
- rework santisize_maps()

TokisanGames · 2025-12-08T21:17:45Z

src/terrain_3d.cpp


+void Terrain3D::set_color_compression_mode(const ColorCompressMode p_color_compression_mode) {
+	SET_IF_DIFF(_color_compression_mode, p_color_compression_mode);
+	switch (_color_compression_mode) {


Better to convert this on the very rare get godot version call, then we don't need to store the extra variable.

TokisanGames · 2025-12-08T21:19:17Z

src/terrain_3d.cpp

+	}
+	LOG(INFO, "Setting compression mode for color maps: ", _image_color_compression_mode);
+	TypedArray<Terrain3DRegion> regions = _data->get_regions_active();
+	for (int i = 0; i < regions.size(); i++) {


We now use ranged for loops

for (Ref<Terrain3DRegion> region : regions) { region->set_modified(true); }

TokisanGames · 2025-12-08T21:20:50Z

project/addons/terrain_3d/src/export_plugin.gd

+
+
+func _customize_scene(scene: Node, path: String) -> Node:
+	var _terrain: Terrain3D = scene.find_child("Terrain3D", true)


_ is our convention for class private variables like _hash.
terrain is local, so does not use _.

TokisanGames · 2025-12-08T21:22:31Z

project/addons/terrain_3d/src/editor_plugin.gd

 # Includes
 const Terrain3DUI: Script = preload("res://addons/terrain_3d/src/ui.gd")
 const RegionGizmo: Script = preload("res://addons/terrain_3d/src/region_gizmo.gd")
+const ExportPlugin = preload("res://addons/terrain_3d/src/export_plugin.gd")


TokisanGames · 2025-12-08T21:25:10Z

src/terrain_3d_data.h

 	// File I/O
 	void save_directory(const String &p_dir);
-	void save_region(const Vector2i &p_region_loc, const String &p_dir, const bool p_16_bit = false);
+	void save_region(const Vector2i &p_region_loc, const String &p_dir, const bool p_16_bit = false, const Image::CompressMode p_color_compression_mode = Image::COMPRESS_MAX);


This should use our color compression mode.

TokisanGames · 2025-12-08T21:29:49Z

src/terrain_3d.h

 		SIZE_2048 = 2048,
 	};

+	enum ColorCompressMode {


This is a property of the maps, so should be in Terrain3DRegion, along with the Image Format.

TokisanGames · 2025-12-08T22:51:43Z

src/terrain_3d_data.cpp

 			// Generate all or only those marked edited
 			if (region && (p_all_regions || region->is_edited())) {
-				region->get_color_map()->generate_mipmaps();
+				region->get_map(TYPE_COLOR)->generate_mipmaps();


Hmm, this cannot be done on a compressed map. So, it must be done on the uncompressed if valid or not at all.

TokisanGames · 2025-12-08T22:55:39Z

src/terrain_3d_region.h

+	void set_compressed_color_map(const Ref<Image> &p_map);
+	Ref<Image> get_compressed_color_map() const { return _compressed_color_map; }
+	void free_uncompressed_color_map();
+	void sanitize_maps(const bool p_free_uncompressed_color_maps);


This should have a default, rather than changing all usage cases of it.

TokisanGames · 2025-12-09T17:20:43Z

src/terrain_3d_region.cpp

 	ClassDB::bind_method(D_METHOD("set_color_map", "map"), &Terrain3DRegion::set_color_map);
 	ClassDB::bind_method(D_METHOD("get_color_map"), &Terrain3DRegion::get_color_map);
-	ClassDB::bind_method(D_METHOD("sanitize_maps"), &Terrain3DRegion::sanitize_maps);
+	ClassDB::bind_method(D_METHOD("set_compressed_color_map", "map"), &Terrain3DRegion::set_compressed_color_map);


This PR doesn't currently sanitize() compressed color maps when set. I don't think we even need to offer the ability to set them to the user. I think we offer: get, clear, or generate. Then we don't need to sanitize, since the raw color map is already sanitized.

Oh, but I see we need this function in order to load the compressed color map from disk.

TokisanGames · 2025-12-09T17:29:58Z

src/terrain_3d_region.cpp

 	_control_map = map;
+	if (p_free_uncompressed_color_maps) {
+		free_uncompressed_color_map();
+		return;


I don't think this is the right function to clear the uncompressed maps. And returning here I'm pretty sure is a bug. The purpose of this function is to ensure the existing data is clean and safe to use, not to free data or optimize memory.

TokisanGames · 2025-12-10T09:27:59Z

src/terrain_3d_region.cpp

 		case TYPE_COLOR:
+			if (!IS_EDITOR && _compressed_color_map.is_valid()) {
+				return *_compressed_color_map;
+			}


This isn't the right place for this either. The expectation should be if you ask for a color map you get the editable color map. We need a new function to get the active one which implies the one you get may be compressed and uneditable.

Other functions like set_maps(), get_maps(), get_data() don't even consider the compressed map. It needs to be all or nothing. I think it's cleaner to be nothing - so the compressed map is only created on save, freed in the editor, only accessed by get_compressed_color_map() or get_active_color_map(), everything else gives you the editable color map, though it may be null.

TokisanGames · 2025-12-10T10:25:45Z

I have reorganized some of the code. Now:

All functions that get the colormap, are unchanged, returning the editable colormap().
Maps and functions all refer to color map and compressed color map. No more "uncompressed".
get_active_color_map() returns the compressed map if not editor and valid, which update_maps() uses as read only.
compressed maps are freed in the editor. Viewing a res file after it has been loaded will show an empty compressed map. Closing a scene and viewing the res will show the compressed and uncompressed map.
color maps are freed in game if the option is chosen and the compressed map is valid.
compress_color_map() is now available

Next, I will pull a commit from my dev mode PR to entirely disable the color map.

Then I'll remove free_color_map and instead introduce a color map mode with these options:

Disabled - saved data is unchanged; editing is disabled; new regions create maps; compressed and uncompressed are freed in editor and game;
Cleared - disabled, and maps are cleared on save
Editable - current - no compressed map saved or used, editable in game
Compressed - editable in editor; compressed freed in editor; uncompressed freed in game and stripped on export. This will also enable the compression options.

Does this look like the full list of desired possibilities?

Xtarsia · 2025-12-10T16:28:57Z

Have exported projects to web/mobile been tested for this yet? Ommiting to test those has caught me out before.

TokisanGames · 2026-01-21T10:15:04Z

There might be an issue with compressed maps on linux.

https://discord.com/channels/691957978680786944/1065519581013229578/1463464825316180061

stan4dbunny changed the title ~~Option to use compressed color map~~ Option to use compressed color map + height map 16 bit bug fix Nov 5, 2025

TokisanGames requested changes Nov 5, 2025

View reviewed changes

stan4dbunny marked this pull request as ready for review November 6, 2025 15:44

stan4dbunny marked this pull request as draft November 6, 2025 15:45

stan4dbunny changed the title ~~Option to use compressed color map + height map 16 bit bug fix~~ Option to use compressed color map Nov 7, 2025

stan4dbunny commented Nov 7, 2025

View reviewed changes

src/terrain_3d_region.cpp Show resolved Hide resolved

stan4dbunny commented Nov 7, 2025

View reviewed changes

src/terrain_3d_region.cpp Show resolved Hide resolved