Implemented possibility to change CARLA's behavior whether the instance segmentation traverses translucent materials or not. (#8909)

DepthCamera and SemanticSegmentationCamera now also use the TaggedSceneProxies instead of the default ones, such that they also benefit from this feature.
Users can set the desired behavior via World::SetAnnotationsTraverseTranslucency (C++) and world.set_annotations_traverse_translucency (Python).

Author: tdegraaff

CHANGELOG.md

@@ -35,6 +35,7 @@
  * Set to default Visual Studio 2022 in Windows
  * Added env CARLA_CACHE_DIR to be able to set CARLA CACHE location
  * Support of masked materials in instance segmentation, resulting in fine-grained annotations on e.g. leaves or fences (as in semantic segmentation)
+ * Added API function `world.set_annotations_traverse_translucency` and implemented functionality to configure, whether depth and semantic + instance segmentation traverse translucent materials or not.
 
 ## CARLA 0.9.15
 

Docs/python_api.md

@@ -3837,6 +3837,10 @@ Retrieves an object containing weather parameters currently active in the simula
     - **Setter:** _[carla.World.set_weather](#carla.World.set_weather)_  
 
 ##### Setters
+- <a name="carla.World.set_annotations_traverse_translucency"></a>**<font color="#7fb800">set_annotations_traverse_translucency</font>**(<font color="#00a6ed">**self**</font>, <font color="#00a6ed">**enable**</font>)  
+Sets CARLA's behavior whether the semantic + instance segmentation and depth annotations traverse translucent materials or not. To find the closest obstacle, this should be set to false (default). If you are interested in annotations that detect objects behind windows, this should be enabled.  
+    - **Parameters:**
+        - `enable` (_bool_) - Enables or disables the traversal of translucent materials in semantic, instance and depth annotations. __Default is `False`__.  
 - <a name="carla.World.set_pedestrians_cross_factor"></a>**<font color="#7fb800">set_pedestrians_cross_factor</font>**(<font color="#00a6ed">**self**</font>, <font color="#00a6ed">**percentage**</font>)  
     - **Parameters:**
         - `percentage` (_float_) - Sets the percentage of pedestrians that can walk on the road or cross at any point on the road. Value should be between `0.0` and `1.0`. For example, a value of `0.1` would allow 10% of pedestrians to walk on the road. __Default is `0.0`__.  

LibCarla/source/carla/client/World.cpp

@@ -394,5 +394,10 @@ namespace client {
     }
   }
 
+  void World::SetAnnotationsTraverseTranslucency(
+      bool enable) {
+    _episode.Lock()->SetAnnotationsTraverseTranslucency(enable);
+  }
+
 } // namespace client
 } // namespace carla

LibCarla/source/carla/client/World.h

@@ -96,7 +96,7 @@ namespace client {
 
     /// Get Gravity value used for IMUI Sensor accelerometer calculation
     float GetIMUISensorGravity() const;
-    
+
     /// Set Gravity value used for IMUI Sensor accelerometer calculation
     void SetIMUISensorGravity(float NewIMUISensorGravity);
 
@@ -237,6 +237,10 @@ namespace client {
         const rpc::TextureFloatColor& normal_texture,
         const rpc::TextureFloatColor& ao_roughness_metallic_emissive_texture);
 
+    /// Enables or disables the traversal of translucent materials in semantic, instance and depth annotations. Defaults to False.
+    void SetAnnotationsTraverseTranslucency(
+        bool enable);
+
     std::vector<std::string> GetNamesOfAllObjects() const;
 
   private:

LibCarla/source/carla/client/detail/Client.cpp

@@ -777,6 +777,11 @@ namespace detail {
     return _pimpl->CallAndWait<return_t>("cast_ray", start_location, end_location);
   }
 
+  void Client::SetAnnotationsTraverseTranslucency(
+      bool enable) {
+    _pimpl->AsyncCall("set_annotations_traverse_translucency", enable);
+  }
+
 } // namespace detail
 } // namespace client
 } // namespace carla

LibCarla/source/carla/client/detail/Client.h

@@ -480,6 +480,8 @@ namespace detail {
     std::vector<rpc::LabelledPoint> CastRay(
         geom::Location start_location, geom::Location end_location) const;
 
+    void SetAnnotationsTraverseTranslucency(
+        bool enable);
   private:
 
     class Pimpl;

LibCarla/source/carla/client/detail/Simulator.h

@@ -300,6 +300,11 @@ namespace detail {
       return _client.CastRay(start_location, end_location);
     }
 
+    void SetAnnotationsTraverseTranslucency(
+      bool enable) {
+        _client.SetAnnotationsTraverseTranslucency(enable);
+      }
+
     /// @}
     // =========================================================================
     /// @name AI

PythonAPI/carla/source/carla/libcarla.pyi

@@ -8056,6 +8056,15 @@ class World:
     # endregion
 
     # region Setters
+    def set_annotations_traverse_translucency(self, enable: bool):
+        """Sets CARLA's behavior whether the semantic + instance segmentation and depth annotations traverse translucent materials or not.
+        To find the closest obstacle, this should be set to false (default). If you are interested in annotations that detect objects
+        behind windows, this should be enabled.
+
+        Args:
+            `enable (bool)`: Enables or disables the traversal of translucent materials in semantic, instance and depth annotations. Defaults to False.
+        """
+
     def set_pedestrians_cross_factor(self, percentage: float):
         """Sets the percentage of pedestrians that can walk on the road or cross at any point on the road.
 

PythonAPI/carla/source/libcarla/World.cpp

@@ -363,6 +363,7 @@ void export_world() {
         self.ApplyTexturesToObjects(PythonLitstToVector<std::string>(list), diffuse_texture, emissive_texture, normal_texture, ao_roughness_metallic_emissive_texture);
       }, (arg("objects_name_list"), arg("diffuse_texture"), arg("emissive_texture"), arg("normal_texture"), arg("ao_roughness_metallic_emissive_texture")))
     .def(self_ns::str(self_ns::self))
+    .def("set_annotations_traverse_translucency", CALL_WITHOUT_GIL_1(cc::World, SetAnnotationsTraverseTranslucency, bool), (arg("enable")))
   ;
 
 #undef SPAWN_ACTOR_WITHOUT_GIL

PythonAPI/docs/world.yml

@@ -814,6 +814,15 @@
         If the layer is already unloaded the call has no effect.
       warning: This only affects "Opt" maps. The minimum layout includes roads, sidewalks, traffic lights and traffic signs.
     # --------------------------------------
+    - def_name: set_annotations_traverse_translucency
+      params:
+      - param_name: enable
+        type: bool
+        doc: >
+          Enables or disables the traversal of translucent materials in semantic, instance and depth annotations. __Default is `False`__.
+      doc: >
+        Sets CARLA's behavior whether the semantic + instance segmentation and depth annotations traverse translucent materials or not. To find the closest obstacle, this should be set to false (default). If you are interested in annotations that detect objects behind windows, this should be enabled.
+    # --------------------------------------
     - def_name: set_pedestrians_cross_factor
       params:
       - param_name: percentage