Instance segmentation (#8334)

* added cpp client build docs (#7942)

* fixed IMU units (#7960)

* Update README.md with new TinyURL links (#7988)

* Added inverse transform (#7999)

Co-authored-by: glopezdiest <[email protected]>

* Aaron/fixwheelchair (#8001)

* Fix OSM2ODR build

* Updated fix wheelchair default value

* Docs/unit updates (#8007)

* fixed IMU units

* updated autitwheel version

* Add a `*.pyi` file for auto-completion & hints.

To enable auto-completion and hints in code editors such as VScode, create a `*.pyi` file. This feature is compatible with `python 3.9` and later versions.

* Fixes and missing Iterators

* Fixed Actor.parent

Can be None or an Actor

* Fixed missing return types

* Updated changelog

needs merge with dev version

* Added DSVEventArray iterator

* Added missing type for Labelled Point

* Fixed spelling misstakes

* Removed wrong unit indication

* Added missing -> World to load_world

* Added missing return value to reload_world

* FIX: __init__ methods do not return

* FIX: added ApplyTransform, fixed ApplyTorque

* Filled in missing information and types.

* ActorList.filter actually returns ActorList

* Fixed CityObjectLabels

* Disambiguated get_waypoint signature

Syntax fix (squased)

* Added undocumented variables

FutureActor
laod_world_if_different

* Corrected Sensor.is_listening

Was changed to a function in 0.9.15. More info see: #7439

* Added type hints for `values` attribute on enums

* Fix intendation shadowing methods

* Fix missing @Property

* Formatted some docstring to be shorter

* Added stubs for HUD drawing

Functions from #7168

* Corrected and more precise type-hints

- fixed carla.Waypoint.next_until_lane_end

* Improved get_waypoint disambiguation

correctly added two overload function

* Fix spelling mistakes

* Better usage of Enum if typing.Self is availiable

Using Self will not report an override / incompatible error.

* Fix: Enum values were tuples. Added Flag or Int to Enums

* Fixes for wrong stubs

- OpendriveGenerationParameter had no init
- missing @Property
- wrong signatures

* Added self parameter to property signatures

* Various fixes

- wrong signatures
- wrong names

* Added setters for VehicleControl

* Improved get_waypoints and Literal type hints

* Corrected [try_]spawn_actor keyword name

* Added Transform.inverse_transform and corrected signature

parameter is called in_point not in_vector

* Improved Callable and callbacks signature

* Corrections and additions

more setters
missing, wrong types corrected
spelling

* Fixed Vector arithmetic

* added digital twins video (#8090)

* use actor ids for instance segmentation

* synchronize bboxes between server and client

* add actor_id attribute to bounding boxes

* navigation information is now loaded when changing maps

* Porting the changes done to UE5 to fix the recording leak to UE4

The slowdown is considerably more noticeable here since the engine runs much smoother. This makes evident that this is a stopgap measure, and should be looked into further down the line.

* Fixed typo in CityScapes palette (#8137)

* Correcting makefile typo to avoid override warning for target "downloadplugins" (#8167)

The downloadplugins target is already defined below (line 162).

* fix typo in title (#8225)

* added unreal coord system, fixed v2x (#8251)

---------


Co-authored-by: Minokori <[email protected]>
Co-authored-by: Daniel <[email protected]>
Co-authored-by: Sergio Paniego Blanco <[email protected]>
Co-authored-by: Ylmdrin <[email protected]>

Author: 5 people

CHANGELOG.md

@@ -21,7 +21,9 @@
  * Added named tuples for BasicAgent.py's detection result to allow for type-hints and better semantics.
  * Added type-hint support for the PythonAPI
  * Added type-hints to GlobalRoutePlanner and use carla.Vector3D code instead of pre 0.9.13 numpy code.
-
+ * If available, use ActorIDs instead of Unreal Engine IDs for instance segmentation
+ * Synchronized actor BoundingBox between server and client
+ * Add actor_id to bounding boxes
 
 ## CARLA 0.9.15
 

Docs/adv_cpp_client.md

@@ -138,4 +138,3 @@ TOOLCHAIN=$(CURDIR)/ToolChain.cmake
 
 
 
-

LibCarla/source/carla/client/Actor.cpp

@@ -32,6 +32,10 @@ namespace client {
     return GetEpisode().Lock()->GetActorAcceleration(*this);
   }
 
+  geom::BoundingBox Actor::GetBoundingBox() const {
+    return GetEpisode().Lock()->GetActorBoundingBox(*this);
+  }
+
   geom::Transform Actor::GetComponentWorldTransform(const std::string componentName) const {
     return GetEpisode().Lock()->GetActorComponentWorldTransform(*this, componentName);
   }

LibCarla/source/carla/client/Actor.h

@@ -26,7 +26,6 @@ namespace client {
       : LIBCARLA_INITIALIZE_LIFETIME_PROFILER(init.GetDisplayId()),
         Super(std::move(init)) {}
 
-    using ActorState::GetBoundingBox;
 
     virtual ~Actor() = default;
 
@@ -60,6 +59,8 @@ namespace client {
     /// acceleration calculated after the actor's velocity.
     geom::Vector3D GetAcceleration() const;
 
+    geom::BoundingBox GetBoundingBox() const;
+
     geom::Transform GetComponentWorldTransform(const std::string componentName) const;
 
     geom::Transform GetComponentRelativeTransform(const std::string componentName) const;

LibCarla/source/carla/client/detail/Client.cpp

@@ -418,6 +418,10 @@ namespace detail {
     _pimpl->AsyncCall("add_actor_torque", actor, vector);
   }
 
+  geom::BoundingBox Client::GetActorBoundingBox(rpc::ActorId actor) {
+    return _pimpl->CallAndWait<geom::BoundingBox>("get_actor_bounding_box", actor);
+  }
+
   geom::Transform Client::GetActorComponentWorldTransform(rpc::ActorId actor, const std::string componentName) {
     return _pimpl->CallAndWait<geom::Transform>("get_actor_component_world_transform", actor, componentName);
   }

LibCarla/source/carla/client/detail/Client.h

@@ -238,6 +238,9 @@ namespace detail {
         rpc::ActorId actor,
         const geom::Vector3D &vector);
 
+    geom::BoundingBox GetActorBoundingBox(
+        rpc::ActorId actor);
+
     geom::Transform GetActorComponentWorldTransform(
         rpc::ActorId actor,
         const std::string componentName);

LibCarla/source/carla/client/detail/Simulator.h

@@ -447,6 +447,10 @@ namespace detail {
       return GetActorSnapshot(actor).acceleration;
     }
 
+    geom::BoundingBox GetActorBoundingBox(const Actor &actor) {
+      return _client.GetActorBoundingBox(actor.GetId());
+    }
+
     geom::Transform GetActorComponentWorldTransform(const Actor &actor, const std::string componentName) {
       return _client.GetActorComponentWorldTransform(actor.GetId(), componentName);
     }

LibCarla/source/carla/geom/BoundingBox.h

@@ -32,6 +32,12 @@ namespace geom {
     // -- Constructors ---------------------------------------------------------
     // =========================================================================
 
+    explicit BoundingBox(const Location &in_location, const Vector3D &in_extent, const Rotation &in_rotation, const uint32_t &in_actor_id)
+      : location(in_location),
+        extent(in_extent),
+        rotation(in_rotation),
+        actor_id(in_actor_id) {}
+
     explicit BoundingBox(const Location &in_location, const Vector3D &in_extent, const Rotation &in_rotation)
       : location(in_location),
         extent(in_extent),
@@ -50,6 +56,7 @@ namespace geom {
     Location location;  ///< Center of the BoundingBox in local space
     Vector3D extent;    ///< Half the size of the BoundingBox in local space
     Rotation rotation;  ///< Rotation of the BoundingBox in local space
+    uint32_t actor_id;
 
     // =========================================================================
     // -- Other methods --------------------------------------------------------
@@ -137,11 +144,12 @@ namespace geom {
     BoundingBox(const FBoundingBox &Box)
       : location(Box.Origin),
         extent(1e-2f * Box.Extent.X, 1e-2f * Box.Extent.Y, 1e-2f * Box.Extent.Z),
-        rotation(Box.Rotation) {}
+        rotation(Box.Rotation),
+        actor_id(Box.ActorId) {}
 
 #endif // LIBCARLA_INCLUDED_FROM_UE4
 
-    MSGPACK_DEFINE_ARRAY(location, extent, rotation);
+    MSGPACK_DEFINE_ARRAY(location, extent, rotation, actor_id);
   };
 
 } // namespace geom

LibCarla/source/carla/rpc/BoundingBox.h

@@ -0,0 +1,17 @@
+// Copyright (c) 2019 Computer Vision Center (CVC) at the Universitat Autonoma
+// de Barcelona (UAB).
+//
+// This work is licensed under the terms of the MIT license.
+// For a copy, see <https://opensource.org/licenses/MIT>.
+
+#pragma once
+
+#include "carla/geom/BoundingBox.h"
+
+namespace carla {
+namespace rpc {
+
+  using BoundingBox = geom::BoundingBox;
+
+} // namespace rpc
+} // namespace carla

PythonAPI/carla/source/libcarla/Geom.cpp

@@ -266,6 +266,7 @@ void export_geom() {
     .def_readwrite("location", &cg::BoundingBox::location)
     .def_readwrite("extent", &cg::BoundingBox::extent)
     .def_readwrite("rotation", &cg::BoundingBox::rotation)
+    .def_readonly("actor_id", &cg::BoundingBox::actor_id)
     .def("contains", &cg::BoundingBox::Contains, arg("point"), arg("bbox_transform"))
     .def("get_local_vertices", CALL_RETURNING_LIST(cg::BoundingBox, GetLocalVertices))
     .def("get_world_vertices", CALL_RETURNING_LIST_1(cg::BoundingBox, GetWorldVertices, const cg::Transform&), arg("bbox_transform"))