in the obb-collider component's updateBoundingBox function, it stores the trackedObject3D's rotation, but restores it by assigning it to this.el.object3D's rotation instead of trackedObject3D
I propose changing this section around line 162:
      // Restore rotations.
      trackedObject3D.parent.matrixWorld.compose(auxPosition, auxQuaternion, auxScale);
      this.el.object3D.rotation.copy(auxEuler); // issue - assigns it to this.el.object3D.rotation
to
      // Restore rotations.
      trackedObject3D.parent.matrixWorld.compose(auxPosition, auxQuaternion, auxScale);
      trackedObject3D.rotation.copy(auxEuler); // fix - assigns it to trackedObject3D