[Bug Report] Is there something wrong with the FrankaKitchen camera config? #290
Description
Describe the bug
Something very strange is happening. If I run the code below with the default_camera_config=default_camera_config line commented out so that it uses the default camera config settings, the view is this:
which doesn't seem right. It's the same whether I use the render mode human or rgb_array_list, etc.
To try and fix it, I used the script below so I could pan/rotate/zoom the view around with my mouse and get a good set of camera config values. However, I noticed that the lookat was always the same exact three numbers, even though the specific number changed with panning.
With a bit more poking and trying to set the lookat value in the camera config passed in, I found that it definitely IS responding to distance, azimuth, elevation as expected, but it appears to take the LAST value of lookat (i.e., lookat[2]), and set the whole array to that value.
For example, with the "lookat": np.array([0.0, 0.0, 0.4]) I have below, when I run it, without changing the view at all, it prints:
Current free-camera config:
distance = 5.0
azimuth = 52.2
elevation = -32.7
lookat = (0.4, 0.4, 0.4), type(cam.lookat) = <class 'numpy.ndarray'>, cam.lookat.shape = (3,)
trackbody = -1
Note the entries of lookat, all the same. Maybe I'm missing something silly, but this doesn't seem intended?
I tried tracking down where it uses the lookat value, which appears to be here, but I don't see anything obviously wrong with that.
Code example
#!/usr/bin/env python3
"""
Minimal FrankaKitchen demo:
- Registers gymnasium_robotics envs
- Creates FrankaKitchen-v1 with render_mode="human"
- Prints MuJoCo camera names (if possible)
- Runs a short random-policy rollout with on-screen rendering
"""
import time
from typing import Optional
import gymnasium as gym
import gymnasium_robotics # noqa: F401 # needed so register_envs can see it
import numpy as np
try:
import mujoco
except ImportError:
mujoco = None
print("[WARN] Could not import `mujoco`. Camera name listing will be skipped.")
def find_mujoco_model(env) -> Optional["mujoco.MjModel"]:
"""Best-effort search for a MuJoCo MjModel inside a Gymnasium-Robotics env."""
if mujoco is None:
return None
u = env.unwrapped
# Common places where the model might live
candidates = [
u,
getattr(u, "sim", None),
getattr(u, "_sim", None),
getattr(u, "mujoco_renderer", None),
getattr(u, "_mujoco_renderer", None),
getattr(u, "renderer", None),
getattr(u, "_renderer", None),
]
for obj in candidates:
if obj is None:
continue
# Directly a model?
if isinstance(obj, mujoco.MjModel):
return obj
# Has a `.model` attribute?
m = getattr(obj, "model", None)
if isinstance(m, mujoco.MjModel):
return m
# Some renderers store sim, which stores model
sim = getattr(obj, "sim", None)
if sim is not None:
m2 = getattr(sim, "model", None)
if isinstance(m2, mujoco.MjModel):
return m2
return None
def print_camera_names(env) -> None:
if mujoco is None:
print("[INFO] mujoco is not available; cannot list camera names.")
return
model = find_mujoco_model(env)
if model is None:
print(
"[INFO] Could not locate MuJoCo model on env; camera listing not available."
)
return
print(f"\nFound {model.ncam} cameras:")
for cam_id in range(model.ncam):
name = mujoco.mj_id2name(model, mujoco.mjtObj.mjOBJ_CAMERA, cam_id)
# mj_id2name can return None for unnamed cameras
if name is None:
name = f"<unnamed_{cam_id}>"
print(f" id {cam_id:2d}: {name}")
print()
def get_viewer(env):
"""Best-effort: get the MuJoCo viewer object from a wrapped env."""
base = env.unwrapped
# Common places Gymnasium / Gymnasium-Robotics put the renderer
candidates = [
getattr(base, "mujoco_renderer", None),
getattr(base, "_renderer", None),
getattr(base, "renderer", None),
getattr(base, "robot_env", None),
]
for rend in candidates:
if rend is None:
continue
rend = getattr(rend, "mujoco_renderer", rend)
viewer = getattr(rend, "viewer", None)
if viewer is not None:
return viewer
# Fallback: some older envs stick viewer directly on the env
viewer = getattr(base, "viewer", None)
if viewer is not None:
return viewer
raise RuntimeError(
"Could not find viewer on env; check dir(env.unwrapped) to see where it lives."
)
def dump_cam(env):
viewer = get_viewer(env)
cam = viewer.cam # mjvCamera
print("\nCurrent free-camera config:")
print(f" distance = {cam.distance}")
print(f" azimuth = {cam.azimuth}")
print(f" elevation = {cam.elevation}")
print(
f" lookat = ({cam.lookat[0]}, {cam.lookat[1]}, {cam.lookat[2]}), {type(cam.lookat) = }, {cam.lookat.shape = }"
)
print(f" trackbody = {cam.trackbodyid}")
def main():
# Register all robotics envs
gym.register_envs(gymnasium_robotics)
default_camera_config = {
"distance": 5,
"azimuth": 52.2,
"elevation": -32.7,
"lookat": np.array([0.0, 0.0, 0.4]),
}
# Create FrankaKitchen with on-screen rendering
env = gym.make(
"FrankaKitchen-v1",
render_mode="human",
default_camera_config=default_camera_config,
)
print("Created env:", env)
print("Action space:", env.action_space)
print("Observation space:", env.observation_space)
# List camera names (best effort)
print_camera_names(env)
_, _ = env.reset(seed=0)
print("\nStarting random rollout...")
for t in range(30000):
_, _, _, _, _ = env.step(env.action_space.sample())
# Small sleep to make the human render visible / not crazy fast
time.sleep(1.0 / 30.0)
if t % 50 == 0:
dump_cam(env)
env.close()
print("Done.")
if __name__ == "__main__":
main()System Info
Describe the characteristic of your environment:
$ pip freeze | grep gymnasium
gymnasium==1.2.1
gymnasium-robotics==1.4.1
$ python --version
Python 3.12.3
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04.3 LTS
Release: 24.04
Codename: noble
Checklist
- I have checked that there is no similar issue in the repo (required)