Camera sensor in mujoco model

Discussion in 'Modeling' started by Adarsh Sehgal, May 1, 2018.

  1. I have a model of AUBO i5 in mujoco. I want to put a camera on the robot and get the images from the camera (like in real world camera). I'm not sure how to add camera to the model and how to get the images from the camera and save it (say in a folder). Also is it possible to access the camera images in mujoco-py?

    Any help will be highly appreciated.

    Thank you,
    Adarsh
     
  2. Emo Todorov

    Emo Todorov Administrator Staff Member

    You can define model cameras with the <camera> element in the XML. Then at runtime, you have to switch to rendering from the model camera instead of the default free camera. See code sample simulate.cpp. Then, after each frame, you have to copy the pixels from the OpenGL frame buffer into your buffer. See function mjr_readPixels in the documentation.

    Note that cameras are not handled as other sensors in MuJoCo, because all "official" sensors are being updates at the simulation rate, while cameras are normally much slower. So getting data from cameras is left to the user, but the key functionality is provided in the API.

    As for the python wrapper, I am not using it but I assume it exposes the necessary functions.