Hi all, I am now working on reading color images from the simulated camera based on MuJoCo Pro, however, I am a little confused about the code: 1. I used the API function mj_printData and acquired different sorts of data. Does it include the raw color data reading from the camera? If so, how can I extract them from the whole data? 2. In the simulate.cpp file, function mjr_getBackbuffer only read data of the depth map. How can I figure out a way to read raw color data instead of depth data? Also, how can I use the API function mjr_showBuffer to show data of the camera instead of the depth map? I really appreciate it if you guys can give me some suggestions on reading raw color data from the simulator. Thanks for your patience. best, Luowei
The pixels are not stored in mjData, but rendered on the fly. If you look in simulate.cpp in the function void render(GLFWwindow* window) after the call to mjr_render(0, rect, &objects, &ropt, &cam.pose, &con); openGL will have written the pixel values. After that you can call glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, buffer); with your own pre-allocated buffer, and you'll get the pixels.
You can do what Yuval said, and in general you can call any OpenGL function. But mjr_getBackbuffer is designed to help you in this case. In the code sample, the first argument in the call to mjr_getBackbuffer is a NULL pointer. If you call it with a pointer different from NULL, it will copy the RGB data into the buffer you provided. Of course internally it just calls glReadPixels, so in terms of performance there is no difference.