I have trained a reinforcement learning policy and it solved to task of grasping by finding the fault in my model. When it performs fast movement it is able to penetrate the object and efectively keep the object in the hand. Video: Is there any way to prevent such a object penetration?
Smaller time steps could do it. If you allow your agent to use physically implausible input (max speed, max input torque ect...), you shouldn't be surprised that your trained agent exhibits physically implausible skills. How do you control the position of your hand. Is it welded to a mocap body and you control the position of that body with your agent?
Hi! In the end I had to switch to another env anyway, so didn't work anymore on that, but have you tried adding the actuator limits as Florian has proposed?
Yeah I tried lowering the actuator ctrlrange, as well as the timestep. The penetration still persists. Don't know what the issue could be. It's even a standard hand model so I doubt if the actuator limits could be the problem.
It is rather the velocity or force limits or something else depending on the actuator type you are using. Try to have a look in the documentation what are the suitable parameters for the actuator you are using.