New feature request: add ability to create "phantom" objects in the VRE To my knowledge, all objects in the simulation must have >0 mass and >0 density. While this makes sense for creating objects with which you want to interact in a physically realistic environment, it creates a shortfall for testing our motor decode algorithms within the VRE with human subjects. Ideally, we would like the ability to create translucent "target spheres" that stay in place (i.e. unaffected by gravity) and have no density (i.e. any object can move through it uninterrupted). These spheres would be used as targets for our subjects to try to reach with a specific decoded finger movement with MoCap deactivated. In the same test, other phantom spheres would be placed over the other finger tips at their rest positions. The subject would have to reach the target sphere with a specific finger while keeping the others at their resting position. These spheres could then change color based on whether the finger tips are in their appropriate positions, either in their resting positions or successfully moving to the target. We have used this experiment previously in another VRE to demonstrate the number of hand DOFs we can achieve with our motor decodes. This experiment is fundamentally doable with MuJoCo in its current state by taking advantage of the hx_read_sensors() function. Whether or not a finger is positioned properly can be determined by returning the amount of motor actuation from an hx_read_sensors() call. However, this is not very helpful for the the subject who would have very little visual feedback as to whether their fingers are oriented properly. Currently, the only cues we can give the subject within the VRE are by updating the message via an mjhx_message() call. In a nutshell, we would like the ability to create zero-mass, zero-density phantom objects in the simulations. Combined with my previously-requested collision detection ability, the aforementioned DOF-determining experiment could be carried out in MuJoCo. Even better, we would also like to use the API to change the position and color of these phantom spheres.
There are two ways one can go about creating "phantom" objects in VRE If the phantom object is a shape, then use non colliding GEOMs - GEOM refers to 'geometric shape'. They are used for visualization and contact detection. A geom lives in its parent's body frame. If you need a phantom shape static in the world, attach it to the root (world) body. Make sure to turn off its collisions using <"contype="0" conaffinity="0"> tags. If the phantom object is a point, use SITE. SITEs are mass-less entities used to model points of interest. For example - sensor location, camera location, special points in a body frame like fingertips etc. SITE lives in its parent's body frame. If you need a phantom point static in the world, attach it to the root body. Example: Find attached an example model with two phantom objects. A phantom green sphere static with respect to the world. A phantom magenta finger tip attached to the distal segment of the index finger. Note: - Change the file extension from .txt to .xml and place it inside the mjhaptix098/model folder next to MPL.xml
As Vikash said, the visible elements in MuJoCo are not bodies, but geoms and sites. You can attach them directly to the world body and achieve the desired effect. However we will have to extend the API to allow changing the geom/site position and color at runtime, and obtaining the fingertip positions (so you can compute the fingertip-object distance). If you want MuJoCo to detect when a fingertip is touching an object, this can be done with the present version of the software, by editing the model file as follows: 1. add a geom to the world, say a sphere 2. add a site at the same position but with slightly larger radius (and assign it to one of the site groups that are hidden by default, if you prefer to keep the site invisible) 3. create a new touch sensor referencing the site; this will select all contact points within the site volume and return the summed normal contact force as the sensor reading; 4. make sure collisions between the new geom and the fingertips are enabled; you probably don't want to sense any other objects or hand segments touching this geom, so you need to explicitly list the pair-wise collisions you want to sense. Let me know if you want to use this and I will make an example model for you. The new sensor will automatically appear in the list of touch sensors exposed by the API. But again, you will not be able to change colors or geom positions at runtime until we extend the API.
I did what you suggested, but there is a dilemma with this approach: the finger becomes obstructed when contact is made with the site-enclosed geom. We would like the option to be able to use the API to detect whether two volumes are overlapped without the need for contact to be made between two geoms. In other words, is it possible to create an additional field within the struct returned from hx_read_sensors() that indicates whether two sites' volumes overlap?
We can of course add fields. However all data structures and functions whose names start with 'hx' are part of a standard API we designed with OSRF and DARPA, and as with most standardization efforts, agreeing took more work than implementing it If we were to add MuJoCo-specific functions (starting with 'mjhx') it will happen a lot faster, but that means we cannot touch the standard API, so it will have to be a separate call... In the meantime, here is a solution to your dilemma: make the contact very soft, so that the touch sensor will still detect a non-zero contact force, but its effect on the physics will be negligible. This can done by adjusting solprm, for example: solprm = "1 1 1 1" I still have to document what exactly this does, but roughly speaking, the contact behaves as an implicit mass-spring-damper, with parameters solprm = (mass0, mass1, damping, stiffness). Smaller numbers mean softer contact. If you are specifying contact pairs explicitly, you can include the above attribute there. If you are relying on the automatic mechanism for detecting all pairwise collisions, this needs to be included in both geoms that are contacting (because in that case the solver takes the average of the geom values to determine the contact-specific values).