Okay, I got it working for real this time.
It was actually pretty complicated. Basically, there are 4 objects. The object you want to grab (the box in my video), which is a RigidBody. The object that is pulling the box, which is a StaticBody. This is what is attached to the PinJoint. A KinematicBody which is what is actually moving, you could think of this as your hand. And the fourth object is just a Vector3, which is the position of where you want to move your hand to.
So basically, you only ever adjust this invisible position vector, and then setup the rest so they follow each other. You can adjust the position freely by any method depending on your game. Then the KinematicBody (which is what handles the wall collision) has a script which follows the invisible position. You must use move_and_slide so that the physics and collision work (but put it only another collision layer so it only collides with the environment and not the things you are grabbing). The move_and_slide finds the difference between the current KinematicBody position and the invisible vector, and creates a pretty fast velocity that will keep them in the same place. That is the first part. And this solves the clipping into the wall. The KinematicBody is just a sphere, with a size equal to the size of the box. If you have different sized objects, then you would need to adjust the physics shape to roughly equal the grabbed object.
Then you set the StaticBody (the thing attached to the object you actually want to grab) to the position of the KinematicBody. You can just set the translation direction equal to each other, this is easy. Now since the RigidBody is attached to the StaticBody already with a PinJoint, it moves automatically without any additional code. So that is the basic setup.
I also set the RigidBody to lock axis when being grabbed, but to unlock once it touches something. So while you are holding it in the air it is stable and solid, but as soon as you brush by the wall, it will allow rotation. It is also unlocked if you let go or throw it.
This was pretty crazy and complex, but it's actually not a lot of code. The project I have is too big, but I could look into making a minimal project as an example, if the directions are not clear. Really glad I got this working.
