Alright I think I get it, you're trying to design a FPS where there's two modes for mouse movement? I'm guessing you want the mouse to be able to switch from being able to direct the players camera to be able to aim a weapon. It's interesting but as to the mechanic itself I'm sure there's a reason I've never seen anything like it in FPS games. Tell me if I understood you right.
As for "messing up near the player" makes sure the player's nodes aren't "input ray pickable".
I tried not to, but I did look at your code a bit. It looks like you're trying to get this done from a mathematical point of view, when IMO in this case it might be better to just let the engine take care of that for you.
Have your game objects forward this function to your targeting node:
func _on_Area_input_event(_camera, event, click_position, click_normal, _shape_idx):
That way each object can just send the targeting node this very useful information:
event, click_position, _click_normal
That click position should be in 3D, you can check for mouse motion like this:
if event is InputEventMouseMotion:
Then do something like if mouseTargeterActive:
adjustPositionOfTargeter(click_position,click_normal)
Remember there in my code there's a _
before _camera
and _shape_idx
just because I'm not currently using them.