Hello! Relatively new user to Godot, but I have worked with other game engine UI editors.
From playing around with the nodes, and from what I gather anchors work with margins to define the position and size of the control nodes (See also Control Nodes Docs Page). These margins are in relation to their parent. So when you scale the entire UI, it causes the margins to count 2 pixels for every 1 that the window gets resized by.
And since margins define position, you get the moving buttons.

As you mention, the quickest solution is to scale the UI nodes individually instead of the whole. What I did was making parents at each anchor point that each elements use. I kept the Parent control as a "full rect" anchor with each direct child to match in size but with the anchors set to their respective positions. Then build the UI inside of them.

Then when you go to scale up the UI, you scale the Anchor parents. Probably not the solution you were looking for, but I hope this helps!
Below is my code I used to adjust the scale on the Control Node.
! extends Control
! var UIScale = 2 # Set the UI scale here
! var UIAnchors # Node Container
! #
! func _ready():
! # Add Anchor Parents to the container
! UIAnchors = [get_node("TopRight"),
! get_node("TopLeft"),
! get_node("BotRight"),
! get_node("BotLeft")]
! # Rescale the Anchors
! for i in UIAnchors:
! scaleFunction(i)
! #
! func scaleFunction(UIAnchor):
! UIAnchor.set_scale(Vector2(UIScale, UIScale))
! UIAnchor.set_size(get_viewport_rect().size/UIScale)