The default project theme in Godot 3.x uses a low-resolution bitmap font, and the default project theme is also made of low-resolution bitmaps. This is why it doesn't look great when the camera goes up close in the GUI in 3D demo.
If you use higher-resolution theme elements, it will look better :)
Godot 4.0 will use a DynamicFont for the default project theme and StyleBoxFlats for the default styleboxes (which means it will be possible to change the default theme's generated scale freely).