I don't think the .gltf comment is necessarily related to your issue, but basically, this is good:

The children while greyed out can still be moved about as required and results in a .tscn file that is 188 bytes and rapid to save/load (sub 1s).
I think the main thing that irritated me was changes to materials/collision shapes would not persist through new imports of the same scene, so I used to think it was better to 'Make Local' the model/meshes. Then it looks like this:

The problem is this is now a ~218mb .tscn file and takes about 11s to save/load (mostly longer saving, but marginally longer loading too).
For whatever reason .gltf format (and I imagine obj/fbx etc.) is just faster and smaller, so now I try to do as much as possible in Blender (including -colonly collision shapes) and basically don't move or alter anything in Godot, just reference the original imported file. Materials inevitably need to be redone in Godot half the time just because Blender != Godot I guess, so whenever I edit them I save them to a separate directory so I can restore them after next batch of overwrites.