Ultimately, it just depends on how you want to handle it and what is most important to you and your project.
I would agree that saving all the blocks generated is probably not the best idea, as the player is unlikely to see or interact with every block, not to mention saving and loading would take a substantial amount of time.
From what I remember of my Minecraft modding days, how Minecraft gets around this issue is that it only saves chunks the player has visited into a single file, along with storing the seed for the world. That way, whatever the player has seen will stay the same regardless of what changes, but the memory footprint is reduced since the N number of unseen chunks are not saved. I'm not sure how often Minecraft saves the world internally, but I think it's on a fixed interval minecraft checks for "dirty" (changed) chunks and updates the save file if needed.
Personally, I would recommend doing something similar to how Minecraft handles it: Saving the chunks that have been visited/generated and saving the seed for the world. That way, everything the player has seen and interacted with will stay the same regardless of changes to the generation code, but you do not have the memory cost of saving the entire world. For knowing when to save, you could implement something like Minecraft does for saving at a set interval, or you could simply save chunks when they are unloaded or when the player quits the game. For the most part, I do not think it would make a huge difference one way or another (and it could always be refined later in development if needed).
