I'm not sure why that setting isn't effective for you in your multiple runset loading scenario (might simply be an oversight when the multiple runset loading capability was originally introduced) but there might be something you could try right away to stop the annoying popup.
In the version you're using, do you see a setting like this:
“Save runset to current directory each time Calibre is run”
(Setup > Preferences, on the Misc tab)
When that is enabled, the runset is saved as runset.autosaved when multiple runsets have been loaded.
I wonder if that would be a temporary workaround while the factory looked into the cause of your setting being ignored when multiple runsets were loaded.
Did you ever find a solution?