5 seconds it's not normal at all. You can try to use the "Anti pop-up" feature on the Addon Manager, and increase its value (default it's 1). By moving the slider up, you increase the loading range of the objects, at the expense of frame rate, but you might reduce pauses. Or, more precisely, the object will load earlier, instead of short final.
If you still have the issue after moving the slider to its maximum, then you can be sure it's not the scenery, but something else, like AI traffic for example.