You should expect *some* startup slowdown, because GSX added a lot of new objects to the sim so, even if they are not loaded, and are not affecting performances when not on screen, the sim it's aware of their existence. It's the same as installing an AI traffic package.
However, the increase should be very noticeable and, in fact, I cannot even see the "Adjusting Scenery Objects" message. Maybe it appears, but I just cannot see it. Are you using an SSD to load the sim ? Have installed the Addon Manager on an SSD drive ?
Try to configure the antivirus to exclude the whole Addon Manager folder from scanning, this could also affect the loading times.
