These OOM can't all be our computers fault. Something in the NGX, Orbx and this scenery don't get along. I will try running the DX10 preview.
If you are saying this, you haven't followed the explanation, really.
First, FSX can't address more than 4GB in total. That's nothing we can do about this. CYVR has been TESTED to work perfectly with the PMDG737 + FTX PNW + UT2 at 100% complexity. Under this configuration, it takes about 3GB of RAM, meaning you have about 1GB of spare room, under a 64 bit OS, of course.
If you try to run MORE stuff than that, you are approaching the 4GB limit and sorry, you'll have to start to make CHOICES, you simply can't just add and add stuff, and expect it will always run.
I turned on the DX10 preview and I have jaggy lines on my aircraft
This means you haven't setup the antialiasing correctly.
. I looked up the settings again for the Nvidia Inspector and while checking that I noticed PMDG does not reccomend running FSX in the DX preview mode. they say it's slower and buggy. Any suggestion other than not flying in with the NGX and PNW?
DX10 was slower and buggy as it was released, but with the recent tweaks that has been made to it, made it very usable and a concrete alternative, chech the DirectX10 forum over at Avsim
http://forum.avsim.net/forum/569-dx-10-discussions-hints-and-help/And, that information might be obsolete by now, since later drivers from nVidia (if you have an nVidia, of course), improved DX10 support a lot. I don't find it any slower than FS9, but it makes for a smoother flight for sure.
Any suggestion other than not flying in with the NGX and PNW?
You can try turning off HD Textures.