CTD Canada

[downscc]

Earlier this year when GSX was modified to inhibit over 250/10000 there was a decrease in the number of CTD events crossing East to West over N Atlantic to Canada.  However, in the past month I have been able to predictably reproduce a stack corruption problem that causes a CTD on a EGLL KLAX route about 186nm NNE of waypoint 5588N.  This location is over the S Hudson Bay approaching the SW Ontario coast.  I was lucky enough to be participating in an on going PMDG QOTSII B747 beta test and one of the PMDG developers evaluated the Windows crash dump files from a couple of these events and the problem was clearly being created by the P3D (v4.4) facilities.dll.  This is no coincidence and was predicted by your discussion on this subject earlier this year.

This problem is not caused by FSDT modules; however, and the only reason I am posting this is to advise that the facilities error (stack corruption) is only triggered when FSDT modules are loaded as add-ons.  When I disable the bglmanx.exe and couatl.dll, and all FSDT sceneries, before launching P3D the problem never occurs.  There appears to be something still related to facility data and FSDT and I cannot even guess what that might be.  Along the way, and as time permits, I've been scanning the APX bgl files in the region and have yet to find the corrupted file I expect to find.

[virtuali]

We still need to maintain a minimal connection to the facilities data, even if GSX is not requesting it.

However, since the problem is clearly related to a specific area, it's clear the problem must be some corrupted data *there* so, the real and final solution would be finding that corrupted data, which might at least allow us to code some kind of workaround to defends ourselves from it.

No, you might not be able to find a corrupted file per se, the file itself might be ok, but might contain corrupted/illegal data somehow. This, because the simulator ITSELF it's crashing when accessing it.

So, the first question would be: have you updated any of the default scenery files, like navaids/intersections ?

[downscc]

but might contain corrupted/illegal data somehow. This, because the simulator ITSELF it's crashing when accessing it.

So, the first question would be: have you updated any of the default scenery files, like navaids/intersections ?

I spent an afternoon looking into all the APX folders in the Scenery\0301\Scenery folder that is relevant to the area using Herve Sors Airport Inspector looking for that corrupted data.  My problem is I do not have a tool that surveys the entire airport container in the bgl other than ADE or BGL2XML tools and those work on one location at at time.

I have not modified any scenery files for intersections globally, and only edited a few navaids in the default scenery (eg delete obsolete markers) in other default scenery folders using BGLNavEditor.  None of those have created problems before with sessions conducted in those scenery regions.

I will keep trying to nail the corrupted data.  Suggestions welcomed and thanks for the response.

[virtuali]

I have not modified any scenery files for intersections globally, and only edited a few navaids in the default scenery (eg delete obsolete markers) in other default scenery folders using BGLNavEditor.  None of those have created problems before with sessions conducted in those scenery regions.

A meaningful test might be testing with a clean install of the sim, with no sceneries added, to see if the problem happens with the default scenery only.

[downscc]

A meaningful test might be testing with a clean install of the sim, with no sceneries added, to see if the problem happens with the default scenery only.

I pondered this for awhile and concluded that the problem only happens when couatl is loaded.  If I remove couatl then there is no problem with any scenery.

I've spent several days looking at the APX and NVX files in the scenery\0301\scenery folder (all of them are original default) using bgl2xml, airport inspector and bgl nav editor tools and I am fairly confident that I have looked at all facility data and not found data values that appear to be incorrect.  I have not ran bgl2xml on all files yet, but so far everything runs through the decompiler without problem.

I then decided to return to flying the simulator instead of troubleshooting and I experienced another stack corruption problem with CTD that had exact same signature as the previous EXCEPT I was flying from VHHH to EDDK and about 5.5 hrs enroute over Western China in the middle of nowhere.  The memory dump file from that is at https://www.dropbox.com/s/bbhvn55601nwjh2/Prepar3D.exe.8196.zip?dl=0.  Since that happened, I have returned to the EGLLKLAS scenario that I have which will ALWAYS terminate in a stack corruption CTD and have twice ran the scenario where there is no crash if I remove couatl.  I've tried it by disabling GSX from the menu but it still crashes.  The only way I can avoid the problem is by commenting out the couatl addon component from the add-on.xml file.

The P3D scenario files for the reliably repeatable crash event are available on request.

[virtuali]

I pondered this for awhile and concluded that the problem only happens when couatl is loaded.  If I remove couatl then there is no problem with any scenery.

It's not possible Couatl can have anything to do with this, since the one that makes queries to the simulator facilities is the Addon Manager.  Couatl simply doesn't have access to that interface, because it's only possible with an in-process .dll.

So, if you have disabled just Couatl, and you think you fixed, you haven't and you simply didn't pass in an area that contained corrupted data but, since this happened in the same session you disabled Couatl, you are assuming it was caused by it.

The P3D scenario files for the reliably repeatable crash event are available on request.

Is the crash happening ONLY during a long flight ? Or can be reproduced by just going to a certain location ? Of course, I already tried multiple times going to coordinates reported by users that caused crashes, but I couldn't replicate any crashes. Of course, I'm using only 100% default scenery, at least when navaids are concerned.

[downscc]

The P3D scenario files for the reliably repeatable crash event are available on request.

Is the crash happening ONLY during a long flight ? Or can be reproduced by just going to a certain location ? Of course, I already tried multiple times going to coordinates reported by users that caused crashes, but I couldn't replicate any crashes. Of course, I'm using only 100% default scenery, at least when navaids are concerned.

The crash only happens in remote locales and on long flights, which seem to go together.  The saved scenario I have is of an enroute flight EGLL KLAS about 100 nm East of the Greenland coast in a PMDG B748 VIR G-VOYG at FL340.  About half the time, there will be a CTD within 10 min, and the other half the session will continue until approaching the SW coast of the Hudson Bay.  This is repeatable.  When I kill couatl there is no CTD, with couatl running there is always a CTD.  I agree couatl may very well not be the problem but the problem will not occur if couatl is not running.  I am killing it with Process Explorer.

I'm looking for a way to kill couatl after departure and get it running again before destination so I'll have an airport then (when destination is a FSDT location).  Haven't figured it out yet, but v4.4 does seem to warm restart when unselecting/selecting addons so I still have to play with that.

[bryn]

Umberto,

I am also getting exactly this crash. Are you subscribing to SimConnect_SubscribeToFacilities or using SimConnect_RequestFacilitiesList call? If so, are you only subscribing to the airports, or to the other facilities too (i.e., where should we be looking)? Can you provide some optional additional logging functionality so we might pin down the problematic facility?

The main issue with this is that it does not crash if you load into that area. It only seems to happen after an ongoing flight across the Atlantic and only when couatl is running.

Bryn.

[downscc]

I just ran another test session using the test scenario previously described with couatl running but excluded FSDT KORD.  Several colleagues in the PMDG beta team have commented on their problems with KORD and although I've no had any it was worth a try.

Without KORD and with couatl running this test session ended exactly as other test sessions with couatl running:  Stackhash_bf3c CTD.

[downscc]

I ran another test session today with the same configuration I've been using with the N Canada sessions except this time I went oceanic all the way:  KATL FAOR.  Other than a few S African locations and Ascension Is in the mid S Atlantic there was little facility data enroute.  All FSDT addons enabled.  This 15 hr session completed normally.

If there was some way, a log perhaps, of facilities queries and responses I bet we could isolate where in the world couatl is colliding with whatever it collides with to cause stack corruption.  I hear you say that couatl cannot be the problem but I never get this problem without couatl.  I seems likely that it could lead us to what the actual problem is.

Your thoughts appreciated.

[Martyjr]

Dan,

FSDT uses a default P3Dv4.4 aircraft (I have not found out which one yet) to check out a SODE problem:
http://www.fsdreamteam.com/forum/index.php/topic,19933.0.html

So, I am wondering if the same default P3D aircraft (rather than e.g. PMDG748) could be used to check out this CDT / Couatl problem?



Note:
Previously I had done a test using the VRS FA/18E, Re: ntdll.dll crash between Greenland and Canada Prepar3D v4

and I did not get a crash but it was on a previous P3D version (4.3) and previous FSDT GSX software version combination.

See here about 11 posts into this topic:
 http://www.prepar3d.com/forum/viewtopic.php?f=6312&t=129041

[CaptKornDog]

...I'm looking for a way to kill couatl after departure and get it running again before destination so I'll have an airport then (when destination is a FSDT location).  Haven't figured it out yet, but v4.4 does seem to warm restart when unselecting/selecting addons so I still have to play with that.

I’d be curious for this as well to continue to isolate.  I hadn’t flown TATL westbound in a while and hoped some recent updates across my software in P3D v4.4 might have fixed things.  Tried my hand LHR-DEN with GSX 1 last night and CTD’ed in the “usual” eastern Canadian regions.  Appeared to be stackhash as well.

[virtuali]

Tried my hand LHR-DEN with GSX 1 last night and CTD’ed in the “usual” eastern Canadian regions.  Appeared to be stackhash as well.

We have released an update in January, asking users for feedback about it:

http://www.fsdreamteam.com/forum/index.php/topic,20063.msg138551.html#msg138551