1. The only AI that can handle custom, hacked mods that have any serious game-design alterations (new units, re-purposed units, etc.) with any real hope of not crashing very quickly was AAI. I did work with JCAI's configurations... no dice. NTAI's current build (and, yes, AF, I am aware you're re-writing the core, etc. but I'm testing what's available) was totally unable to handle anything except for the mods it has been specifically configured for... and that's only if they're left pretty much 100% pure. Even minor alterations, like changing the speed at which the Commander moves, made NTAI unstable

AAI required that I delete all learn and cache files, then ran, but when I got to the final sets of experiments, it also crashed, sometimes with memory leaks that crashed Windows

Even AAI has some serious problems, however. It cannot handle game designs that do not include Metal Extractors or Geos- lacking both things causes serious instability in the AI. None of the other AIs were even remotely stable, however, when presented with radical changes to the game design, so I have to give Submarine the prize here.
2. None of the AIs were able to find me if I just made big enough rings of DTs around my base. You can win almost any map by simply using a Necro/Fark and a Com to DT a large portion of a map. Boom, you win- the AI won't ever eat the DTs, nor will it scout you. I don't think it even matters if the AIs eventually build Air, if they can't handle that. DTs are a lot more important in XTA (as is resurrection) than they were in TA- the changes the SYs have made to get rid of micro have really changed a lot of fundamentals about how TA works, and I don't get the impression that most folks really understand that yet.
3. None of the AIs really held their own if I could build halfway-decent defenses. None of them ever changed the direction they attacked from, for example. AAI gets the prize, however, because it... and only it... built custom units and used them. It was quite nice to see my Mecha getting sent to attack me, but it sent them in random drips and drabs, which wasn't effective.
4. None of the AIs build any defenses worth a darn. Period. Random collections of defenses are just a waste of resources against a good player.
5. None of the AIs cleaned up battlefield wreckage, especially in their bases, with enough speed for it to matter. This is something that a player might, for example, even use the D-Gun for, if time was desperately short- and the very least, players will re-task Fark/Nec/whatever.
6. None of the AIs resurrect.
7. None of the AIs help factories.
8. None of the AIs build formations that make any sense.
The only games I played where an AI was remotely a threat were games where I gave the AI 100% advantage, played 3 on 1, and even then, only on small maps where the AI could rush me before I had defenses that mattered. After that... none of them were remotely challenging. AAI, 3 vs. 1, with a 100% advantage, is challenging on small maps, so long as I don't break it by using DTs to put my base outside its scouts' sight radii. NTAI was challenging only on maps like Ashap Plateau where its sheer rushing speed to get mexxes placed helped it. Even then, if I could hold two and throw up Mohos, I was fine. I suspect that adding Air to the equation will not change things greatly, but I'm not happy with the way Air is balanced in XTA, period, so that's another issue.
So... there you go. Hopefully this was useful to all of you. Point no. 1 is especially important- if you want to see the full range of possible game designs that could be built for Spring, then this awkward limitation needs to be looked at. My suspicion is that every current AI has code in it that looks for Mexxes in the FBI files (or hard-coded buildtree) and if it can't find it, it crashes. My guess is that AAI will require the least amount of alteration before it can run without this problem, but the other ones are going to have serious problems handling anything that doesn't basically look like XTA.
Which, if it wasn't already obvious, is what I'm going to build.