OK, very strange, opened up the setup client and Carefree WAS on, and also my auto-leveling smoothness had changed from 30% to 48%. I guess that must be the issue. I was SURE it was off, but maybe I was mistaken, or it changed after I did a second accel calibration.
So when carefree is on, it must be activated when Auto-Leveling is engaged? Good thing it didn't really screw me up and I was flying relatively straight out.
That would explain why it was not flying right, because my compass on the gps board is sitting in the middle of the ESC's, with a power harness around it and the batteries above. If I use mag hold it will suddenly yaw, especially when applying power. I tried Position hold too and it went into a toilet bowl pretty quickly. Guess I have to put the external mag back on.
Phew. now, of course I can't test-fly because it's snowing...