Long arms, big props, low weight, and a DJI controller is the cause, at least in the case of what I've built.
On this one the arms and motor mounts are pinned in place and can't be in misalignment, still does the yaw twitch if there's the slightest bit of wind. Only in the calmest conditions...