What you have to understand is that in GPS and Atti mode, your 'throttle' stick isn't so much a throttle control more a rate of descent/ascent control. So when your stick goes over the half way point the WKM sees the input as a climb rate of x ft/s and then sets the throttle accordingly. In manual mode you are dealing direct with throttle so yes it reacts much quicker to stick inputs- think how fast the motors have to react for stability and you will understand why the throttle in manual mode is so responsive.
In the WKM there is v little you can do about this. In your TX you could change your throttle curves so they are flatter around the hover point but this will also affect GPS and Atti unless you use a separate curve for each mode. The easiest and best solution to this all though is just practice flying in manual. Practice practice practice. You will soon get the hang of it. Its a bit like the heli pilot saying ' I just cant get the hang of nose in hovering, what am I doing wrong' The answer is invariably nothing, the pilot just needs to practice more.
