What is it about manipulation that we fear? Is it the loss of control, or the high uncertainty surrounding it, or something else?
In general we have a probability distribution over events that have happened, or those that we anticipate will happen. This probability distribution forms what we call our beliefs and expectations about the world and, crucially, our expectations about the behavior of other agents. The moment that we begin doubting our own belief facility beyond some threshold, we go nuts! If your brain cannot trust its most intimate core, then what can it trust, to any level of certainty.
The fundamental problem with manipulation is that it systematically takes advantage of our internal probability distribution. A determined manipulator will, almost by definition, attack you where you least suspect it. The problem therein lies. If we assume that to be so, then the event which we least suspected becomes our prime suspect. But perhaps the manipulator considers this, and decides to exploit yet another unsuspecting event. This exercise may be iterated several times. All that your mind is left with is the sounding of alarm, cries of panic, and deep scathing paranoia.