One of the markers of good public speaking is to avoid 'ums' & 'erms' we are told. Filler words distract the listener from your message and make your communication less effective. 

In practice however, the research below using call analyzing software by Gong.io, suggests that in one-to-one communications people "seem to be fairly blind to them" - and there is little or no correlation between use of the filler words and the sales outcome.

Maybe its time to worry a little more about what you say, not how you say it.