Yes. Your business competitors can attack your trademark on the basis of non-use. Non-use is a valid ground for cancellation.