Nothing. By constantly rubbing the inside of the ear, you're irritating it and removing the oils, which in turns makes the ear produce it more, which in turns makes you clean it constantly.
It's unnecessary. When you stop, the ears will get to their normal balance and self-cleaning.
If the ears do get dirty or truly well stuck with wax, there are ear cleaning sprays you can buy, or make a solution of hydrogen peroxide, warm water and alcohol, then gently rub in some olive oil.
If you still have excess wax you can of course use a tissue or something to clean your outer ear, but yeah sticking hard objects in your ear canal isn't great.
I think that if you feel you must clean them, you're supposed to use some kind of water pick or something, but you're not really supposed to clean them much at all.
I once heard someone say "you should never put anything smaller than your elbow inside your ear". Not certain if that meant just because of the size. For me at least, it's rather difficult to get my elbow anywhere near my ear which I think is the point.