It would be considered rude if another person touched your magickal items without your permission IF they knew what those tools where. Believe it or not, some people are completely clueless when it comes to things like that. Make sure you tell the person not to do it again.
Our magickal tools do accumulate energy each time we use them, but we should always cleanse any negative energy off of them a few times a year. I would cleanse and reconsecrate your items before using them again, but have no fear... they'll be just as powerful as they were the last time you used them.