You have a rubber pad on top of the swing arm?
My chain touches the swingarm a little when it's not under load. Under load it's clear though.
There's also a lot of build up fling there too, all helps. Had to scrape gunk out of my front srpoket cover last weekend. Damn it was messy.