A while back I started a thread looking for ideas about how to remove rust stains from my trunk mat; I think that thread was lost in the last purge. The consensus at the time was that it would be very difficult to do if it could be done, and that I'd be better off getting another mat.
Since I've never been after a show car, and since I have a lot of time of my hands right now, I decided to keep after it. Long story short, my research lead me to a substance called oxalic acid which is used to bleach wood and to remove rust stains from concrete. I got a small amount, tried it, and it works. I will say, though, that it was a lot of work, and from a time-investment sense it probably wasn't worth it. If you have something with a rust stain that's very expensive or irreplaceable, however, this would be one way to go.
Pics show before and after in roughly the same area. The after pic is still wet from a soapy wash to get rid of the rest of the acid.