Just going with a small patch in those areas would be faster, but my question is: What does the inside of the quarter look like? It looks like the rust is coming from the other side of the panel (IMHO), in which case you're only delaying the inevitable. I'd do some more poking around behind there before commiting one way or the other.