Did you already have all the sheet metal installed and lined up? There is no way it will all just fit together without a lot of fine tunning first. It took me a very long time to get all the panels close enough before using body filler to get them straight from panel to panel, and that is with the original doors, NOS fenders and an original hood (off another car).
Anyways, as to your question, I would go over the panels with 80 grit, then use a good etching primer, then filling primer.
Just my two cents.