I agree, if across the taillight panel it's 2 colors, silver taillights with a center section that's black, or vice versa, the car was painted at one point or other.
As far as 73 and 74 Rallyes having a silver taillight panel and black around the taillight pockets, I'm no expert, I dunno.
I guess the only way to tell is to get factory info. In my '72 facts and features book, it lists the base Challenger as having an Argent Grille and medium astrotone lower deck panel. (I take that as the taillight panel.) The Rallye has a Black Grille and black organosol lower deck panel.
I just found this image. Its the last page from a 1973 Dodge Challenger dealer brochure. States Rallye package has black-out painted grille and lower deck panel. On the second page of the brochure it APPEARS to be a base Challenger with a silver taillight panel with black taillight pockets. It could be a Rallye with a 318, steel wheels and hub caps and a delete on the side scoops and strobe tape since it has the body side molding. Can't say for sure.