I have 275/40/17's on my Challenger with 17x9's with 5" backspace. No rubbing on the frame, but I did have to remove the fender braces when I was running the car at 23" at the top of the wheel opening to the ground because it would rub under compression with the wheels turned. I also partially rolled my passenger fender lip, and run -.9º camber to keep it off the passenger fender lip. Not sure why I had issues with that side, but I did. No issues at all on the driver's side. Currently I run the car closer to 23.5" from the ground to the top of the wheel opening, but that's just because of header clearance. Too many speed bumps around here, and I got tired of crawling over them at a 45º angle at 3mph.
I'd say 275/40/17 is as big as you can go in the front. Body tolerances on these cars vary a bit, so it may be easier for some folks to run that size and harder for others. Although it took some fiddling, I haven't had any rubbing since I went with a bit more camber. I also run 1.12" torsion bars, Hotchkis UCA's, adjustable strut rods, and 11.75" rotors. Haven't had any clearance issues in close to 30k miles, my Challenger has been my daily for 6 years now and I've had that rim/tire set up on there since 2011.
As far as the 255/45/17's, they'd be a safer bet for sure. I doubt you'd have any issues at all with the exception of the fender braces if you're running that low. I'd still run them on a 9" rim, that way you can always upsize later. Plus, most 255/45/17's have a section width of ~10" and a tread width of 9", so they'd be a little squeezed on an 8" rim. Some are even wider than that, the specs vary quite a bit with manufacturer and tire model. Not saying you couldn't run them on an 8" rim, quite a few of the tire manufacturers say you can, but a 9" rim would probably be a better fit.
