How is hand buffing any different from a slow speed power buffer as far as trapping dirt? I've done it 4 times in the past year and a half and no problems or scratches.
I've seen plenty of hand buff jobs,and it's very easy to scratch with some dirt trapped under the rag if doing by hand. Obviously you have never had the paint so slick,you make scratches by simply dragging brand new microfiber towel and it leaves marks,but you can only see that under certain lighting conditions,and if your eye is not trained to catch that,you just won't see it.
I've been doing custom paint and body for over 20 years and I still will never understand why would anyone would want to waste time rubbing paint by hand and job will never be as good as polished by a machine,no matter what products they use .
I hand rubbed lacquer paint job I did on my first car, and have came up with few tricks to save time when doing lacquer and come up with slick as glass jobs, but that was 20 years ago
I have got my sanding/polishing down so good, that even on black base/clear job I can start and finish with 3M Perfect-it II Rubbing Compound with Meguiars Burgundy Cutting foam pad and not leave any swirls,