I also had a problem with Rickards' Toughness. It does state each set of 'hits'. Retreats happen 'after' normal hits, so in theory, failed retreat hits should be seperate and Toughness effects both. Harsh, but correct if using the rules as written. I know house rules are becoming very numerous and the official FAQs add many optional rules that new players may not be aware of. Players like to add ALL hits done in one combat action together, including other units 'flanking' an engaged unit. This, of course, weakens Toughness/Cover and makes it easier to 'Capture' commanders. Good or bad is a personal choice, but what about the Command card No 3 of Greatjon? That allows a unit to take 1hit to increase Rank by 1. Toughness could ignore this as a seperate set of hits, and most Commanders could as well, as they need 2+ to capture!
I can see a myriad of house rules just covering this one aspect, with no 2 players playing the same, until FFG nail this issue down with something official. It is OK for regular players to play their own set of rules, but no one likes to spend half their playing time just agreeing on which rules they all like. Still a good game, but with more expansions coming out quick and fast, this game could easily fragment into as many different versions as there are players. Cheers!