There is a difference on whether you are the attacker or defender.
But generally, both players can retreat but it must be declared at the begining of the combat round and 1 round of fighting MUST occur is the essential rule to remember.
The action card "Skilled Retreat" essentially allows you to skip the fighting part. That is why this is such a valuable Action card.
This does get to be an involved subject if you want to know all the nuances of it.
1. Only Defending players can Retreat. Attacking players must do a Tactical Withdrawel, if I remember right.
If you look in the rules, you will see there is a slight difference between the two options.
But generally, most players just say they are retreating. Even thought that is not officially correct.