A player controls an area if they have at least one unit or a Stronghold in that area. If a player moves all his troops out of the area, he no longer controls that area if there is no Stronghold there. Activation markers have no bearing on control, only the presense of units/strongholds. Heroes also do not impact who controls an area.
There are two exceptions to this:
1) Home realm areas are ALWAYS controlled by the relevant faction (unless an enemy has units or a stronghold there). Even empty home realm areas are considered to be controlled by the faction whose home realm it is.
2) With the expansion, the Daqan Lords have a Development Card for "Peasants", which allow them to put Peasant tokens in areas they vacate, which basically allow them to retain control of the area even without units.
The rules are actually pretty explicit with this, once you know where to look. On page 15 of the Revised Rulebook is a list of "Frequently Used Terms", which describes, among other things, a "Controlled Area".