A skirmisher infantry for example, could be considered to even wear the same armor as a red unit, but operate in smaller numbers and being more dispersed.
This type of formation, covers more ground (staying in formation slows units), is used to fighting dispersed (wont miss the lack of mutual support from harsh ground like woods) and is expected to fulfill a particular role (flanking, rushing into places, or simple reconesence). They still take the same amount of space (4 units in a hex) but each figure could be assumed to have less fighting potential then a red one.
This type of unit could be elite rather then new recruits, but becouse it serves a particular function its designated green. Maege Mormont (with her green infantry) is a good example of this (expecially with her ability to attack twice). Most likely the the low number of her elite bodyguards is what causes them to take such heavy casulties from attacks (even a few kills will increase the proportion of casulties making them the equivilant of a green unit).
For the most part however, we can still assume that lighter armor or heavier armor is what causes the change.
This is one of the things I really love about this game. It deals with rather complex millitary considerations (diffrent troop qualities) in a simple way that isnt burdened with extra rules but still gives you all the tactical potential of having unit variety. Even if heavy units have the most staying power, then armies (on purpose and not due to lack of equipment) used a mix of units to give them tactical flexability on the battlefield.