What percentage of opponents are bigger than a dwarf? A herd of snotling - then he's outnumbered. Hordes of hunchmen are rather common too. Really the occasions where he can't use Trollfeller are rare...
There's a lot of room for interpretation, and that's what GMs are for. I wouldn't let it count against a regular human. Just the name "Trollfeller" is a good hint that "bigger" is not a matter of a few inches in height. Trollfeller Strike is for killing trolls, giants and dragons.
In a similar way, I wouldn't say that 3 snotlings outnumber a single dwarf. 30 snotlings might. If you're in a group of 5 fighting a group of 6, you're not sufficiently outnumbered in my opinion. You actually have to feel outnumbered.
Although personally I think it'd make more sense if only size mattered. Fighting a very big opponent is very different from fighting a lot of opponents.