My answer to the OPs question would be "As big as you want!"
In terms of sources from the background, the one that immediately springs to mind is the Rogue Trader dynasty from the Matt Farrer BL library novel "Legacy", which seemed to have a very large number of ships. I can't remember exactly how many, but it certainly seemed to be over a dozen, and indeed the Segmentum Fleet seemed to be treating them with wary respect. The Rogue Trader himself had a flagship and a palace/pleasure ship as a spare, and it seemed that each of the major figures in the fleet (head of the Mechanicus delegation etc) had their own ship, too.
The recent Horus Heresy FW books are pretty explicit about RT fleets in 30k being very large. Given the gradual formalisation and stagnation of the Imperium, I'd hazard a guess that "modern" 40k RT fleets tend to be typically smaller and rather less likely to be accompanied by Astartes elements. "Legacy" scale fleets are likely rarer than they once were.
In the 40k setting, I've always felt that a typical sector fleet would probably have roughly 1 Imperial Navy starship for each inhabited planet, and a merchant fleet of 9-10 civilian ships per military ship. (There's no canon source for this, it's just a gut "feel"). I would imagine that the Imperial Navy would therefore start to get a bit twitchy/nervous if any Rogue Trader fleets reached a size where they posed a serious threat to a sector or even subsector fleet. This would suggest a tentative 15-25 ship limit for "normal" RT fleets. I personally feel that anything above that size would cause Inquisitors to begin to conduct investigations into potentially Sector-shattering power being held in the hands of one individual.
(Mind you, thinking it through, that argument falls apart a bit in a genuine "frontier" region like the Koronus Expanse where there are so many canonical RT dynasties right on the border of the Imperium that the Navy would probably struggle to defeat them if they got together and decided to invade the Calixis Sector.)
Anyway, beside-the-point musings aside, the whole point of the RPG is that there should be no limit to your ambitions! If you want to be Darth Trader stomping around your Super-Dreadnought pre-heresy battleship executing underlings by the dozen as you raze worlds to cinders using your 300 strong fleet, then hell, go for it! There's absolutely nothing wrong with a larger scale chronicle, and indeed a lot to recommend it.
Edited by Lightbringer, 28 May 2014 - 07:38 AM.