Overall compatibility for game-play is 100%. Most overlapping rules are indentical (except RT rules are generally the version from the errata to DH and not the pre-errata mess.)
There are many added rules/systems for things not covered in DH (Starship combat, Navigator powers, etc. etc.)
The biggest change is that RT uses a different game mechanic for triggering/using psyker powers.
Because the outcomes both use systems and terms of the compatible rules it would be possible to play a RT rules psyker side-by-side with a DH psyker and just let them each use their own rules without having any game mechanics problems.
My preference however is to be consistent in this matter, and with some minor effort it is simple enough to convert psykers and powers from whichever rules you prefer to the other. There is even a sidebar in the RT psyker chapter on how to do so. (although there is an aspect that is left out, its not that big a deal. you can find it discussed in detail in this thread: www.fantasyflightgames.com/edge_foros_discusion.asp )
So game mechanics-wise - nigh fully compatible.
Playing a mixed party DH and RT PCs of the same number of XP, I have some concerns about scaling as characters advance, however will need to do some play-testing to get a better idea. Basically the issue comes in when the same skills and talents which cost 100xp for a DH character cost 300-500 for a RT character. So if RT characters and DH characters in a mixed group end up recieving the same amount of XP it looks like the DH characters will end up with far more skills and talents than their equal XP RT compatriots. Then again the DH advance tree at present ends 20,000 XP before the RT advance tree, however I expect that will be corrected in Acension.