Referring to the original post, I know it's old but I wanted to clarify a few things.
Librarian starts with a Psy Rating of 3, only upon obtaining a new Rank may the Librarian gain a stronger Psy Power rating. At Rank 8, they can finally have a Psy Rating of 10. Rank 8 is basically a Chief Librarian and hero of the chapter.
When using Psychic Powers the Librarian chooses how powerful - Fettered, Unfettered or Push - this is very important but no matter what just using a power could cause psychic phenomena (see page 187 of Deathwatch).
I have not seen many "anti-peril" talents. Even "favored by the warp" is perilous because you are still rolling on the chart. "Warp Affinity" is the same way. It doesn't prevent the psychic phenomena roll, it just gives a chance at making it less deadly. I have not really come across too much that let's you resist the effects of Psychic Phenomena. Even then, the Librarian might get a reroll but gains corruption points. FFG did a very good job limiting the Librarian. They are scary to have in your group and one that is throwing powers around spells doom for himself and his kill-team.
Fettered: Half Power. Strength of power is half (rounded up). They make a focus test as normal. No chance of triggering Psychic Phenomena.
Unfettered: Full power. Strength of the power uses full Psy Rating. Make a focus test to manifest like normal. If doubles are rolled on the dice the Librarian triggers psychic phenomena, regardless of whether he passed or failed his test… it is always dangerous.
Push: +3 to Psy Rating. Make focus test to manifest power. Regardless of what happens, Psychic Phenomena roll is automatic. If doubles are rolled the Librarian suffers one level of fatigue also.
Keep in mind if the Librarian is sustaining powers it gets worse. THE ODDS OF ROLLING DOUBLES ON A D10 IS PRETTY HIGH ACTUALLY. I think it is something around a 5% chance, about the same as rolling a 20 on a d20. Not exact as number crunching is my strongest point.
The Peril Involved
When rolling on the psychic phenomena table there is a 25% chance you are going right to the Perils of the Warp roll. There is basically a 50% of something very, very bad happening. Even some of the simple things on the psychic phenomena table can be killers. Insanity and corruption are constant threats. Librarians that ignore corruption, insanity and the threats will die quickly.
The Librarian Powers at a Glance
Even getting to Rank 3 takes time. A Librarian will only have a Psy Rating of 5 at Rank 3. Nearly all experience must be spent on psychic powers to really push into the realm brought up by the original question. Sticking to the rules is the most important thing to restrict a Librarian. There is no need to add or change things, just read the rules. Since it was the first game you were playing, I doubt you were even Rank 3. But it's possible to get into Rank 2 pretty quick so we will say you were dealing with a Rank 2 Librarian using Smite.
To use Smite, you make a Focus Test modified like ranged attack. Lets use it Unfettered (PR 4). Let us assume the Focus Test succeeds. He deals 4d10 energy damage with a penetration of 4. It will affect everything (even allies) within 4 meters and he has a range of 40 meters with the ability. So an 8 meter radius explosion of energy occurs at impact. The Smite will have a penetration of 4. That pen is going to break even with most standard Xenos armor. Not the Tyranid warrior or anything above a grunt… who will have around 8 or so. You will deal 4d10 minus 4 to a tyranid warrior, it will definitely not kill him and you just risked your life to do it by not rolling doubles on your focus test. Now the great thing is this… the player might start loving the damage… and fail a few rolls here or there and cause some insanity and corruption… that's what it does. I'm telling you right now… that in three different games I have seen the Librarian become a problem more than once. In one occassion a Librarian was drug off into the warp after pushing to get a massive hit… when you push you always roll on the table.
The rules in the book work. A Librarian will get corruption points and most likely a few insanity points along the way. The rules that govern librarians are in place and they work great - you should always use them. There are three things that really affect the Librarian - the psychic phenomena table when using powers and stronger than 1/2 Psy Rating, the insanity gained and the corruption gained. At 100 Corruption the character is lost forever and corruption never goes away. You can remove insanity points but not corruption. There are other rules outside of the Core book that you can do more with too.
It would take a very powerful Librarian to fire 10d10 smites without losing his soul to corruption after just a few game sessions. Librarians are not too strong. The best thing you can do if you want to limit them is focus on the buildup of corruption if you wanted to.