I replied to the BGG thread but here is a copy of the reply for those here that might be interested.
As the rules stand, it is not clear how this should be resolved.
Both methods have merit, both methods can be "justified" thematically one way or the other.
In my previous post
So a die could be removed from the Spell and added to your dice pool but, if it is the yellow or red die then you will need to "pay" for it by discarding the appropriate Common or Unique Item.
I should have added that there is an alternate interpretation and that was when removing a yellow or red dice from a Spell it doesn't need to be "paid" for again. My bad.
Before we go on, lets make sure we are talking about the same situation. Note: I changed the example so the die in question doesn't show a Wildcard result, it wouldn't make much sense to discard such a die
During Player A's turn he had spent an Item (e.g. Dynamite) and added the red die to his die pool. After rolling the dice, the red die was showing a Peril result. Player A then plays a Spell card (e.g. Enchant Weapon) and places the red die (Peril face up) on the Spell card.
Player A's turn has now ended and it is Player B's turn.
Player B doesn't want the Peril result for the Adventure he wants to try (perhaps a more important Adventure has appeared and needs to be attempted right now, e.g. Transported by Magic). For whatever reason, Player B decides to remove the red die from the Spell.
The FAQ makes it clear that he is allowed to do this.
Before any roll, a player may remove some or all of
the dice held on spells in order to roll them. Any spell
that has no dice remaining on it is discarded. New
dice cannot be added to a spell to replace those that
were removed from it.
The question is if the red die is now part of Player B's dice pool?
One viewpoint says that since the red die has been paid for during Player A's turn then it was part of the dice pool then and is still part of the dice pool now. The fact that it had been placed on a Spell doesn't change this. Since Player B is allowed to remove dice held on Spells then is returns to the dice pool.
Another viewpoint says that as soon as the red die is removed from the Spell card then the Spell has been cancelled and the red die is simply available to be used again. Yellow and red die do not remain in the dice pool between player's turns (as stated on page 9 of the rules "In any event, the die is removed from the dice pool at the end of the active player's turn"). If Player B want to use the red die then he needs to spend an Item to add it to his dice pool.
Which viewpoint is correct? I don't really think there is a "correct" answer. Both methods work. Even if FFG officially rule this one way or the other, either method would be fine.
Personally, I think that the first method makes the game a little easier therefore that's a reason not to use it. I prefer the consistency of saying that the dice pool doesn't "carry over" to the next player's turn. Thematically, the Spell has been broken so it disperses to no effect. Whatever cost that had been paid is now lost.
An alternative viewpoint is that since the red dice has been paid for twice (with the Item that originally added it to the dice pool, and with a Spell) then there is no need to pay for it a third time. It also allows some player interaction as it is a nice way for Player A to "pay" for a die to be used in Player B's turn. Elder Sign doesn't have much interaction so it would be a shame to remove this as well.
One final note: All this only applies to the red and yellow dice. Green dice are always part of your dice pool so if it was a green die being removed from the Spell, then the player would be allowed to roll it for no cost (since you don't need to "pay" for green dice in the first place).
I'm sure there are many other options, logic and theme that could be applied to push your choice one way or the other. Feel free to play which ever way you are more comfortable with.