The problem is that the dice are used for completely different events, so saying you can no longer roll 6 is pointless: it will happen because anyone else has rolled a 6 for any action, and not just because you already rolled a 6 in this situation. So it feels pretty out of place there.
I think that the deck may get reshuffled prior to it being all used up, so it's not going to be an absolute. The above assessment is also rooted in the 1d6 scenario, which is a bit limited in scope. The deck is a more of a pressure-release valve instead of an airtight seal...instead of 1d6 and saying you can't reroll a number until all of them have been rolled, it's saying, as an example, that you can't roll a number more than 6 times until all of them have been rolled 6 times - which would balance out the situation that an "unfair" die was appearing to favor a result; it gets reined back in a bit if it's looking bizarre, but once the bizarrity has been balanced out, it can happen again, if it's going to.