All depends on how you created the conditional format. If you selected all cells that you want formatted, then you'll have this issue.
What version of Excel are you using?
In Excel 2010, when you create your new conditional formatting rule, if you select a single cell in the pivot table first, it will ask you if you want to set the rule up on "Selected cells", or "All cells showing xxxx values", or "All cells showing xxxx value for yyyy"
It's one of those latter two you're looking for, as that will apply a rule that deals with changing data.