When you create custom page template for category you can create a rule to use it for all categories of a particular taxonomy. However, you can also create another custom category template without rules in it. Instead, on edit category page in wp admin, you can create a custom rule to use the template you need for that particular category only.
