Restricting assignment, comment, and tagging by group

We had an issue where someone was inadvertently tagged on a comment for a confidential project. Is there a way to prevent anyone outside a certain group from gaining visibility/access to a project through an assignment, comment, or tag?