Increase available timesheet permissions/restrictions for managers and above by department

Currently, timesheet permissions is in a bit of an "all-or-nothing" state, where managers/directors/SVPs/etc can only see the timesheets of their direct reports, or as we've done, if you allow admin access to timesheets and hours, that level of management can see ALL timesheets for the whole company. We would like the ability to restrict this by group/dept as exposing the entire company's timesheets to management can still be quite sensitive. 

We are unclear if this exposure happened prior to the Timesheet UI overhaul, but we noticed it after the changes to the Timesheet UI were made.