Can upper-level managers see the performance reviews of all the employees under them?

Yes, they can. Managers can see the performance reviews information from My Team, My Entire Team, and My Secondary Team, under the Manage Reviews tab.