The glass-and-concrete High Court building sits on the plaza between the capital's two main art galleries, and, not to be outdone, is home to a large collection of art of its own (pick up a handout inside). It's definitely worth a peep. There's also a small display on the history of the Australian judicial system, and visitors are welcome to view court proceedings when in session.
The High Court sits for two weeks of every month except January and July, with the usual sitting hours from 10.15am to 12.45pm and 2.15pm to 4.15pm daily. Check the schedule online before your visit.