This historic, though run-down district, was once the heart of Dutch, and later British, Matara. There are no real sights, but it's an intriguing area architecturally, and a quick wander will reveal many fine old colonial mansions in various states of disrepair. The river bank at the west corner is serene; see if you can spot one of the rumoured crocodiles.
The Matara Fort area is just west of the Dutch Rampart. Strolling along Court St to the riverbank and back to the rampart may take 20 minutes or so.