Your best chance of spotting rare yellow-eyed penguins (hoiho) is from a hide at Roaring Bay, 8km south of Kaka Point's main drag. The hide is accessible throughout daylight hours but suggested viewing times are posted on a noticeboard at the car park. Penguin behaviour changes seasonally but times are usually before 7am or after 3pm or 4pm; ask locally or at your guesthouse.
Don't linger outside the hide and obey all warning signs: as you can see, penguins lead a precarious existence. There's no beach access after 3pm for their protection. If no penguins waddle into sight, console yourself with the majestic views of cliffs and rolling surf.