Menemsha Inn is a big old sprawling inn perched in the woods on a cliff overlooking the Menemsha piers. There's a huge variety of rooms; all are fairly spartan, but many feature stunning ocean views. You can also rent cottages and even houses (per week $7010 to $7500), perfect for groups or families.
For a slightly more upscale experience try its sister property, the Beachplum Inn.