Inside Uparkot Fort and close to the Jama Masjid, these are not actually caves but rather monastic quarters carved out of rock around the 2nd century AD. Descend into the eerie, three-storey complex to see the main hall and its pillars, with weathered carvings.