Udayagiri and Khandagiri Caves have derived their names, owing to their location on two hills, Udayagiri and Khandagiri. Mentioned as Kumari Parvat in Hathigumpha inscription, the two caves face each other across the road. At the site, tourists can find a number of ornately carved caves. Legend states that during the reign of King Kharavela, most of these caves were carved out as residential blocks for the Jain monks.
Khandagiri has 15 caves, while Udayagiri meaning 'Sunrise Hill', has 18 caves. These partly natural and partly artificial caves of archaeological, historical and religious importance are called lena or lena in the inscriptions. Dug out mostly during the reign of Kharavela for being the abode of Jain ascetics, the caves are located at a distance of 8 km from the destination. A double storeyed monastery, Ranigumpha in Udayagiri is the most important of this group.
Caves have been numbered according to the enumeration of the Archaeological Survey of India (ASI). Due to art treasures of their sculptures and relief, Hathigumpha (cave 14) and Ganeshagumpha (cave 10) in Udayagiri are especially well known. Sculptural friezes adorn Rani ka Naur (Queen's Palace Cave, cave 1). Carved figures of women, elephants, athletes and geese carrying flowers embellish the Ananta Cave (cave 3).