Though many of the world’s most impressive churches feature high ceilings and soaring spires, sometimes you have to go underground to find the real gems. Jesus called the apostle Peter, the rock on which “I will build my church,” but these churches are built inside the rock or carved into it.

Cave churches and monasteries have been around since the beginning of the church, often as a way to seek out spiritual succor in a remote and solitary place. Carved into the rock, or located inside caves, they have a raw appeal that often contrasts with urban cathedrals with stained glass windows.

Explore them here:

monastery of saint simon
A general view of St Samaans (Simon) Church also known as the Cave Church in the Mokattam village, nicknamed as ‘Garbage City,’ is seen on July 26, 2012 in Cairo. Once a week hundreds gather at the Cave Church in Moqattam, after the prayer, a coptic priest performs exorcism or healing blessing to some of the believers. With a cross and holy water he fights spiritual entities and demons. The Monastery of St. Simon the Tanner is the largest and it has an amphitheater with a seating capacity of 20,000 making it the largest church in the Middle East. It is named after the Coptic Saint, Simon the Tanner, who lived at the end of the 10th century, when Egypt was ruled by the Muslim Fatimid Caliph Al-Muizz Lideenillah. Simon the Tanner is the Coptic Saint who is associated with the legend of the moving of the Mokattam Mountain. AFP PHOTO/GIANLUIGI GUERCIA

coober pedy church
A general view of the Underground Serbian Orthodox Church on May 5, 2009 in Coober Pedy, Australia. (Photo by Quinn Rooney/Getty Images)

cave church
Panoramic shot taken in small chapel built in cave, in Western Syros, Cyclades. (V. Paravas / Getty)

cave church
The interior of the Panagria Hrysospiliotissa (Chapel of Our Lady of the Cavern) in a grotto under the Temple of Dionyso, in Greece.

zipaquira
The Salt Cathedral of Zipaquirá is an underground Roman Catholic church built within the tunnels of a salt mine 200 meters underground in a Halite mountain near the town of Zipaquirá, in Cundinamarca, Colombia.

cave church
France, Provence, Sainte-Baume Mountain: woman burning candle at Troglodyte Sainte-Marie Madeleine Holy Cave.

church goreme
The Cave Churches of Goreme, Turkey

cave church
Italy, Calabria, Pizzo, Piedigrotta church

callao cave
Callao Cave Chapel, Philippines

cave church
The Hermitage Saint Antoine and his troglodyte chapel, located at the bottom of the Gorges Galamus. The gorge is the border between the Aude and Pyrenees-Orientales, Languedoc-Roussillon, France, Europe (P. Eoche / Getty)

cave church
Cave Church of St. Peter, Hatay, Turkey

cave church
Agioi Saranta, a cave church near Protaras, Cyprus.

geghard
Tomb at back of Geghard Monastery complex in Armenia.

cave church
A man gets out of a chapel built on the rock inside the Orthodox Christian monastery of the Temptation near the West Bank city of Jericho, on February 22, 2014. The monastery of the Temptation was built during the 6th century CE above the cave traditionally said to be that where Jesus spent forty days and forty nights fasting and meditating during the temptation of Satan. (THOMAS COEX/AFP/Getty Images)

mihalich church
Mihalich Cave Church, Bulgaria

cave chapel
Chapel Of the Blessed King, Wieliczka Salt Mine, UNESCO World Heritage Site, Krakow, Poland

budapest cave church
St. Stephen’s Cave Church, Budapest, Hungary

cave church
Jonas caves, Auvergne, Frace

cave chapel
Small underground chapel carved into the rock of the Barranco de Guayadeque valley.

cave church
Man praying at Cave Chapel at Modderpoort Prior, near Ladybrand, Free State, South Africa