The caves are on the western side of the Carboniferous limestone headland known as Humphrey Head. In the picture the doline (collapsed area on the top of the headland) is a clue to the large cave beneath it. The cave entrance which is dangerous and inaccessible can be seen centre right on the cliff face.
It seems likely the cave was formed well before the Ice Age look in SEARCH box for 'Llanddulas' where there are pictures of caves with similar characteristics.