Discovered by Michael Horner in 1837 and excavated by Joseph Jackson and the British Association. Victoria Cave was originally known as the Fox Holes. During excavations 9m of deposits were removed, changing the entrance from the original narrow crawl to 10m high by 30m wide. The original entrance was to the top left of the current entrance. A smaller entrance is to the north.

Several passages lead off from the main chamber, which was originally two caves, Chamber A to the left and Chamber D to the right. At the far end of the right wall a 2m climb up leads to a crawl, which eventually becomes too tight.

From the main chamber, the first passage left passes the second smaller entrance and drops down to join the other two passages left in Chamber B. Two passages from here, one a crawl and the other accessed by a 1m climb, lead into a low chamber with a pool, Chamber C.

Birkbeck Gallery is a narrow passage at the back of the main chamber, which passes over several holes and lowers to a crawl before reaching a 10m pitch.

Remains of animals found include the bones of mammoth, brown bear, reindeer, hippopotamus, straight-tusked elephant, spotted hyena, lion, narrow-nosed rhinoceros and giant deer. Other finds include Bronze Age pottery, Romano-British jewelry, tools and over 100 coins dating from 83 BC to 346 AD.

A sign outside the cave indicates the roof is unsafe.