If you enter the dungeon and the loading is done, the game crashes. Be warned!
Update: Every single “secret” cave causes my game to crash no matter what I do.
Update 2: Turns out that on EUW I can do these caves normally so I am guessing it had something to do with server stability on Zinnervale (EUC).