Scaring Your Players

My attitude as DM is that you're there to present challenges that your party can overcome. While you need to provide a big enough challenge that they're going to be engaged, you also don't want to make things so hard that they don't have a reasonable chance of surviving.

Naturally, unless your group is super-hardcore, any player whose PC is killed and can't be brought back will simply be able to roll up a new character, so the actual penalty to killing a character off isn't actually that high.

I've never actually killed a PC in D&D, which is either due to my players really knowing what they're doing, rolling luckily, or my hesitance to make really deadly encounters.

What I've found, though, is that it makes it hard for me to scare my players.

Currently, the party is hovering around the tail end of tier 2 - some players are level 10, and the others are just a couple levels behind. Recently, I had the players enter "Kazejen Library," a super-secret building owned by House Dimir as a repository for some of their many, many books of secret lore. The revelation they discovered there was that the major Dimir villain of the campaign (there's one for each guild, part of a vast conspiracy) is an Elder Brain (yes, Mind Flayers aren't canon in MtG, but they seemed like a good fit for the Dimir, which seems to be the most aberration-friendly guild.) So I built the adventure to have them solve some relatively simply puzzles to get into the library and then face off against a group of Star-Spawn Manglers led by a Mind Flayer (who, sadly, used Mind Blast once, and all the players succeeded on their saves, and he was dead before he could do anything else.)

From there, they had to explore a section of the library. I kept them in initiative because there were monsters lurking around some corners of the library. The idea was for them to run from the monsters as they searched for the exit.

Instead, they mostly stood and fought.

This, for one thing, made the encounter last way longer than it was supposed to, but also I think failed to present the tone I was looking for - the party was slaying monsters, using their various abilities and attacks, and not really going into the labyrinth I'd designed.

Eventually, as a failsafe, I had monsters start to "respawn" to force the party to flee, but it hadn't really worked as well as I wanted it to.

So how do we scare our players?

I think, for one thing, that my players feel relatively safe with me - I throw them fights that they can win, and even if there are moments that get scary - like when my homebrewed Phyrexian Negator gets a sneak attack in a surprise round and takes the Barbarian nearly down before they can rage - the fight is still clearly doable.

Player characters in D&D are fairly tough, though, and their damage output can surprise you. I was looking at the Lich stat block while making a homebrew variant based on the Monk class rather than the Wizard, and realized that the thing has less than 200 health, which means that some high-damage-specced characters could, with some luck, really blast the thing apart before it could do much (especially if encountered at the level you're supposed to be to fight a Lich, which is roughly 15ish.)

So this makes me wonder how one can even scare a player character in the higher levels.

To an extent, this is intentional, of course. Gaining power in a world filled with monsters is the way to no longer fear them - a Vampire is an absolute menace to a level 4 character, but by the time the players are level 13 or so, they're a pushover.

The Ravnica game isn't meant, overall, to be a horror story, even if the main bad guys, the Phyrexians, are horrific. But it does make me wonder a lot about building horror into the narrative of the game. Horror is a tool, just like NPCs with quest objectives and hidden maps, to direct your players in one way or another. I'm struggling to make the players feel scared enough to play through what I want them to see.