How does surprise work when both sides of an encounter are attempting to surprise the other?

Question

I an a total noob in D&D and just started LMoP (the Starter Set adventure).

We are now in a room before Klarg's cave (the Twin Pools, I think). One goblin escaped to tell Klarg that there's an attack coming. The book says Klarg, the Wolf, and the Goblins hide to prepare to ambush the PCs.

Now, my PCs want to suprise Klarg, the wolf, and the goblins since they deduced that there are monsters in this cave.

How does surprising work if two groups are trying to ambush/surprise one another?

Answer 1

Each character can be surprised individually

The rules for Surprise states:

The DM determines who might be surprised. If neither side tries to be stealthy, they automatically notice each other. Otherwise, the DM compares the Dexterity (Stealth) checks of anyone hiding with the passive Wisdom (Perception) score of each creature on the opposing side. Any character or monster that doesn't notice a threat is surprised at the start of the encounter.

Roll Dexterity (Stealth) for each of the PCs, and for Klarg, the Wolf, and Goblins.

For each PC, compare their Passive Perception to the value of the Dexterity (Stealth) check for each enemy. If a PC doesn't notice any enemies, then they are Surprised.

For each enemy, do the same, compare their Passive Perception to each PC's Dexterity (Stealth) check. If that enemy doesn't notice any PCs, then they are Surprised.

What is Passive Perception?

Passive Perception is a characters general awareness. It is calculated as follows:

10 + all modifiers that normally apply to the check

If the character has advantage on the check, add 5. For disadvantage, subtract 5. The game refers to a passive check total as a score.

For example, if a 1st-level character has a Wisdom of 15 and proficiency in Perception, he or she has a passive Wisdom (Perception) score of 14.

Consider using a group check

Instead of rolling Dexterity (Stealth) for each character, perhaps consider making a Group Check for each side:

When a number of individuals are trying to accomplish something as a group, the DM might ask for a group ability check. In such a situation, the characters who are skilled at a particular task help cover those who aren't.

To make a group ability check, everyone in the group makes the ability check. If at least half the group succeeds, the whole group succeeds.

Instead of rolling for each individual character, make 1 group Dexterity (Stealth) check for each side, and make 1 group Passive Perception check for each side. Then compare the values and decide if either side are surprised.

There is more info in the Hiding section

If you want to know more about how Stealth functions, check out the Hiding section.

Keep in mind the environment

Where and when the ambush takes place may affect how easy it is to hide or spot others. You may want to award Disadvantage or Advantage depending on how favorable the conditions are. Hiding in the dark, in shadows, or in fog can all make someone harder to spot with sight:

A given area might be lightly or heavily obscured. In a lightly obscured area, such as dim light, patchy fog, or moderate foliage, creatures have disadvantage on Wisdom (Perception) checks that rely on sight.

A heavily obscured area--such as darkness, opaque fog, or dense foliage--blocks vision entirely. A creature effectively suffers from the blinded condition when trying to see something in that area.

Keep in mind that locating a hidden enemy isn't a purely sight check. You could also locate them by hearing their armour clink, listening for footsteps, or smell them.

Answer 2

No Surprise Here

jgn's answer covers the rules related to Dexterity (Stealth) and Perception (Wisdom), and does it well, so I won't rehash them. Instead, I present an alternative: surprise isn't really a factor in this situation.

Trying to be stealthy is all well and good, but we're talking about a forewarned enemy laying in wait for a forewarned party. You're allowed to declare nobody is going to be surprised, because everybody is ready for a coming fight.

The DM determines who might be surprised. [...] Any character or monster that doesn't notice a threat is surprised at the start of the encounter.

In the subject case, the warning has just happened. Everybody is going to be amped up and ready to go. If there's no doubt a fight is coming, and coming soon, it's not really a surprise. If the party were to pull back and wait for a bit, give the kobolds a chance to calm down, create some doubt as to when or if the party is coming, there might be a chance for surprise then.

I reserve surprise for situations where one group definitely has the drop on another. I adjudicate surprise based on the level of wariness when the situation switches to "combat time" and initiative is rolled. For example:

• The party successfully sneaks up to a closed door, hears people on the far side, and bursts through it. The Stealth checks determine how quiet the party is in that instant before the boot hits the door: Did somebody's armor jingle? Did a weapon being drawn scrape coming out of the sheath? Did somebody talk too loudly arranging getting the rogue away from his trap check and the fighter in place to bust the door?
• The party teleports into a location they were expecting to be empty, everybody involved is surprised without any rolls. In many cases, the first round may pass by with nobody being able to do anything, but there are abilities that allow characters to ignore surprise - the Alert feat, Barbarians with Feral Instinct, and so on.
• The party is travelling and being accosted by bandits is fairly straightforward, though I have been known to flip things around and use Passive Stealth for the bad guys and active Wisdom (Perception) checks for the party. Players like throwing dice, after all.
• A social scene going sideways can use Wisdom (Insight) and Deception (Charisma) instead; the Red Wedding from Game of Thrones, for example.