So I've been in a drawn out conversation with Alexis of The Tao about whether or not all DM rolls should be in the open or not. The most recent exchange in the comments of my previous post.
One of the original examples I gave of why sometimes it's better for the game to keep some rolls secret is in the case of secret doors in an old school game.
The way I see it, there's a better choice analysis/trade off in old school D&D. Searching takes time, 1 Turn per 10' searched. And every Turn (or two Turns, depending on the rules used) the DM makes a wandering monster check. So every time players make a choice to search an area for secret doors is them gambling on facing the next low payoff random encounter.
Random encounters provide some monster XP, but rarely have any treasure worth scooping up. And they risk losing hit points, spells, flasks of oil, potions, magic item charges, etc. to deal with.
So that's the situation. Players suspect an area might (or even must) have a secret door. Do they want to spend time searching for it, possibly failing, and risk more wandering monsters? Or do they want to just move on to the next area? If they can see the results of the search roll (d20+ style player Perception skill rolls, or old school style DM rolls made in the open) they have less uncertainty. If they roll well but find nothing, the question is answered. If they roll poorly, they are in the dark. By keeping the roll secret, the players are always in the dark if no door is found, and the choice remains on the table.
Alexis pointed out that the DM rolling in secret was functionally identical to the DM rolling in advance to see if the secret door would even be found. And if so, why wouldn't the DM save themselves some effort and roll in advance, and if negative, not even draw/stock/create the contents beyond the door?
Now, I have been formulating ideas in my head for the greater question - should some rolls be kept secret from the players? But Alexis wanted this specific question addressed. And when he repeated the question in the comments yesterday, he actually gave me the answer I was looking for.
He further specified a situation in which a dungeon would be visited once and then forgotten (like in a lot of modern adventure path gaming). And I have to say, in that specific situation, he's not wrong.
For example, back in October/November of 2007 (or maybe it was 2006, after I got married but before my first son was born and we were still living in Japan, pretty sure it was 2007 though), I intended to run Ravenloft as a one-shot. It turned into a 3 or 4 shot. Before running it, to speed things along, I made a time chart and rolled all the random encounters and their reaction rolls in advance. Partly this was because according to the module, at certain times, Strahd will be aware of the PCs and attack or send minions to attack. But I also wanted to just save a bit of time in the session.
This resulted in a few interesting encounters. For one, I'd rolled spectres, but friendly reactions! And when that encounter came up, the party were resting for the night in the chapel (which they incorrectly thought was still hallowed ground and safe). Thinking on the spot, it's the chapel, the middle of the night, spectres, but not hostile. It was a ghostly Black Mass being celebrated. Creeped the players out, built up the proper Gothic mood, but also allowed them to avoid what could have been an adventure ending encounter if it had devolved into combat.
I mention that to point out that I'm not against the idea of the DM making some rolls in advance. There's a time and place for that.
But back to the secret door thing. In my answer to Alexis, I pointed out to him that in my current West Marches campaign (actually also true for my play-by-post megadungeon game on RPOL), players often decide to return to partially explored dungeons. And as players come and go, and characters die and get replaced, it's not always the same party exploring.
In a game like mine, players knowing there definitely ISN'T a secret door at a certain location becomes a form of metagaming. But if the players themselves aren't sure, then their characters are also unsure. And they remain with the trade-off of searching for the secret door and risking wandering monsters, or not.
Now, it should go without saying that whatever is behind the door shouldn't be vital to the success or failure of the adventure. If the only way to get to the BBEG or rescue the prisoners or escape the fiendish Bond villain deathtrap is to find the secret door, don't roll. If the players search, they find it.
But if the secret door is just a shortcut from A to B, or has some extra loot or nonessential but helpful clues or strangeness that would just make for a cool moment, whether they find the door or not is irrelevant. It's an Easter Egg. In that case, why should the roll be in the open? Perceived fairness of the DM is the only reason why someone would argue that it should.
I'd argue that DM fairness will be known by other things than by whether the results of some rolls are kept secret or not.