I more or less do this for stealth or deception checks. I get the players to tell me their modifier, and then roll behind the screen. And then I'll give them a description like "try as you might, you can't seem to make your armor stop squeaking" or "to the best of your knowledge, you are quiet and unseen" or whatever. But I don't actually tell them what they rolled, and let the scenario play out.

My players seem to actually prefer this, since it allows them to blissfully ignore the metagaming elements.