What matters is the total mass of the black hole, not its density. If you replaced Earth's core with a black hole of the same mass, the gravity you'd feel at the surface (or beneath the surface) would be the same. You'd only notice a difference if you were in the hollow region formed by removing the core.

The way I see it, the real problem with a planet like Earth is that because the inside is so hot, the inner parts are too soft to support their own weight, and the crust is probably too fragile to support its own weight. That's not a problem, though, in an asteroid or a planet that's solid all the way through.