I had to find the damn chuck key first, to change the drill bit out, to let me drill the hole in the chuck key, so I could attach the chuck key to the lanyard holder, so I wouldn't lose the chuck key again... All in, it was a 20 minute search for a 30 second operation but that says more about me than it does about the project...
I think it would depend on how close to the chuck it is. Way better than keeping it in my pocket and then finding it when I'm folding laundry though....