LotR magic is interesting. Sauron bred all kinds of monsters essentially "hacking the system" to produce something in a world where no one but Illuvatar can create life, and Melkor got around the whole life/sentience thing by convincing lesser spirits to inhabit this creations. Balrogs, werewolves, etc.

Other creations Melkor imbued with his own vast power, essentially spreading his "divine fire" out amongst his thralls, and thereby diminishing himself. Sauron eventually does the same, in creating the One Ring.

So, when Gandalf became Gandalf, his power to affect things was explicitly limited. , The basic difference between The Istari and Sauron: they did not seek to usurp their position in 'the song of Illuvatar'. In fact the five Istari (Gandalf, Saruman, Radagast and the other unnamed two) were maiar spirits that had until that point stayed out of Arda (Earth) and all its shenanigans on purpose!

So, the staff served as a badge of office mostly. And thought it never stated, it's kind of implied that some of their power is imbued within it. This is why Gandalf looks totally different when he comes back as the White, he took Saruman's job as the head of the order, and had some more of his power unlocked as a result. The main strength of the Istari was their wisdom, knowing how the world worked and to a point, how "the song" was supposed to go. They were trusted to do the job (of taking down Sauron) because they wouldn't use their knowledge and power to 'hack the system'.

IMHO, Gandalf needed his staff not as a conduit of power here, but as an emblem of his station: He had to show Grima that his boss had just been fired.