Jiggle the handle a bit, and see if it stops.
Also, check if the water level is like, right at the lip of the empty pipe in the middle. If it's one of the old style "ball on a metal arm" floats, bend the arm down a little bit so the water shuts off before it reaches the lip of that pipe. If it's one of those new floats there should be a little screw that lets you set the water level down a little bit.