It is true that conductors who move very little will usually use something quite large and those who move a great deal will tend to use something smaller. Partly this is a question of visibility. A larger baton makes a bigger picture. But this is also a good thing because some pretty awful accidents have happened—injuries to others and to the conductors themselves. See the paragraph on the death of Jean-Baptiste Lully at Wikipedia, but there are more.

This leads to a related subject which is the material and shaping of a baton. Mine are custom-made and of a light-weight and flexible balsa wood; they are also blunt rather than pointed. I have a variety of batons of various lengths and switch around depending on the repertory and the performing conditions (though none is very large or very small and switching is not frequent).