Personally I have almost no affinity towards babies or any child under about 3, I find more of them ugly than I do cute, and to me they are just socially and mentally inept midgets. My family believe I am a sociopath as a result.
I'll probably become a big bundle of baby love once there's one in my close family, but at this point they're rarely anything other than a large (well, small) annoyance.
For me, there's a period of between 4-10 years old where most kids annoy the **** out of me.