I call everyone baby. Young? Baby. Old? Baby. Man, woman, black, white, purple with green polka dots? "What's up, baby? How are you?" I'm a middle aged white guy.
There's a 60-odd year old black guy that comes to one of the bars I frequent. I always greet him with "What's up, baby?" He always says "Same old, baby, how you doin'?" I have no idea how that became normal because it feels so weird explaining it but it feels super normal saying it.
Nope. It may be because of the tone. I don't do an Austin Powers thing. I say it the same way I'd say buddy or friend.
Thinking harder about it, generally if I'm meeting someone for the first time I use their name because I'm trying to get the hamster that runs the wheel in my brain to put their name somewhere I'll remember it. By the time we've interacted a while it's very likely they've seen me say it to half a dozen of my friends.