When he starts getting clingy, then it's time to leave. Once he shows signs of clingy-ness, they will stay there. If he gets upset that you talk to other men, then that's a problem. I have a lot of guy friends that I talk to and occasionally meet up with. It's innocent.
I don't freak out every time he talks to a girl. If he's possessive and controlling, then leave him. Jealously can be cute, but it can also become a problem. Trust has to be there from the start.