As someone already said - first start by examining your attitude towards your son. It doesn't matter whether your son knows that your boyfriend doesn't want him anywhere near him. Your son FEELS that. Kids, and adults, can feel things that they don't see.
Having a child is a responsibility. You were a child once. You would've died without your parents care. It is a God given job! Be happy you have a child in the first place! I am sure there are boyfriends out there who love kids; I am also sure that whatever caused your breakup with your son's father (and God I hope it wasn't a fatal accident or something), will cause you another break-up! It is really important that we learn from these experiences - not about others and how bad they are, but about ourselves!
Your first challenge is understanding that for a few years your son was given to you by God to take care of him - love him, and above all: help him become independent and feel welcome on this planet. Subconscously he gets a message that he is not welcome with your boyfriend, and that later creates a person who doens't feel equal to others, and who feels that his life is not worth much; that he is NOT welcome on this planet. Then we start wondering why are people on drugs, or how come people become suicidal - they just don't feel they are worth the life they are living.
I've gone through therapy (Transactional analisys, combined with other techniques), and also studied psychology for many years (that part of me I had to shut down during therapy to really benefit from it). Relationships are our greatest chance to grow. Especialy bad ones.
One example: if you can forget about your son so easily, you can most certainly forget about yourself just as easy! This means you were desensitized to your needs as a kid - probably pushed to the side and not treated with enough respect, just as you did with your son. Nothing to feel guilty about, but plenty to learn from.
I hope you can take this with open mind - it is not a criticism, it is a pointer that your dillema with that boyfriend is small compared to some real issues.
Your child will grow and the way you treat him stays with him forever. In the meantime, make him your #1 priority, as he has no other mother then you. There are not that many examples of good parenting, but search for them, and learn from them! And if you can, and if you had a broken relationship or divorce it is good to get a good therapist - otherwise the whole thing was a waste. Breakups can be turned around to benefit us more then anything else!
With your son, make sure you let him do on his own all the things he can - like prepare some food, or clean after him, or do whatever else. And another important aspect is patience: just take a look at lions or other animals and what they allow their kids to do them! There is an infinite amount of patience in the animal world towards their offsprings; we have lost that as we apply our economical values to children - how much time and money they "cost" and so on.
Give patience and support to your son to become independent and you'll be the best mom that ever existed.