As a father, he should want to be in his children's lives. The reason your children go to family members other than their father, when they are crying, is because they feel safe and comforted. Children sense more than we give them credit. They know who loves them and cares for them. It is hard to be a single mother, but you can't continue to make the father be a father. He has to be the one to step up and want to be a part of his children's lives. He has to want to be an active part in their lives. When I got divorced 15 yrs ago, I made sure I had sole physical & legal custody and the father had visitation. I had the final decision on all issues because I was looking out for my son's best interest. You have to make the best decision for the welfare of your children.
Stay in prayer and God will guide you.
This is a tough place to be. I have been there and here are some things I did. I am not trying to give you advice, just want to share my experience. We divorced when my daughter was five and my son was four. I never wanted to keep my kids from their father and it wasn't my place. I also never made myself responsible for the father's behavior. If he came to a birthday or special event, he came of his own free will. If he didn't come, that was also his choice. I never spoke bad about him to my children and I never defended him. Kids are a lot smarter than we give them credit for, but they also want to please BOTH parents. Please try to allow him to be in their lives without placing any regulations on him. He is an adult and he is free to make his own choices. Your kids know they can count on you and their grandparents, but please allow them to have a relationship with their father, even if it is different than what you want it to be. I parent my children much differently than their dad does, but that doesn't mean either of us is wrong. Trust that God has a plan and take heart in your relationship with the kids. Everything else is out of your control. Good luck!