You ask how you can stop the conflict, if you can't would you accept this is the way it is and always will be?
You don't say how old your daughter is so I don't know how many years until she's an adult and you don't have to have any communications with him. It's a tragic situation. And if he's unreasonable like my x there is no way to communicate without conflict, it's impossible.
I've just called the police because after 12 years of being separated from my x he still can't leave me alone. Any excuse it's ridiculous. For example a random text message last night on my home phone (he doesn't think I have a mobile now so the abusive text are less frequent) the text said 'You own me money and everyone will find out you're a prostitute"... For a start I don't owe him money and I'm not a prostitute either. The thing is our 14yr old son was the person who first read that, my son would have deleted it as he's refused to pass on his abusive messages to me in the past but I was there also so I read it too. My 14year old more of an adult than he will ever be.
It's really hard, I haven't spoken to him in years, he's just made it difficult for me to bring up his children, I returned home once to find a for sale sign in the garden, the house is mine, I bought it, it's just one thing after another for 12 years, always putting obstacles in the way of me raising his children. Insane, spiteful, bitter and twister, he's never paid money for them but buys loads of property (I think he believed acquiring houses would make him more happy than contributing to his childrens upkeep) he's even tried to claim child benefit and child tax credit which was put on hold for 8 months because of the dispute which meant his children went without food and heating. What sort of people are these? Communication isn't possible with these people, they are unreasonable. I would call him say his daugther was in hospital and it would end with me hanging up after it descended into 'You're a fucking slag you, everybody hates you, bla bla'...
I'm not sure there's a solution to your problem. I refused communications because it was guarenteed to become verbal abuse. So then letters of abuse would come through my door and text messages, to which I've never responded. But sure they can get you down especially after 12 years there is no let up. Looking back I wish I'd upt and left the area. It hasn't been beneficial my children knowing their father. However bad that sounds and I know them not knowing him would have had consequences also, but I do believe it would have been better if we'd moved away. That would have been my solution to the problem in hindsight. But hope things improve & try not to let it get you down.