Keep a dated and times diary of things he does that are abusive or would be negative to an impressionable child. Keep it factual and quote anything your child says/does in response also.
If you apply for sole custody use it as evidence of why you don’t want unregulated contact between him and her.
But stress you are willing and encourage for him to have supervised contact at a contact centre or in the presence of a support worker or similar.
Just try to keep as calm as you can when around him, but also show your daughter it is good to embrace any sad/angry/confused feelings in a productive way. If she is angry and shouts when calmed down a bit get her to draw out the frustration and talk to her while she is focused on something else.