If it’s even a question, there’s a problem that you shouldn’t ignore: lack of trust. Trust is the foundation of any healthy relationship. Without it, the relationship is dysfunctional and not worth preserving.
Now, you have to self reflect here. Is your lack of trust a problem you have with trusting people, or is it because he has actually acted in a way that betrays trust?
What you do next depends on the answer to that question.