Sorry for your hurt.
You will decide what to do. Just be courageous and honest. Maybe keeping a private record of the conflicts could help break away at the feeling low for days stuff by defining it for what it is. This is how I mean it, write what happened right before, what was the main offense/hurt/conflict, what happened after.
This is best really brief and really honest (not mean) and don’t worry about who had an excuse to do what (oh he had a bad day or oh I was just so tired of that way he says that). Just the facts and not editorials. Review it regularly when you’re not emotional, and when you’re ok knowing what the real deal is. You have to be willing for it to come out, be courageous & you’ll see it.
I’m sorry for the hurt you’re feeling. The more you have combined responsibilities the more intense the struggles can be and more vulnerable it feels and that can make bigger reactions that are “automatic” (unquestioned & untested for reality).
When there’s conflict, it’s responsible and caring to address it. Your feelings are your own, but in a relationship, good or bad or in-between, each person has an influence on the other. In the good sense, that’s why people choose to be together in the first place. Maybe he feels shame and doesn’t know how to resolve it when he has struggled & failed and disappointed you the same way again. Maybe he is distancing to protect himself (and not you) & not have to deal with it. Could be a good counseling topic?
Your patience is for you. It truly benefits him, but it is for you. Never doubt patience is helpful. It doesn’t mean walk all over me. It means learning to think & assess clearly in real time.
If you sense he would like you to lose patience with him, that’s another thing. It can be an avoidance strategy. If you do end up keeping track with writing, another helpful thing to note is what the immediate result is from his outburst (or whatever) that might bring relief from some pressure he is experiencing. Doesn’t make it excusable, or even acceptable, but explainable which is different. That explanation is info that can help you decide how you want to deal with it in yourself and towards him.
Be strong enough to tell yourself the truth and keep seeing it. It takes courage. Best wishes to you. Don’t lose heart about your own self management and future, no matter what you choose.