Even though you haven’t found success with the shared calendar yet, I still think it’s your best bet, since making plans when you already have a commitment is a problem.
So, what can you do to try to make this work for everyone? Have you tried color coding events. So his activities are one color and yours are another? Visually, he’d only have to worry about his color (maybe his is a vibrant color and yours is grey or white so it kind of fades into the background).
My husband and I don’t share a calendar, but I do send him calendar invites for anything he will also be participating in or needs to fill in in my absence for (like being the only parent with kids if I’m out of town). That way, I’m only adding to his calendar my activities which are a necessity for him to know about. That means far less on his calendar to keep up with.
With google calendar (I think it’s google calendar), if you set an address for a calendar item, you can set a notification for “time to leave.” This is so helpful in planning for those with time blindness.
ADDitude Community Moderator, Parenting ADHD Trainer & Author, Mom to teen w/ ADHD, LDs, and autism