The three things overlapping: ADHD, specific math LD, and normal increase in memory problems with aging. In my experience ADHD acts like an amplifier. ADHD amplifies symptoms of other problems like anxiety or memory problems.
The medication for ADHD is an important part of an effective intervention plan. It sounds like you were not taking the medication regularly.