When you described your off-days, that was me on Adderall period. I had severe mood swings, depression, and irritability without any increased focus. I have since switched to 10mg of Ritalin that I take as-needed, usually just once a day when I need the extra focus. Not all of us respond well to stimulants (some see no benefit at all), and some respond better to some stimulants than others. I am also on Strattera, which is a mood stabilizer, to help with the mood swings and general irritability associated with ADHD. I have found this to be the best mix for me, low-dose of stimulant as-needed with a mood stabilizer every day. I would talk to your doctor about your symptoms, and maybe look into a different stimulant or adding a mood stabilizer to the mix. ADHD is a tricky thing to treat, and may take a few tries to find the right combination for you.
Good luck! I hope you find something that works well for you!