Yes, stimulant medication is very safe. Much safer, in fact, than having untreated ADHD where your chances of early death double over non-ADHD peers.
Even among the non-ADHD groups that use it recreationally, it’s safer than cigarettes and alcohol. The thing that makes it dangerous, is using it when you have a pre-existing heart condition. Many people who have heart conditions don’t know they have heart conditions, so using stimulant medication without a doctor checking you out first can be very risky. Thus, you need a prescription for it.
The bad reputation of stimulant medication comes from ignorance and an aversion to treating mental disorders with medication in general. It’s the false dichotomy of “natural” vs. “artificial.” And when it comes to the brain, laymen want to believe that everyone’s brain is perfect and problems can be overcome with willpower.