This is a point that the educated (but never the educators) understand very well.
The fact remains that doing math is hard. Being truly great requires that (a) your brain be wired in a particular way, and (b) that you put in the hard slog of 10,000hours to achieve the perspective necessary to conceive new mathematical truths.
Paradoxically, most people who pursue the path of math don't need to hear this from me - to them, the task is inherently rewarding enough to deserve their utmost attention.
For the rest of us, we've got to view it as a privileged tool, and nothing more.