We are so brainwashed by the media to think that sexiness can been be defined by a set of physical attributes, with age a major factor. It is not true. Sexual attraction, even if it only leads to a one night stand, is never just about two bodies. It is about two people, two personalities, two minds, two souls...
It took me along time to understand and accept this. This probably led up to my splitting up with the younger man who fell in love with me and with whom I have only just reconnected. After splitting up I had some flings and they were all with younger males. I don't regret them because they taught me to believe in my power of attraction.
Read past threads and you will see what a lot of people say. It really helped me. Yes, he could leave you for a woman his age, a woman younger than him, a woman even older than you! This is the risk in any exclusive relationship but the real danger is to make your fear a self-fulfilling prophecy.
Have faith in yourself and in life!