For me, I'd have to say the best advice is to ignore the noise. Clive and I have a 38 year age gap. We've been together 2 years so we've both heard remarks in public places.
"She must be a sugar baby"
"She's only with him for his money"
"He's a cradle snatcher"
It's hard not to bite but you need to block it out. Don't let strangers make you question what you have. We have a loving relationship and that is what matters. You and your partner are what's important, not the gossip from people you don't know. And thankfully for us, our families have been very supportive. Admittedly surprised at first, but supportive.
So focus on yourselves and not what other people think. Otherwise you'll let it get you down and it will interfere with what could be the best thing in your life!
The other tip I would give is to find some common interests. There is a good chance you may enjoy different things, no different to any relationship but more likely in an AGR. If this is the case find some new interests that you both enjoy. And discovering new activities together is very rewarding, even if turns out its not your thing. At the very least you'll have an experience together.