Hang in there TMAN...just continue to take it slow and see what happens. Sheila gave good advice...be honest with yourself and with your lady friends.
I'm quite sure if you do a little digging around on this site you will find age gaps like yours that have worked/are working. If it's right for you, you won't have any doubts, or at least they won't be big enough to make much difference.