And next time refuse it. Don't take gifts from weirdos. It might feel like the more dangerous option to refuse, but assholes may treat is as you owing them some kind of debt.
Have you mentioned the degree to any other neighbours? He might have heard about it that way, and is trying to make conversation, very awkwardly. But still weird to mention it to you. And donβt eat the chocolates.