If you really want to start from "scratch", I suggest you to look first at following project on sourceforge:
This is pure java implementation of MAIL Server with of support POP3/SMTP. Sources are included so you'll get chance to REUSE some of the classes or patterns and maybe you'll even contribute that project.
Next thing I would look at before implementation of SMTP server is following article - specification of SMTP protocol:
Hope this help a bit.
Cheers and good luck brave soul