I thought I just let you know that I solved the issue. It was the sendmail path in php ini that caused the script to go incredibly slow.
i changed it to:
sendmail_path = /usr/lib/sendmail.postfix -t -i
No the mails fire real fast. However I am having another problem now. In my maillog file I frequently see messages like this:
Feb 1 19:44:29 server1 postfix/smtp: 88B9F14A81EE: to=<email@example.com>, relay=none, delay=20, delays=0.02/0.01/20/0, dsn=4.4.3, status=deferred (Host or domain name not found. Name service error for name=gmail.com type=MX: Host not found, try again)
Same thing happens with yahoo or hotmail. It doesnt happen with every mail but quite often. Any ideas on this one?