Okay I fooled myself.
After the reaktivation of phpmyadmin by deleting the squirrelmail part in the nginx vhost configuration. I logged in as ispconfig and saw a different username for the ftp login and that worked. Sorry for beeing so stupid, and bother you with such stuff.
But the email problem is still alive. Are there any suggestion what to check. I read the topic about pam problems but I have to admit not to understand what to do.
Thanx in advance