Compromised Email

Discussion in 'ISPConfig 3 Priority Support' started by blinden, Sep 23, 2013.

  1. blinden

    blinden Member

    I'm trying to find the source of this compromise.

    My server is being blacklisted and it's sending out a ton of email from accounts that don't exist, they do not show up in ISPconfig interface OR directly in the DB.

    They all seem to be from one domain name, but I'm having trouble finding out how they are authenticating, just tons of [email protected] are flying out of my server.


    It's coming from several different IP addresses

    Ubuntu 12.04

    The whole system is up to date.
     
    Last edited: Sep 23, 2013
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    First check if your server is a open relay. Post the output of:

    postconf -n | grep mynetworks

    and use a open relay check, you find several sites that provide this check for free when you search with goole fr these terms.

    The most likely reason for such spam is a hacked website, so they dont need an email account on your server to send them, they just use the php mail() function to do it.

    First you should check if the emails are send by a php script, recent php versions add some info in the mail header if the email is sent by a php script. You can view the emails in the queue with the postcat command.

    1) get a email ID from mailqueue with

    postqueue -p

    email ID's are the cryptic numer / char combination at the begnning. Lets assume for the example that the mail ID is A9A1F23B47DC

    2) The mails are most likely in the deferred queue, so you can view it with:

    postcat /var/spool/postfix/deferred/A/A9A1F23B47DC | more

    In the deferred queue, emails are organized in this way that you have a folder with the name of the first char of the ID which contains a file with the full ID as name.
     
  3. blinden

    blinden Member

    mynetworks = 127.0.0.0/8 [::1]/128
    proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $virtual_mailbox_limit_maps
    smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, check_recipient_access mysql:/etc/postfix/mysql-virtual_recipient.cf, reject_unauth_destination

    Not an open relay according to mxtoolbox

    Working on the second part of your post currently.
     
  4. blinden

    blinden Member

    Here is one of the records

    *** ENVELOPE RECORDS /var/spool/postfix/deferred/9/9AF9F34C2B6E ***
    message_size: 1644 1057 5 0 1644
    message_arrival_time: Sun Sep 22 07:46:15 2013
    create_time: Sun Sep 22 07:46:15 2013
    named_attribute: log_ident=9AF9F34C2B6E
    named_attribute: rewrite_context=local
    sender: [email protected]
    named_attribute: encoding=7bit
    named_attribute: log_client_name=localhost
    named_attribute: log_client_address=127.0.0.1
    named_attribute: log_client_port=53193
    named_attribute: log_message_origin=localhost[127.0.0.1]
    named_attribute: log_helo_name=localhost
    named_attribute: log_protocol_name=ESMTP
    named_attribute: client_name=localhost
    named_attribute: reverse_client_name=localhost
    named_attribute: client_address=127.0.0.1
    named_attribute: client_port=53193
    named_attribute: helo_name=localhost
    named_attribute: protocol_name=ESMTP
    named_attribute: client_address_type=2
    named_attribute: dsn_orig_rcpt=rfc822;[email protected]
    original_recipient: [email protected]
    done_recipient: [email protected]
    named_attribute: dsn_orig_rcpt=rfc822;[email protected]
    original_recipient: [email protected]
    done_recipient: [email protected]
    named_attribute: dsn_orig_rcpt=rfc822;[email protected]
    original_recipient: [email protected]
    done_recipient: [email protected]
    named_attribute: dsn_orig_rcpt=rfc822;[email protected]
    original_recipient: [email protected]
    done_recipient: [email protected]
    named_attribute: dsn_orig_rcpt=rfc822;[email protected]
    original_recipient: [email protected]
    recipient: [email protected]
    *** MESSAGE CONTENTS /var/spool/postfix/deferred/9/9AF9F34C2B6E ***
    Received: from localhost (localhost [127.0.0.1])
    by mailserver.wpa.net (Postfix) with ESMTP id 9AF9F34C2B6E;
    Sun, 22 Sep 2013 07:46:15 -0400 (EDT)
    DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lvgraphics.net;
    s=default; t=1379850375;
    bh=rNXB+HjfDVbNjE4G57h7HqyTDlO65VBepKtPFJiFmNo=;
    h=From:To:Subject:Date;
    b=GUlHYWzOrdlVoRfdzTKWER6TJa60atmR+OWGgtBgJ4pfdFyoDjvdMyD/kOqjH6/+2
    5GlQLwTctIuBun8Qr802s9XmkRwrvN3Z4eRDLoNp+f3v8i8dHA5iQTvFejidh5vXhb
    ITTig9FBJ5sP8mLFa6OQDOTTQx1JYHlIwxGIp1Hc=
    X-Virus-Scanned: Debian amavisd-new at mailserver.wpa.net
    Received: from mailserver.wpa.net ([127.0.0.1])
    by localhost (mailserver.wpa.net [127.0.0.1]) (amavisd-new, port 10024)
    with ESMTP id OsyoET80l1Ns; Sun, 22 Sep 2013 07:46:15 -0400 (EDT)
    Received: from fwppgi (unknown [2.133.66.45])
    (Authenticated sender: [email protected])
    by mailserver.wpa.net (Postfix) with ESMTPA id 0162C34C3019;
    Sun, 22 Sep 2013 06:52:24 -0400 (EDT)
    DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lvgraphics.net;
    s=default; t=1379847147;
    bh=rNXB+HjfDVbNjE4G57h7HqyTDlO65VBepKtPFJiFmNo=;
    h=From:To:Subject:Date;
    b=H9wuNVQAbHztFOKiJ2dFskgsDcGFtgp2bN0jy7HoYyj25s2dVvu0FsgcQHXPfuX+P
    mi9Ld7Gy50OPP3A6dUq0+XxInBQtC9VDYf0FV70KgoJAUFzABd31u3Bukx+kth7uUN
    0cYeP0L210wYUos0f6eRzMWTd7WJeVWkTC/JeZlU=
    From: "q te" <[email protected]>
    To: <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>
    Subject:
    Date: Sun, 22 Sep 2013 11:42:11 -0700
    MIME-Version: 1.0
    Content-Type: text/plain;
    format=flowed;
    charset="iso-8859-7";
    reply-type=original
    Content-Transfer-Encoding: 7bit
    X-Priority: 3
    X-MSMail-Priority: Normal
    X-Mailer: Microsoft Outlook Express 6.00.2900.5931
    X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157
    Message-Id: <[email protected]>

    http://lire-et-merveilles.fr/movie.htm?rafuty



    *** HEADER EXTRACTED /var/spool/postfix/deferred/9/9AF9F34C2B6E ***
    named_attribute: encoding=7bit
    *** MESSAGE FILE END /var/spool/postfix/deferred/9/9AF9F34C2B6E ***

    Just noticed that it does list a partciuarl "authenticated user" so maybe that account is the one compromised? There are no websites on this server, other than roundcube and phpmyadmin, it is strictly mail.

    I changed that users password, but there are TONS of messages in the queue, how can I remove them:

    postqueue -p | tail -n +2 | awk 'BEGIN { RS = "" } / lvgraphics\.net/ { print $1 }' | tr -d '*!' | postsuper -d -

    I've used this to remove individual users before, can I do this domainwide?
     
    Last edited: Sep 23, 2013
  5. till

    till Super Moderator Staff Member ISPConfig Developer

    Yes, thats very likely. You should set a new password for that acount.

    To delete all emails in the queue that were from a specific sender, use this tiny script:

    Code:
    mailq | tail -n +2 | awk 'BEGIN { RS = "" }
    # $7=sender, $8=recipient1, $9=recipient2
    { if ($7 == "[email protected]")
    print $1 }
    ' | tr -d '*!' | postsuper -d -
    just replace the email address inside with the sender address, then copy the script as it is on the shell and hit return.
     
  6. blinden

    blinden Member

    Will that script queue off of the "authenticated user" I would rather just wipe them out for the entire domain since it has the appearance of coming from a huge number of different fake addresses
     
  7. till

    till Super Moderator Staff Member ISPConfig Developer

    It will use the sender address.

    Here is a script that uses a regex instead:

    Code:
    mailq | tail -n +2 | awk 'BEGIN { RS = "" }
    # $7=sender, $8=recipient1, $9=recipient2
    { if($7 ~ [email protected]\.tld/)
    print $1 }
    ' | tr -d '*!' | postsuper -d -
    so you can delete by sender domain.
     
  8. blinden

    blinden Member

    Well, I ran this, and it appears to be doing stuff

    Tons of logs reading like:

    Sep 23 11:29:01 mailserver postfix/postsuper[28013]: 2F7BB368412A: removed
    Sep 23 11:29:01 mailserver postfix/postsuper[28013]: 1E60C3662526: removed
    Sep 23 11:29:01 mailserver postfix/postsuper[28013]: 862D93503F42: removed
    Sep 23 11:29:01 mailserver postfix/postsuper[28013]: F2A9B3700E3E: removed
    Sep 23 11:29:01 mailserver postfix/postsuper[28013]: 1DA8634E2F73: removed
    Sep 23 11:29:01 mailserver postfix/postsuper[28013]: 5EAAB34E7A41: removed


    However, the files in are remaining, is that to be expected?

    Additionally, there appears to be new ones still being created after I changed the users password. I completely deleted the user at this point.

    confirmed new files created in /var/spool/postfix/deferred/ with the same username listed as "Authenticated user" despite his account being deleted from the server, they were just created as of 2 minutes ago.

    Postfix is currently stopped but I need to get my server back up
     
    Last edited: Sep 23, 2013
  9. till

    till Super Moderator Staff Member ISPConfig Developer

    Postfix will remove them, this might not be in realtime but when it cleanes up its queue directories the next time.

    Thats not good. Did you check the new emails, are they really now or just requeued and when they are new, which user was used for the smtp login?
     
  10. blinden

    blinden Member

    Well, I was just doing an ls -ltr and seeing what were the newest files, I just opened one up and it has:

    create_time: Mon Sep 23 00:43:59 2013 which would have been a little over 11 hours ago my time so perhaps they aren't new?

    Still plowing through removing these queued messages.

    Any other thoughts/logs/etc I should be looking into while I'm waiting for this to process that script?

    EDIT-

    Finished that up:
    postsuper: F3E4A34C0D1C: removed
    postsuper: F419434D59AE: removed
    postsuper: F23A534CA1BF: removed
    postsuper: Deleted: 473337 messages
     
    Last edited: Sep 23, 2013
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    I would recommend to check with:

    postqueue -p

    if there get new mails added. Another option is to check the mail.log file in /var/log.
     
  12. blinden

    blinden Member

    So, that took a long time, but things seem to be a bit more normal now, other than a HUGE amount of traffic due to Postfix being turned off for an hour or so.

    So, Huge thanks Till, I appreciate the super fast responses.

    Now the matter of getting off blacklists.

    The big question here though is, what can I do better to help prevent this.

    In our environment, it wouldn't be unreasonable to cap users at a certain limit if possible, like 300 emails/24 hours or something, so any thoughts on what I can do to better prevent issues with compromised accounts. I really upped the password strength requirements but sometimes they still get compromised.
     
  13. till

    till Super Moderator Staff Member ISPConfig Developer

    ISPConfig 3.0.5.4 will ship with support for policyd which allows to set email sending quotas like max. 50 mails / hour.
     
  14. blinden

    blinden Member

    fantastic, I looked into PolicyD but was having a hard time configuring it
     

Share This Page