XMPP with ISPConfig

Discussion in 'Developers' Forum' started by theWeird, Feb 20, 2015.

  1. theWeird

    theWeird Member

    At the moment I am writing on a modification of the code in ispconfig master branch to enable ISPConfig to manage XMPP domains and users.
    @till: can you estimate a planned release date for v3.1? Maybe I can place my first merge request in time for 3.1.

    Details:
    Server Software:
    The XMPP server managed by my modification is Metronome, a lightwight, easy to configure and extend XMPP service forked from prosody im. It is also the preferred XMPP daemon for use with Jappix Webchat or Movim.
    All required steps to get metronome up and running on a debian based host will be included in my merge request.

    ISPConfig:
    My Modification adds new client- and reseller-limits for XMPP domain and user count.
    It adds a new menu section to "E-Mail" Module: "XMPP" with the two menu links "XMPP Domains" and "XMPP Accounts". (Maybe renaming to "Communication" or "Mail & Chat"?)
    Of course it also adds required steps to the install routine of ISPConfig and server configuration options.

    Release v1. (merge request placed):
    Supported Domain Features (per Domain configuration):
    - List of Domain Admins (Jabber IDs)
    - Pubsub (Microblogging via XMPP)
    - Bytestream Proxy (used to bypass Firewalls for Firetransfer and Voice-/Videochat)
    - Status Host per Domain (Status of Server Components in JSON/XML format and 'Webpresence' for integration of users chat status on websites)
    - Anonymous Host per Domain (for WebChat with guest access)
    - Global BOSH support
    - MUC support (Multiuser Chatrooms) with pastebin (autoshorten or manuel triggerd link to server-stored paste instead of long text) and http_archive (Chatroom history)
    - Creation of SSL certificates and CSRs for entire server and every single hosted domain
    - Auto-Setup of required DNS records

    Supported Account Features:
    - active/inactive
    - server module for SQL authentication to identify users by their password from ispconfig database

    Release v1.1:
    New XMPP domain option: Account management method (Normal / By Mail Domain)
    Option "By Mail Domain" requires the corresponding mail domain to be created before the xmpp domain.
    OnInsert and OnUpdate this option will sync all mail accounts to xmpp accounts of this domain. It will not be possible to create/delete XMPP accounts for this domain or to change their passwords. They will be created, updated and deleted by management of corresponding mail accounts.
    Furthermore the mail domain cannot be deleted as long as there is a linked xmpp domain. First the XMPP must be deleted or switched back to management "normal".

    Further Ideas:
    Support of password changes and public registration
    Only if I can manage to create a plugin to handle this using ISPC Remote Api.
    Maybe, if this modification gets included to ISPC, I can get some support by the developer of Metronome to implement this.
     
    Last edited: Mar 5, 2015
    till likes this.
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    This looks like a really nice addon!

    I'am currently working on bugfixes and hope that I can make a first alpha or beta version sooon. If you can do a merge request for your code in the next few weeks, then its likely that we can add it in the 3.1 release. If not, then thats no real problem as I've planned to have releases more often in future anyway. My idea is to switch to a release cycle like ubunt with 2 major releases every year.

    As you might have noticed, we have splitted the remote api into separate files, one for each module. so it should be easier now to extend the api for new modules.
     
  3. theWeird

    theWeird Member

    The problem is not ISPConfig. But there is no server module for the xmpp server yet to handle password changes and registrations with external data sources.
     
  4. dharman

    dharman ISPConfig Developer ISPConfig Developer

    I'm trying another solution. I'm using PROSODY:
    - Adding a new field, in table "mail_user", called disablexmpp
    - Enabling XMPP authentication through dovecot
    - Adding a new checkbox to module mail, on mailbox section, to achieve XMPP login
    Can we join our solutions?
     
  5. theWeird

    theWeird Member

    By adding only a field to the mail accounts you cannot control things like MUC or domain admins.
    Also you cannot decouple XMPP accounts from mail accounts. Perhaps some customers want to have seperate accounts for both services.
    Does your solution require to run dovecot and prosody on the same machine? I think this would break the multiserver concept to be able to run different servers on different machines.

    Furthermore, it is not possible to merge back from metronome to prosody but vice versa.
    Metronome has a more efficient storage structure for saving the account/chat data on disk and automatically converts from "old" prosody data structure.
     
  6. theWeird

    theWeird Member

    I placed merge request #170 with release v1.
     
    till likes this.
  7. CubAfull

    CubAfull Member HowtoForge Supporter

    Hello @theWeird
    First I want to thank you for your wonderful work!
    I'm really interested in this new feature for ISPConfig, actually I need to implement a communication system and this is what I been searching for, but after 2 days playing with metronome I have found some problems/bugs with the ISPConfig templates.

    Server environment:
    - Debian Jessie
    - ISPConfig 3.1.1p1
    - Metronome v3.7.9

    1. In metronome_conf_host.master, Anonymous login must be activated using the "use_anon_host" variable. Actually using "use_proxy"
    3. In metronome_conf_host.master, "proxy65_interfaces" and "proxy65_ports" must be defined in metronome_conf_global.master https://prosody.im/doc/modules/mod_proxy65
    3. In metronome_conf_global.master, "bosh_ports" have no effect because this option was removed in prosody v0.9 https://prosody.im/doc/modules/mod_bosh

    I can assume that you make this implementation for an older version of metronome/prosody?
    Are you planning to release/commit the 1.1 version? (I need the Account management method (Normal / By Mail Domain)) feature.

    Regards,
    Michael Epstein
     
  8. theWeird

    theWeird Member

    Hi Michael,
    I'm glad you like it.
    You're right, my implementation is from march '15. It seemed to be a high chance for a new release in the first month of the last year so I finished my commits by then. For 1.1 I wanted to wait until 1.0 is out with the new release to fix bux first before I start 1.1 development.
    There is still another bug I'm working on at the moment:
    Metronome should spawn a process for the auth script to work with, but on systems installed with perfect server setup tutorial it doesn't.
    So at the moment I try to fix the bugs first.
    The timing of this major release is really bad for me, because I start my bachelor theses this week. So it unfortunatly will take some time to fix the bugs. Furthermore I don't think that I will be able to place a merge request for 1.1 this year.
    But I definitely will continue my work as soon as the known bugs and problems are removed.

    Regards,
    Michael
     
  9. ccoudsi

    ccoudsi New Member

    Hi guys,
    I installed several weeks back "The Perfect Server - Ubuntu 16.04" with Metronome, and I spent a lots of time trying to connect to it from XMPP client without any luck, I checked for info everywhere even the xmpp core RFCs no luck, I would really appreciate any help I can get, here's my configuration:
    I setup the DNS for 2 srv records and 1 txt record, also I have setup an XMPP domain & account on my ISPconfig, when I connect from an Xmpp client (Pidgin) to my server I get the following error:
    ====================================
    (12:20:04) jabber: Recv (ssl)(137): <iq id='purple926531ff' type='error'><error type='cancel'><service-unavailable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/></error></iq>
    (12:20:04) connection: Connection error on 04675888 (reason: 0 description: Service Unavailable)
    ========================================

    Best regards,
    Charles
     
  10. CubAfull

    CubAfull Member HowtoForge Supporter

    Hi ccoudsi,
    As Michael say, metronome should spawn a process for the auth but it doesn't.
    I spend 5 days reading, changing config..... and nothing. I think the problem is the compatibility for mod_auth_external and the actual metronome version. I ended uninstalling metronome and installing Prosody 0.10. Now everything is working fine, but I have to configure everything by hand and use another auth method, mail account.

    I really appreciate Michael and ISPConfig team work, but this metronome implementation is unusable.
     
  11. ccoudsi

    ccoudsi New Member

    Hi Michael,
    Thaks for your reply, I thought I was doing something wrong !!
    Can you please share how did you "use another auth method, mail account." ??
    Thanks
     
  12. CubAfull

    CubAfull Member HowtoForge Supporter

    Hi ccoudsi,
    With Prosody, you can use other plugins for authentication. Please check https://hg.prosody.im/prosody-modules/file (all the mod_auth_*)
    Right now Im using mod_auth_imap:
    Code:
    authentication = "imap"
    auth_append_host = true;
    imap_auth_host = "localhost";
    auth_imap_verify_certificate = false;
    imap_auth_port = 993;
     
  13. theWeird

    theWeird Member

    I am still hunting for the bug with the developer of metronome. Hope, we will have a solution soon.
    If you still have metronome installed, please try the following command as root:
    It should spawn the auth script that reads single lines in the form
    Code:
    isuser:username:domain
    auth:username:domain:password
    (replace username, domain and password with real xmpp user credentials)
    It should answer to each command with 0 or 1 and also write to /var/log/metronome/auth.log (without password). If it does, the auth script is working as expected and your bug is the same I am working on at the moment.

    And by the way, mod_auth_imap should also work with metronome as metronome is a prosody fork. Download it to /usr/lib/metronome/isp-modules and edit the generated host configs in /etc/metronome/hosts/ to use it. Afterwards run 'metronomectl restart' to restart with imap authentication while we fix the bug with the isp authentication.
     
  14. CubAfull

    CubAfull Member HowtoForge Supporter

    Hi Michael,
    I made this test some time ago and when I manually execute the .sh script, it works, and I get the 0/1 response, but with metronome I never pass the auth process.

    Now that I'm running Prosody 0.10, I test the ispconfig script (only for testing purpose) with the auth external module and the script is executed by prosody, but prosody don't like the script response and I can't restart prosody anymore with command "service prosody restart" or "prosodyctl restart", I have to KILL the .sh process first.

    Other thing to note (don't know if usefull for you):
    With auth imap, when I execute "netstat -tupln | grep 52" I see for example:
    Code:
    tcp        0      0 0.0.0.0:5280            0.0.0.0:*               LISTEN      32945/lua5.1
    tcp        0      0 0.0.0.0:5281            0.0.0.0:*               LISTEN      32945/lua5.1
    tcp        0      0 0.0.0.0:5222            0.0.0.0:*               LISTEN      32945/lua5.1
    tcp        0      0 0.0.0.0:5269            0.0.0.0:*               LISTEN      32945/lua5.1
    but when I use the auth external and after the first login attemp, I see something like this:
    Code:
    tcp        0      0 0.0.0.0:5280            0.0.0.0:*               LISTEN      32945/bash
    tcp        0      0 0.0.0.0:5281            0.0.0.0:*               LISTEN      32945/bash
    tcp        0      0 0.0.0.0:5222            0.0.0.0:*               LISTEN      32945/bash
    tcp        0      0 0.0.0.0:5269            0.0.0.0:*               LISTEN      32945/bash
     
  15. ccoudsi

    ccoudsi New Member

    Hi Guys,
    I still have Metronome installed and I'm using it with "internal_hashed", I created user accounts and folders manually and it seems working fine, now I'm trying to get "stun service" working for audio & video, no luck.

    Last night I enabled "external" authentication and added 1 xmpp user account (user1: mydomain.com) to ISP config, I tried to run the commands above as root I got (Command not found), from which folder to run the commands??
    /opt/metronome # isuser:username:domain
    /opt/metronome # auth:user1:mydomain.com:Mypassword

    Cheers,
     
  16. CubAfull

    CubAfull Member HowtoForge Supporter

    1- STUN/TURN:
    Code:
    apt-get install rfc5766-turn-server
    Edit /etc/default/rfc5766-turn-server and uncomment:
    Code:
    TURNSERVER_ENABLED=1
    Edit /etc/turnserver.cfg and uncomment/set:
    Code:
    lt-cred-mech
    use-auth-secret
    static-auth-secret     YoUrSecReTAuthStrinG
    realm=turn.your.domain
    cert=/path/to/cert.crt
    pkey=/path/to/key.key
    Restart:
    Code:
    service rfc5766-turn-server restart
    Create DNS A record for turn host and open port 3478 and 5349 TCP/UDP

    2- You must execute the authenticate_isp.sh script and then send the auth params
     
  17. ccoudsi

    ccoudsi New Member

    CubAfull,
    Thank you, I will give it a try.
    I was looking at the sample configuration from metronome, I thought I can just do the following:
    Enable --> "extdisco"; -- External Service Discovery
    Add -->
    Code:
    -- External Service Discovery (mod_extdisco)
    external_services = {
        ["stun.mydomain.com"] = {
            [1] = {
                port = "3478",
                transport = "udp",
                type = "stun"
            },
    
    
            [2] = {
                port = "3478",
                transport = "tcp",
                type = "stun"
            }
        }
    };
    
    Then open port 3478, and create "A" record for stun.Mydomain.com

    Cheers,
     
  18. CubAfull

    CubAfull Member HowtoForge Supporter

    Oh sorry... I forgot to say that to make it work with Prosody I have to use the "turncredentials" module and set:
    Code:
    turncredentials_secret = "YoUrSecReTAuthStrinG";
    turncredentials_host = "turn.your.domain";
    turncredentials_port = "3478";
    turncredentials_ttl = "3600";
    This module set the "External Service Discovery" and allow only registered users to use the STUN/TURN server.
     
  19. ccoudsi

    ccoudsi New Member

    CubAfull,
    Thanks for the info I'll give it a try.

    Michael,
    Here's my test results, I renamed the user & domain:

    Code:
    ==================
    [email protected]:/usr/lib/metronome/isp-modules/mod_auth_external# ./authenticate_isp.sh
    isuser:username:domain
    0
    auth:user1:mydomain.com:mypassword
    1
    ^C
    [email protected]:/usr/lib/metronome/isp-modules/mod_auth_external#
    ========================
    Auth LOG file :
    Date: Tue Nov 15 09:48:29 PST 2016 Action: isuser User: username Host: domain
    ISUSER FAILED
    Date: Tue Nov 15 09:48:58 PST 2016 Action: auth User: user1 Host: mydomain.com
    AUTH OK
    =========================
     
  20. ccoudsi

    ccoudsi New Member

    Hi CubAfull,
    I tried to install TURN server I got the following error message:
    Code:
    [email protected]:/home/ccoudsi# apt-get install rfc5766-turn-server
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    Package rfc5766-turn-server is not available, but is referred to by another package.
    This may mean that the package is missing, has been obsoleted, or
    is only available from another source
    
    E: Package 'rfc5766-turn-server' has no installation candidate
    [email protected]:/home/ccoudsi#
    Cheers,
     

Share This Page