Talking SOAP With Exchange

Want to support HowtoForge? Become a subscriber!
 
Submitted by ErikCederstrand (Contact Author) (Forums) on Mon, 2008-12-08 17:29. :: Email | PHP

Talking SOAP With Exchange

Previously, talking to Exchange without using Microsoft products was pretty much out of the question. The binary MAPI protocol is proprietary and poorly documented. Exchange supports IMAP and POP, but these protocols only give acesss to emails, not the calendar, address book, todo lists etc. But beginning with version 2007, Exchange now ships with a SOAP interface called Exchange Web Services, or EWS. This interface gives us access to the functions necessary to write clients in any programming language on any platform.

This article describes a PHP program to look up, delete and insert items in an Exchange calendar.

Overview

SOAP is an XML-based standard for web services. PHP supports SOAP in a separate module. One part of the SOAP specification is WSDL, an XML-based web service definition language which defines the data types and the functions available. The functions and data types in EWS are actually very well documented on MSDN: http://msdn.microsoft.com/en-us/library/bb204119.aspx. EWS uses the HTTPS protocol for communication, but instead of basic authentication, it uses Microsoft-specific NTLM authentication. PHP doesn't support this protocol with SOAP but, as we shall see, we can work around this.

The script

A normal SOAP communication in PHP goes something like this:

$wsdl = "http://example.com/webservice/definition.wsdl"; $client = new SoapClient($wsdl); $request = 123; $response = $client->MyFunction($request); # Do something with the response

On an Exchange 2007 server, the WSDL file is usually located at https://exchange.example.com/EWS/Services.wsdl. To access this file, we need a username and password for a valid user on the Exhange server. However, since Exchange uses NTLM authentication, we need to make a wrapper for SoapClient. The CURL library (also found as a PHP library) supports NTLM authentication, so we'll use this to make the wrapper:

class NTLMSoapClient extends SoapClient { function __doRequest($request, $location, $action, $version) { $headers = array( 'Method: POST', 'Connection: Keep-Alive', 'User-Agent: PHP-SOAP-CURL', 'Content-Type: text/xml; charset=utf-8', 'SOAPAction: "'.$action.'"', ); $this->__last_request_headers = $headers; $ch = curl_init($location); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_POST, true ); curl_setopt($ch, CURLOPT_POSTFIELDS, $request); curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1); curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_NTLM); curl_setopt($ch, CURLOPT_USERPWD, $this->user.':'.$this->password); $response = curl_exec($ch); return $response; } function __getLastRequestHeaders() { return implode("n", $this->__last_request_headers)."n"; } }

This class overrides the doRequest function of SoapClient to use CURL to fetch the WSDL file. Depending on you PHP installation, you might need to install the PHP CURL module for this to work. Edit: If you experience SoapClient errors, you may need to disable SSL certificate validation. I haven't found the real cause for these errors (it's not just an expired certificate), and obviously it's a security risk to disable validation, but it might what you need to get around the errors. Add these options to the __doRequest() method above:

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);

Edit2: If you get a "looks like we got no XML document" SoapFault, it may be because the server is responding with a non-XML document. In my case, the response was an HTML 401 authentication error page. Printing out the $request and $response objects in the doRequest function above is a big help when debugging. I solved the auth error by deleting the line containing "CURLAUTH_NTLM", so apparently NTLM authentication is not always used. Oh well.

 

We supply the username and password in another wrapper:

class ExchangeNTLMSoapClient extends NTLMSoapClient { protected $user = 'john.doe@example.com'; protected $password = 'secret'; }

Now we can call EWS:

$client = new ExchangeNTLMSoapClient($wsdl);

However, this will fail for two resaons. The first reason is that the WSDL file should contain a soap:address element describing where to find the location of the SOAP web service. The WSDL file served by Exchange does not contain such an element. There are possibly other ways to do this, but one solution is to download the WSDL file and add the following at the end:

<wsdl:service name="ExchangeServices"> <wsdl:port name="ExchangeServicePort" binding="tns:ExchangeServiceBinding"> <soap:address location="https://exchange.example.com/EWS/Exchange.asmx"/> </wsdl:port> </wsdl:service> </wsdl:definitions>

This tells SoapClient where to find the actual web service. This solution requires that two files referenced by the WSDL file, types.xsd and messages.xsd, are also downloaded and placed locally. This is not a problem of you're only contacting one Exchange server, but it's not an elegant solution if you need to contact many servers.

The other reason the call to ExchangeNTLMSoapClient will fail is that the wrapper only adds NTLM support to the initial download of the WSDL file. When SoapClient proceeds to contact the web service, it switches back to basic authentication. To work around this, we create a new stream object which uses CURL:

class NTLMStream { private $path; private $mode; private $options; private $opened_path; private $buffer; private $pos; public function stream_open($path, $mode, $options, $opened_path) { echo "[NTLMStream::stream_open] $path , mode=$mode n"; $this->path = $path; $this->mode = $mode; $this->options = $options; $this->opened_path = $opened_path; $this->createBuffer($path); return true; } public function stream_close() { echo "[NTLMStream::stream_close] n"; curl_close($this->ch); } public function stream_read($count) { echo "[NTLMStream::stream_read] $count n"; if(strlen($this->buffer) == 0) { return false; } $read = substr($this->buffer,$this->pos, $count); $this->pos += $count; return $read; } public function stream_write($data) { echo "[NTLMStream::stream_write] n"; if(strlen($this->buffer) == 0) { return false; } return true; } public function stream_eof() { echo "[NTLMStream::stream_eof] "; if($this->pos > strlen($this->buffer)) { echo "true n"; return true; } echo "false n"; return false; } /* return the position of the current read pointer */ public function stream_tell() { echo "[NTLMStream::stream_tell] n"; return $this->pos; } public function stream_flush() { echo "[NTLMStream::stream_flush] n"; $this->buffer = null; $this->pos = null; } public function stream_stat() { echo "[NTLMStream::stream_stat] n"; $this->createBuffer($this->path); $stat = array( 'size' => strlen($this->buffer), ); return $stat; } public function url_stat($path, $flags) { echo "[NTLMStream::url_stat] n"; $this->createBuffer($path); $stat = array( 'size' => strlen($this->buffer), ); return $stat; } /* Create the buffer by requesting the url through cURL */ private function createBuffer($path) { if($this->buffer) { return; } echo "[NTLMStream::createBuffer] create buffer from : $pathn"; $this->ch = curl_init($path); curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($this->ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1); curl_setopt($this->ch, CURLOPT_HTTPAUTH, CURLAUTH_NTLM); curl_setopt($this->ch, CURLOPT_USERPWD, $this->user.':'.$this->password); echo $this->buffer = curl_exec($this->ch); echo "[NTLMStream::createBuffer] buffer size : ".strlen($this->buffer)."bytesn"; $this->pos = 0; } }

... and a second wrapper over this stream to supply the password for NTLMStream:

class ExchangeNTLMStream extends NTLMStream { protected $user = 'john.doe@example.com'; protected $password = 'secret'; }

Now we need to tell PHP to use this stream instead while calling the web service:

stream_wrapper_unregister('https'); stream_wrapper_register('https', 'ExchangeNTLMStream') or die("Failed to register protocol"); $wsdl = "/usr/local/www/Services.wsdl"; $client = new ExchangeNTLMSoapClient($wsdl); /* Do something with the web service connection */ stream_wrapper_restore('https');

Now we have a working communication with EWS. Let's do something with it:

print_r($client->__getFunctions());

This lists the available functions. Let's use the FindItem function. It fetches all items in a specific folder on the Exchange server. But how do we compose a request? Looking at the list of functions, we se that they define the data types of the argument and the return value. EWS data types are fairly detailed and complex, and there are more than 400 data types. Let's look up what these data types look like:

print_r($client->__getTypes());

This describes the individual data types in a general C-like syntax.

Let's create a request. The MSDN documentation is helpful to determine required fields and their possible values. First, we'll list the folders in the top level of the account:

$FindFolder->Traversal = "Shallow"; $FindFolder->FolderShape->BaseShape = "Default"; $FindFolder->ParentFolderIds->DistinguishedFolderId->Id = "root"; $result = $client->FindFolder($FindFolder); $folders = $result->ResponseMessages->FindFolderResponseMessage->RootFolder->Folders->Folder; foreach($folders as $folder) { echo $folder->DisplayName."n"; }

Now, let's find all items in the calendar:

$FindItem->Traversal = "Shallow"; $FindItem->ItemShape->BaseShape = "AllProperties"; $FindItem->ParentFolderIds->DistinguishedFolderId->Id = "calendar"; $FindItem->CalendarView->StartDate = "2008-12-01T00:00:00Z"; $FindItem->CalendarView->EndDate = "2008-12-31T00:00:00Z"; $result = $client->FindItem($FindItem); $calendaritems = $result->ResponseMessages->FindItemResponseMessage->RootFolder->Items->CalendarItem; foreach($calendaritems as $item) { echo $item->Subject."n"; }

This gets us a list of all John Doe's calendar items for december 2008. Now let's delete all items on this list. For this, we need Id and a ChangeKey for all items:

$ids = array(); $changeKeys = array(); foreach($calendaritems as $item) { $ids[] = $item->ItemId->Id; $changeKeys[] = $item->ItemId->ChangeKey; } if(sizeof($ids) > 0) { $DeleteItem->DeleteType = "HardDelete"; $DeleteItem->SendMeetingCancellations = "SendToNone"; $DeleteItem->ItemIds->ItemId = array(); for($i = 0; $i < sizeof($ids); $i++ ) { $DeleteItem->ItemIds->ItemId[$i]->Id = $ids[$i]; $DeleteItem->ItemIds->ItemId[$i]->ChangeKey = $changeKeys[$i]; } $result = $client->DeleteItem($DeleteItem); print_r($result); }

And finally, let's create a new item in the calendar:

$CreateItem->SendMeetingInvitations = "SendToNone"; $CreateItem->SavedItemFolderId->DistinguishedFolderId->Id = "calendar"; $CreateItem->Items->CalendarItem = array(); for($i = 0; $i < 1; $i++) { $CreateItem->Items->CalendarItem[$i]->Subject = "Hello from PHP"; $CreateItem->Items->CalendarItem[$i]->Start = "2010-01-01T16:00:00Z"; # ISO date format. Z denotes UTC time $CreateItem->Items->CalendarItem[$i]->End = "2010-01-01T17:00:00Z"; $CreateItem->Items->CalendarItem[$i]->IsAllDayEvent = false; $CreateItem->Items->CalendarItem[$i]->LegacyFreeBusyStatus = "Busy"; $CreateItem->Items->CalendarItem[$i]->Location = "Bahamas"; $CreateItem->Items->CalendarItem[$i]->Categories->String = "MyCategory"; } $result = $client->CreateItem($CreateItem); print_r($result);

There are many other functions available and many other attributes for the objects I have used in this tutorial.

Advanced

If you need to extend the classes defined in the WSDL with e.g. a function, it is possible to do this using the NTLMSoapClient class. Add a constructor to the class which registers the WSDL classes as PHP classes:

function __construct($wsdl, $options = null) { $client = new NTLMSoapClient($wsdl, $options); $types = array(); foreach($client->__getTypes() as $type) { # Match the type information using a regular expession preg_match("/([a-z0-9_]+)s+([a-z0-9_]+([])?)(.*)?/si", $type, $matches); $qualifier = $matches[1]; $name = $matches[2]; if($qualifier == "struct") { # Store the data type information in an array for later use in the classmap $types[$name] = $name; # Check that the class does not exsit before creating it. We only need to create empty classes. if (! class_exists($name)) { eval("class $name {}"); } else { echo "[ExchangeNTLMSoapClient::__construct] Class $name already exists.n"; } } } # Add the classmap to the options array and call the parent constructor if(is_null($options)) { $options = array(); } $options['classmap'] = $types; parent::__construct($wsdl, $options); }

This loads empty class definitions for classes not already defined in the PHP script. Now it's possible to define a class that overrides the one automatically loaded:

class EmailAddressDictionaryEntryType { function validate() { # Lame email validator return stristr("@", $this->Value); } }

Finally

That's all. There's still a long way from this sample script to an Outlook replacement, but this can be very useful for e.g. integration purposes and data migration.

Thanks to Thomas Rabaix for his article on NTLM authentication in SOAP and PHP: http://rabaix.net/en/articles/2008/03/13/using-soap-php-with-ntlm-authentication. Thanks to Adam Delves for his article on WSDL and PHP: http://www.phpbuilder.com/columns/adam_delves20060606.php3.


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by Anonymous (not registered) on Mon, 2011-06-06 08:02.

From what I can see, this article seems to suggest that the SoapClient calls the __doRequest() method when requesting the wsdl.

 This is not what I'm experiencing with php 5.3.5.

Can anyone confirm / deny? Should this be in the forum instead?

 eg.

class TestSoap extends SoapClient
{

  function __doRequest($request, $location, $action, $version, $oneWay)
  {
// DOES NOT GO THROUGH HERE ON OBJECT INSTANTIATION
    die('HERE');
    return parent::__doRequest($request, $location, $action, $version, $oneWay);
  }

}
$wsdl = 'http://host/service?wsdl';
$options = aray( /* */ );
$client = new TestSoap($wsdl, $options);
Submitted by Jeroen Aarts (not registered) on Fri, 2010-09-10 20:51.

Great Article! Thanks for sharing the code.

 It works great on Windows machines (Win XP/Server, tested with different versions of PHP 5.2.8+).

However, on Mac machines (Mac OS 10.6/10.6 Server), using different versions of PHP (5.2.11/5.3.1), curl_exec() in the overridden __doRequest() method only returns an empty string. Has anyone any clue, or can anyone share experience on non-Windows machines, as I am executing exactly the same code on all machines?

 - Jeroen

Submitted by Andi (not registered) on Fri, 2010-07-23 09:50.

...for the information on NTLM. I use NuSOAP and simply pass the ntlm information on:

$client = new nusoap_client(ENDPOINT, false);
$client->setCurlOption(CURLOPT_PROXY, "");
$client->setCurlOption(CURLOPT_HTTPAUTH, CURLAUTH_NTLM);
$client->setCurlOption(CURLOPT_USERPWD, 'user:pass');
etcetc.

Btw. I had to unset the proxy because we had some local environment variables set (http_proxy and https_proxy).

Thank you again :-)

Andi

 

Submitted by zehhaf (not registered) on Tue, 2010-07-06 11:04.

Hi

when you delete un item from outlook, it goes to deleteditems folder but it doesn't keep the same Id before deleting.

 I have a real problem with my synchronization

 ex.

before (in contact folder)

AAAmAHRlc3RAY2FyYS1tZWwuZW1lYS5taWNyb3NvZnRvbmxpbmUuY29tAEYAAAAAAEo5vPb1H6pPkUGStDYeaCYHADRCVMaUGCxKlgdjf/vdoLwAHo47I/sAADRCVMaUGCxKlgdjf/vdoLwAHo5qWbwAAA==

after (the same item in deleteditems folder)

AAAmAHRlc3RAY2FyYS1tZWwuZW1lYS5taWNyb3NvZnRvbmxpbmUuY29tAEYAAAAAAEo5vPb1H6pPkUGStDYeaCYHADRCVMaUGCxKlgdjf/vdoLwAHo472hIAADRCVMaUGCxKlgdjf/vdoLwAHo5qY+UAAA==

 please help

Submitted by Taras Kozlov (not registered) on Wed, 2010-06-09 15:51.

Erik,

I can't find a words to show my gratitude to you. It's the best article for the subject in the whole net. It's simply amazing tutorial.

Thank you so much!

Submitted by Baz (not registered) on Wed, 2010-03-17 16:04.
Is there anyway to change distribution lists also known as mailing lists through soap?
Submitted by R. Veenbrink (not registered) on Sun, 2010-02-14 01:21.

Hello all,

for the ones where the script is not working. Check if you're running PHP version 5.2.6.
I was running on PHP 5.1.x and this was the problem why the script was not working.

What i found it has to do something with the cUrl module en a bug in PHP.

Hope you enjoy my comment.

Submitted by jyoti (not registered) on Fri, 2009-09-11 13:27.

Hi

I  implemented this  fc8 and php5.2.6 its giving error. I know its login issue i tried all the possibles .it didn't work . please help me in this 

<output>:

PHP Fatal error:  Uncaught SoapFault exception: [Client] SoapSlient::__doRequest() returned non string value in /home/jyoti/contacts/php/calender.php:179
Stack trace:
#0 [internal function]: SoapClient->__call('FindFolder', Array)
#1 /home/jyoti/contacts/php/calender.php(179): ExchangeNTLMSoapClient->FindFolder(Object(stdClass))
#2 {main}
  thrown in /home/jyoti/contacts/php/calender.php on line 179
 </output>

 


 

<code>

class NTLMSoapClient extends SoapClient {
    function __doRequest($request, $location, $action, $version) {
        $headers = array( 'Method: POST', 'Connection: Keep-Alive', 'Content-Type: text/xml; charset=utf-8', 'SOAPAction: "'.$action.'"', );
        print $request;
        $this->__last_request_headers = $headers;
        $ch = curl_init($location);
        curl_setopt($ch, CURLOPT_USERAGENT, "PHP-SOAP-CURL");
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
        curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
        curl_setopt($ch, CURLOPT_POST, true );
        curl_setopt($ch, CURLOPT_POSTFIELDS, $request);
        curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
        curl_setopt($ch, CURLOPT_SSLKEY, "/tmp/test.txt");
        curl_setopt($ch, CURLOPT_USERPWD, $this->user.':'.$this->password);
        curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
        curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
        $response = curl_exec($ch);
        print_r($response);
        return $response;
    }
    function __getLastRequestHeaders() {
       print implode("n", $this->__last_request_headers)."n";
        return implode("n", $this->__last_request_headers)."n";
    }
}

 <code>

 thanks

jyoti

Submitted by ludo42 (not registered) on Wed, 2010-05-26 12:44.

I have the same problem...

Do you have a solution ?

Submitted by Anonymous (not registered) on Thu, 2010-07-15 05:36.

 Make sure you use your actual wdsl address location, not the example 

<wsdl:service name="ExchangeServices"> <wsdl:port name="ExchangeServicePort" binding="tns:ExchangeServiceBinding"> <soap:address location="https://exchange.example.com/EWS/Exchange.asmx"/> </wsdl:port> </wsdl:service> </wsdl:definitions>

Submitted by Ryan (not registered) on Thu, 2009-08-20 01:49.
Great article.  An example or two on accessing Public Folders with this method would be fabulous. I would recommend an example on enumerating the contents of a public folder or creating new contacts in a public folder.  I have a need to take email/address information from a 3rd party app and dump it into a public folder on a nightly basis so its available to employees through Outlook.
Submitted by Strawp (not registered) on Wed, 2009-05-13 15:19.

Hi, This is great stuff - I used Thomas Rabaix's code to create a data abstraction class for SharePoint but now I'm going to pick through this and create a sync script for Exchange and Google Calendar. Anyway, it seems that some of the formatting is a bit broken on this post (e.g. newlines). A zip of all the code would be nice so I don't have to copy-paste and then pick through fixing bits. Cheers, Strawp

Submitted by Brian (not registered) on Tue, 2009-04-28 22:22.

Any clues on how to get to the message count of the Inbox, as well as the count of unread messages in there?  I'm having a hard time getting my mind around the MSN docs and not making much progress.

 Changing the $FindFolder->ParentFolderIds->DistinguishedFolderId->Id to 'Inbox' throws an error, and changing it to 'inbox' returns a ResponseClass : Success, as if it found the Inbox, but TotalItemsInView => 0 (which I know isn't true for my Inbox).

Submitted by Brian (not registered) on Wed, 2009-04-29 21:39.

Yes, I'm replying to myself...

Turns out that 'inbox' (or 'Inbox') isn't one of the blessed folders that can be called by name, you have to dig for the Id and ChangeKey for 'Top of Information Store' and take a look in there to find the folder with a display name of 'Inbox'.

 <code>
$FindFolder->Traversal = 'Shallow';
$FindFolder->FolderShape->BaseShape = 'AllProperties';
$FindFolder->ParentFolderIds->DistinguishedFolderId->Id = 'root';
$result = $client->FindFolder($FindFolder);
$folders = $result->ResponseMessages->FindFolderResponseMessage->RootFolder->Folders->Folder;
foreach ($folders as $folder)
{
    if ('Top of Information Store' == $folder->DisplayName)
    {
        $tois_folder = $folder;
    }
}
$FindFolder = null;
$FindFolder->Traversal = 'Shallow';
$FindFolder->FolderShape->BaseShape = 'AllProperties';
$FindFolder->ParentFolderIds->FolderId->Id = $tois_folder->FolderId->Id;
$FindFolder->ParentFolderIds->FolderId->ChangeKey = $tois_folder->FolderId->ChangeKey;
$result = $client->FindFolder($FindFolder);
$folders = $result->ResponseMessages->FindFolderResponseMessage->RootFolder->Folders->Folder;
foreach ($folders as $folder)
{
     if ('Inbox' == $folder->DisplayName)
     {
          // $folder is now the Inbox
     }
}
</code>

Submitted by Erik (not registered) on Mon, 2009-03-09 11:19.

Hi,

This solution is really the best I've ever seen!!! I'm already searching for 14 hours or so for a solution, saw this page several times but thought I didn't need it, because NTLM Authentication was changed to Basic Authentication...

My Exchange Webservices still didn't work and the __getFunctions didn't return result and now it does!

Thanks for this great article!

Erik.

p.s. a tip, make the pre boxes wider for better readability

Submitted by ErikCederstrand (registered user) on Mon, 2009-03-16 21:22.

Thanks for the kind words. When I wrote the howto, the default CSS mangled my attempts at making readable code excerpts. I found a way around this now, as you can see.

Submitted by PMC (not registered) on Mon, 2009-06-08 09:41.

I agree that this HowTo is extremely useful for the most part, unfortunately, calls to the Exchange server that contain the updateItem function do not work and instead return a path error - (see http://bugs.php.net/bug.php?id=47924&thanks=6)

Any ideas how to get around this?

 

Thanks

Submitted by Heratech (not registered) on Thu, 2010-03-25 13:40.

Hello,

 I had the UpdateItem bug to and apparently using the types.xsd from 

http://code.google.com/p/php-ews/source/browse/trunk/wsdl/types.xsd

 

fixes it- Just testing it now, so far so good.

Regards,

Heratech