A linux server should always run at least a minimal mail server instance, so i wont disable postfix. What you can do to free up memory is this:
Disable mailscanning with amavisd in postfix, see article in ispconfig faq for detailed steps.
Stop and disable amavisd, clamav, freshclam and dovecot or courier
This should free most of the memory of the mail system and leaves just postfix on.