I see no problems in removing those packages.. all of them are perl modules.
the only one i don't get is "logrotate" so better install it again after you removed it.
i'd use apt-get remove amavisd-new spamassassin clamav clamav-daemon clamav-docs
if you'd rather disable them instead of uninstalling then use:
update-rc.d -f amavis remove
update-rc.d -f spamassassin remove
update-rc.d -f clamav-daemon remove
update-rc.d -f clamav-freshclam remove
this way the init scripts are not run when the server boots ..