I have used a Perfect Server How To on this site to run a web, dns, ftp, and email server using Ubuntu 8.04 LTS without using ISPConfig. The Perfect Server setup really just got me a base server with web.
I then used another How To to setup an email server using a MySQL backend with virtual users.
I then used another How To to setup the FTP server again using a MySQL backend with virtual users.
For the DNS server, I turned to a How To not on this site.
Before running 8.04, I had done the same with 7.04 and 7.10. So it can be done, but it took a lot of trial and error and a lot of research. In my scenario, it has been working well for nearly two years. I am currently building a new machine based on one of the Perfect Server How To's and will be installing ISPConfig3.
Although I have a firm understanding of the command line now from not using a control panel, the setup and management of domains and users on the machine takes much more time than it should and I could benefit greatly from ISPConfig3.
If you would like me to list the How To's I used, let me know.