I basically ruled out that its my ISP blocking ports because I have a VM image running Ubuntu 6.10 and ISPCONFIG 2.2.29 (Used Perfect Setup) and I have no problems telnetting in externally when I have it up and running.
I can't seem to figure it out though...