No, not without DNS. It's not the way that ISPConfig was designed. Since ISPConfig was meant to handle multiple websites it uses virtual sites in apache. When doing this the one ip address handles multiple sites, and hence why you cannot access it just by ip address.
I suggest maybe you uninstall ISPConfig and simply use apache so that you can browse to the ip address and have it go directly to the site.
I apologize I'm just offering work-arounds but it's simply not how ISPConfig was designed. You need DNS.