well I ran into the same problems, I have a thread going on somewhere in the foru mwith exactly the same questions: http://www.howtoforge.com/forums/showthread.php?t=4271
Here are some issues I can remember right now that might help you:
if you have a vhost pointing here: /var/www/web1 and your project is called test you copy everything inside test/public to /var/www/web1/web and delete the folder test/public. Then you copy everything from test/ into /var/www/web1 and then you can delete your project test. (I guess it might also be possible to work with symlinks instead of all this copying but I am not sure about that and neither how to do it, although using symlinks you could play around with more projects and keep your directory much more clean)
Thats how I got it working. If you do that you end up with a .htaccess inside /var/www/web1/web which in my case contained "Options +ExecCGI +FollowSymLinks" which you have to remove from the .htaccess file as you should have put it into the directives field of ispcfg anyway. Then you add
SetEnv RAILS_ENV production
Allow from all
at the top of that .htaccess file.
I still had some issues with folder rights, I had to do a chown -R www-data:web1 /var/www/web1/tmp so that the temporary files could be written there.
AND check inside the .htaccess file that dispatcher.fcgi is called not dispatcher.cgi
thats all I can remember right now but I guess it will bring you on the right track.