Application running very slow on public side

I have created a Flexviewer application and I am having problems with it rendering very slowly on the public viewing side. Depending on the computer and Internet Speed, it takes up to 30-60 seconds for the page to fully load. Our ArcGIS server is not public so I set up a virtual directory on our Counties webserver connecting to the application on our ArcGIS server. I am using a proxy page to get to the data out.

Does any else have this problem? Could the proxy be what is slowing this down? I have most of the data in the mapped cached so I don't think that is the problem.