The problem here is that ISA/TMG isn't a web server and it's not particularly talented at efficiently serving files. The only affected files will be the login/agreement pages and any items they refer to (images, script, css, etc.)
Some customers have worked around this latency by placing the css, javascript, and images on an anonymously available IIS server so that the only thing ISA has to send is the very small html itself.
If you are experiencing this behavior, the above solution will probably be your best bet. You could also simply remove all images and the css declaration from the html and write an html yourself that only uses hex colors and other simple elements to save on total transfer size.