Don't know how much you've done or not done, and it's been ages since I set up ICS in windows (And that was on 2k server) However, on the client end it should be simply the matter of setting the sharing box as your default gateway, and plugging your DNS server IP addresses in as appropriate.
Although I will say that when I went to DSL, I had to adjust the MTU on my client machines to 1450 instead of the default 1500. at 1500 I could access some sites, but not all. it was spotty as to what I could or could not access. I've never had any problem at 1450.
Poet and seeker of knowledge