Curious, have you tried a different s-video cable? Tried hooking it up to a different tv? The s-video outputs differ from a composite out in that the color and black & white components are separated by the video card before being sent to the tv.
So while I've never seen your scenario myself, it's not that hard to imagine this happening, if the tv wasn't receiving the color signal.