Most of VS 6.0 won't install (or run properly) on a post-XP system. So normally today one would only be trying to install VB 6.0, some of the 6.0 Tools, etc. anyway.
People do this all the time. Just leave out the parts of VS you don't need.
Ideally you'd do this before installing a later version of VS though just to be sure to avoid conflicts. They were never designed to be installed out of sequence.
And be aware that when installing these VB parts of VS 6.0 there are a few tricks required to get it right, such as running both phases of setup elevated and clicking through the MS JVM warning.