Keep in mind that on Solaris the directories [tt]/var/run[/tt] and [tt]/tmp[/tt] use swap, so you can look for some very large files in these directories. A big file in [tt]/tmp[/tt] could be your culpret.
If that's not the problem, you can use the command "[tt]prstat -s size[/tt]" to list processes sorted by memory used, or "[tt]prstat -s rss[/tt]" to list the processes by resident memory size.
Also, when you have a suspect process, the command [tt]pmap[/tt] can be used to get details of a processes memory usuage.
Hepe this helps.