This discussion has appeared in the VB6 forum more than once in the last few months.
The #1 problem was too many controls on a form. Microsoft claims the maximum number of controls that can be placed on a VB6 form is 255. If you are certain you haven't exceeded this number, never mind. Otherwise, consider the alternatives: eliminate all unnecessary controls, move some controls to another form or place similar controls in a control array (You can use many more than 255 controls on a form when they are in an array. Due to the extra programming, leave this as a last resort.)
#2 Problem: inadvertent recursion. For example, you have code that modifies the contents of a text box in the text box change event. It won't take long to hog 99% of system resources and run out of memory (if you are low on disk space) on a Win9x system. There are countless ways that this bug can appear in an application and none of them are easy to track down and kill. Check the events on all forms and controls. Make sure that you haven't created a "feed-back loop".
#3 Problem: corrupted form. If you are using VB 5 or 6 your forms will be saved as ASCII text. If you can't find an easy way to view the forms outside the VB IDE, use the MSDOS editor. Shell to DOS, enter EDIT and open the suspect *.FRM file. You shouldn't see any extended ASCII characters in the file. If you do, depending on the context, you may be able to delete them and use Mike's suggestion to recreate the form.
My suggestion would be to go through each form until you find the problem. Create a new project. Place a button on a form and add code to show a form. Add your modules, if any. Then add the first form to the project, changing the code in the button to show the form according to its name. Run the project and click the button. Make note of every error, especially errors that reference subs in other forms. Comment-out each error line and restart the project.
Repeat until there are no error. Remove the form from the project (make sure you don't save the changes!) and then add the next form and repeat the process.
Debugging the "Out of Memory" error can be very time-consuming. In the end, you will find it quite gratifying.