If I run the following code, it results in a memory leak. I monitor some internal websites, and after a few hours or days,the memory can become outrageous, sometimes running the machine out of the memory! I didn't put in a sleep or delay just so that you can see the problem, but when you run...
I have an array with 5 elements. With each fork, I want the next element to pass to it and then exit. Here's what I've got so far. Could someone please help? :)
Here's what I get:
child 0 k: test1
child 0 k: test2
child 0 k: test3
child 0 k: test4
child 0 k: test5
child 1 k: test1
child 1 k...
It seems to be a problem with LWP, or the way I'm using LWP. If I create a loop where I'm pulling a URL's content into an array, it will continually eat up more and more memory each and every time. I'll post the code in a bit...
I was able to do something like:
$content = parse_html( get $url)->format;
which converts the HTML to text. This works great, but when running in a loop, parse_html crashes my machine because it runs out of memory!
Anybody know how I can fix this, or is there a better way to convert HTML...
Thanks rharsh. I appreciate the response. Let me work with that some.
I wasn't too worried about memory or CPU, the script is running on a quad core 3.2Ghz with 8gb of ram, its just painfully slow. :)
The biggest file I have to work with is about 100k, so much smaller than the example I provided.
What it does:
runs through a file, sorts, and splits it alphabetically into files with words that are equal or less than 200.
In this example I'm using the dict file, and I use it several times, so excuse the redunancy since I didn't want to include the actual source files.
It works, but it...
I'd like to save a single page automatically. The page uses javascript, and therefore I can't seem to use links, elinks, lynx, w3m or curl (I've tried them with spidermonkey but it keeps telling me to enable javascript) to do so properly, it always saves everything except the javascript rendered...
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.