Memory leak on unix but not windows

[cjwuea]

Hi,

I have an app (C++) that runs on windows fine, and all memory appears to be released correctly (by observing the mem usage in Windows Task Manager).

When I run the same app on Unix / Linux, the memory does not release properly, and eventually goes on to the disk.

Is this a problem people have seen before? Are there any "techniques" to try and locate this problem?

Thanks in advance.

 

[Salem]

So how does the same code run on both platforms?

Is it a console program constrained to just those interfaces required by ISO?

Is it a GUI program using a portable toolkit such as Wxwidgets to achieve the portability?

Is the portability achieved by having a completely separate home-grown portability layer, using say MFC on windows, and Qt on Linux?

If there is some portability layer in there, then the problem could well be within that layer rather than in any code you've written.


--
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.

 

[cjwuea]

Sorry, I was a bit vague:

It's just a standard "console" application i.e. there's no gui stuff.

I recompile on unix using g++. I use mainly stl, but I do use the string class from boost.

The application is a speech recogniser - it works by passing thousands of "tokens" around a network - I'm constantly creating new tokens and deleting them - as far as I can see, all tokens are deleted at the end of the iteration - which is supported by Windows Task Manager.

Thanks.

 

[xwb]

I've seen it on copy constructors but that was 10 years ago. It was call by value not tidying up properly. When we changed from

void xxx (string yyy)
{
   ...
}

to

void xxx (const string& yyy)
{
   ...
}

it magically went away. That was on the Solaris 2.6 compiler. Don't know if it is the same with gcc.

 

[cpjust]

Do you have any platform specific code, like:

#ifdef WIN32
  ...
#else // UNIX
  ...
#endif

Do you know exactly where the memory leak is happening?

 

[cjwuea]

As far as I know there are no platform specific pieces of code.

I do use inheritance on those "Token" objects that I was talking about - I use virtual destructors - in this program I actually only create derived objects, and thus delete the derived objects.

 

[jstreich]

Are you using new and delete or malloc and free? Are you mixing new and free or malloc and delete? What are your compilation flags?

[plug=shameless]

http://game-master.us/phpx-3.4.0/

[/plug]

 

[cjwuea]

I'm just using new and delete (where possible I always use the c++ functions / library) - I compile with -DNDEBUG and -O3, but I've tried changing the flags with no success.

 

[cjwuea]

Hi again guys,

I have some more info - I decided to use boost's shared_ptr on my token objects, but still a memory leak (on unix) - here's the output from the memory tool with Valgrind:

14,659,544 bytes in 17 blocks are still reachable in loss record 19 of 19
==27212==    at 0x4A19257: operator new(unsigned long) (vg_replace_malloc.c:167)
==27212==    by 0x4BAA4CD: std::__default_alloc_template<true, 0>::_S_chunk_alloc(unsigned long, int&) (in /usr/lib64/libstdc++.so.5.0.5)
==27212==    by 0x4BAA5A9: std::__default_alloc_template<true, 0>::_S_refill(unsigned long) (in /usr/lib64/libstdc++.so.5.0.5)
==27212==    by 0x4BAA90D: std::__default_alloc_template<true, 0>::allocate(unsigned long) (in /usr/lib64/libstdc++.so.5.0.5)
==27212==    by 0x4186C3: std::__simple_alloc<int, std::__default_alloc_template<true, 0> >::allocate(unsigned long) (stl_alloc.h:232)
==27212==    by 0x418694: std::_Vector_alloc_base<int, std::allocator<int>, true>::_M_allocate(unsigned long) (stl_vector.h:127)
==27212==    by 0x41797E: std::vector<int, std::allocator<int> >::_M_insert_aux(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, int const&) (vector.tcc:236)
==27212==    by 0x4169AD: std::vector<int, std::allocator<int> >::push_back(int const&) (stl_vector.h:603)
==27212==    by 0x44BAAE: VNSA_Token::Update_Token(double, VNSA_State*, int, bool) (VNSA_Token.cpp:132)
==27212==    by 0x42BFBC: Token_Passing_VNSA::Pass_Token_DTW(boost::shared_ptr<VNSA_Token>, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, int, double, double) (Token_Passing_VNSA.cpp:169)
==27212==    by 0x42C12A: Token_Passing_VNSA::Pass_Token_DTW(boost::shared_ptr<VNSA_Token>, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, int, double, double) (Token_Passing_VNSA.cpp:186)
==27212==    by 0x42BDB2: Token_Passing_VNSA::Pass_Token_DTW(boost::shared_ptr<VNSA_Token>, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, int, double, double) (Token_Passing_VNSA.cpp:142)
==27212==    by 0x42C3D4: Token_Passing_VNSA::Pass_Token_DTW(boost::shared_ptr<VNSA_Token>, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, int, double, double) (Token_Passing_VNSA.cpp:213)
==27212==    by 0x42B8AD: Token_Passing_VNSA::Decode(VNSA*, std::vector<Template*, std::allocator<Template*> >&, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, std::vector<std::string, std::allocator<std::string> >*, int, double, double) (Token_Passing_VNSA.cpp:70)
==27212==    by 0x422DD6: Template_Recogniser::Match_Templates(std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >&, std::vector<std::string, std::allocator<std::string> >*, int, double, double) (Template_Recogniser.cpp:343)
==27212==    by 0x406360: main (Main.cpp:331)

which basically refers to this function:

void VNSA_Token::Update_Token(double prob, VNSA_State *new_state, int move, bool update_history)
{
	if(this->cur_state != NULL && update_history)
	{
		if(this->cur_state->is_lg_layer)
		{
			int word_id = ((VNSA_State *)this->cur_state)->Get_Accept_Symbol();
			this->word_history.push_back(word_id);  //line 132
		}
	}
	this->partial_prob += prob;
	this->cur_state = new_state;
	this->last_move = move;
}

VNSA_Token are the token objects that I create - these are allocated with the smart_ptr.

word history is simply a vector<int> stored privately in the VNSA_Token class.

This leaks on unix, but not windows...the leak is "small" here, but that's because I'm using a smaller dataset to test - it's also just one iteration of thousands (so imagine the problem I have with swapping later...)

I can give more information if necessary.

Thanks for your time.

 

[jstreich]

Did you compile Boost on your own or did you get a precompiled version? Maybe the leak is in Boost (while I like the Boost guys, and their code is generally sound... maybe they slipped a bit)?

[plug=shameless]

http://game-master.us/phpx-3.4.0/

[/plug]

 

[cjwuea]

The Boost classes I use only require the headers - so no precompiling - I'm using the latest version of boost - 1.34.1.

 

[Salem]

If you've narrowed it down to that line, it should be possible to construct a small test program (say <50 lines) which tests your hypothesis.

If that test program leaks just as bad on Linux, but doesn't leak at all on windows, then you have a really good example to submit as a bug to the Boost people.

If you do replicate the problem, you also have a nice example with which to test possible workarounds.


--
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.