DevX Home    Today's Headlines   Articles Archive   Tip Bank   Forums   

Page 2 of 3 FirstFirst 123 LastLast
Results 16 to 30 of 33

Thread: Event hook causing memory leak

  1. #16
    Join Date
    Jul 2010
    Posts
    30
    The compiler didnt issue any warning or error about the delete statements following the return statement. so, how are you supposed to properly dispose of a variable if it is needed all the way to the return statement? I mean if you had 2 temp variables (tempvar1 & tempvar2) and had

    return tempvar1 + tempvar2;

    how do you delete them after they are not needed(which would be after the return statement)?

  2. #17
    Join Date
    Jul 2010
    Posts
    30
    I didnt even think of this till just now, in addition to my last post, why does it seem to work correctly without the events? the delete statements are in the same place in both versions of the class (in fact I put a bool variable in the base class to enable/disable the events so I wouldnt have to keep changing the code), but with events enabled it eats memory?

  3. #18
    Join Date
    Dec 2003
    Posts
    3,366
    if you return a pointer out of a function, the user must delete it. This is sort of old school; this sort of thing is often considered a bad design in modern code (either use a smart pointer that self destructs or a different design). Putting the burden onto the user to delete memory causes problems because at some point a user WILL fail to delete it. The answer then, is, you *don't* do things after the return statement, you change the code so that everything that needs to happen is done before the return statement!

    There are a variety of ways around the problem. Pass into your function the destination pointer, and fill it in is one approach. Smart pointers work, as mentioned. If the objects are not too huge, you can just create them directly (no pointers at all). If the objects have internal pointers, the dtor should clear them out safely (this happens after the return statement, but no code is written).

    -------------
    It works without the events because the events are the problem, most likely? That question cannot be answered without answering your full question: where is the memory leak... which we do not know. If it leaks when you turn on the events, then something is wrong in the events, but more I cannot say.

  4. #19
    Join Date
    Jul 2010
    Posts
    30
    I apprciate yalls help on this, I think I am going to try to rwrite the class using native c++, Ill use it in its current state till I finish that. Im not going to give up trying to find the problem with it now, but I am wandering into areas that Ive not been to before and using tools Ive never used before, so..... its off to reading ( probably for quite a few hours) and even though youre not used to managed code, do you think you might be able to point me in a good direction as to which tool might be best suited for tracking the problem with the events down? Ive allready found good material (I think) on native c++ but that direction is putting me right back at square one with learning this language so is going to take some time to port it over, besides, I cant leave the problem unanswered lol. I really do appreciate all the help youve givin, thanks...

  5. #20
    Join Date
    Nov 2003
    Posts
    4,118
    Quote Originally Posted by AKRichard View Post
    The compiler didnt issue any warning or error about the delete statements following the return statement. so, how are you supposed to properly dispose of a variable if it is needed all the way to the return statement? I mean if you had 2 temp variables (tempvar1 & tempvar2) and had

    return tempvar1 + tempvar2;

    how do you delete them after they are not needed(which would be after the return statement)?
    You can store the result in a temporary object, delete the two pointers and return the temp by value. If you can't use value objects, try to replace the pointer with a smart pointer or have the caller delete that pointer (the latter is the worst option). You can also pass an extra argument to the function: a pointer to an object into which the result will be written. This way, the callee wouldn't have to bother deleting that pointer.
    Danny Kalev

  6. #21
    Join Date
    Dec 2003
    Posts
    3,366
    Quote Originally Posted by AKRichard View Post
    I apprciate yalls help on this, I think I am going to try to rwrite the class using native c++, Ill use it in its current state till I finish that. Im not going to give up trying to find the problem with it now, but I am wandering into areas that Ive not been to before and using tools Ive never used before, so..... its off to reading ( probably for quite a few hours) and even though youre not used to managed code, do you think you might be able to point me in a good direction as to which tool might be best suited for tracking the problem with the events down? Ive allready found good material (I think) on native c++ but that direction is putting me right back at square one with learning this language so is going to take some time to port it over, besides, I cant leave the problem unanswered lol. I really do appreciate all the help youve givin, thanks...
    Managed or not, its all code. My single complaint about managed code is simply that it does not accept some legal c++ code. That makes it invalid --- microsoft is telling me what I can and cannot do with the language. If I wanted that, I would use java or some other very limited languaged. I use C++ because it has no limitations imposed on the user.

    Finding a memory leak, all I can say is keep reducing it. Find a function with a leak, and write a stub program that does nothing except call this function. Then stub around until you stumble onto another function (Called by the first one) that contains the leak. Repeat until you find the deepest level function that is not doing what is expected. This may not be possible in a very complex design, though. If you cannot actually trim out the unimportant code, try to find a way (in the debugger) to ignore it.

    Somewhere in the event stuff is a routine that allocates memory. Somewhere else, most likely, this is called frequently, but the memory is not destroyed. Finding it is going to take time and effort, but you know a LOT already. You know its in the event section of your code. You know its called a lot, as you run the memory out rather fast. You just have to muddle thru the event code looking for something that is called every iteration of the program, or nearly so. Maybe the outermost event handling area, where things are polled or whatever? Or something called by that section?

  7. #22
    Join Date
    Jul 2010
    Posts
    30
    Quote Originally Posted by Danny View Post
    You can store the result in a temporary object, delete the two pointers and return the temp by value. If you can't use value objects, try to replace the pointer with a smart pointer or have the caller delete that pointer (the latter is the worst option). You can also pass an extra argument to the function: a pointer to an object into which the result will be written. This way, the callee wouldn't have to bother deleting that pointer.
    I looked into the smart pointer, it looks like what the gc is supposed to do in managed c++ and not sure they would solve my problem since (I believe) that the problem with the event hooks is that the delegate has a reference to that instance which is not being released, and if thats the case the refernce count in smart pointers would never reach zero. I was wondering though if returning by value instead of by ref would solve it, but am worried about the impact on the speed of the class (especially in multi threaded apps) as Ive read that by value is a lot more costly in a class with a lot of interal variables. I hadnt even thought about passing the function a ref to a variable to be used as the return variable.

    Quote Originally Posted by jonnin View Post
    Somewhere in the event stuff is a routine that allocates memory. Somewhere else, most likely, this is called frequently, but the memory is not destroyed. Finding it is going to take time and effort, but you know a LOT already. You know its in the event section of your code. You know its called a lot, as you run the memory out rather fast. You just have to muddle thru the event code looking for something that is called every iteration of the program, or nearly so. Maybe the outermost event handling area, where things are polled or whatever? Or something called by that section?
    This is where I have hit a brick wall. I know that with events turned off there is not a problem, with them turned on there is. The only section of code called with events enabled that does not get called when they are enabled, is the section of code that hooks the instance into the delegates (and the section that is supposed to remove the hooks). The events are not even being fired right now since all the events are there for is to notify EVERY instance when a change has been made to the base class (which just holds how the class as a whole is supposed to act), and in my testing program I am just using the default settings. I have debugged the event handling routines and events themselves, I know that they will fire when they are supposed to and that the event handlers are operating as they should, but as I said, none of this is being hit in my test program because I am not making changes to the base programaticcally,not even to enable the events, when I want to try it with events enabled, I change a bool value in the base class then recompile it.

    I am just not sure how to proceed debugging this, I have been reading quite a bit about some tools Ive never used before (like the clr profiler) but am not sure how to use these tools to solve the problem. I know the problem has to be in my code somewhere, but have been unable to find the offending section (or I have failed to recognize it because of my limited understanding of c++)

    I do have another question though, what is the difference between the & and the ^ symbols, from what Ive read they both mark the variable as a pointer and the * will dereference both, but I have a hard time using the & symbol in my class, everything is marked with the ^(tophat?) symbol. I assume there must be a difference between the two but I can find very little information on the ^ symbol and how it is related to everything else in c++.

  8. #23
    Join Date
    Nov 2003
    Posts
    4,118
    A smart pointer will solve the problem because its destructor releases the memory, and that destructor is called at a predictable time. Besides, smart pointers such as auto_ptr don't use reference counting at all.
    With respect to value objects: the myth about their performance overhead is rarely true. Actually, the overhead of gcnew is probably much higher, but the question is what the size of your object is and how complex is its initialization.
    Danny Kalev

  9. #24
    Join Date
    Dec 2003
    Posts
    3,366
    Quote Originally Posted by AKRichard View Post

    I do have another question though, what is the difference between the & and the ^ symbols, from what Ive read they both mark the variable as a pointer and the * will dereference both, but I have a hard time using the & symbol in my class, everything is marked with the ^(tophat?) symbol. I assume there must be a difference between the two but I can find very little information on the ^ symbol and how it is related to everything else in c++.

    & is address of. It returns a pointer to the argument.

    ^ is the logical operator xor. I am unaware of it being anything else. However, remember that C++ allows overloading of operators, so you may have, in your code, a class that overloads the ^ operator to do something else, for example I could write a number class and overload the ^ operator to perform the POW function. (this goes with any operator, for a specific class they can be changed, for example << is binary shift but it is overloaded for cin and cout to do something else!)

    * is pointer dereference, and is approximately the same as x[y] or x->data. There are slight differences, but 99% of the time, these are the same:
    *x
    x[0]
    and if x is a class, struct, etc, then:
    *x.data
    x[0].data
    x->data
    are the same.

    -------------------------
    Debugging tools can be great, but in the end, its a skill you have to learn. Print statements are all you really need to debug a module. Larger programs that hide a bug in the interaction of modules are tricky, but for memory leaks and errors, logic errors, etc isolation of a module into a stubbed test program with lots of print statements is all it takes to debug it. Debugging tools have a common problem: they were designed to find bugs. This creates a problem where the tools will point at any "strange looking" or "commonly bad" code and claim a bug when none exists, wasting a ton of your time as you verify the stuff it points at, looking for a bug that is not there. Other times, it points out something that really needs to be fixed, and in those cases the software pays for itself.

    ------------------------
    You need to grow your test program to exercise the problem areas then. If the code that could be creating a problem is not tested in the test program, the test program is not helpful and needs to do more stuff, or different stuff. Not sure how to tell you to do that, but you must call the offending code if you are ever going to find it.

  10. #25
    Join Date
    Jul 2010
    Posts
    30
    Thanks for the tip on the auto_ptr, I hadnt come across it yet and sounds like it may take care of some of the problems (and be easy to implement yay).

    As for the ^ operator, When I first started this project I kept running into situations at compile time that would say that a variable required a top level ^ (this usually came up in variable declarations and function defnitions), after messing around I did overload the ^ operator, but that wasnt until recently.

    Something I found on msdn when I first came across this said"Quote>What does ^ do and why must I use it to delcare a String?



    It's part of the C++/CLI language. From the spec:



    handle - A handle is called an "object reference" in the CLI specification.
    For any CLI class type T, the declaration T^ h declares a handle h to type T,
    where the object to which h is capable of pointing resides on the CLI heap.
    A handle tracks, is rebindable, and can point to a whole object only.
    (See also type, reference, tracking.)



    type, value class, boxed - A boxed value class is an instance of a value class
    on the CLI heap. For a value class V, a boxed value class is always of the form V^.



    unboxing - An explicit conversion from type System:: Object^ to any value class type,
    from type System::ValueType^ to any value class type, from V^ (the boxed form of a value
    class type) to V (the value class type), or from any interface class type handle to any value
    class type that implements that interface class. (See also "boxing".)

    "

    and I could find very little else on the subject, I took this as a hint that it is a form of pointer, the * operator appears to dereference the handle, and I have had a hard time getting the & form of pointer to work correctly, which I thought was because I am not completely undestanding pointers. Unfortunately, the class make exclusive use of the ^ operator where, I would have thought at least, the & should be used. Even though the class works very well (without the events), I am more concerned with writing correct code and actually utilizing the language.

    By the way, I am finally zeroing in on the problem, I found where the leak was and plugged it, but now as the test program is running instead of eating memory I am getting spurts of page faults. Running the test app without events, it runs steadily at about 24k mem and very very few page faults, with events it is now running steadily at about 27k mem but I start getting spurts of page faults causing the app to look jittery (I had noticed the page faults before but had assumed they were because of the memory leak), I know a page fault is when the cpu needs data that is not in memory, but with the app running at 27k it seems that the entire app would be in memory when running. any thoughts?

  11. #26
    Join Date
    Dec 2003
    Posts
    3,366
    there is memory, and then there is memory =)

    The fastest memory is your registers, of course, but thats like 100 or so bytes (!!) and not really useful. If an entity is not in a register, the cpu has to LOOK for it:

    the next place to look is in the cache, level 1, then 2, then 3, and so on, depending on the CPU design. 1 is faster than 2 is faster than 3 and so forth.

    However, so far, all this memory is FAST and if the desired thing was located, you suffer only the most tiny of performance hits, or effectively none at all (because its impossible to keep all your data in registers, the next best thing is to keep it all in cache, and that is pretty much optimal which means that, in practice, your program is pretty optimal).

    If its not in the cache, you are in trouble if its data that needs to be accessed a LOT in a tight loop. Now, it has to swap pages of memory, which means write and read of kilobytes of memory to and from your actual RAM chips, or, worse, if those are full it may have to hit your virtual memory (read the hard disk!!!) which is extremely slow (compared to the CPU). These are your page faults --- when the CPU sits idle waiting while a page of memory is swapped to an from ram, or the disk, etc, into the cache.

    The way to reduce page faults, then, is to lock pages into the cache (there are operating system, platform specific, etc libraries and calls to do this) if you know you want it to stay there. Another way is to allocate your memory yourself. For example, rather than code like this:

    int *x = new int[100];
    ..
    double *d = new double[3000];
    ...
    myclass *blah = new myclass[1000];
    ...

    which could put EACH OF THOSE into a different "PAGE" of memory, you could do this:

    unsigned char* cp = new unsigned char[20000];

    and allocate from it:

    int dloc = 1000;
    d = (double *)&(cp[dloc]);
    blah = (myclass*) &(cp[dloc+ 3005*sizeof(double)); //3005 gives a little extra space for safety and debugging, but you can jam it up tight if you prefer.

    etc. This forces all these variables to be in the same page or pages (if large enough, they will span pages). However such a method is complex and requires a lot more than just this example code, you have to track your memory and manage it (unless, by design, all the objects exist for the lifetime of the program, that makes it simple!).

    There are other ways to try to manage your memory, or you can try to use less memory if possible (do you need all that?), or make some of the variables not be dynamic if possible (this is a gotcha: the program itself, and its stack memory, are usually in your cache already, so if you can put variables into that stack then they are less likely to page fault).

    Page size varys on different systems, so if you use the char* do it yourself approach, you should know what 1 page of memory on the target system is (or detect this with operating system queries if multiple target systems).

    You can try other things, like placement new or your own heaps etc as well (same idea as the char * approach, just using someone else's tools to put the variables into the same memory region).

    If you need to use more memory than your cache can hold at a given time, try to divide and conquor the problem so it WILL fit. If that is not possible, page faults are unavoidable but you can still reduce them. More than a few PF per second, like 10 per second, and you really begin to impact performance.

  12. #27
    Join Date
    Jul 2010
    Posts
    30
    unfortuneately with the events running, page faults are running in the thousands at times (even though I am not firing the events), so I guess the first question I need to ask, how do I go about finding the part of my code where the page faults are happnening? I havent been able to identify where they happen, and I am not aware of a way to make the program break when a page fault happens.

  13. #28
    Join Date
    Dec 2003
    Posts
    3,366
    Its the same issue as a memory leak. You remove part of the code, observe the page faults per second in the task manager, repeat until you isolate the problem.

    Page faults are a little easier to find, though. Look for a tight loop that processes a data structure. To get a page fault problem, you have to have a decently sized wad of data (kilobytes or more) in some sort of storage that is being touched in a loop. Or, you have a disorganized memory storage (linked list or graph or tree or the like, any data structure that adds data dynamically, one item at a time...!) so that a simple iteration over the data requests pages all over your memory (this means your system's memory is heavily fragmented, which means the machine needs more memory or needs to be cleaned up (too many background apps running) or the like).

    Do you have any data structures that are large? Do you even know, since you are using so many built in tools? Is your system low on memory? Do you have a rapid insert/delete issue (lots of data created and destroyed in a short period of time)?

    Still, unless you know where your big pile of data is and what is touching it, the easy way is to stub out sections of your code until the page faults drop to a reasonable level, then search the section of code that causes it for something like what I listed.

  14. #29
    Join Date
    Jul 2010
    Posts
    30
    Sorry, life got in the way to working on my project, Actually, my project really does not have any large data structures. I use javas BigInteger as a holder for the number (and java holds it in a sign magnitude array which I rarely push above 20 or so elements (I am working with relatively small numbers for testing purposes of only 50 to 100 decimal digits or so and only keeping 50 decimal places for scientific calculations) and the class has its own version of that same array (since its numbers are in screwed up order, little endian?), the rest of the members in the class are a few bools and a few ints which rarely change(except for the int that keeps track of the decimal point).

    I spent the last few days completely redoing how the class works internally. The way I had it the class was creating a ton of temporary versions of itself, so I got rid of all that. Now, It is running about ten times faster then it was (at least without the events enabled), but the code size grew by a factor of five, its now well over 200,000 lines of code and doesnt look as clean as it did the other way, but, it completely got rid of the memory leak and got rid of most of the page faults when events are enabled, however, it drops from doing about 14,000 modular exponentiations per second (on a 20 digit number) to less then 100 per second when I enable the events. Ive got the code down to where it only hits 3 methods during the modular exponentiation (the main method, the multiply, and the modular methods) and cant figure out for the life of me why the events are giving me so much trouble.

    From everything I am reading, I am not the only person thats had trouble with events in the dot net framework, but, it also sounds like most of the problems people have is that they do not unhook from the delegate (which I know its unhooking correctly).

    About your comment on me using so many built in tools, could you explain a little more what you mean by that? are you talking about the visual studio evironment? or that I use java as a holder? or that I use forms to organize the data when testing? is it possible something outside of the class is clashing with the events (which arent even firing and still causing me a headache)?

  15. #30
    Join Date
    Jul 2010
    Posts
    30
    oh ya, and I killed most of the background apps. I am running a dual core machine with 2 gigs of memory, not really much running in the background to begin with, and like I said, it runs absolutely stable without the events. As much as I hate to say it, I have started looking for another way to notify all the instances when a change has been made to the base class, lik maybe instead of a pure abstract class , could I run it as a singleton and fire pure virtual methods from the base class? or something like that?

Similar Threads

  1. Possible Memory Leak in Java application
    By esi-eric in forum Java
    Replies: 3
    Last Post: 02-24-2005, 07:58 PM
  2. Announcing a memory leak in javax.comm
    By Jason Jakob in forum Java
    Replies: 0
    Last Post: 06-30-2002, 01:14 PM
  3. help memory
    By kathy in forum Java
    Replies: 0
    Last Post: 11-02-2001, 05:22 PM
  4. Interface leaks in C#?
    By Kevin Burton in forum .NET
    Replies: 12
    Last Post: 10-09-2000, 10:29 AM
  5. Re: MS Provider For Oracle : Memory Leak ?
    By John Grandy in forum VB Classic
    Replies: 0
    Last Post: 04-08-2000, 08:35 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center
 
 
FAQ
Latest Articles
Java
.NET
XML
Database
Enterprise
Questions? Contact us.
C++
Web Development
Wireless
Latest Tips
Open Source


   Development Centers

   -- Android Development Center
   -- Cloud Development Project Center
   -- HTML5 Development Center
   -- Windows Mobile Development Center