> As I mentioned at the top of this section, this VC++ 2010 ‘buffer overflow’ is apparently not a bug and would never cause crashes during normal usage.
That's the problem when you try to track down bugs in other people's programs. Just give the best info you have, trying to debug a binary black box should be a last resort rather than an attempt to help.
From what I can tell, the program will cause memory access violations during normal usage. Presumably it installs a handler to "fix" the violation and resume the program. You can use this technique to make a buffer that automatically expands without having to put bounds checks in the code. However, getting it right is extremely tricky. You rarely see these kind of tricks outside of language runtimes.
For an example of this kind of trick, imagine writing a heap allocator that simply decremented a pointer and returned it. Heap allocations would be ridiculously fast, requiring one or two instructions per allocation. It does no bounds checking. Eventually it will return a pointer to a bad page which will trigger a signal when the program writes to it, and the signal handler fixes it by doing a garbage collection cycle.
Neat stuff! I've experimented a bit with the analysis features of VS but only through the one provided with the Xbox 360 SDK, I'll make a point of checking out VS2012's even if it is more crash-prone.
Does the application verifier move all allocations, especially those made through C/C++ heap allocation functions, to page boundaries or just those made with HeapAlloc et al? I've had good luck with valgrind on both Linux and OS X for diving into difficult bugs and shining a light on what may eventually be a problem.
App Verifier just moves HeapAlloc allocations, but the C/C++ functions default to using these, so it grabs pretty much everything.
For game development I wrote a custom allocator that redirects all of our allocations to the Windows heap instead of using our custom allocator, in order to support App Verifier and xperf memory profiling.
It's not similar to valgrind. It's more like Electric Fence.
That's the problem when you try to track down bugs in other people's programs. Just give the best info you have, trying to debug a binary black box should be a last resort rather than an attempt to help.
From what I can tell, the program will cause memory access violations during normal usage. Presumably it installs a handler to "fix" the violation and resume the program. You can use this technique to make a buffer that automatically expands without having to put bounds checks in the code. However, getting it right is extremely tricky. You rarely see these kind of tricks outside of language runtimes.
For an example of this kind of trick, imagine writing a heap allocator that simply decremented a pointer and returned it. Heap allocations would be ridiculously fast, requiring one or two instructions per allocation. It does no bounds checking. Eventually it will return a pointer to a bad page which will trigger a signal when the program writes to it, and the signal handler fixes it by doing a garbage collection cycle.