Assertion error in CRT calling _osfile() in VS 2008?

Question

I have a C++ code base that has been working for a long time. The code base was a legacy VS 2003 set of projects that I recently migrated to VS 2008. The migration seemed to be successful in that the resulting program built, and run.

I reinstalled my OS and all applications on a fresh drive, and now when I attempt to debug the program within the debugger, I receive an assertion error inside the CRT's chsize (really, _chsize_s). Specifically (cropped to essentials, ignoring safety checks):

FILE * testfile = fopen("P:\\_Dan\\local\\foogoo.txt", "w");
int filehandle = fileno(testfile);
chsize(filehandle, 0);
fwrite("goohoo", 1, 6, testfile);
fclose(testfile);

The debug assertion occurs within chsize - specifically, within the CRT's source code file chsize.c, at the following line:

 _VALIDATE_CLEAR_OSSERR_RETURN_ERRCODE((_osfile(filedes) & FOPEN), EBADF);

... where filedes matches filehandle.

I thought possibly the problem might result from not having an older version of VS installed on the new system (only VS 2008), because some 3rd-party libraries require VS 8.0 redistributable - even though on the old system things seemed to be building and running just fine using VS 2008. I therefore installed VS 2005 (not 2003). However, the problem continues to occur.

Any suggestions would be immensely welcome.

* Update - The issue is unrelated to chsize. See my answer below.

Answer 1

The issue is resolved - and unrelated to chsize. The linking model to the c-runtime libraries chosen for code generation was set to multi-threaded debug (/MTd) for the main project, but multi-threaded debug DLL (/MDd) for all of the projects in the solution that it linked to. Changing to /MDd resolved the issue.

I am familiar with these linking issues and am generally careful to set them properly, but because this was an upgrade of a working project from an earlier version of Visual Studio with no changes, I did not think to look down this road. I did not investigate how or why the setting was changed (or even if it was set this way in the previous version but did not cause problems).

Answer 2

Found this problem in my code too. Main program need to linked with shared library which build with MT. When file handler opened in main program passed to shared library's function, _VALIDATE_RETURN((_osfile(fh) & FOPEN), EBADF, -1) in setmode.c of CRT crashed the program.

Debug the _osfile in assembly mode, osfile lookup file handler in table __pioinfo (01802EEDB0h). Well it's fixed area in statically linked CRT. And another __pioinfo in the main program is another address 01E619540h. In one word, if two module need to shared global data, can not build with MT model.

I just want to optimize the shared library with static compilation, some hard to notice bugs could happen. Seems GCC's force shared or static makes sense in most situation.