slow getc() / putc() in VC++

[Matt Mahoney]

The following code copies a 100 MB file in about 0.5 seconds when compiled with g++ -O3 (MinGW 4.7.0) and 22 seconds when compiled with cl /O2 (visual C++ 16.00.30319.01 for 80x86) under 32 bit Vista on a 2.0 GHz T3200.

  FILE* in=fopen(argv[1], "rb");
  FILE* out=fopen(argv[2], "wb");
  for (int c; (c=getc(in))!=EOF;)
    putc(c, out);

Replacing getc() and putc() with fgetc() and fputc() slows g++ to about 25 seconds, but has no effect on speed in VC++. Anyone else notice this?

I can work around this using fread() and fwrite(), but I'm curious if other versions of VC++ do this.

[encode]

Try

#define _CRT_DISABLE_PERFCRIT_LOCKS

(Before #include <stdio.h>)

http://msdn.microsoft.com/en-us/libr...(v=vs.80).aspx

[Shelwien]

Also try adding #include "getcputc.inc" -- http://nishi.dreamhosters.com/u/getcputc.inc
Its actually even better than MS's old getc/putc macros.

[Matt Mahoney]

Thanks. That works perfectly.

[schnaader]

I wrote a small benchmark to test this on my machine. The code has been tested in MinGW gcc and g++ under Windows 7 and g++ under Ubuntu 11.04. Guess it has to be modified slightly to run in MSVC, didn't test this.

The code doesn't use any C++ specific features, so it can be compiled with both gcc and g++, although I didn't manage to compile it using gcc under Ubuntu (problems with timing routines). Every test writes 100 MiB of data to the disk and reads it afterwards, checking the data for integrity. The first two tests use (f)(get/put)c and single byte reads/writes, the last test uses fread/fwrite with a block size of 1 KiB. There are two compiler switches that change the behaviour: "LINUX" for Linux systems (timing routines use GetTickCount on Windows, gettimeofday if LINUX is set) and "USEINC" to enable inclusion of getcput.inc that Shelwien posted above. Two #define statements (BUFSIZE and LENGTH) can be used to adjust the fread/fwrite block size and the total amount of data.

[1] Windows 7 64-Bit, MinGW gcc/g++ 4.6.1, -O3 -s
[2] Windows 7 64-Bit, MinGW gcc/g++ 4.6.1, -O3 -s -DUSEINC
[3] Ubuntu 11.04 32-Bit, g++ 4.5.2-8ubuntu4, -O3 -s -DLINUX
[4] Ubuntu 11.04 32-Bit, g++ 4.5.2-8ubuntu4, -O3 -s -DLINUX -DUSEINC

                [1]             [2]             [3]             [4]
fgetc/fputc	12900-13800 ms  12800-13500 ms  2900-3300 ms    2800-3200 ms
getc/putc       700-900 ms      800-1000 ms     2900-3700 ms    900-1000 ms
fread/fwrite    300-500 ms      300-500 ms      750-1100 ms     750-1100 ms

Conclusions:

• fgetc/fputc is the worst thing you can use in MinGW (~7 MiB/s), simply replacing it with getc/putc is both safe and gives a huge speed boost (> 100 MiB/s)
• For the original gcc/g++, it seems there is no huge difference between fgetc/fputc and they are faster than MinGW's, although they are still quite slow (30 - 40 MB/s)
• The file getcputc.inc that Shelwien posted above doesn't do much for MinGW, but is perfect for the original gcc/g++
• Using fread/write or buffered I/O is the best choice, but depending on the task, might lead to some programming work.
• The observations match those from Matt, VC++ seems to act similar to gcc/g++ under Linux (no difference between fgetc/fputc and getc/putc, but boost from getcputc.inc)

[Shelwien]

Yes, though there's always http://www.cplusplus.com/reference/cstdio/setvbuf/

[encode]

Nice thing:
FILE* f=fopen(filename, "rbS");

"S" is the same as _O_SEQUENTIAL flag.

More info:
http://msdn.microsoft.com/en-us/library/yeby3zcb.aspx

Some results:
http://www.nicolaslelong.fr/2009/06/...equential.html

[Shelwien]

I think these results in the blog are weird.
33MB/s read speed on a modern hdd?
Also I tested these sequential and no-buffer flags with winapi and they didn't really help.
I think that guy did something wrong, like maybe tested reads with a totally wrong buffer size
(too small or too large).

[m^2]

If you're going to make it Windows-specific then why not use WinAPI?

[Bulat Ziganshin]

m^2, because it will need to change much more than single char