Re: Diff crash when comparing 2 big files with error message "diff.exe: memory exhausted"

[Bob Proulx]

Kissinger Chen wrote:
> This is Kissinger, a Gnuwin32 user. Today I use diff to compare some big
> files and try to retrieve the differences between files. 
>  
> For some big files, the diff work well and I can get the differences.

Sounds good.

> But for some other big files, the diff will crash with 
> such error message:"diff.exe: memory exhausted". These files are more
> then 500 MB, pure text files. 
> And I use the diff of 2.8.7, my OS is windows XP.

GNU diff is a program and requires a certain amount of memory to
analyze files.  Bigger files with complex differences may consume more
memory than smaller files.  When diff encounters an error it will
report it.  Running out of memory is an error and is being reported.
Running out of memory in the case of processing large files is not a
program crash.  The program did not crash.  The program detected that
it was out of resources and reported this error condition.

> It seems there is an issue in the Gnuwin32 diff tool. Has this issue
> been reported? I can not search this issue from help.

The error message you have received is stating "memory exhausted"
meaning that there is not enough virtual process memory for the
program to run to successful completion.  This is a physical
limitation of your platform.  You have run out of memory.

How much memory do you have on your system?

How much memory is diff currently consuming before running out?

> If convenient, can you tell me how to resolve this issue? Is there any
> another diff-like tool? Thanks.

Here are some suggestions:

1. If your 32-bit system has less than 4G of memory then adding more
   memory to your system may allow this program to complete.  32-bit
   systems have a maximum addressable memory of 4G so that is the
   upper limit possible.

2. Run this task on a different system that has more memory than your
   current machine.

3. Run this task on a 64-bit system with more than 4G of memory.  A
   64-bit operating system has much more capability to address very
   large amounts of physical memory.

Note that in extreme cases diff may consume very large amounts of
memory and still may exhaust all available memory.  You may need to
change the problem statement in order to reduce the system resources
needed to accomplish what you want to accomplish.  You may need to
reorganize what you are trying to do in order to make the problem
tractable within current system memory limitations.

Bob