Re: [Bug-wget] performace issues when saving a http iptv stream on a SD card

[Bykov Aleksey]

Greetings, gstream
It is possible to add caching writes to Wget. May be as options (default -  
BUFSIZ, or - user request?).
I check it with very old flash that had problem with writing by little  
blocks. Set buffer to 8 Mb and downloading speed increased from 10-20 kb/s  
(limited by speed of writing small chunks) to 600 kb/s(internet limit)).

Think, that it (large write buffer) can be much better for write directly  
on flash and SSD.

PS. BUT: There exist some reason for developers that they sometime decided  
to force flush...

--
Best regars, Alex

Ugly changes for fast check
diff --git a/src/ftp.c b/src/ftp.c
index b585631..0eb31e7 100644
--- a/src/ftp.c
+++ b/src/ftp.c
@@ -1243,6 +1243,7 @@ Error in server response, closing control  
connection.\n"));
           fd_close (local_sock);
           return FOPENERR;
         }
+      setvbuf (fp, NULL, _IOFBF, 8*1024*1024);
     }
   else
     fp = output_stream;
diff --git a/src/http.c b/src/http.c
index 1d55ade..0cd5005 100644
--- a/src/http.c
+++ b/src/http.c
@@ -2883,6 +2883,7 @@ read_header:
           xfree_null (type);
           return FOPENERR;
         }
+      setvbuf (fp, NULL, _IOFBF, 8*1024*1024);
     }
   else
     fp = output_stream;
diff --git a/src/retr.c b/src/retr.c
index 50b9c0d..290d76a 100644
--- a/src/retr.c
+++ b/src/retr.c
@@ -170,25 +170,6 @@ write_data (FILE *out, FILE *out2, const char *buf,  
int bufsize,
     fwrite (buf, 1, bufsize, out2);
   *written += bufsize;

-  /* Immediately flush the downloaded data.  This should not hinder
-     performance: fast downloads will arrive in large 16K chunks
-     (which stdio would write out immediately anyway), and slow
-     downloads wouldn't be limited by disk speed.  */
-
-  /* 2005-04-20 SMS.
-     Perhaps it shouldn't hinder performance, but it sure does, at least
-     on VMS (more than 2X).  Rather than speculate on what it should or
-     shouldn't do, it might make more sense to test it.  Even better, it
-     might be nice to explain what possible benefit it could offer, as
-     it appears to be a clear invitation to poor performance with no
-     actual justification.  (Also, why 16K?  Anyone test other values?)
-  */
-#ifndef __VMS
-  if (out != NULL)
-    fflush (out);
-  if (out2 != NULL)
-    fflush (out2);
-#endif /* ndef __VMS */
   if (out != NULL && ferror (out))
     return -1;
   else if (out2 != NULL && ferror (out2))