strtok() segfault

[Fumitoshi UKAI]

Hi,

While I'm building and testing w3mmee, a text web browser based on w3m
written by Akinori ITO, Kiyokazu SUTO and others, I got segfault from
latest w3mmee.  I looked into this problem and I found the reason of
this is in strtok().

If first call of strtok() in program returns NULL, then subsequent 
search by calling strtok(NULL, "...") will segfault.  For example,

  #include <string.h>

  int
  main()
  {
        char *p,*s;
        p = strdup("\n");
        s = strtok(p, " \n\t\r");
        printf("s=%p<%s>\n", s, s?s:"NULL");
        s = strtok(NULL, " \n\t\r")
        printf("s=%p<%s>\n", s, s?s:"NULL");
        printf("\n");
        exit(0);
  }

I'm not sure but it is better to run without segfault.
If another call of strtok() is done before strtok(p, " \n\t\r"), then
it would work fine:

  #include <string.h>

  int
  main()
  {
        char *p,*s;
        p = strdup("abc\ndef\n");
        s = strtok(p, " \n\t\r");
  #if 1 /* if 0, then last strtok() will return "def" */
        s = strtok(NULL, " \n\t\r");
        s = strtok(NULL, " \n\t\r");
        s = strtok(NULL, " \n\t\r");
  #endif
        p = strdup("\n");
        s = strtok(p, " \n\t\r");
        printf("s=%p<%s>\n", s, s?s:"NULL");
        s = strtok(NULL, " \n\t\r");
        printf("s=%p<%s>\n", s, s?s:"NULL");
        printf("\n");
        exit(0);
  }

In this example, if first three of strtok(NULL, " \n\t\r") is removed,
last call of strtok(NULL, " \n\t\r") will return "def", which is not
subsequent search of "\n"!

I think this can be fixed by the following patch:

--- sysdeps/generic/strtok.c.orig       Wed Aug 16 03:16:25 2000        
+++ sysdeps/generic/strtok.c    Fri Feb 23 03:40:41 2001
@@ -44,8 +44,10 @@
 
   /* Scan leading delimiters.  */
   s += strspn (s, delim);
-  if (*s == '\0')
+  if (*s == '\0') {
+    olds = s;
     return NULL;
+  }
 
   /* Find the end of the token.  */
   token = s;

Of course, it should be applied for sysdeps/generic/strtok_r.c.
And same logic might be needed in sysdeps/i386/{strtok.S,strtok_r.S}

Or, is such use of strtok(), that is call strtok(NULL, "...") after
strtok() return NULL, illegal?

Thanks,
Fumitoshi UKAI