Re: [Bug-gnubg] Bug in sigmoid?

[Joseph Heled]

I am not sure about that. I have approximated exp(x) with 1+x, which is 
mathematically sound and has an o(x^2) error, which is nice in the [01] 
range the approximation is used. I think this is not the case for 1/(1+x)

Second, we need to re-train (we can't keep 2 nets around for practical 
reasons).

Third, I need to see numbers on how faster this is on non scalar, 
regular x86 machine.

-Joseph

Olivier Baur wrote:
> Le jeudi, 17 avr 2003, à 22:04 Europe/Paris, Joseph Heled a écrit :
>
> > Hi,
> >
> > Can you please post the code for the new sigmoid? I did not understand 
> > the new formulation.
>
> Sure, here it goes:
>
>
> #define SIG_Q 10.0f    /* reciprocal of precision step */
> #define SIG_MAX 100    /* half number of entries in lookup table */
> /* note: the lookup table covers the interval [ -SIG_MAX/SIG_Q  to  
> +SIG_MAX/SIG_Q ] */
>
> static float Sig[2*SIG_MAX+1];
>
> static void ComputeSigTable (void)
> {
>     int i;
>
>     for (i = -SIG_MAX; i < SIG_MAX+1; i++) {
>         float x = (float) i / SIG_Q;
>         // Sig[SIG_MAX+i] = 1.0f / (1.0f + exp (x)); /* more accurate, 
> but fails gnubgtest */
>         Sig[SIG_MAX+i] = sigmoid(x);  /* use the current sigmoid 
> function instead :-)  */
>     }
> }
>
>
> static inline float sigmoid2 (float const x)
> {
>     float x2 = x * SIG_Q;
>     int i = x2;
>
>     if (i > - SIG_MAX) {
>         if (i < SIG_MAX) {
>             float a = Sig[i+SIG_MAX];
>             float b = Sig[i+SIG_MAX+1];
>             return a + (b - a) * (x2 - i);
>         }
>         else
>             return 1.0f / 19931.370438230298f;  /* warning: this is 
> 1.0f/(1.0f+exp(9.9f)) */
>     }
>     else
>         return 19930.370438230298f / 19931.370438230298f;  /* warning: 
> this is 1.0f/(1.0f+exp(-9.9f)) */
> }
>
>
>
> -- Olivier
>
>
>
> _______________________________________________
> Bug-gnubg mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/bug-gnubg