[Octave-bug-tracker] [bug #57113] fminunc return wrong Hessian matrix

[Nir Krakauer]

Follow-up Comment #4, bug #57113 (project octave):

For minimizing a quadratic form like @(x) sum (x.^2), BFGS [1] converges in
one iteration from any starting point, but the approximate Hessian is
generally not the right value when it's initially set to the identity (which
is the case in fminunc). This is expected. If you need the Hessian for
something like estimating confidence intervals, you should compute it at the
minimum in some other way.

We could explore initializing the Hessian more intelligently in fminunc using,
e.g., the ideas here [2]. These would give the right answer using BFGS for
this particular problem, and could conceivably be what Matlab does. fminunc in
Octave actually combines BFGS with a double dogleg trust region minimization
(which is different from Matlab's method, based on their documentation).
Although unfortunately there's no reference provided in the fminunc code to
the algorithm used, there are some details in an old mailing list thread [3].
But in general, you shouldn't count on the approximate Hessian from this sort
of solver to be accurate.

[1]
https://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm
[2] https://scicomp.stackexchange.com/q/11324
[3] https://octave.1599824.n4.nabble.com/algorithm-for-fminunc-td4648956.html

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57113>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/