Re: [Octave Forge] Possible addition of gradient-descent framework for optim package

[Olaf Till]

On Fri, Oct 23, 2015 at 08:59:54AM +0200, Daniel Kraft wrote:
> ...
> and very different from
> optimisation in vector spaces where such algorithms are usually
> formulated.  This is much more similar to optimisation on manifolds, as
> far as I understand the general optimisation theory (which is not my
> main topic of research).
> 
> For instance, the concepts of "point" and "direction" are very different
> from each other.  You cannot simply do something like
> 
>   x_new = x_current + step * direction".
> 
> I assume that all existing algorithm code would have to be rewritten for
> that (but I may be wrong).

Unfortunately I can't comment on this, since I'm not familiar with
your field of research ...

> ...
> Which callbacks do you mean here -- the function evaluation and gradient
> computation and so on?  I'm not sure if this can be used to extract all
> of the information I use -- which includes not just all points along the
> optimisation, but also things like accepted line-search steps.  Is this
> available from the callbacks?

There is also the callback function(s) in the option
"user_interaction" (similar to Matlabs "OutputFcn"). There are some
(optional) standard fields in the informational structure it gets as
an argument. But each algorithm can choose to add additional
information to it; it might be good, however, to decide on the names
of the fields and their contents in community. As yet we have no
algorithm with line-search.

> Summarising the discussion so far, I think that the problem I'm tackling
> with shape optimisation is quite different from the usual numerical
> optimisation stuff (mostly because the setting is not a vector space).
> I think that one cannot easily fit it into existing optimisation
> backends; since it is still "optimisation", I believe that my code could
> still fit to the optim package -- as a separate category for
> optimisation on manifolds, for instance.  But I also understand if you
> think this is too far off the scope of the package; after all, maybe it
> fits best to level-set for shape optimisation unless a separate package
> for manifolds or shape optimisation is created.

Since you say your problem needs an algorithm with line search, a new
algorithm would indeed have to be added. But as you say your problem
can't be transformed into one in which just vector elements are
optimized, this would indeed be a blocker from using existing
frontends of optim.

If that is so, I'd favour leaving your code in 'levelset' or in a
separate package. A similar issue are neural nets, where optimization
is performed, too, but the code is not in the 'optim' package.

Olaf

-- 
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc
Description: Digital signature