Re: LLM Experiments, Part 1: Corrections

[T.V Raman]

1. code rewrite and prose rewrite just feel very different to me --
   starting with simple things like white-space formatting etc.
2. Code rewrites therefore require a different type of mental activity
   -- side-by-side diff, whereas prose rewrite are more about has the
   meaning being  preserved -- and that is not conveyed by ws as directly.
3. You're likely right about js parsing and follow-on steps as being
   "atomic" actions in some sense from the perspective of using AI as
   a tool, but I still feel it too early to connect too many steps
   into one because it happens to work sometimes at present; it'll
   likely both get better and change, so we might end up abstracting
   early and perhaps erroneously at this stage. So if you do
   pool/group steps -- eep that an implementation detail.
   

Andrew Hyatt writes:
 > On Tue, Jan 23, 2024 at 08:17 PM "T.V Raman" <raman@google.com> wrote:
 > 
 > > All very good points, Kartik!
 > >
 > > Some related thoughts below:
 > >
 > > 1. I think we should for now treat  prose-rewriting vs code-rewriting as
 > >    separate flows -- but that said, limit our types of "flows"
 > >    to 2. More might emerge over time, but it's too early.
 > 
 > How do you see the code and prose rewriting requiring different UI or 
 > processing?
 > 
 > > 2. Multi-step flows with LLMs are still early -- or feel early to me; I
 > >    think that for now, we should just have human-in-the-loop at each
 > >    step, but then leverage the power of Emacs to help the user stay
 > >    efficient in the human-in-the-loop step, start with simple things
 > >    like putting point and mark in the right place, populate Emacs
 > >    completions with the right choices etc.
 > 
 > It can't be at every step, though. Maybe you wouldn't consider this a
 > step, but in my next demo, one step is to get JSON from the LLM, which
 > requires parsing out the JSON (which tends to be either the entire
 > response, or often in a markdown block, or if none of the above, we
 > retry a certain amount of times). But agreed that in general that we do
 > want humans to be in control, especially when things get complicated.

--