Re: SMIE implementation for the C-like languages

[Arthur Evstifeev]

Stefan Monnier writes:

>> When in trying to indent such code:
>
>> if true {
>>     |bar
>> }
>
>> Token "bar" is positioned incorrectly and I see such requests from smie
>> and lexer and indentation rules:
>
>> forward: 15 -> 18 = bar
>> backward: 15 -> 10 =
>> forward: 9 -> 9 =
>> backward: 9 -> 4 = true
>> backward: 4 -> 1 = if
>> forward: 9 -> 9 =
>> :after '{'; sibling-p:nil parent:(nil 4 if) hanging:t == nil
>> forward: 9 -> 9 =  [2 times]
>> backward: 9 -> 4 = true
>> backward: 4 -> 1 = if
>> forward: 9 -> 9 =
>> :before '{'; sibling-p:nil parent:(nil 4 if) hanging:t == nil
>> forward: 9 -> 9 =
>> backward: 9 -> 4 = true [2 times]
>> backward: 4 -> 1 = if [3 times]
>> :list-intro 'if'; sibling-p:nil parent:nil hanging:nil == nil
>> forward: 4 -> 8 = true
>> :elem 'args'; sibling-p:nil parent:nil hanging:nil == nil
>> forward: 4 -> 8 = true
>> :elem 'basic'; sibling-p:nil parent:nil hanging:nil == 4
>> forward: 9 -> 9 =
>> :elem 'basic'; sibling-p:nil parent:nil hanging:t == 4
>
>> This logging output and indentation requests don't seem to be respecting
>> defined grammar.
>
> I don't see anything out of the ordinary here.  Which part seems odd
> to you?

I'm a bit suspicious about :list-intro 'if', :elem 'args' and that :elem
'basic' was requested 2 times, especially if you compare against second
example I sent yesterday (which has a proper indentation). What is
specifically looking confusing to me is the whole situation with
braces. I don't understand why smie produces different indentation
results for cases when we tokenize braces and when we don't. I
think both cases shouldn't be treated differently in grammars or
indentation rules. From implementation perspective I understand why it
happens at the moment.

>
>> But blink-matching-open calls for the simple code block:
>
>> {
>> }|
>
>> will return "Mismatched parenthesis" error.
>
> That's because it tries to match "}" with an opening "if" (since your
> grammar states ("if" exp "{" insts "}") which implies that "{" is an
> infix terminal).
>

Yes that's true, but braces are also a part of the syntax table. Since smie
allow any sequence of sexp anywhere, I think this code block should be
handled. I'm not sure what is the best way to do that but maybe
something like a fallback logic to the syntax tables for such cases?

>> smie output will still contain request for indentation of "}" even if
>> lexer didn't return such token:
>
>> forward: 19 -> 20 = end
>> backward: 20 -> 19 = end
>> backward: 19 -> 18 = ;
>> backward: 18 -> 15 = bar
>> backward: 15 -> 9 = begin
>> backward: 9 -> 4 = true
>> backward: 4 -> 1 = if
>> :close-all '}'; sibling-p:t parent:(nil 1 if) hanging:nil == nil
>
> Indeed.  Please send this via M-x report-emacs-bug so we get a number to
> track this problem.
>

I submitted a new bug report.

>> But it doesn't change behavior of the blinking: for the same if
>> construction blinking happens for the "if" token.  Is there a different
>> way of altering this behavior?
>
> My guess would be that the default blinking code uses forward-sexp which
> goes through forward-sexp-function which SMIE sets up as well.  Try set
> this var back to nil in swift-mode buffers, see if that helps.
>

I tried to reset forward-sexp-function and it works when I'm not
tokenizing braces. When braces handled in lexer, I'm getting "Mismatched
parenthesis" error similar to one of the previous examples.

Thank you,
Arthur