Re: [Help-smalltalk] [PATCH 06/15] stinst: Fix the sourcecode extraction in RBScanner>>#scanNumber

[Paolo Bonzini]

Il 08/04/2013 11:30, Holger Hans Peter Freyther ha scritto:
> From: Holger Hans Peter Freyther <address@hidden>
> 
> Fix two problems introduced by the RBNumberLiteralToken change.
> The first one is to use >>#copyFrom:to: instead of re-positioning
> the stream to fix an issue with the ConcatenatedStream and the
> second is to pick the right stop for the token.
> 
> Code like '(Delay forSeconds: 3) wait' got re-formatted to
> '(Delay forSeconds: 3)) wait'. The code that extracts the
> sourcecode for the RBNumberLiteralToken should have operated on
> the result of >>#previousStepPosition but instead the current
> position of stream was used as an end.
> 
> Use self previousStepPosition to find the end of the number literal,
> remove unused variables and remove the trimSeparators as we are now
> using the right place to stop.
> 
> 2013-02-17  Holger Hans Peter Freyther  <address@hidden>
> 
>       * RBParser.st: Fix RBScanner>>#scanNumber.
>       * RewriteTests.st: Add testcase for RBScanner>>#scanNumber.
>       * package.xml: Add the new test to the testsuite.
> ---
>  packages/stinst/parser/ChangeLog       |    6 ++++
>  packages/stinst/parser/RBParser.st     |    9 ++---
>  packages/stinst/parser/RewriteTests.st |   57 
> +++++++++++++++++++++++++++++++-
>  packages/stinst/parser/package.xml     |    1 +
>  4 files changed, 66 insertions(+), 7 deletions(-)
> 
> diff --git a/packages/stinst/parser/ChangeLog 
> b/packages/stinst/parser/ChangeLog
> index ed27e41..94dab52 100644
> --- a/packages/stinst/parser/ChangeLog
> +++ b/packages/stinst/parser/ChangeLog
> @@ -1,3 +1,9 @@
> +2013-02-17  Holger Hans Peter Freyther  <address@hidden>
> +
> +     * RBParser.st: Fix RBScanner>>#scanNumber.
> +     * RewriteTests.st: Add testcase for RBScanner>>#scanNumber.
> +     * package.xml: Add the new test to the testsuite.
> +
>  2013-02-10  Holger Hans Peter Freyther  <address@hidden>
>  
>       * OldSyntaxExporter.st: Reformat the method node in
> diff --git a/packages/stinst/parser/RBParser.st 
> b/packages/stinst/parser/RBParser.st
> index 6f2b1cb..7be57d6 100644
> --- a/packages/stinst/parser/RBParser.st
> +++ b/packages/stinst/parser/RBParser.st
> @@ -1320,16 +1320,13 @@ Stream subclass: RBScanner [
>      ]
>  
>      scanNumber [
> -        | start stop val string |
> +        | stop val string |
>       <category: 'private-scanning'>
> -        start := tokenStart.
>          val := self scanNumberValue.
> -        stop := stream position.
> +        stop := self previousStepPosition.
>  
>          "Get the parsed source"
> -        stream position: tokenStart - 1.
> -        string := (stream next: stop - start + 1) trimSeparators.
> -        stream position: stop.
> +        string := stream copyFrom: tokenStart - 1 to: stop - 1.
>  
>       ^RBNumberLiteralToken
>              value: val
> diff --git a/packages/stinst/parser/RewriteTests.st 
> b/packages/stinst/parser/RewriteTests.st
> index 4c42cd8..d38b69b 100644
> --- a/packages/stinst/parser/RewriteTests.st
> +++ b/packages/stinst/parser/RewriteTests.st
> @@ -7,7 +7,7 @@
>  
>  "======================================================================
>  |
> -| Copyright (C) 2007 Free Software Foundation, Inc.
> +| Copyright (C) 2007,2013 Free Software Foundation, Inc.
>  | Written by Stephen Compall.
>  |
>  | This file is part of the GNU Smalltalk class library.
> @@ -278,6 +278,61 @@ TestCase subclass: TestFormat [
>       res := RBFormatter new formatAll: (Array with: inp).
>       self assert: res = '#(16r01 2r01 16rFF)'.
>      ]
> +
> +    testParanthesis [
> +        | inp |
> +        inp := RBParser parseExpression: '(2r1)'.
> +        self assert: inp formattedCode = '2r1'.
> +    ]
> +
> +    testNumberLiteralFloatRewrite [
> +        | inp |
> +        inp := RBParser parseExpression: '2q'.
> +        self
> +            assert: inp value = 2.0;
> +            assert: inp formattedCode = '2q'.
> +
> +        inp := RBParser parseExpression: '2d'.
> +        self
> +            assert: inp value = 2.0;
> +            assert: inp formattedCode = '2d'.
> +
> +        inp := RBParser parseExpression: '2e'.
> +        self
> +            assert: inp value = 2.0;
> +            assert: inp formattedCode = '2e'.
> +
> +        inp := RBParser parseExpression: '2q-'.
> +        self
> +            assert: inp value = 2.0;
> +            assert: inp formattedCode = '2q-'.
> +
> +        inp := RBParser parseExpression: '20q-1'.
> +        self
> +            assert: inp value = 2.0;
> +            assert: inp formattedCode = '20q-1'.
> +    ]
> +]
> +
> +TestCase subclass: TestScanner [
> +    <comment: 'Test aspects of the RBScanner'>
> +
> +    testScanner [
> +        | scanner num |
> +        scanner := RBScanner on: '3' readStream.
> +        num := scanner next.
> +        self assert: num value = 3.
> +    ]
> +
> +    testScannerConcatStream [
> +        | scanner num |
> +        "This is different to >>#testScanner by using a different kind of 
> stream with
> +        a different species."
> +
> +        scanner := RBScanner on: (Kernel.ConcatenatedStream with: '3' 
> readStream).
> +        num := scanner next.
> +        self assert: num value = 3.
> +    ]
>  ]
>  
>  TestCase subclass: TestRewrite [
> diff --git a/packages/stinst/parser/package.xml 
> b/packages/stinst/parser/package.xml
> index b54d14a..f14f6e0 100644
> --- a/packages/stinst/parser/package.xml
> +++ b/packages/stinst/parser/package.xml
> @@ -30,6 +30,7 @@
>     <sunit>STInST.Tests.TestStandardRewrites</sunit>
>     <sunit>STInST.Tests.TestFormat</sunit>
>     <sunit>STInST.Tests.TestRewrite</sunit>
> +   <sunit>STInST.Tests.TestScanner</sunit>
>     <sunit>STInST.Tests.TestDefaultPoolResolution</sunit>
>     <sunit>STInST.Tests.TestClassicPoolResolution</sunit>
>     <filein>RewriteTests.st</filein>
> 

Ok.

Paolo