Re: [PATCH 6/7] qapi: Split up scripts/qapi/common.py

[Markus Armbruster]

Eric Blake <address@hidden> writes:

> On 10/2/19 10:16 AM, Markus Armbruster wrote:
>> Eric Blake <address@hidden> writes:
>>
>>> On 10/1/19 2:15 PM, Markus Armbruster wrote:
>>>> The QAPI code generator clocks in at some 3100 SLOC in 8 source files.
>>>> Almost 60% of the code is in qapi/common.py.  Split it into more
>>>> focused modules:
>>>>
>>>> * Move QAPISchemaPragma and QAPISourceInfo to qapi/source.py.
>>>>
>>>> * Move QAPIError and its sub-classes to qapi/error.py.
>>>>
>>>> * Move QAPISchemaParser and QAPIDoc to parser.py.  Use the opportunity
>>>>     to put QAPISchemaParser first.
>>>>
>>>> * Move check_expr() & friends to qapi/expr.py.  Use the opportunity to
>>>>     put the code into a more sensible order.
>>>
>>> Code motion can be easier to review when it is 1:1 (using 'diff -u
>>> <(sed -n '/^-//p' patch) <(sed -n '/^\+//p'patch)', which is quite
>>> small if code moved wholesale).  Reordering things breaks that
>>> property.
>>
>> True.  But see below.
>
>> Perhaps a bit of shell wizardry can increase your confidence.
>>
>> Before this patch:
>>
>> 1. Split into classes and functions (crudely!):
>>
>>      $ csplit scripts/qapi/common.py '/^\(class\|def\) /' '{*}'
>>
>> 2. Rename the parts:
>>
>>      $ for i in xx*; do n=`sed -n '1s/^[a-z]* \([A-Za-z0-9_]*\).*/\1/p' $i`; 
>> [ "$n" ] && mv $i xx-$n; done
>>
>> 3. Stash them:
>>
>>      $ mkdir o
>>      $ $ mv xx* o
>>
>> After this patch:
>>
>> 1. Split:
>>
>>      $ csplit <(cat 
>> scripts/qapi/{common,source,error,parser,expr,schema,gen}.py) 
>> '/^\(class\|def\) /' '{*}'
>>
>> 2. Rename:
>>
>>      $ for i in xx*; do n=`sed -n '1s/^[a-z]* \([A-Za-z0-9_]*\).*/\1/p' $i`; 
>> [ "$n" ] && mv $i xx-$n; done
>>
>> 3. Stash & diff:
>>
>>      $ mkdir n
>>      $ mv xx* n
>>      $ diff -rup o n
>
> Slick.
>
>>
>> Output of diff appended for your reading pleasure.
>>
>>> Reviewed-by: Eric Blake <address@hidden>
>>
>> Thanks!
>>
>>
>> diff -rup o/xx-QAPIDoc n/xx-QAPIDoc
>> --- o/xx-QAPIDoc     2019-10-02 17:02:35.984552694 +0200
>> +++ n/xx-QAPIDoc     2019-10-02 17:06:17.930607336 +0200
>> @@ -273,5 +273,31 @@ class QAPIDoc(object):
>>                   self.info,
>>                   "the following documented members are not in "
>>                   "the declaration: %s" % ", ".join(bogus))
>> +#
>> +# Check (context-free) QAPI schema expression structure
>> +#
>
> New boilerplate is obviousl
>
>> +
>> +# Names must be letters, numbers, -, and _.  They must start with letter,
>> +# except for downstream extensions which must start with __RFQDN_.
>> +# Dots are only valid in the downstream extension prefix.
>> +valid_name = re.compile(r'^(__[a-zA-Z0-9.-]+_)?'
>> +                        '[a-zA-Z][a-zA-Z0-9_-]*$')
>>   
>
> The crude split shows that this blurb changed in relation to which
> class/def it was closest to (but you did say it was a crude split),
> which isn't fatal.
>
>>   diff -rup o/xx-QAPIGen n/xx-QAPIGen
>> --- o/xx-QAPIGen     2019-10-02 17:02:35.987552655 +0200
>> +++ n/xx-QAPIGen     2019-10-02 17:06:17.932607309 +0200
>> @@ -43,4 +43,3 @@ class QAPIGen(object):
>>           f.close()
>>     -@contextmanager
>> diff -rup o/xx-QAPIGenH n/xx-QAPIGenH
>> --- o/xx-QAPIGenH    2019-10-02 17:02:35.987552655 +0200
>> +++ n/xx-QAPIGenH    2019-10-02 17:06:17.933607296 +0200
>> @@ -7,3 +7,4 @@ class QAPIGenH(QAPIGenC):
>>           return guardend(self.fname)
>>     +@contextmanager
>
> Another victim of the crude split (this line logically begins with the
> next line in the original file(s), not the previous class/def).
>
>> diff -rup o/xx-QAPISchema n/xx-QAPISchema
>> --- o/xx-QAPISchema  2019-10-02 17:02:35.986552668 +0200
>> +++ n/xx-QAPISchema  2019-10-02 17:06:17.932607309 +0200
>> @@ -297,9 +297,26 @@ class QAPISchema(object):
>>                       visitor.visit_module(module)
>>                   entity.visit(visitor)
>>           visitor.visit_end()
>> -
>> -
>>   #
>> -# Code generation helpers
>> +# QAPI code generation
>> +#
>
> More whitespace and boilerplate.
>
>
>> +++ n/xx-QAPISchemaParser    2019-10-02 17:06:17.930607336 +0200
>> @@ -268,14 +268,3 @@ class QAPISchemaParser(object):
>>           raise QAPIParseError(self, "documentation comment must end with 
>> '##'")
>>     -#
>> -# Check (context-free) schema expression structure
>> -#
>> -
>> -# Names must be letters, numbers, -, and _.  They must start with letter,
>> -# except for downstream extensions which must start with __RFQDN_.
>> -# Dots are only valid in the downstream extension prefix.
>> -valid_name = re.compile(r'^(__[a-zA-Z0-9.-]+_)?'
>> -                        '[a-zA-Z][a-zA-Z0-9_-]*$')
>> -
>> -
>
> We already visited this above.
>
>
>>   Only in o: xx-camel_case
>
> Interesting change, but not detrimental.

I forgot to mention I moved camel_case() to the bit bucket, because it's
unused.  I'll fix that.

> Overall, the interdiff is fairly representative of clean code motion
> in spite of any reordering.  Thanks for doing that!
>
> R-b still stands.

Thanks!