Re: About scoring

[Mike Sheldon]

On Thu, 2011-04-07 at 12:34 +0100, Daniel Thomas wrote:
> <snip>
> Unfortunately that doesn't solve the problem as it favours projects for
> which lots of people feel like they can score over more obscure (but
> still good) projects for which only their mentor knows what they are
> talking about well enough to score. Additionally the mentors of the
> project will also be basing the score on the community interaction which
> preceded the application which other people will not know about.
> 
> It might be helpful to determine what exactly we mean by 1,2,3,4,5 and
> then for the mentors of the project to pick a value using private
> comments and then one of them vote that. (yes this is an ugly hack)
> 
> Or to say "two people can score" and hope everyone can find two people
> (though that makes it harder for me as there is only one of me but I can
> probably prod the official maintainer into helping if necessary.)

 In previous years with other projects (GNOME and GStreamer), the
approach taken by the org admins was to mostly use the mentor's scores
as a rough guideline. We then had a couple of IRC meetings towards the
end of the rating period to discuss how people felt about the various
projects, after which the admins modified the scores to reflect the
result of the IRC discussions. Previously I think this was a bit easier
due to the availability of negative scoring, but apparently the range of
scores available to admins is much greater than to mentors (based on a
few emails on the google-summer-of-code-mentors-list a couple of weeks
back), so I think this should still be possible.

 Perhaps a similar process could be employed here to allow for a level
of human judgement to solve the various possible problems that may
arise?

Cheers,
 Mike.