Re: [MIT-Scheme-devel] non-recursive mutexes

[Micah Brodsky]

[TLDR: I think we can do better, and just because compatibility gets no respect 
as an engineering problem doesn't mean it doesn't deserve it. Also, boy it 
would be nice to have a package depot.]

Hey, Alexey.
Here's my take:

People can be expected to rely on essential services (and even some inessential 
ones) whether or not they are documented. The only exception is if how to 
*avoid* needing to rely on them is straightforward and well documented. (ask me 
how I know. :P ) Saying "don't rely on it, it's not documented" in reference to 
an essential service is basically saying "our system is an unfinished toy, 
we're sorry you decided to use it".

While email lists are all we presently have, they are not the only means of 
communication with third party developers for the purpose of platform evolution 
-- and, in general, they are woefully inadequate. Other methods of outbound 
communication include compile-time or runtime deprecation warnings shipped 
prior to breaking changes, a pre-release test module to probe for potentially 
breaking dependencies, or a summary of expected breaking changes in a single, 
easy to follow newsletter prior to release (as opposed to tons of little 
messages of questionable relevance spread throughout the year). Other methods 
of *inbound* communication include dependency usage statistics tracked by the 
platform and reported back with the user's consent, or a central repository of 
third-party projects that can be tested en masse by the platform developers 
prior to deciding on a change. (For the record, I think utility of having a 
package depot like you suggest is impossible to overstate!)

Now, I'm not saying that MIT Scheme needs any of these things (besides a 
package depot). A pony would be nice too. We have a rather small community, and 
a spit-and-polish job on compatibility is not in the cards. But, the absence of 
a more sophisticated means of communication is a hobble, not a license to 
ignore the need to close the feedback loop. Relying on "well, it was 
undocumented behavior and we sent out a email" should be a last resort for 
critical fixes, not standard operating procedure. It does nothing for Scheme's 
reputation as being unsuited to the "real world". 

I'm not trying to pile on Taylor -- I should note that I have been cringing in 
terror at the SMP-related breaking changes to the runtime (e.g. fluid vars) --  
but do I believe the sort of cosmetic breaking change in question here is a 
mistake, no matter how common a sin it may be. If it sounds like I'm 
particularly passionate about this stuff, well, it's because I am. The "worse 
is better" approach to compatibility in the free software world has gone too 
far and is costing too many grey hairs multiplied by too many users, all 
debugging isomorphic problems. MIT Scheme has too few users to suffer from that 
sort of multiplier, but we're never going to get over this problem collectively 
unless we both learn and teach better strategies.
(With some luck I will be pushing a research agenda on 'software evolution' in 
the next few years.)

[End flame]

--Micah

-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of Alexey Radul
Sent: Monday, June 08, 2015 6:57 PM
To: Micah Brodsky
Cc: Taylor R Campbell; Arthur A. Gleckler; MIT Scheme Developers
Subject: Re: [MIT-Scheme-devel] non-recursive mutexes

Micah,

Respectfully, in general, users who rely on subtleties of the detailed 
semantics of undocumented procedures [*] should expect occasional incompatible 
changes.  And what communication channel do we have except postings on the 
devel mailing list?

As to ecosystem stewardship, do we as a community have the need and resources 
to maintain a package repository, such as hackage.haskell.org, 
planet.racket-lang.org, or even interoperate with snow.iro.umontreal.ca ?  That 
seems a more basic step than permitting fear of potential unknown users' 
potential unknown uses of system internals for nefarious purposes [+] to impede 
progress.

Best,
~Alexey

[*] It is regrettable that the thread subsystem happens to consist entirely (as 
far as I can tell) of undocumented procedures.

[+] Such as spawning threads only to lock them up reentrantly

On Mon, Jun 8, 2015 at 5:54 PM, Micah Brodsky <address@hidden> wrote:
> Hi, Taylor.
> +1 for Arthur on this. I don't know whether my RPC library relies 
> +anywhere
> on recursive locking, but I'm not currently maintaining it and I'd 
> really rather not have the most difficult parts of it (i.e. thread 
> management) rot out from under me while I'm busy with other projects.
>
> In general, deciding whether to make breaking changes based on whether 
> or not there is a reply to one or two postings on an obscure mailing 
> list is a lousy strategy for software ecosystem stewardship. Speaking 
> as an alum of MS's application compatibility group, if you don't want 
> your users hating your guts, the default assumption needs to be that 
> breaking changes are unacceptable until proven otherwise. "Encouraging 
> good coding practice" is hardly justification to wing it.
>
> --Micah
>
> -----Original Message-----
> From: address@hidden
> [mailto:address@hidden On 
> Behalf Of Taylor R Campbell
> Sent: Monday, June 08, 2015 3:02 PM
> To: Arthur A. Gleckler
> Cc: MIT Scheme Developers
> Subject: Re: [MIT-Scheme-devel] non-recursive mutexes
>
>    Date: Sun, 7 Jun 2015 14:37:17 -0700
>    From: "Arthur A. Gleckler" <address@hidden>
>
>    I'm just upgrading to MIT Scheme 9.2, and my code is breaking in several
>    places because of your change to Remove support for recursion in
>    WITH-THREAD-MUTEX-LOCKED
>
> <http://git.savannah.gnu.org/cgit/mit-scheme.git/commit/?id=d7241d6fe8
> b151f6
> d15db9cac8fba44b074ca215>.
>    Can you tell me why you made that change?  I can't see a correctness or
>    performance advantage, and the change list comment gives no 
> justification.
>    Not supporting recursion is a nuisance in cases where one has utility
>    procedures that may or may not be run in a context where the lock is
>    already held.
>
> Hi, Arthur!  I asked the list last year whether anyone relied on it:
>
> https://lists.gnu.org/archive/html/mit-scheme-devel/2014-11/msg00005.h
> tml
>
> Nobody replied that they relied on it, and since the thread system is 
> undocumented I expected its use outside the tree to be limited.
>
> The reason I decided to do this is that it is practically always a 
> mistake to rely on recursion: for each mutex, you should know whether 
> or not you hold it at each point when you need access to the resource 
> it protects, and you should know what other locks are held in order to 
> determine a consistent lock order.
>
> Utility routines whose callers might or might not hold the lock are 
> suspect and tend to suggest insufficiently considered design:  What is 
> the invariant the lock provides?  If the caller doesn't need to hold 
> the lock, why does the lock need to be there at all?
>
> There are legitimate cases where recursive locks make sense, but most 
> uses are best served by a nonrecursive lock abstraction.  You can 
> always build a recursive lock out of a nonrecursive lock and a condition 
> variable.
>
> _______________________________________________
> MIT-Scheme-devel mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/mit-scheme-devel
>
>
> _______________________________________________
> MIT-Scheme-devel mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/mit-scheme-devel