Re: From Gnus to mu4e

[Suvayu Ali]

On Thu, Aug 27, 2015 at 05:24:33PM -0700, Ian Zimmerman wrote:
> On 2015-08-28 02:14 +0200, Suvayu Ali wrote:
> 
> > No, it's not.  AFAIK on most (all?) *nix filesystems, basic operations
> > are atomic.  See for example this (outdated) list:
> > 
> > http://rcrowley.org/2010/01/06/things-unix-can-do-atomically.html
> > 
> > E.g., this breaks down when you talk about network mounts like NFS (not
> > sure about SSHFS), but it is a valid assumption as long as your storage
> > is local.
> 
> Assume there's a message ~/Mail/inbox/cur/1440718280.10956_2.ahiker:2,S
> 
> MUA 1 wants to set the "tagged" flag, which means renaming
> 
> 1440718280.10956_2.ahiker:2,S -> 1440718280.10956_2.ahiker:2,FS
> 
> MUA 2 wants to set the "replied" flag, which means renaming
> 
> 1440718280.10956_2.ahiker:2,S -> 1440718280.10956_2.ahiker:2,RS
> 
> Only one of them can succeed, depending on the order they try.  And
> final state also depends on that order.

I think you are confusing failures on the MUA side with race conditions
leading to email corruption.  Take your example above, when MUA 1 flags
a message, and succeeds, atomicity of filesystem operation ensures
setting the reply flag by MUA 2 will fail.  It is up to MUA 2 to handle
this failure.  The maildir is guaranteed to be in a consistent state by
the atomicity of filesystem operations.

When MUA 2 fails, it is easily handled by reporting to the user and not
trying to commit the changes again.  E.g. in mutt, this is handled by
telling the user something like: file does not exist (I don't recall the
exact phrasing), and keeping the folder state as is.  The user now has
to do two things, mark the folder read-only (so mutt stops trying to
write state changes back to the maildir), and reread the present
maildir.  During all this, your maildir has _not_ been corrupted in any
way, and both MUAs are reporting things correctly.

It is an entirely different story however if your maildir is over NFS
(and possibly SSHFS, actually maybe this is true for any network based
filesystem).  In that case, say the MUAs are accessing the maildir from
different locations.  MUA 1 writes the "flagging" action, the NFS client
commits the change.  However the real files on disk have not been
changed, and when MUA 2 tries to write its "replying" action, it
succeeds!  Now you have a race condition, MUA 1 thinks the message is
flagged, MUA 2 thinks it's read.  And no one knows what it is in reality
other than the NFS daemon.  Technically the email is not corrupted, only
the meta information about the flag is.  This is easily recoverable by
quiting both MUAs.  And essentially this is the beauty of the maildir
format, data integrity.

Hopefully I have explained myself clearly enough.  Or maybe you were
trying to say something else?  In that case please feel free to stop me.

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.