Re: [Gluster-devel] [Gluster-users] RPMs for Samba 4.1.3 w/ Gluster VFS plug-in for RHEL, CentOS, etc., now available

[Dan Mons]

Thanks for that, Ira.

So far the combined effect of:

        kernel oplocks = no
        oplocks = yes
        level2 oplocks = yes

on a FUSE-mounted GlusterFS share appears to keep everything happy.
So far no data corruption, and a mostly-happy user base.  That doesn't
work with vfs_glusterfs, and other combinations result in locking
problems (again, Photoshop, Word and Excel are the three most
troublesome).

A real bummer too, because vfs_glusterfs is noticeably faster,
particularly at directory traversal.

-Dan

----------------
Dan Mons
Skunk Works
Cutting Edge
http://cuttingedge.com.au


On 17 February 2014 13:12, Ira Cooper <address@hidden> wrote:
> Fake oplocks opens up the possibility of file corruption.  You'd be better 
> off disabling oplocks if you care about your data.
>
> Thanks,
>
> -Ira
>
> ----- Original Message -----
>> More testing of vfs_glusterfs reveals problems with Microsoft and
>> Adobe software, and their silly locking.  (For example, Microsoft
>> Excel opening a .xlsx, or Adobe Photoshop opening a .jpg or .psd).
>> Dozens of other production applications we use are all fine, for what
>> it's worth (the bigger the vendor, the more likely it seems their
>> software is broken.  How amusing).
>>
>> The following config (without vfx_glusterfs) is the only way I can
>> make these particular applications play ball.  Still playing with
>> "fake oplocks" as well to see what broader effect that has.
>>
>> [prodbackup]
>>         vfs object = streams_xattr
>>         path = /prodbackup
>>         Comment = prodbackup
>>         browseable = yes
>>         writable = yes
>>         guest ok = no
>>         valid users = +prod
>>         create mask = 0660
>>         force create mode = 0660
>>         directory mask = 0770
>>         force directory mode = 0770
>>         hide dot files = no
>>         security mask = 0660
>>         force security mode = 0660
>>         directory security mask = 0770
>>         force directory security mode = 0770
>>         kernel share modes = no
>>         kernel oplocks = no
>>         ea support = yes
>>         oplocks = yes
>>         level2 oplocks = yes
>>
>> -Dan
>>
>> ----------------
>> Dan Mons
>> Skunk Works
>> Cutting Edge
>> http://cuttingedge.com.au
>>
>>
>> On 17 February 2014 07:58, Dan Mons <address@hidden> wrote:
>> > Thank you Daniel and Lala,
>> >
>> > "kernel share modes = no" was the magic incantation.  This is working
>> > nicely for me now.  Much appreciated.
>> >
>> > -Dan
>> >
>> > ----------------
>> > Dan Mons
>> > Skunk Works
>> > Cutting Edge
>> > http://cuttingedge.com.au
>> >
>> >
>> > On 14 February 2014 16:52, Daniel Müller <address@hidden> wrote:
>> >> HI again,
>> >>
>> >> I had the same issue. I got it working by:
>> >> Adding -- kernel share modes = No to the sahres
>> >> EX.:
>> >> [home]
>> >> comment=gluster test
>> >> vfs objects=glusterfs
>> >> glusterfs:volume= sambacluster
>> >> glusterfs:volfile_server = 172.17.1.1
>> >> path=/ads/home
>> >> read only=no
>> >> posix locking =NO
>> >> kernel share modes = No
>> >>
>> >> By the way running Centos6.4 Samba 4.1.4, gluster 3.4.1qa2.
>> >>
>> >>
>> >> Good Luck
>> >> Daniel
>> >>
>> >> EDV Daniel Müller
>> >>
>> >> Leitung EDV
>> >> Tropenklinik Paul-Lechler-Krankenhaus
>> >> Paul-Lechler-Str. 24
>> >> 72076 Tübingen
>> >> Tel.: 07071/206-463, Fax: 07071/206-499
>> >> eMail: address@hidden
>> >> Internet: www.tropenklinik.de
>> >> "Der Mensch ist die Medizin des Menschen"
>> >>
>> >>
>> >>
>> >>
>> >> Von: address@hidden
>> >> [mailto:address@hidden Im Auftrag von Lalatendu
>> >> Mohanty
>> >> Gesendet: Freitag, 14. Februar 2014 07:25
>> >> An: Dan Mons; Kaleb KEITHLEY
>> >> Cc: address@hidden; Jose Rivera; Gluster Devel
>> >> Betreff: Re: [Gluster-users] [Gluster-devel] RPMs for Samba 4.1.3 w/
>> >> Gluster
>> >> VFS plug-in for RHEL, CentOS, etc., now available
>> >>
>> >> On 02/14/2014 06:37 AM, Dan Mons wrote:
>> >> Hi,
>> >>
>> >> This is failing for me.  I've had the same problems after trying to build
>> >> my
>> >> own vfs_glusterfs from source.  I'm certain I'm doing something stupid.
>> >>
>> >> Client is a Windows Server 2008R2 64bit machine with AD authentication.
>> >>
>> >> Server is CentOS 6.4, Gluster 3.4.1GA, Samba 4.1.4 with matching
>> >> samba_vfs_glusterfs as per this thread, AD authentication (sssd for the
>> >> Linux/PAM/nsswitch side, and Samba is configured as a member server).  The
>> >> Gluster volume is working fine (this is our production test/backup
>> >> cluster,
>> >> and has been in operation for over a year).
>> >>
>> >> Samba works fine when pointing to a local FUSE mount (this is how we run
>> >> in
>> >> production today for Windows and Mac clients).  When I change to
>> >> vfs_glusterfs, it all goes wrong.
>> >>
>> >> Client errors include:
>> >>
>> >> Action: On Windows regular windows explorer directory browsing
>> >> Result: All good.  Much faster than regular Samba to FUSE.
>> >>
>> >> Action: On Windows: Right click -> New ->  Text Document
>> >> Result: "Unable to create the file "New Text Document.txt".  The system
>> >> cannot find the file specified.
>> >>
>> >> Action:
>> >> On a Linux box: dmesg > test.txt
>> >> On Windows: double-click test.txt
>> >> Result: The process cannot access the file because it is in use by another
>> >> process
>> >>
>> >> Action: On Windows: drag and drop a text file to a share
>> >> Result: nothing (file not copied, no error dialog).
>> >>
>> >> Samba logs:
>> >> [2014/02/14 10:44:52.972999,  0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >>   prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:46:31.020793,  0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >>   glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:47:03.326100,  0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >>   prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:47:08.449040,  0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >>   glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:48:21.007241,  0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >>   prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:48:21.068066,  0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >>   glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:51:36.683883,  0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >>   prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:51:36.743577,  0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >>   glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:53:14.160588,  0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >>   glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:53:57.229060,  0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >>   prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:53:57.288750,  0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >>   glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:54:47.062171,  0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >>   prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:54:47.121809,  0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >>   glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:55:16.602058,  0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >>   prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:55:16.670562,  0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >>   glfs_stat(./..) failed: No data available
>> >>
>> >>
>> >>
>> >> The log seems similar to bug
>> >> https://bugzilla.redhat.com/show_bug.cgi?id=1062674
>> >>
>> >> Please put "kernel share modes = No" for the shares and let us know if it
>> >> works for you.
>> >>
>> >> -Lala
>> >>
>> >> /etc/samba/smb.conf:
>> >>
>> >> [global]
>> >>         workgroup = BLAH
>> >>         server string = Samba Server Version %v
>> >>         log file = /var/log/samba/log.%m
>> >>         max log size = 50
>> >>         security = ads
>> >>         passdb backend = tdbsam
>> >>         realm = BLAH
>> >>         domain master = no
>> >>         domain logons = no
>> >>         wins support = yes
>> >>         wins proxy = yes
>> >>         dns proxy = yes
>> >>         load printers = no
>> >> ## Shares
>> >> [prodbackup]
>> >>         vfs object = glusterfs
>> >>         glusterfs:volume = prodbackup
>> >>         glusterfs:volfile_server = localhost
>> >>         path = /
>> >>         Comment = prodbackup
>> >>         browseable = yes
>> >>         writable = yes
>> >>         guest ok = no
>> >>         valid users = +prod
>> >>         create mask = 0660
>> >>         force create mode = 0660
>> >>         directory mask = 0770
>> >>         force directory mode = 0770
>> >>         hide dot files = no
>> >>
>> >> Any pointers would be appreciated.
>> >>
>> >> -Dan
>> >>
>> >>
>> >>
>> >> ----------------
>> >> Dan Mons
>> >> Skunk Works
>> >> Cutting Edge
>> >> http://cuttingedge.com.au
>> >>
>> >> On 7 February 2014 07:12, Dan Mons <address@hidden> wrote:
>> >> Fantastic work, thank you.
>> >>
>> >> -Dan
>> >> ----------------
>> >> Dan Mons
>> >> Skunk Works
>> >> Cutting Edge
>> >> http://cuttingedge.com.au
>> >>
>> >>
>> >> On 6 February 2014 22:50, Kaleb KEITHLEY <address@hidden> wrote:
>> >>>
>> >>> YUM repo at http://download.gluster.org/pub/gluster/glusterfs/samba/ now
>> >> has
>> >>> el6 RPMs for RHEL6, CentOS6; along with RPMs for Fedora 18 and Fedora 19.
>> >>>
>> >>> Fedora 20 and Fedora 21 (rawhide) have Samba 4.1.3 by default.
>> >>>
>> >>> --
>> >>>
>> >>> Kaleb
>> >>> _______________________________________________
>> >>> Gluster-users mailing list
>> >>> address@hidden
>> >>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Gluster-devel mailing list
>> >> address@hidden
>> >> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>> >>
>> >>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>>