gluster-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Gluster-devel] [PATCH v7] vfs_glusterfs: Samba VFS module for glust


From: Anand Avati
Subject: Re: [Gluster-devel] [PATCH v7] vfs_glusterfs: Samba VFS module for glusterfs
Date: Wed, 29 May 2013 04:21:04 -0700
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130509 Thunderbird/17.0.6

On 5/28/13 7:50 PM, Andrew Bartlett wrote:
On Tue, 2013-05-28 at 18:33 -0700, Anand Avati wrote:



On Tue, May 28, 2013 at 4:47 PM, Anand Avati <address@hidden> wrote:
         On 05/28/2013 03:42 PM, Anand Avati wrote:




                 On Tue, May 28, 2013 at 2:46 PM, Andrew Bartlett
                 <address@hidden

                 <mailto:address@hidden>> wrote:

                     On Tue, 2013-05-28 at 17:36 -0400, Anand Avati
                 wrote:
                      > Implement a Samba VFS plugin for glusterfs
                 based on gluster's gfapi.
                      > This is a "bottom" vfs plugin (not something to
                 be stacked on top of
                      > another module), and translates (most) calls
                 into closest actions
                      > on gfapi.

                     Thank you for your patience here.

                      > Signed-off-by: Anand Avati <address@hidden

                     <mailto:address@hidden>>

                      > ---
                      >
                      > Hi,
                      >
                      > I hope the waf changes are fine. I could not
                 test them because
                     builds were failing generally on samba.git master
                 HEAD.

                     You need to resolve whatever is causing your
                 problems here, as it is not
                     a general issue. Our autobuild system ensures that
                 git master and
                     v4-0-test always compile and test in at least
                 Ubuntu 10.04, and it is
                     incredibly rare that it not work on at least Linux
                 without it being a
                     specifically local issue.

                     +
                     +bld.SAMBA3_MODULE('vfs_glusterfs',
                     +                  subsystem='vfs',
                     +                  source=VFS_GLUSTERFS_SRC,
                     +                  deps='samba-util',
                     +                  init_function='',
                     +

                 internal_module=bld.SAMBA3_IS_STATIC_MODULE('vfs_glusterfs'),
                     +

                 enabled=bld.SAMBA3_IS_ENABLED_MODULE('vfs_glusterfs'))

                     You should ensure your module depends on
                 glusterfs, by changing that to
                     adding deps='glusterfs samba-util' (and making
                 that match what the
                     conf.check_cfg finds and stores).


                 I tried that. I added uselib_store='glusterfs' and
                 included 'glusterfs'
                 in deps= line. However while building I get this
                 error:

                 address@hidden samba]$ make
                 WAF_MAKE=1 python ./buildtools/bin/waf build
                 ./buildtools/wafsamba/samba_utils.py:397:
                 DeprecationWarning: the md5
                 module is deprecated; use hashlib instead
                    import md5
                 Waf: Entering directory `/home/avati/work/samba/bin'
                 Selected embedded Heimdal build
                 Checking project rules ...
                 Unknown dependency 'glusterfs' in 'vfs_glusterfs'
                 make: *** [all] Error 1


                 I am not familiar with waf, and trying to figure out
                 how it works. I
                 don't understand how the check_cfg() in wscript and
                 source3/wscript_build's bld.SAMBA3_MODULE() actually
                 integrate internally.

                      > diff --git a/source3/wscript b/source3/wscript
                      > index dba6cdc..0d07692 100644
                      > --- a/source3/wscript
                      > +++ b/source3/wscript
                      > @@ -59,6 +59,11 @@ def set_options(opt):
                      >                     help=("Directory under
                 which libcephfs is
                     installed"),
                      >                     action="store",
                 dest='libcephfs_dir',
                     default=None)
                      >
                      > +    opt.add_option('--enable-glusterfs',
                      > +                   help=("Enable building
                 vfs_glusterfs module"),
                      > +                   action="store_true",
                 dest='enable_glusterfs',
                     default=True)
                      > +    opt.add_option('--disable-glusterfs',
                 help=SUPPRESS_HELP,
                      > +                   action="store_false",
                 dest='enable_glusterfs')

                     In general we would prefer not to have options for
                 every possible
                     module.  Instead, please just make it automatic
                 based on finding
                     glusterfs.


                 OK, I will just leave things to check_cfg() detection
                 then?

                      >  def configure(conf):
                      > @@ -1709,6 +1714,13 @@ main() {
                      >      if
                 conf.CHECK_HEADERS('cephfs/libcephfs.h', False, False,
                     'cephfs') and conf.CHECK_LIB('cephfs'):
                      >          conf.DEFINE('HAVE_CEPH', '1')
                      >
                      > +    conf.env.enable_glusterfs =
                 Options.options.enable_glusterfs
                      > +
                      > +    if Options.options.enable_glusterfs:
                      > +
                  conf.check_cfg(package='glusterfs-api',
                     args='"glusterfs-api >= 4" --cflags --libs',
                      > +
                 uselib_store='GLUSTERFS', msg='Checking
                     for glusterfs-api >= 4',
                      > +                       mandatory=False)
                      > +
                      >      conf.env.build_regedit = False
                      >      if not Options.options.with_regedit ==
                 False:
                      >
                  conf.PROCESS_SEPARATE_RULE('system_ncurses')
                      > @@ -1797,6 +1809,9 @@ main() {
                      >      if conf.CONFIG_SET("HAVE_CEPH"):
                      >
                  default_shared_modules.extend(TO_LIST('vfs_ceph'))
                      >
                      > +    if conf.CONFIG_SET('HAVE_GLUSTERFS'):
                      > +
                  default_shared_modules.extend(TO_LIST('vfs_glusterfs'))
                      > +
                      >      explicit_shared_modules =
                     TO_LIST(Options.options.shared_modules,
                 delimiter=',')
                      >      explicit_static_modules =
                     TO_LIST(Options.options.static_modules,
                 delimiter=',')

                     This much looks good, assuming it is tested and
                 works.   You might need
                     to make uselib_store='glusterfs' (not sure).


                 As mentioned above, I tried this but still get the
                 dependency error. Not
                 sure what I am missing..



         I tried to look up documentation on the web, but I couldn't
         find anything specific. My understanding so far is:

         conf.check_cfg(...) does the 'detection' and stores the result
         in variables. You can specify a variable name prefix with
         uselib_store=NAME parameter.

         You can then use the test results while building, with a
         construct like:

         bld.new_task_gen(..., uselib=NAME, ...)

         However, samba is using bld.SAMBA3_MODULE(...) for building,
         and that does not seem to be accepting uselib= parameter.

         What I am trying to achieve is:

         - detect for presence of glusterfs headers/libs in the
         buildsystem. This is detected with the presence of
         glusterfs-api.pc

         - use the --libs and --cflags from pkg-config on
         glusterfs-api.pc to build vfs_glusterfs.c

         I know how to do this in autotools. I am not familiar with
         waf, and it doesn't seem to be intuitive enough to be self
         understood. There seems to be quite a lot of macro hackery
         around it which makes the techniques you get from a google
         search "not applicable".

         Any help from the waf experts will be most appreciated!


Looks like the issue was that there had to be atleast one call to
conf.CHECK_HEADER in context of the package (unlike autotools where
this doesn't seem to be necessary). After adding a line to
CHECK_HEADERS (in v8 patch) everything else just worked.

That shouldn't be needed, but at this point I don't want to hold you up
any longer.  If you can happen to point me at Fedora 18 packages and/or
instructions I'll do an incremental patch to fix it further, not because
it matters massively for glusterfs, but because this will I think be the
canonical example for similar requirements in the future, where we know
that there must be a pkg-config.

It turned out CHECK_HEADER wasn't the reason. I was changing two variables at a time and came to the wrong conclusion. The reason seems to be that I need to explicitly do conf.CHECK_LIB() in order to make things work. I checked the source of CHECK_LIB and also the code which prints 'Unresolved dependency' error. The latter checks for SET_TARGET_TYPE and the former does the SET_TARGET_TYPE, so it seems pretty clear why the error was coming (not obvious from the surface at all).

I am sending out a v9 patch (hopefully final) and we can probably figure out better wscript integration later?

You can install the following glusterfs RPMs on a Fedora 18 system and this will let samba.git master branch build vfs_glusterfs.so:

http://shell.gluster.com/~avati/glusterfs-smb/

Thanks,
Avati

The only other thing is: do we really need the configure option?

Even with these concerns, I'm happy at this point for you to consider
'[PATCH v8] vfs_glusterfs: Samba VFS module for glusterfs' to be

Reviewed-by: Andrew Bartlett <address@hidden>

Thanks for your patience, and I'm sure we can address these final issues
in the next little while, without blocking getting this into master.

Andrew Bartlett





reply via email to

[Prev in Thread] Current Thread [Next in Thread]