gluster-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Gluster-devel] Performance question.


From: Anand Avati
Subject: Re: [Gluster-devel] Performance question.
Date: Wed, 21 Nov 2007 22:03:12 +0530

>
>
>       Caching won't help in the real appication I don't believe.
> Mostly it's read, crunch, write.  If I'm wrong here please let me
> know.  Although I don't believe it will hurt.  I'll give moving
> write-behind and io-cache to the client and see what happens.  Does it
> matter how they're stacked, i.e. the which comes first?


the order of them on the client side should not matter. (atleast with their
current featuresets)

avati


> You should also be loading io-cache on the client side with a decent
> > cache-size (like 256MB? depends on how much RAM you have to spare). this
> > will help re-read improve a lot.
> >
> > avati
> >
> > 2007/11/21, Anand Avati <address@hidden>:
> >>
> >> Chris,
> >>  you shoud really be loading write-behind on the client side, that is
> wht
> >> improves write performance the most. do let us know the results with
> >> writebehind on the client side.
> >>
> >> avati
> >>
> >> 2007/11/21, Chris Johnson <address@hidden>:
> >>>
> >>>       Hi, again,
> >>>
> >>>       I asked about stack building philosophy.  Apparently there isn't
> >>> one.  So I tried a few things.  The configs are down the end here.
> >>>
> >>>       Two systems, CentOS5, both running fuse-devel-2.7.0-1 gluster
> >>> enhanced, glusterfs-1.3.5-2.  Both have gigabit ethernet, server runs
> >>> a SATABeast.  Currently I ge the following from from iozone.
> >>>
> >>> iozone -aN -r 32k -s 131072k -f /mnt/glusterfs/sdm1/junknstuff
> >>>
> >>>
> >>> random  random    bkwd  record  stride
> >>>                KB  reclen   write rewrite    read    reread    read
> >>> write    read rewrite    read   fwrite frewrite   fread  freread
> >>>            131072      32     589     587      345      343     818
> >>> 621     757     624     845      592      591     346      366
> >>>
> >>> Now, a similar test using NFS on a CentOS4.4 system running a 3ware
> >>> RAID card gives this
> >>>
> >>> iozone -aN -r 32k -s 131072k -f /space/sake/5/admin/junknstuff
> >>>
> >>>
> >>> random  random    bkwd  record  stride
> >>>                KB  reclen   write rewrite    read    reread    read
> >>> write    read rewrite    read   fwrite frewrite   fread  freread
> >>>            131072      32      27      26      292
> >>> 11      11      24     542       9     539       30       28     295
> >>> 11
> >>>
> >>> And you can see that the NFS system is faster.  Is this because of the
> >>> hardware 3ware RAID or is NFS really that much faster here?  Is there
> >>> a better way to stack this that would improve things?  And I tried
> with
> >>> and without striping.  No noticable difference in gluster performance.
> >>>
> >>>       Help appreciated.
> >>>
> >>> ============  server config
> >>>
> >>> volume brick1
> >>>    type storage/posix
> >>>    option directory /home/sdm1
> >>> end-volume
> >>>
> >>> volume brick2
> >>>    type storage/posix
> >>>    option directory /home/sdl1
> >>> end-volume
> >>>
> >>> volume brick3
> >>>    type storage/posix
> >>>    option directory /home/sdk1
> >>> end-volume
> >>>
> >>> volume brick4
> >>>    type storage/posix
> >>>    option directory /home/sdk1
> >>> end-volume
> >>>
> >>> volume ns-brick
> >>>    type storage/posix
> >>>    option directory /home/sdk1
> >>> end-volume
> >>>
> >>> volume stripe1
> >>>   type cluster/stripe
> >>>   subvolumes brick1 brick2
> >>> # option block-size *:10KB,
> >>> end-volume
> >>>
> >>> volume stripe2
> >>>   type cluster/stripe
> >>>   subvolumes brick3 brick4
> >>> # option block-size *:10KB,
> >>> end-volume
> >>>
> >>> volume unify0
> >>>   type cluster/unify
> >>>   subvolumes stripe1 stripe2
> >>>   option namespace ns-brick
> >>>   option scheduler rr
> >>> # option rr.limits.min-disk-free 5
> >>> end-volume
> >>>
> >>> volume iot
> >>>   type performance/io-threads
> >>>   subvolumes unify0
> >>>   option thread-count 8
> >>> end-volume
> >>>
> >>> volume writebehind
> >>>    type performance/write-behind
> >>>    option aggregate-size 131072 # in bytes
> >>>    subvolumes iot
> >>> end-volume
> >>>
> >>> volume readahead
> >>>    type performance/read-ahead
> >>> #  option page-size 65536 ### in bytes
> >>>    option page-size 128kb ### in bytes
> >>> #  option page-count 16 ### memory cache size is page-count x
> >>> page-size per file
> >>>    option page-count 2 ### memory cache size is page-count x page-size
> >>> per file
> >>>    subvolumes writebehind
> >>> end-volume
> >>>
> >>> volume server
> >>>    type protocol/server
> >>>    subvolumes readahead
> >>>    option transport-type tcp/server     # For TCP/IP transport
> >>> #  option client-volume-filename /etc/glusterfs/glusterfs-client.vol
> >>>    option auth.ip.readahead.allow *
> >>> end-volume
> >>>
> >>>
> >>> ============  client config
> >>>
> >>> volume client
> >>>    type protocol/client
> >>>    option transport-type tcp/client
> >>>    option remote-host xxx.xxx.xxx.xxx
> >>>    option remote-subvolume readahead
> >>> end-volume
> >>>
> >>>
> -------------------------------------------------------------------------------
> >>>
> >>> Chris Johnson               |Internet: address@hidden
> >>> Systems Administrator       |Web:
> http://www.nmr.mgh.harvard.edu/~johnson
> >>> <http://www.nmr.mgh.harvard.edu/%7Ejohnson>
> >>> NMR Center                  |Voice:    617.726.0949
> >>> Mass. General Hospital      |FAX:      617.726.7422
> >>> 149 (2301) 13th Street      |A compromise is a solution nobody is
> happy
> >>> with.
> >>> Charlestown, MA., 02129 USA |     Observation, Unknown
> >>>
> >>>
> -------------------------------------------------------------------------------
> >>>
> >>>
> >>> _______________________________________________
> >>> Gluster-devel mailing list
> >>> address@hidden
> >>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
> >>>
> >>
> >>
> >>
> >> --
> >> It always takes longer than you expect, even when you take into account
> >> Hofstadter's Law.
> >>
> >> -- Hofstadter's Law
> >
> >
> >
> >
> > --
> > It always takes longer than you expect, even when you take into account
> > Hofstadter's Law.
> >
> > -- Hofstadter's Law
> >
>
>
> -------------------------------------------------------------------------------
> Chris Johnson               |Internet: address@hidden
> Systems Administrator       |Web:
> http://www.nmr.mgh.harvard.edu/~johnson
> NMR Center                  |Voice:    617.726.0949
> Mass. General Hospital      |FAX:      617.726.7422
> 149 (2301) 13th Street      |For all sad words of tongue or pen, the
> saddest
> Charlestown, MA., 02129 USA |are these: "It might have been".  John G.
> Whittier
>
> -------------------------------------------------------------------------------
>



-- 
It always takes longer than you expect, even when you take into account
Hofstadter's Law.

-- Hofstadter's Law


reply via email to

[Prev in Thread] Current Thread [Next in Thread]