Re: [Gluster-devel] Problems with ioc again

[Dan Parsons]

I'd love to use 1.4rc4 but are there any issues in it that would  
effect me? I have 4 glusterfs servers, each with 2gbit ethernet  
(bonded), provididing sustained 8gbit/s to 33 client nodes. Below is  
my entire config file. If you see anything in there using a system  
that is either buggy or non-optimal in 1.4rc4, or would be difficult  
to upgrade, please let me know. If not, I can possibly upgrade.

Below is my current config file. The one I was using when gluster was  
using all memory is identical except for 'cache-size' was changed to  
4096MB and 'page-size' was changed to 512KB.

-----------
### Add client feature and attach to remote subvolume of server1
volume distfs01
 type protocol/client
 option transport-type tcp/client     # for TCP/IP transport
 option remote-host 10.8.101.51      # IP address of the remote brick
 option remote-subvolume brick        # name of the remote volume
end-volume

### Add client feature and attach to remote subvolume of server2
volume distfs02
 type protocol/client
 option transport-type tcp/client     # for TCP/IP transport
 option remote-host 10.8.101.52      # IP address of the remote brick
 option remote-subvolume brick        # name of the remote volume
end-volume

volume distfs03
 type protocol/client
 option transport-type tcp/client
 option remote-host 10.8.101.53
 option remote-subvolume brick
end-volume

volume distfs04
 type protocol/client
 option transport-type tcp/client
 option remote-host 10.8.101.54
 option remote-subvolume brick
end-volume

volume stripe0
  type cluster/stripe
  option block-size *.gff:1KB,*.nt:1KB,*.best: 
1KB,*.txt3:1KB,*.nbest.info:1KB*:1MB
  option scheduler alu
  option alu.order read-usage:write-usage:disk-usage
  option alu.read-usage.entry-threshold 20%
  option alu.read-usage.exit-threshold 4%
  option alu.write-usage.entry-threshold 20%
  option alu.write-usage.exit-threshold 4%
  option alu.disk-usage.entry-threshold 2GB
  option alu.disk-usage.exit-threshold 100MB
  subvolumes distfs01 distfs02 distfs03 distfs04
end-volume


volume ioc
  type performance/io-cache
  subvolumes stripe0         # In this example it is 'client' you may  
have to change it according to your spec file.
  option page-size 1MB      # 128KB is default
  option cache-size 2048MB    # 32MB is default
  option force-revalidate-timeout 5 # 1second is default
  option priority *.psiblast:3,*.seq:2,*:1
end-volume

volume fixed
  type features/fixed-id
  option fixed-uid 0
  option fixed-gid 900
  subvolumes ioc
end-volume

Dan Parsons


On Dec 17, 2008, at 2:09 PM, Anand Avati wrote:

> Dan,
> Is it feasible for you to try 1.4.0pre4? We have fixed a couple of
> ioc leaks in that branch. Backporting them to the 1.3.x tree is a bit
> hard for us at the moment since all the developers are maxed out on
> tasks. Please let us know if there is no way around than upgrading to
> 1.4rc for you.
>
> Avati
>
> 2008/12/18 Dan Parsons <address@hidden>:
> > I am once more having a problem with the ioc translator.  
> > Specifically, I
> > upgraded the RAM in my gluster client from 4GB to 12GB. Under the 4GB
> > configuration, I had cache-size set to 2048MB. That was working  
> > alright.
> > Once I upgraded the server to 12GB, I changed cache-size to 4GB.  
> > After doing
> > that, glusterfs proceeded to use every last drop of RAM, and all  
> > swap, on
> > the system. Not right away of course, only after I read about,  
> > well, 12GB of
> > data.
> >
> > This behavior was repeated across all 33 of my gluster clients.  
> > Even back
> > when it was set to 2GB, there were times when it would ignore that  
> > setting,
> > and if you look through the mailing list, you'll find at least 2  
> > people that
> > had the same problem, and didn't get much support.
> >
> > The ioc translator is an extremely essential part to the operation  
> > of my
> > department. I really need this fixed ASAP- if there is anyone I can  
> > pay to
> > do this, please contact me off-list, as it needs to be done ASAP.
> >
> > address@hidden conf]# glusterfs --version
> > glusterfs 1.3.11 built on Aug 21 2008 11:26:38
> > Repository revision: glusterfs--mainline--2.5--patch-795
> >
> > volume ioc
> > type performance/io-cache
> > subvolumes stripe0         # In this example it is 'client' you may  
> > have to
> > change it according to your spec file.
> > option page-size 1MB      # 128KB is default
> > option cache-size 2048MB    # 32MB is default
> > option force-revalidate-timeout 5 # 1second is default
> > option priority *.psiblast:3,*.seq:2,*:1
> > end-volume
> >
> >
> >
> > Dan Parsons
> >
> >
> >
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/gluster-devel