[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Gluster-devel] Crawling and indexing hardware
From: |
Daniel Maher |
Subject: |
Re: [Gluster-devel] Crawling and indexing hardware |
Date: |
Fri, 9 May 2008 10:37:50 +0200 |
On Wed, 7 May 2008 20:06:40 +0200 "Marcus Herou"
<address@hidden> wrote:
> 1. Big index files ~x Gig each
> 2. Many small files in a huge amount of directories.
Do you plan to do any AFR (automatic file replication) ? If so,
consider that even a one-byte change to your "big index files" will
cause the /entire/ file to be AFR'd between all participating nodes.
> Finally what tools would suite to test zillions of small files ?
> Bonnie++ ? Fewer big files ? Still Bonnie++ or perhaps IOZone ?
IOZone is an interesting tool, assuming you can interpret the
results. :P I have been using Bonnie++ and FFSB extensively over the
past couple of weeks to stresstest / benchmark Gluster. Both have the
advantage of producing easily interpretable results, and FFSB is highly
configurable, depending on what sort of tests you'd like to run (read /
write / both, small / large files, lots / few files, etc..).
The following page contains some sample FFSB configs to work from :
http://tastic.brillig.org/~jwb/zfs-xfs-ext4.html
(see "Step 8".)
Cheers !
--
Daniel Maher <dma AT witbe.net>
- [Gluster-devel] Crawling and indexing hardware, Marcus Herou, 2008/05/07
- Re: [Gluster-devel] Crawling and indexing hardware,
Daniel Maher <=
- Re: [Gluster-devel] Crawling and indexing hardware, Marcus Herou, 2008/05/09
- Re: [Gluster-devel] Crawling and indexing hardware, Krishna Srinivas, 2008/05/09
- Re: [Gluster-devel] Crawling and indexing hardware, Daniel Maher, 2008/05/09
- Re: [Gluster-devel] Crawling and indexing hardware, gordan, 2008/05/09
- Re: [Gluster-devel] Crawling and indexing hardware, Daniel Maher, 2008/05/09
- Re: [Gluster-devel] Crawling and indexing hardware, Kevan Benson, 2008/05/09
- Re: [Gluster-devel] Crawling and indexing hardware, Marcus Herou, 2008/05/09
- Re: [Gluster-devel] Crawling and indexing hardware, Anand Avati, 2008/05/09
- Re: [Gluster-devel] Crawling and indexing hardware, Krishna Srinivas, 2008/05/09
- Re: [Gluster-devel] Crawling and indexing hardware, Marcus Herou, 2008/05/09