Re: [Gluster-devel] mysql on glusterfs

[Jeff Humes]

Anand,

I put back the TLA version:

SERVER:
address@hidden <mailto:address@hidden> build]# glusterfsd --version
glusterfs 1.3.8 built on Nov 23 2007

CLIENT:
address@hidden <mailto:address@hidden> build]# glusterfs --version
glusterfs 1.3.8 built on Nov 23 2007

# command run:
glusterfs --no-daemon -l /dev/stdout -f 
/etc/glusterfs/glusterfs-client_knworksmysql.vol /var/lib/mysql

#mountpoint on client:
glusterfs on /var/lib/mysql type fuse 
(rw,nosuid,nodev,allow_other,default_permissions,max_read=1048576)

# first start:
address@hidden <mailto:address@hidden> ~]# /etc/init.d/mysqld start
Starting MySQL:  [  OK  ]
# and stop:
address@hidden <mailto:address@hidden> ~]# /etc/init.d/mysqld stop
Stopping MySQL:  [  OK  ]

# second start:
Starting MySQL:  [  FAILED  ]

# MySQL log:
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
071123 23:28:23  InnoDB: Unable to open the first data file
InnoDB: Error in opening ./ibdata1
071123 23:28:23  InnoDB: Operating system error number 11 in a file 
operation.
InnoDB: Error number 11 means 'Resource temporarily unavailable'.
InnoDB: Some operating system error numbers are described at
InnoDB: 
http://dev.mysql.com/doc/refman/5.0/en/operating-system-error-codes.html
InnoDB: Could not open or create data files.
InnoDB: If you tried to add new data files, and it failed here,
InnoDB: you should now edit innodb_data_file_path in my.cnf back
InnoDB: to what it was, and remove the new ibdata files InnoDB created
InnoDB: in this failed attempt. InnoDB only wrote those files full of
InnoDB: zeros, but did not yet use them in any way. But be careful: do not
InnoDB: remove old data files which contain your precious data!

* *
*# glusterfs log:*
*2007-11-23 23:26:43 E [fuse-bridge.c:2103:fuse_setlk_cbk] 
glusterfs-fuse: 1900: ERR => -1 (11)*
*< same errors clipped>  (101 times)*
*2007-11-23 23:28:23 E [fuse-bridge.c:2103:fuse_setlk_cbk] 
glusterfs-fuse: 2029: ERR => -1 (11)*


-JPH



Anand Avati wrote:
> Jeff,
>  the debug log you have posted does not have any entries about the 
> failures. probably because it has too much of valid entries after the 
> last error. Can you reproduce this without -LDEBUG and post the client 
> log?
>
> avati
>
> 2007/11/24, Jeff Humes <address@hidden <mailto:address@hidden>>:
>
>     *
>     Here is the procedure I used to install:
>
>     # Get source:*
>     cd /usr/src/INSTALL_GLUSTER_UPGRADE
>
>     /usr/local/arch/bin/tla register-archive
>     http://arch.sv.gnu.org/archives/gluster/
>     <http://arch.sv.gnu.org/archives/gluster/>
>     <http://arch.sv.gnu.org/archives/gluster>
>
>     /usr/local/arch/bin/tla get -A address@hidden
>     <mailto:address@hidden>
>     <mailto: address@hidden <mailto:address@hidden>>
>     glusterfs--mainline--2.5 glusterfs
>
>     cd glusterfs
>     ./autogen.sh
>     mkdir build
>     cd build
>     # SERVER ONLY:
>     ../configure --disable-ibverbs
>     # CLIENT ONLY:
>     ../configure --disable-ibverbs --disable-server
>
>     make install
>
>
>
>     *#----------------------------------------
>     Here is the output of the checkout:*
>
>     address@hidden INSTALL_GLUSTER_UPGRADE]# /usr/local/arch/bin/tla
>     get
>     -A address@hidden <mailto:address@hidden>
>     glusterfs--mainline--2.5 glusterfs
>
>     WARNING: no rule found for checking signatures from
>     address@hidden <mailto:address@hidden>
>
>       Consider creating
>     ~/.arch-params/signing/address@hidden
>     <mailto:params/signing/address@hidden>
>       or ~/.arch-params/signing/=default.check
>
>     * from archive cached:
>     address@hidden/glusterfs--mainline--2.5--patch-516
>     <http://address@hidden/glusterfs--mainline--2.5--patch-516>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-517
>     <http://address@hidden/glusterfs--mainline--2.5--patch-517>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-518
>     <http://address@hidden/glusterfs--mainline--2.5--patch-518>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-519
>     <http://address@hidden/glusterfs--mainline--2.5--patch-519>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-520
>     <http://address@hidden/glusterfs--mainline--2.5--patch-520>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-521
>     <http://address@hidden/glusterfs--mainline--2.5--patch-521>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-522
>     <http://address@hidden/glusterfs--mainline--2.5--patch-522>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-523
>     <http://address@hidden/glusterfs--mainline--2.5--patch-523>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-524
>     <http://address@hidden/glusterfs--mainline--2.5--patch-524>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-525
>     <http://address@hidden/glusterfs--mainline--2.5--patch-525>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-526
>     <http://address@hidden/glusterfs--mainline--2.5--patch-526>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-527
>     <http://address@hidden/glusterfs--mainline--2.5--patch-527>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-528
>     <http://address@hidden/glusterfs--mainline--2.5--patch-528>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-529
>     <http://address@hidden/glusterfs--mainline--2.5--patch-529>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-530
>     <http://address@hidden/glusterfs--mainline--2.5--patch-530>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-531
>     <http://address@hidden/glusterfs--mainline--2.5--patch-531>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-532
>     <http://address@hidden/glusterfs--mainline--2.5--patch-532>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-533
>     <http://address@hidden/glusterfs--mainline--2.5--patch-533>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-534
>     <http://address@hidden/glusterfs--mainline--2.5--patch-534>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-535
>     <http://address@hidden/glusterfs--mainline--2.5--patch-535>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-536
>     <http://address@hidden/glusterfs--mainline--2.5--patch-536>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-537
>     <http://address@hidden/glusterfs--mainline--2.5--patch-537>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-538
>     <http://address@hidden/glusterfs--mainline--2.5--patch-538>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-539
>     <http://address@hidden/glusterfs--mainline--2.5--patch-539>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-540
>     <http://address@hidden/glusterfs--mainline--2.5--patch-540>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-541
>     <http://address@hidden/glusterfs--mainline--2.5--patch-541>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-542
>     <http://address@hidden/glusterfs--mainline--2.5--patch-542>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-543
>     <http://address@hidden/glusterfs--mainline--2.5--patch-543>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-544
>     <http://address@hidden/glusterfs--mainline--2.5--patch-544>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-545
>     <http://address@hidden/glusterfs--mainline--2.5--patch-545>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-546
>     <http://address@hidden/glusterfs--mainline--2.5--patch-546>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-547
>     <http://address@hidden/glusterfs--mainline--2.5--patch-547>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-548
>     <http://address@hidden/glusterfs--mainline--2.5--patch-548>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-549
>     <http://address@hidden/glusterfs--mainline--2.5--patch-549>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-550
>     <http://address@hidden/glusterfs--mainline--2.5--patch-550>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-551
>     <http://address@hidden/glusterfs--mainline--2.5--patch-551>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-552
>     <http://address@hidden/glusterfs--mainline--2.5--patch-552>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-553
>     <http://address@hidden/glusterfs--mainline--2.5--patch-553>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-554
>     <http://address@hidden/glusterfs--mainline--2.5--patch-554>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-555
>     <http://address@hidden/glusterfs--mainline--2.5--patch-555>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-556
>     <http://address@hidden/glusterfs--mainline--2.5--patch-556>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-557
>     <http://address@hidden/glusterfs--mainline--2.5--patch-557>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-558
>     <http://address@hidden/glusterfs--mainline--2.5--patch-558>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-559
>     <http://address@hidden/glusterfs--mainline--2.5--patch-559>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-560
>     <http://address@hidden/glusterfs--mainline--2.5--patch-560>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-561
>     <http://address@hidden/glusterfs--mainline--2.5--patch-561>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-562
>     <http://address@hidden/glusterfs--mainline--2.5--patch-562>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-563
>     <http://address@hidden/glusterfs--mainline--2.5--patch-563>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-564
>     <http://address@hidden/glusterfs--mainline--2.5--patch-564>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-565
>     <http://address@hidden/glusterfs--mainline--2.5--patch-565>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-566
>     <http://address@hidden/glusterfs--mainline--2.5--patch-566>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-567
>     <http://address@hidden/glusterfs--mainline--2.5--patch-567>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-568
>     <http://address@hidden/glusterfs--mainline--2.5--patch-568>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-569
>     <http://address@hidden/glusterfs--mainline--2.5--patch-569>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-570
>     <http://address@hidden/glusterfs--mainline--2.5--patch-570>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-571
>     <http://address@hidden/glusterfs--mainline--2.5--patch-571>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-572
>     <http://address@hidden/glusterfs--mainline--2.5--patch-572>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-573
>     <http://address@hidden/glusterfs--mainline--2.5--patch-573>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-574
>     <http://address@hidden/glusterfs--mainline--2.5--patch-574>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-575
>     <http://address@hidden/glusterfs--mainline--2.5--patch-575>
>     * patching for revision:
>     address@hidden/glusterfs--mainline--2.5--patch-576
>     <http://address@hidden/glusterfs--mainline--2.5--patch-576>
>     * making pristine copy
>     * tree version set address@hidden/glusterfs--mainline--2.5
>     <http://address@hidden/glusterfs--mainline--2.5>
>
>
>     **
>
>
>
>     -JPH
>
>
>
>
>     Anand Avati wrote:
>     > Jeff,
>     >  what patchset did you try with?
>     >
>     > avati
>     >
>     > 2007/11/24, Jeff Humes <address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>>:
>     >
>     >     Anand,
>     >
>     >     I also had this same issue after building version checked
>     out from:
>     >     /usr/local/arch/bin/tla register-archive
>     >     http://arch.sv.gnu.org/archives/gluster/
>     >     <http://arch.sv.gnu.org/archives/gluster>
>     >     /usr/local/arch/bin/tla get -A address@hidden
>     <mailto:address@hidden>
>     >     <mailto:address@hidden <mailto:address@hidden>>
>     >     <mailto: address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>>
>     >     glusterfs--mainline--2.5 glusterfs
>     >
>     >
>     >     Should I be checking out mainline--2.6?
>     >
>     >     -JPH
>     >
>     >
>     >     Anand Avati wrote:
>     >     > Jeff,
>     >     >  this bug has been fixed in the TLA version. You could either
>     >     checkout
>     >     > and use the snapshot, or await the next release.
>     >     >
>     >     > avati
>     >     >
>     >     > 2007/11/24, Jeff Humes < address@hidden
>     <mailto:address@hidden> <mailto:address@hidden
>     <mailto:address@hidden>>
>     >     <mailto:address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>>>:
>     >     >
>     >     >     Hello.  I would like to ask about having mysql data
>     hosted on
>     >     >     glusterfs.  Please see my issue below.  I have posted
>     DEBUG log
>     >     >     information on pastebin.
>     >     >
>     >     >     Thanks in advance!
>     >     >     -JPH
>     >     >
>     >     >     *
>     >     >     SERVER Version:*
>     >     >     glusterfsd --version
>     >     >     glusterfs 1.3.7 built on Nov  3 2007
>     >     >
>     >     >     *CLIENT Version:*
>     >     >     glusterfs --version
>     >     >     glusterfs 1.3.7 built on Nov  3 2007
>     >     >
>     >     >     *ISSUE:*
>     >     >     When hosting mysql data on glusterfs I have an issue:
>     >     >     The first time I start the glusterfsd server, and
>     mount the
>     >     glusterfs
>     >     >     client (/var/lib/mysql) mountpoint, I can start the
>     mysql server
>     >     >     just fine.
>     >     >     Upon subsequent stop and restart of mysql it says in
>     the mysql
>     >     >     logs that
>     >     >     it cannot lock the ibdata1 file:
>     >     >     InnoDB: Unable to lock ./ibdata1, error: 107
>     >     >     InnoDB: Unable to lock ./ibdata1, error: 107
>     >     >     InnoDB: Unable to lock ./ibdata1, error: 107
>     >     >     InnoDB: Unable to lock ./ibdata1, error: 107
>     >     >     InnoDB: Unable to lock ./ibdata1, error: 107
>     >     >     InnoDB: Unable to lock ./ibdata1, error: 107
>     >     >     InnoDB: Unable to lock ./ibdata1, error: 107
>     >     >     InnoDB: Unable to lock ./ibdata1, error: 107
>     >     >     ...
>     >     >
>     >     >     This does not happen on local disk.
>     >     >
>     >     >     http://gluster.pastebin.com/m2357e117
>     >     >     < http://gluster.pastebin.com/m2357e117>
>     >     >
>     >     >
>     >     >     _______________________________________________
>     >     >     Gluster-devel mailing list
>     >     >     address@hidden
>     <mailto:address@hidden> <mailto:address@hidden
>     <mailto:address@hidden>>
>     >     <mailto: address@hidden
>     <mailto:address@hidden> <mailto:address@hidden
>     <mailto:address@hidden>>>
>     >     >     http://lists.nongnu.org/mailman/listinfo/gluster-devel
>     <http://lists.nongnu.org/mailman/listinfo/gluster-devel>
>     >     <http://lists.nongnu.org/mailman/listinfo/gluster-devel>
>     >     >
>     >     >
>     >     >
>     >     >
>     >     > --
>     >     > It always takes longer than you expect, even when you take
>     into
>     >     > account Hofstadter's Law.
>     >     >
>     >     > -- Hofstadter's Law
>     >
>     >
>     >
>     >
>     > --
>     > It always takes longer than you expect, even when you take into
>     > account Hofstadter's Law.
>     >
>     > -- Hofstadter's Law
>
>
>
>
> --
> It always takes longer than you expect, even when you take into 
> account Hofstadter's Law.
>
> -- Hofstadter's Law