Re: [Gluster-devel] regarding execution times of gluster commands

[Anand Avati]

We cannot afford to acknowledge the user that the configuration is changed before actually making those changes on stable media. If it takes longer with O_SYNC and fsync(dir), it just does. I do see the benefits of reduced regression run test times without the O_SYNC/fsync. But I don't think just that justifies removing O_SYNC / fsync(dir). It would be ridiculous for a filesystem's management software to not follow the safety requirements of storing data on a filesystem :-). Let's just work on making the scripts more efficient and intelligent (removing unnecessary sleeps, reducing iterations or file counts which may be unnecessarily large etc.) for bringing down runtime.

Avati

On Wed, Feb 6, 2013 at 5:31 AM, Pranith Kumar Karampuri <address@hidden> wrote:

hi,
     Here is the comparison of command execution times of 'volume set' on v3.3.0 and master:

On master:
⚡ time gluster volume set r2 performance.read-ahead off
volume set: success

real    0m1.250s
user    0m0.059s
sys     0m0.019s

On v3.3.0:
⚡ time gluster volume set r2 performance.read-ahead off
Set volume successful

real    0m0.081s
user    0m0.051s
sys     0m0.016s

The contributing factors to the delay are opening files in gluster-store with O_SYNC and fsync of the directory fd. With the following changes, the execution times came down to:

⚡ time gluster volume set r2 read-ahead off
volume set: success

real    0m0.130s
user    0m0.063s
sys     0m0.019s

⚡ g d
diff --git a/xlators/mgmt/glusterd/src/glusterd-store.c b/xlators/mgmt/glusterd/src/glusterd-store.c
index 863b70c..a225dfd 100644
--- a/xlators/mgmt/glusterd/src/glusterd-store.c
+++ b/xlators/mgmt/glusterd/src/glusterd-store.c
@@ -89,7 +89,7 @@ glusterd_store_mkstemp (glusterd_store_handle_t *shandle)
         GF_ASSERT (shandle->path);

         snprintf (tmppath, sizeof (tmppath), "%s.tmp", shandle->path);
-        fd = open (tmppath, O_RDWR | O_CREAT | O_TRUNC | O_SYNC, 0600);
+        fd = open (tmppath, O_RDWR | O_CREAT | O_TRUNC, 0600);
         if (fd <= 0) {
                 gf_log (THIS->name, GF_LOG_ERROR, "Failed to open %s, "
                         "error: %s", tmppath, strerror (errno));
@@ -121,12 +121,12 @@ glusterd_store_sync_direntry (char *path)
                 goto out;
         }

-        ret = fsync (dirfd);
-        if (ret) {
-                gf_log (this->name, GF_LOG_ERROR, "Failed to fsync %s, due to "
-                        "%s", pdir, strerror (errno));
-                goto out;
-        }
+        //ret = fsync (dirfd);
+        //if (ret) {
+        //        gf_log (this->name, GF_LOG_ERROR, "Failed to fsync %s, due to "
+        //                "%s", pdir, strerror (errno));
+        //        goto out;
+        //}

         ret = 0;
 out:

run-tests.sh executed in 20 minutes with this change as opposed to 30 minutes without this change on my machine.
If we remove the O_SYNC flag alone and keep fsync of dir as is, here is the execution time:

⚡ time gluster volume set r2 read-ahead off
volume set: success

real    0m0.653s
user    0m0.054s
sys     0m0.015s

Do you feel we need to change anything based on these execution time differences?
It will save quite a bit of time at least for gerrit builds if there is a way to turn them off.
Let me know your views.

Pranith.

_______________________________________________
Gluster-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/gluster-devel