[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[GNUnet-SVN] r19508 - in Extractor: . doc src/main
From: |
gnunet |
Subject: |
[GNUnet-SVN] r19508 - in Extractor: . doc src/main |
Date: |
Sun, 29 Jan 2012 17:28:19 +0100 |
Author: grothoff
Date: 2012-01-29 17:28:19 +0100 (Sun, 29 Jan 2012)
New Revision: 19508
Modified:
Extractor/ChangeLog
Extractor/doc/extractor.texi
Extractor/src/main/extractor.c
Log:
document threading issue found today (see also discussion on #gnunet
Modified: Extractor/ChangeLog
===================================================================
--- Extractor/ChangeLog 2012-01-29 15:48:23 UTC (rev 19507)
+++ Extractor/ChangeLog 2012-01-29 16:28:19 UTC (rev 19508)
@@ -1,3 +1,7 @@
+Sun Jan 29 17:27:08 CET 2012
+ Documented recently discovered issues with pthreads and
+ out-of-process plugin executions in the manual. -CG
+
Tue Nov 29 12:55:40 CET 2011
Improved IPC code on W32 to use APIs correctly and make it work
on NT 6.1. -LRN
Modified: Extractor/doc/extractor.texi
===================================================================
--- Extractor/doc/extractor.texi 2012-01-29 15:48:23 UTC (rev 19507)
+++ Extractor/doc/extractor.texi 2012-01-29 16:28:19 UTC (rev 19508)
@@ -15,7 +15,7 @@
GNU libextractor is a GNU package.
-Copyright @copyright{} 2007, 2010 Christian Grothoff
+Copyright @copyright{} 2007, 2010, 2012 Christian Grothoff
@quotation
Permission is granted to copy, distribute and/or modify this document
@@ -535,28 +535,33 @@
@cindex thread-safety
@tindex enum EXTRACTOR_Options
-All of the functions for loading and unloading plugins, including
address@hidden|EXTRACTOR_plugin_add_defaults|} and
@verb{|EXTRACTOR_plugin_remove_all|},
-are thread-safe and reentrant. However, using the same plugin list
-from multiple threads at the same time is not safe. Creating multiple
-plugin lists and using them concurrently is supported as long as
-the @code{EXTRACTOR_OPTION_IN_PROCESS} option is not used.
+Using @gnule{} from a multi-threaded parent process requires some
+care. The problem is that on most platforms @gnule{} starts
+sub-processes for the actual extraction work. This is useful to
+isolate the parent process from potential bugs; however, it can cause
+problems if the parent process is multi-threaded. The issue is that
+at the time of the fork, another thread of the application may hold a
+lock (i.e. in gettext or libc). That lock would then never be
+released in the child process (as the other thread is not present in
+the child process). As a result, the child process would then
+deadlock on trying to acquire the lock and never terminate. This has
+actually been observed with a lock in GNU gettext that is triggered by
+the plugin startup code when it interacts with libltdl.
-Generally, @gnule{} is fully thread-safe and mostly reentrant.
+The problem can be solved by loading the plugins using the
address@hidden option, which will run @gnule{}
+in-process and thus avoid the locking issue. In this case, all of the
+functions for loading and unloading plugins, including
address@hidden|EXTRACTOR_plugin_add_defaults|} and
address@hidden|EXTRACTOR_plugin_remove_all|}, are thread-safe and reentrant.
+However, using the same plugin list from multiple threads at the same
+time is not safe.
+
All plugin code is expected required to be reentrant and state-less,
but due to the extensive use of 3rd party libraries this cannot
-be guaranteed. Hence plugins are executed (by default) out of
-process. This also ensures that plugins that crash do not cause
-the main application to fail as well.
+be guaranteed.
-Plugins can be executed in-process by giving the option
address@hidden when loading the plugin. This
-option is only recommended when debugging plugins and not for
-production use. Due to the use of shared-memory IPC the
-out-of-process execution of plugins should not be a concern for
-performance.
-
@deftp {C Struct} EXTRACTOR_PluginList
@tindex struct EXTRACTOR_PluginList
Modified: Extractor/src/main/extractor.c
===================================================================
--- Extractor/src/main/extractor.c 2012-01-29 15:48:23 UTC (rev 19507)
+++ Extractor/src/main/extractor.c 2012-01-29 16:28:19 UTC (rev 19508)
@@ -1096,8 +1096,7 @@
static int
-read_all (
- int fd,
+read_all (int fd,
void *buf,
size_t size)
{
@@ -1688,6 +1687,9 @@
fflush (plugin->cpipe_in);
while (1)
{
+ fprintf (stderr, "Reading header from PID %u (plugin %s)\n",
+ plugin->cpid,
+ plugin->short_libname);
if (0 != read_all (plugin->cpipe_out,
&hdr,
sizeof(hdr)))
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [GNUnet-SVN] r19508 - in Extractor: . doc src/main,
gnunet <=