[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL 04/16] call_rcu: stop using mb_set/mb_read
From: |
Paolo Bonzini |
Subject: |
[PULL 04/16] call_rcu: stop using mb_set/mb_read |
Date: |
Tue, 9 May 2023 11:04:41 +0200 |
Use a store-release when enqueuing a new call_rcu, and a load-acquire
when dequeuing; and read the tail after checking that node->next is
consistent, which is the standard message passing pattern and it is
clearer than mb_read/mb_set.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
util/rcu.c | 45 ++++++++++++++++++++++++++++++++-------------
1 file changed, 32 insertions(+), 13 deletions(-)
diff --git a/util/rcu.c b/util/rcu.c
index e5b6e52be6f8..30a7e220264a 100644
--- a/util/rcu.c
+++ b/util/rcu.c
@@ -189,8 +189,22 @@ static void enqueue(struct rcu_head *node)
struct rcu_head **old_tail;
node->next = NULL;
+
+ /*
+ * Make this node the tail of the list. The node will be
+ * used by further enqueue operations, but it will not
+ * be dequeued yet...
+ */
old_tail = qatomic_xchg(&tail, &node->next);
- qatomic_mb_set(old_tail, node);
+
+ /*
+ * ... until it is pointed to from another item in the list.
+ * In the meantime, try_dequeue() will find a NULL next pointer
+ * and loop.
+ *
+ * Synchronizes with qatomic_load_acquire() in try_dequeue().
+ */
+ qatomic_store_release(old_tail, node);
}
static struct rcu_head *try_dequeue(void)
@@ -198,26 +212,31 @@ static struct rcu_head *try_dequeue(void)
struct rcu_head *node, *next;
retry:
- /* Test for an empty list, which we do not expect. Note that for
+ /* Head is only written by this thread, so no need for barriers. */
+ node = head;
+
+ /*
+ * If the head node has NULL in its next pointer, the value is
+ * wrong and we need to wait until its enqueuer finishes the update.
+ */
+ next = qatomic_load_acquire(&node->next);
+ if (!next) {
+ return NULL;
+ }
+
+ /*
+ * Test for an empty list, which we do not expect. Note that for
* the consumer head and tail are always consistent. The head
* is consistent because only the consumer reads/writes it.
* The tail, because it is the first step in the enqueuing.
* It is only the next pointers that might be inconsistent.
*/
- if (head == &dummy && qatomic_mb_read(&tail) == &dummy.next) {
+ if (head == &dummy && qatomic_read(&tail) == &dummy.next) {
abort();
}
- /* If the head node has NULL in its next pointer, the value is
- * wrong and we need to wait until its enqueuer finishes the update.
- */
- node = head;
- next = qatomic_mb_read(&head->next);
- if (!next) {
- return NULL;
- }
-
- /* Since we are the sole consumer, and we excluded the empty case
+ /*
+ * Since we are the sole consumer, and we excluded the empty case
* above, the queue will always have at least two nodes: the
* dummy node, and the one being removed. So we do not need to update
* the tail pointer.
--
2.40.1
- [PULL 00/16] Misc patches for 2023-05-09, Paolo Bonzini, 2023/05/09
- [PULL 02/16] test-aio-multithread: do not use mb_read/mb_set for simple flags, Paolo Bonzini, 2023/05/09
- [PULL 01/16] rcu: remove qatomic_mb_set, expand comments, Paolo Bonzini, 2023/05/09
- [PULL 03/16] test-aio-multithread: simplify test_multi_co_schedule, Paolo Bonzini, 2023/05/09
- [PULL 04/16] call_rcu: stop using mb_set/mb_read,
Paolo Bonzini <=
- [PULL 07/16] include/qemu/osdep.h: Bump _WIN32_WINNT to the Windows 8 API, Paolo Bonzini, 2023/05/09
- [PULL 05/16] tb-maint: do not use mb_read/mb_set, Paolo Bonzini, 2023/05/09
- [PULL 09/16] target/i386: Add new EPYC CPU versions with updated cache_info, Paolo Bonzini, 2023/05/09
- [PULL 06/16] MAINTAINERS: add stanza for Kconfig files, Paolo Bonzini, 2023/05/09
- [PULL 08/16] target/i386: allow versioned CPUs to specify new cache_info, Paolo Bonzini, 2023/05/09
- [PULL 12/16] target/i386: Add missing feature bits in EPYC-Milan model, Paolo Bonzini, 2023/05/09
- [PULL 11/16] target/i386: Add feature bits for CPUID_Fn80000021_EAX, Paolo Bonzini, 2023/05/09
- [PULL 13/16] target/i386: Add VNMI and automatic IBRS feature bits, Paolo Bonzini, 2023/05/09
- [PULL 10/16] target/i386: Add a couple of feature bits in 8000_0008_EBX, Paolo Bonzini, 2023/05/09
- [PULL 16/16] meson: leave unnecessary modules out of the build, Paolo Bonzini, 2023/05/09