[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
30/33: Check the WAL size for each chunk
From: |
Christopher Baines |
Subject: |
30/33: Check the WAL size for each chunk |
Date: |
Wed, 14 Aug 2024 05:01:30 -0400 (EDT) |
cbaines pushed a commit to branch master
in repository data-service.
commit 97b76000ec2eeb7905a7cc1de4e81e91b966e918
Author: Christopher Baines <mail@cbaines.net>
AuthorDate: Fri Aug 9 15:47:49 2024 +0100
Check the WAL size for each chunk
Rather than for each system and target, as this should mean the
response (pausing and allowing inferiors and store connections to be closed)
is quicker.
---
guix-data-service/jobs/load-new-guix-revision.scm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/guix-data-service/jobs/load-new-guix-revision.scm
b/guix-data-service/jobs/load-new-guix-revision.scm
index 2d6af34..5e28ba0 100644
--- a/guix-data-service/jobs/load-new-guix-revision.scm
+++ b/guix-data-service/jobs/load-new-guix-revision.scm
@@ -1891,7 +1891,7 @@ SELECT 1 FROM derivation_source_file_nars WHERE
derivation_source_file_id = $1"
(define chunk-size 5000)
- (define (process-system-and-target system target)
+ (define (check-wal-size)
(let loop ((wal-bytes
(catch #t
(lambda ()
@@ -1914,6 +1914,7 @@ SELECT 1 FROM derivation_source_file_nars WHERE
derivation_source_file_id = $1"
(with-time-logging
(simple-format #f "getting derivations for ~A" (cons system
target))
(let loop ((start-index 0))
+ (check-wal-size)
(let* ((count
(if (>= (+ start-index chunk-size) packages-count)
(- packages-count start-index)
- 22/33: Tweak timeouts in resource-pool-stats, (continued)
- 22/33: Tweak timeouts in resource-pool-stats, Christopher Baines, 2024/08/14
- 24/33: Stream the render-revision-package-derivations JSON response, Christopher Baines, 2024/08/14
- 28/33: Parallelise inserting package derivation distribution counts, Christopher Baines, 2024/08/14
- 29/33: Start trying to handle GC happening while processing revisions, Christopher Baines, 2024/08/14
- 33/33: Use system-ids for inserting distribution counts, Christopher Baines, 2024/08/14
- 31/33: Reduce chunk size, Christopher Baines, 2024/08/14
- 32/33: Add more logging to backfilling guix_revision_package_derivation_distribution_counts, Christopher Baines, 2024/08/14
- 23/33: Fix the pool metrics, Christopher Baines, 2024/08/14
- 17/33: Reduce the WAL size threshold, Christopher Baines, 2024/08/14
- 25/33: Add the fiberize utility, Christopher Baines, 2024/08/14
- 30/33: Check the WAL size for each chunk,
Christopher Baines <=