Re: [buildbots] False pass results for segmentation fault in test

[Markus Mützel]

Am 13. Januar 2020 um 16:21 Uhr schrieb "Markus Mützel":
> It looks like a segmentation fault in the last step of running the test suite 
> on the build bots is still marked as "success" if there were no failed test 
> up to that point. E.g.:
> http://buildbot.octave.org:8010/#/builders/12/builds/1525/steps/6/logs/stdio
> 
> Maybe any output to stderr in that step should result in it to be marked as 
> "failure"?

I only have a very vague idea of how configuring the buildbots works. And I 
have no way of testing this. But maybe something similar to this change would 
work:

diff -r e02079ee69d1 master.cfg
--- a/master.cfg        Fri Feb 08 15:58:06 2019 +0000
+++ b/master.cfg        Mon Jan 13 19:44:13 2020 +0100
@@ -324,6 +324,7 @@
     self.passed = 0
     self.failed = 0
     self.warnings = 0
+    self.errors = 0
     self.summary_found = False
     self.complete = False
 
@@ -352,6 +353,11 @@
           self.rc = FAILURE
           self.failed += num
 
+  def errLineRecived (self, line)
+    self.total += 1
+    self.errors += 1
+    self.rc = FAILURE
+
 
 class octave_test (Test):
 
@@ -365,7 +371,7 @@
       passed = self.observer.total - self.observer.failed
 
       self.setTestResults (total = self.observer.total,
-                           failed = self.observer.failed,
+                           failed = self.observer.failed + 
self.observer.errors,
                            passed = self.observer.passed,
                            warnings = self.observer.warnings)