bug#37631: service zabbix-server (and zabbix-agent) fails starting (cannot run as root!)

[Giovanni Biscuolo]

Hi Guix,

executive summary: do we really need to start zabbix_server in
foreground mode?

I have a Guix System in which I defined this services (thanks to the
work in guix-maintenance):

--8<---------------cut here---------------start------------->8---
           ;; For the Zabbix database.  It was created by manually
           ;; following the instructions here:
           ;; 
https://www.zabbix.com/documentation/4.2/manual/appendix/install/db_scripts
           (postgresql-service)

           ;; Monitoring

           (service zabbix-agent-service-type)

           (service zabbix-server-service-type
                    (zabbix-server-configuration
                     (include-files '("/root/secrets/zabbix-server-dbpass"))
                     (log-type "file")))

           (service zabbix-front-end-service-type
                    (zabbix-front-end-configuration
                     (nginx (list
                             (nginx-server-configuration
                              (root #~(string-append #$zabbix-server:front-end 
"/share/zabbix/php"))
                              (listen '("7878"))
                              (index '("index.php"))
                              (locations
                               (let ((php-location (nginx-php-location)))
                                 (list (nginx-location-configuration
                                        (inherit php-location)
                                        (body (append 
(nginx-location-configuration-body php-location)
                                                      (list "
fastcgi_param PHP_VALUE \"post_max_size = 16M 
                          max_execution_time = 300\";
"))))))))))
                       (db-secret-file 
"/root/secrets/zabbix-front-end-dbpass"))))
--8<---------------cut here---------------end--------------->8---

The zabbix frontend service is running well but the zabbix-server
refuses to start

--8<---------------cut here---------------start------------->8---
$ herd start zabbix-server
Service zabbix-server could not be started.
herd: failed to start service zabbix-server
--8<---------------cut here---------------end--------------->8---

looking in the current system profile (built with a guix master branch
on 27 Sept)

--8<---------------cut here---------------start------------->8---
Generation 12   Sep 27 2019 21:18:26    (current)
  file name: /var/guix/profiles/system-12-link
  canonical file name: /gnu/store/h03qdv70sgndclgp04dpkka4rqlk9fg3-system
  label: GNU with Linux-Libre 5.2.17
  bootloader: grub
  root device: UUID: 9862e534-946d-4323-b7ce-9937661bdb7d
  kernel: /gnu/store/bjs8k11phqhn39n7cs1wix5x147fwhnn-linux-libre-5.2.17/bzImage
--8<---------------cut here---------------end--------------->8---

I found the shepherd uses
/gnu/store/lm1d60d0kra3z86hcjmav828cfxjcgi8-shepherd-zabbix-server.scm
with this (partial) parameters:

--8<---------------cut here---------------start------------->8---
#:start (make-forkexec-constructor (list 
"/gnu/store/qcm5j0wk8rs6ykn6b10vg8awf2v6kvx1-zabbix-server-4.2.0/sbin/zabbix_server"
 "--config" "/gnu/store/w1vgvlbzs3jks014r5dra7ih6g7r26n7-zabbix_server.conf" 
"--foreground") #:user "zabbix" #:group "zabbix" #:pid-file 
"/var/run/zabbix/zabbix_server.pid"
--8<---------------cut here---------------end--------------->8---

and if I try to start it from the command line:

--8<---------------cut here---------------start------------->8---
/gnu/store/qcm5j0wk8rs6ykn6b10vg8awf2v6kvx1-zabbix-server-4.2.0/sbin/zabbix_server
 --config /gnu/store/w1vgvlbzs3jks014r5dra7ih6g7r26n7-zabbix_server.conf 
--foreground
--8<---------------cut here---------------end--------------->8---

I get:

--8<---------------cut here---------------start------------->8---
zabbix_server [879]: cannot run as root!
--8<---------------cut here---------------end--------------->8---

I had a look in upstream bug reports but was not able to find nothing
strictly related to zabbix_server, but I was able fo find this for
zabbix_agentd https://support.zabbix.com/browse/ZBX-10611 (fixed since
4.2.1rc1)

actually if I start zabbix_server without ``--foreground'' the server
starts without problems

I thought upgrading to the last stable release of zabbix was the
solution, so I submitted a patch (bug#37629) to upgrade to 4.2.7 and now
I'm using a custom channel with that patch applied:

--8<---------------cut here---------------start------------->8---
(list (channel
        (name 'guix)
        (url "https://gitlab.com/gbiscuolo/guix.git";)
        (branch "wip-zabbix-update")))
--8<---------------cut here---------------end--------------->8---

but if I switch to my last system generation (built with the above channel):

--8<---------------cut here---------------start------------->8---
Generation 13   Oct 05 2019 10:24:28
  file name: /var/guix/profiles/system-13-link
  canonical file name: /gnu/store/bmmjbk6sidqjahq0i53mgp38b342lnda-system
  label: GNU with Linux-Libre 5.3.2
  bootloader: grub
  root device: UUID: 9862e534-946d-4323-b7ce-9937661bdb7d
  kernel: /gnu/store/b28yv4dww1fha0hdhxibfg0k1f50sy0f-linux-libre-5.3.2/bzImage
--8<---------------cut here---------------end--------------->8---

and reboot...

--8<---------------cut here---------------start------------->8---
$ sudo herd status zabbix-server
Status of zabbix-server:
It is stopped.
It is enabled.
Provides (zabbix-server).
Requires ().
Conflicts with ().
Will be respawned.
--8<---------------cut here---------------end--------------->8---

this time if I start zabbix_server from the command line:

--8<---------------cut here---------------start------------->8---
 $ 
/gnu/store/94w5smlc86xac7v1vz4wbqkq43b2fia8-zabbix-server-4.2.7/sbin/zabbix_server
 --config /gnu/store/w1vgvlbzs3jks014r5dra7ih6g7r26n7-zabbix_server.conf 
--foreground
Starting Zabbix Server. Zabbix 4.2.7 (revision 624fb7497b).
Press Ctrl+C to exit.

--8<---------------cut here---------------end--------------->8---

the server starts but the console remains open (as expected by using of
foreground) and if I press Ctrl+C obviously the server is terminated

AFAIU the foreground process never exits and shepherd times it out:
right?

the same thing **almost** apply to zabbix-agent that is also
started with "--foregroud": it is started by shepherd, the console gets
busy (so I cannot login for exemple) but if I press Ctrl+C the process
keeps running

--8<---------------cut here---------------start------------->8---
$ sudo herd status zabbix-agent
Status of zabbix-agent:
  It is started.
  Running value is 277.
  It is enabled.
  Provides (zabbix-agent).
  Requires ().
  Conflicts with ().
  Will be respawned.
--8<---------------cut here---------------end--------------->8---

strange behaviour

anyway back to the question: do we need to start zabbix_server and
zabbix_agentd in foreground?

Thanks! Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures