[Maposmatic-dev] [task #10040] Gracefuly handle maposmaticd job timeouts

[David Decotigny]

URL:
  <http://savannah.nongnu.org/task/?10040>

                 Summary: Gracefuly handle maposmaticd job timeouts
                 Project: MapOSMatic
            Submitted by: daviddecotigny
            Submitted on: Tue 05 Jan 2010 08:52:47 AM GMT
         Should Start On: Tue 05 Jan 2010 12:00:00 AM GMT
   Should be Finished on: Tue 05 Jan 2010 12:00:00 AM GMT
                Category: maposmaticd
                Priority: 8
                  Status: None
                 Privacy: Public
        Percent Complete: 0%
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
                  Effort: 0.00

    _______________________________________________________

Details:

We experienced a general failure of the rendering after one ocitysmap job
(from maposmaticd) timed out. The failed job appeared to have been stuck (and
killed) in the middle of the big street_index queries, and left the
corresponding postgresql worker process eating 100% CPU.

This had the exceptional benefit of preventing any other rendering coming
afterwards to time out too. The most annoying feature of this behavior being
that, if a daily planet update appears, then it will probably take a veeeeery
long time to complete. Note that it could be too dangerous to kill the faulty
pgsql worker process eating the CPU while the update has already been started
(because in that case, I bet nothing would guarantee the consistency of the
DB, so we'd better restart the whole pgsql server and, hence, the daily
update; but I'm afraid the update process is not idempotent).

One solution we could try: instead of executing the big street_index queries
synchronously, run them asynchronously and attach a process
atexit/int/quit/term signal handler that calls pg_cancel_query. This /might/
(?) help... maybe.





    _______________________________________________________

Reply to this item at:

  <http://savannah.nongnu.org/task/?10040>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.nongnu.org/