[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: buildbot server down?
From: |
Kai Torben Ohlhus |
Subject: |
Re: buildbot server down? |
Date: |
Sun, 6 Sep 2020 01:40:12 +0900 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 |
On 9/6/20 12:32 AM, John W. Eaton wrote:
> On 9/5/20 10:17 AM, John W. Eaton wrote:
>> On 9/4/20 9:59 PM, John W. Eaton wrote:
>>
>> Thinking that maybe we were just running near the edge on resource
>> limits, I resized the droplet from 2GB to 4GB RAM. When buildbot
>> starts, the python process uses about 80MB. But it seems to grow
>> fairly rapidly. Here is the output from top after a few hours:
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
>> COMMAND
>> 931 buildbot 20 0 3141096 3.0g 9780 R 103.0 76.4 111:40.02
>> python3
>>
>> So it seems something is wrong there. I don't think the buildbot
>> server process should need that much memory and I don't recall this
>> being a problem previously.
>>
>> I updated to a newer version. We were using 2.7.0-1, now we have
>> 2.8.2-3. I'll watch and see if that improves things.
>
> Hmm, seems to be doing the same thing again after 90 minutes or so we
> are up to
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
> COMMAND
> 1148 buildbot 20 0 3142408 3.0g 9636 S 75.0 76.5 56:44.73
> python3
>
> I have no idea why this is happening. Is it buildbot? Python 3?
> Something wrong with our configuration?
>
> jwe
>
Do I get the setup right, that only the Buildbot Master runs in that
Digital Ocean droplet? Or anything else?
To my experience, there is no need for the Buildbot Master to use up
that much CPU and RAM. My instance is not as busy as yours, but all
files (sqlite database, log files, etc.) are below 1 MB and the CPU is
mostly sleeping, RAM consumption is minimal.
In another thread [1], a tool called buildbot-profiler [2] is used.
Maybe this can be of some help to find the problem?
Kai
[1] https://github.com/buildbot/buildbot/issues/3444
[2] https://pypi.python.org/pypi/buildbot-profiler/