gnunet-developers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [GNUnet-developers] MYSQL module


From: Hendrik Pagenhardt
Subject: Re: [GNUnet-developers] MYSQL module
Date: Sun, 11 Jan 2004 14:06:26 +0100
User-agent: KMail/1.5.4

On Friday 09 January 2004 23:48, you wrote:
> No. See the note before estimateAvailableBlocks(). As those
> two fields do not update in real time, relying on them can
> result in gnunetd trashing hundreds of megabytes from the
> database if it gets full. And that did happen before.

This is not completely correct. The "Data_length" and "Index_length"
fields are correct always in regard to the size of the files on the disk.
What makes it seem not reliable is the fact, that MySQL doesn't shrink
the data file when rows are deleted. That only happens when OPTIMIZE
TABLE is run. So to get an amount which can be used in the quota
calculation one has to subtract the "Data_free" column from the sum of
the other two. This is more accurate then the current solution, as the
"Avg_row_length" column is just calculated from "Data_length" and the
number of records in the table (including deleted rows), which when
multiplied with the actual row count is nothing more than an estimate of
the consumed space in the data file based on outdated row length
information.

BTW, running OPTIMIZE TABLE on the gnunet tables is a good idea after big
insertions as it balances the indexes (and in most cases shrinks them)
and removes the deleted rows from the data file. Maybe this could be made
an option of gnunet-check.

> A question before dwelving further into optimization hacks,
> is the database really a bottleneck on your system? That is,
> can you afford so much upstream bandwidth that mysql can't
> deliver blocks fast enough? If thats not the case, there
> is not much point in optimizing further, imo, unless
> you really wish to save a few cpu cycles or a few blocks from
> the disk.

Well, as the upstream limit of my gnunetd is reached most of the time, I
have to answer: no, probably the database is not really the bottleneck
for my system. It might be different on other setups, though. What I
can't stand is wasting ressources. And as GNUnet is not the only
application running on the system (as it is probably the norm for most
current and future users) every feasible reduction in ressource usage
should be implemented once the implementation of the concerned module is
reasonably stable.

And what I suggested reduces the number of SQL-queries done every time a
block is inserted into the database by the number of buckets (in my case
24). And when I come around to implement the reduction of calls to
estimateAvailableBlocks, this is further reduced by number of buckets -
1. Admitted, the queries affected are quite fast, but nevertheless they
don't need to be executed and the overhead of inserting a block will be
reduced by quite some margin.

> If you really wish to make gnunet more efficient, you might
> look into the routing/trust business and suggest enhancements
> for that. I suspect thats the place where extra efficiency
> is called for most at the moment.

If you can point me to a place in the source were it makes sense to start
looking into, I'll try my best ;-). But I'm afraid, that I'm better in
making optimizations on a local scale, than on the larger design issues.

Ciao,
        Hendrik





reply via email to

[Prev in Thread] Current Thread [Next in Thread]