Re: Patch to support big buffers

[Samuel Thibault]

Hello,

Philippe SWARTVAGHER, le mar. 25 févr. 2020 11:51:04 +0100, a ecrit:
> * it fixes buffer sizes bigger than 2 GB (size was stored in 32 bits
> variables -> changed to 64 bits variables).

In fut_flush, we need to make size a size_t as well. On 32bit systems,
with will allow, when built with LFS flags, to have 64bit values.

> * it adds a parameter to the fut_setup function to define a callback

I'm Ok with this, but call it another name, and restore an fut_setup
function that just calls this extended function, for compatibility with
all applications using FxT :) StarPU can then detect in configure.ac
whether the new function name is available or not.

That being said, see fut_flush: it already writes
FUT_START_FLUSH_CODE/FUT_STOP_FLUSH_CODE flush events.

> * is it possible to add a FUT routine to force the flush, even if the buffer
> isn't full and tracing isn't finished ? I don't know if calling directly
> fut_flush() is a proper way of doing it.

That won't be enough: you also need to prevent threads from writing
to tbe buffer. See fut_getstampedbuffer for the way it is currently
implemented. In the spinlock method you'd just need to additionally keep
the spinlock held. In the compare-and-swap method, you need to reset the
pointers to the other buffer, wait for other threads to have finished
filling their slot, and then write. And of course take the flush lock to
avoid mixing with a concurrent flush triggered by some thread.

> Are there events which clearly define the beginning and the end of a
> trace ?

Not events, no. And not in already-recorded traces.

There isn't a way to determine whether the write() happened to be
complete other than just checking its return value, unless adding yet
another even in the already-full buffer. Or reserve some room right
after the buffer just for that etc. But then since you fix the bug,
there is no point in twisting the code to catch it.

> Or an event counter stored in the trace ?

That would add another contention point.

Samuel