Re: Patches for --vnlog support

[Tim Rice]

Hey Dima,

On Fri, Aug 05, 2022 at 04:16:36PM -0700, Dima Kogan wrote:
> Tim Rice <trice@posteo.net> writes:
>
> > Or are there scenarios where vnlog will be true but vnlog_prologue
> > could turn out to be false?
>
> Yep. When looking for the header, vnlog_prologue=true. After that, when
> we're reading data, vnlog_prologue=false. vnlog=true being true in both
> cases

Fair enough, thanks for explaining.


> > Even if we do need the vnlog_prologue toggle inside that section of
> > code, there are a couple of alternative ways to do it which I might
> > prefer:
> > <snip>
>
> We could do that, but I'm not clear on what problem that would solve,
> and it would make stuff worse, I think. Right now you can see the logic
> with your eyes. If you add indirection with function pointers, you
> can't.

The problem it would solve is the duplication of lengthy boolean toggles. My 
count is already three in the patch set you provided, and it's still just early 
days ;)

I'm not only thinking of how to patch in vnlog support, but how to do it in a 
future-proof way. Eg, what if we want to support some other format, like "vnlog 
2.0", or any other format that becomes popular? Better to get the right skeleton 
now, and then arbitrary meat can be hung off it later.

So, I claim we should toggle the preferred behavior at exactly one place in the 
code, rather than smearing the logic around all over the place.


> If you want to have a clearer separation between vnlog-specific logic
> and everything else, we can have two flavors of _line_record_fread() for
> the two cases, with a single if() in line_record_fread().

I'm in two minds about how much to separate it. I wouldn't say that's my number 
one goal, unless it's a means to an end. Given that we support vnlog, we should 
support it properly, which doesn't necessarily mean keeping its code paths 
entirely disjoint from everything else.

I am more interested in things like robustness and correctness. People come and 
go from software projects. You might lose interest in datamash, the vnlog 
support might come to seem like a burden, and meanwhile someone else comes 
along and wants their bespoke format supported too. What is the most correct 
and robust way of setting up our future selves to handle these situations?

~ Tim