Re: [GNUnet-developers] Proposal: Make GNUnet Great Again?

[Christian Grothoff]

On 2/2/19 12:41 PM, Schanzenbach, Martin wrote:
> Hi,
> 
>> On 2. Feb 2019, at 11:12, Christian Grothoff <address@hidden> wrote:
>>
>> Signed PGP part
>> Hi Martin,
>>
>> I'm honestly not sure about your idea, but mostly because I totally
>> don't see where to draw the line, or whether it would help or hurt.
>> Maybe you have a great idea here, but I just see issues:
>>
>> For example, is GNS part of GNUnet "core" or an application? What about
>> gnunet-dns2gns? Is ReclaimID part of GNS?
> 
> 
> Yes, I see that problem. The line would have to be drawn more or less on  a 
> "best effort"/"least overhead" basis for existing applications.
> For _anything_ new, always default to a new repo and then, if consensus 
> and/or common sense agrees, merge it into the core package.

There is certainly nothing wrong with new stuff maturing out-of-tree
first, instead of our current approach of --enable-experimental.

> For example:
> 
> I _should_ have built reclaim (src/reclaim) as a separate package. It would 
> have included the attribute-based encryption functionality as well (src/abe).
> This would have allowed the dependencies of the gnunet core package to stay 
> the same at first.

Or you could have tested in configure.ac whether the reclaim
dependencies were present on the system, and only then built reclaim/ABE
logic. See what we do for libextractor for FS or libogg/libvorbis/etc
for conversation: if the dependency is not available, we 'warn' the
user, but build the parts we can without it.  I understand this can be
suboptimal, say when users cannot tell whether they actually want
libjansson or not, but at least with the correct build system support
optional dependencies are not a major issue IMO.

> If at any time in the future we think the ABE functions are useful to other 
> services/apps, we coult merge it into the core.
>  Regarding the dns2gns: as it requires little additional dependencies, I 
> would argue that it can sta part of GNS.
> However, the proxy might as well be a completely separate package.

DEB package or TGZ package?

> Think:
> 
> $ apt install gnunet(-core)
> $ apt install gnunet-gns-proxy
> $ apt install gnunet-fs
> $ apt install gnunet-conversation
> $ apt install gnunet-gtk
> 
> Subjectively, this feels right to me.
> Is separation like this even possible with the current repo structure??

Oh, but this is for me a *completely* different issue. Git repositories
should IMO match release TGZs reasonably closely (modulo submodules),
but *DEB* and *RPM* archives have no need to at all resemble the TGZ
structure! I totally am for Debian more-or-less atomizing GNUnet into
packages like you write above, possibly many more.

> Now we also have to look at how devs look at this (see below).
> 
>>
>> Similarly, Lynx's vision for SecuShare incluced file-sharing. How do you
>> manage such dependencies? It seems to be possible to split up the
>> sources into anywhere between 1 and 50 packages!
> 
> Yes, in particular the dependency management is a bit non-trivial. I see the 
> problem here.
> 
>>
>> What do we do with gnunet-gtk? Already some people miss out on
>> gnunet-gtk because they don't get that it's a separate download. Do we
>> create a gnunet-fs with or without gtk? Do you then have to configure
>> and install
>> gnunet-core+gnunet-gtk-base+gnunet-conversation+gnunet-conversation-gtk
>> in that order? Or do we merge all gnunet-gtk stuff with gnunet-core
>> and/or the respective applications? But gnunet-gtk-common cannot even be
>> tested right now without some application.  Is splitting the sources
>> into 30+ sub-sources really going to make it _easier_ to install!??
> 
> The build order should not matter for the average user anyway, as he will 
> install packages. Not from source.

Yes, but I read your proposal as reorganizing our source (Git!).

> So let us think about the package scenario first.
> I think there are two different "pain points":
> 
> - Separation into tiny packages
> - Monolithic "one size fits all"

My feeling: monolithic source, maybe experimental stuff in a separate
Git, but tiny packages for distros with dependency management.

> Now, the problem is that tiny packages also result in a lot of packages for 
> distributors, for example.

While with a monolithic source, packages can _choose_ how big (or small)
they want to make the binary packages.

> The monolith, however, results in a _huge_ unreasonable dependency tree 
> (Think "apt-get install gnunet" on a headless server that pulls gstreamer for 
> conversation).#
> Not all package managers allow dynamic dependencies like e.g. gentoo where 
> compiler flags are passed at installation time.
> We need to find a good balance. I am just saying that currently, the balance 
> is skewed towards the monolith.

But that could easily be solved with a split DPKG package, we don't need
to re-organize our sources for this. And I totally agree: Debian should
split the package whenever a particular sub-part drags in some larger
dependencies (texlive, gstreamer, any multimedia library, libmicrohttpd,
libjansson, PostgreSQL, MySQL, etc.).

> The other side is when we think about the development scenarios.
> Why is development of completely unrelated functionality (reclaim, secushare) 
> done in a repository alongside the code functions?
> This is prone to lead to FTBFS situations and also makes me have sources I 
> never touch or use as a dev.

Right, that's why there I tend to prefer the monolithic source code,
with if possible optional dependencies so I don't have to have ABE to
build it.

> My builds are triggered by unrelated code and maybe prevents my testing in a 
> CI.
> Those are not things a dev wants. Separation of concerns as much as possible.
>

That might be a question of setting up the CI correctly. Maybe it could
trigger building tests in certain directories only if there were changes
in certain submodules but not others?  I must admit I've not read up
enough on BB/Gitlab triggers to say if this is easy or hard to do.

> 
> See above. Users should be made to think in packages anyway.
> Only devs should care about repos.
> I agree with the docs, but my argument is that the docs will stay confusing 
> if they try to explain to a user that he needs to
> 
> 1. Build from source
> 2. Build in a specific way to add/remove functionality

Above, I discussed TGZ vs DEB. It is even possible that we might want to
discuss Git vs. TGZ vs DEB and have different answers for all three...

Happy hacking!

Christian

signature.asc
Description: OpenPGP digital signature