Re: [Libreboot-dev] Suggestions and proposal for easier Libreboot development

[Ministry of Freedom]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Op 20/04/16 om 17:19 schreef Paul Kocialkowski:
> After spending some hours updating C201 support in Libreboot, here
> are some suggestions I'd like to submit to make Libreboot
> development easier and more comfortable.
> 
> First, there were talks about moving the build system to something
> other than scripts. Is it still planned? What is the new
> alternative going to be? I would strongly urge you to keep a
> script-based approach. It's a perfect fit for such a project and a
> Makefile-based approach should like a train-wreck for this task, if
> doable at all.

the current build system is fine and will be kept, but we want to
start using Makefiles around that. see libreboot.org/gnu

> Regarding the current script-based build system, I noticed how
> multiple versions of coreboot are handled, to have a different
> revision per payload-device association. I think it's a very nice
> thing to have, but the way it's done current seems quite painful to
> me. Instead of duplicating the tree, I'd suggest using git and
> branches, for each devices (even ones that don't need it).
> 

impossible, because of blobs in coreboot git history. this is why we
create directories for each revision of coreboot used, and then run
git-init again and apply patches for each board, making a branch per
board per revision, with patches applied per branch(board)

> Also, the various scripts would be much easier to handle with some
> included library, instead of duplicating lots of aspects in each
> script, making it hard to maintain each and creating disparity
> between projects.
> 
> For instance, a library that handles generic project-related
> aspects for all the other scripts would be very welcome. This would
> also make Libreboot more consistent. Such functions could handle: *
> Cloning a particular project from given URLs (with a primary URL
> and mirrors)

This would be nice

> * Creating a branch for each payload-device pair and reverting it
> to the appropriate revision. Similarly, having a branch for
> "modules", used for building host tools (crossgcc, cbfstool in
> coreboot, flashrom, bucts, etc). * Patching each branch with the
> appropriate patches, both with patches common to the payload and
> specific to the device. * Releasing each project's source code for
> each payload-device pair, making a copy for each and removing the
> git history.
> 

This needs further thought.

I agree with the overall premise; the current build system is very
inflexible.

> This way, all downloaded projects would only have one copy and a
> branch for each needed source tree. For coreboot, it would imply
> keeping the git history. Removing it is very inconvenient when
> developing and there is about no need to do it prior to releasing.
> 
> In order to make everything even more simply, I think it would be
> best to switch to a project-based structure, where each project
> simply defines a few functions with given names, such as
> "download", "build", with relevant parameters. The global scripts
> would simply include the appropriate scripts and call the functions
> with the right arguments. This would severely reduce the
> dissipation of files for a particular project. Each file would
> actually be quite simple thanks to the use of a library for common
> operations.
> 

Could you whip up a basic proof of concept for all of this? Even just
diagrams/concepts are fine, I find it hard to understand text because
we're talking about massive changes to the build system.

> Projects could either be tools (flashrom, bucts), the bootup
> software (coreboot) or a payload (GRUB, depthcharge). Since one
> might surely depend on the other, each project's script would have
> a "depends" function calling other scripts and building the
> required components.



> Regarding the naming of projects, I find some to be quite
> confusing. Libreboot sometimes refers to the whole build system or
> to the deblobbed coreboot, which is sometimes also called
> "libreboot-libre". This kind of inconsistency makes it hard to
> figure out exactly which component does what.
> 
> Overall, here is a suggested directory structure example matching
> these changes: ressources/projects/coreboot/libreboot (script with
> the functions) 
> ressources/projects/coreboot/config/depthcharge/veyron_speedy/config
> (coreboot configuration) 
> ressources/projects/coreboot/config/depthcharge/veyron_speedy/revision
> (coreboot revision) 
> ressources/projects/coreboot/patches/depthcharge/0001-foo.patch 
> ressources/projects/coreboot/patches/depthcharge/veyron_speedy/0001-ba
r.patch
>
> 
ressources/projects/vboot/libreboot
> ressources/projects/vboot/config/depthcharge/veyron_speedy/revision
>
> 
ressources/libs/git (git-related library)
> ressources/libs/common (common operations library)
> 

I think rather, we just need better documentation for maintenance.
Having a more logical structure makes more sense. I've considered
something like what you're proposing.

We're focussing on a release at the moment. Organisational/structural
changes to the project are not on the table at the moment, but we'll
revisit this discussion thread later.

> Also, it would be best to build each project out-of-tree, or to
> ensure that the tree is cleaned between different builds (something
> that could be done in a clean() step, either called before each
> build() for in-tree build or skipped for out-of tree build).
> 
> This is really just a mockup at this point, but if you don't hate
> the idea or see a reason not to do this, I'll probably be spending
> some time working in that direction, because I find the Libreboot
> build system quite painful to deal with as it is now and it won't
> scale well for more projects anyway.
> 
> Cheers,
> 

See above. You hit the nail on the head; it's a mockup. Let's get a
new stable release done first, before thinking about changing the
entire structure of the libreboot project.

(this response is hastily written, because paulk pinged me on IRC and
told me to respond quickly)

- -- 
Libreboot developer
Freenode IRC nick (#libreboot): vimuser

Use free software. Free as in freedom.
https://www.gnu.org/philosophy/free-sw.html

Use a free operating system, GNU/Linux.
https://www.gnu.org/

Use a free BIOS.
https://libreboot.org/

Support freedom. Join the Free Software Foundation.
https://fsf.org/

Minifree Ltd, trading as Ministry of Freedom | Registered in England,
No. 9361826 | VAT No. GB202190462
Registered Office: 19 Hilton Road, Canvey Island, Essex SS8 9QA, UK |
Web: http://minifree.org/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJXF+zeAAoJEP9Ft0z50c+ULpIIAKMyP1vdzC7qoB6YAXUVaxEK
8MWxJ8nP51mgaT2Z/lAn2LXjjOqQc7Nnbw22FtG5VAI6hXWt/1gPIImHKdkVs6tY
MKm8WWLvEItmlG6wh21aT4rh/Fyklf2FmYVaaciV4zBwUhVn8dV/0DQK4ZFSUUzS
ovA1my4CcGEp3sRRBWmTBC/4JlATVvQyqkpNiiLspIOQz7NPgdQV9c4qZvNAgjsB
0zmAVlQLF+FQJzafsi5uhMFeWuyepA+0bd7Ji9LrlgHu2x0sN7nRwgZ36mj+G1Zx
GtpHtGQiAdnXyO1PvOBaMAts+7M8cdkTYZQmBtGz8O8a19jnrc701hwoBs5WuMU=
=fF28
-----END PGP SIGNATURE-----