dragora-users
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Dragora-users] Proposal: flexible script to create different images


From: DustDFG
Subject: Re: [Dragora-users] Proposal: flexible script to create different images
Date: Thu, 12 May 2022 18:59:09 +0100

On Thu, May 12, 2022 at 12:46 PM Matias Fonzo <selk@dragora.org> wrote:
El 2022-05-12 02:01, DustDFG escribió:
> On Wed, May 11, 2022 at 10:51 PM Matias Fonzo <selk@dragora.org> wrote:
>
>> El 2022-05-11 15:45, DustDFG escribió:
>> > On Wed, May 11, 2022 at 5:38 PM Matias Fonzo <selk@dragora.org> wrote:
>> >
>> >> Hello DustDFG!,
>> >>
>> >> El 2022-05-10 14:29, DustDFG escribió:
>> >> >
>> >> > On Tue, May 10, 2022 at 1:47 PM Matias Fonzo <selk@dragora.org>
>> wrote:
>> >> >
>> >> >> El 2022-05-10 01:32, DustDFG escribió:
>> >> >> > On Tue, May 10, 2022 at 1:12 AM Matias Fonzo <selk@dragora.org>
>> >> wrote:
>> >> >> >
>> >> >> >> El 2022-05-06 03:58, DustDFG escribió:
>> >> >> >> > Hello Matias!
>> >> >> >> >
>> >> >> >> > I am sorry for the delay
>> >> >> >>
>> >> >> >> No problem!
>> >> >> >>
>> >> >> >> > On Thu, Apr 28, 2022 at 7:00 PM Matias Fonzo <selk@dragora.org>
>> >> >> wrote:
>> >> >> >> >
>> >> >> >> > The size of the ISO matters, since we have to create the images
>> for
>> >> >> >> >> several CDs, in 700mb maximum. To achieve this you have to
>> adjust
>> >> or
>> >> >> >> >> change the output of the packages for the files containing the
>> >> build
>> >> >> >> >> orders. For example, the packages generated from 00-core.order
>> >> would
>> >> >> >> >> be
>> >> >> >> >> installed to /var/cache/qi/packages/cd1/ with the rest
>> continuing
>> >> to
>> >> >> >> >> wrap their output for the next CD number. So from stage 2 you
>> can
>> >> >> >> >> create
>> >> >> >> >> the images for the CDs. It also gives the possibility of doing
>> >> what
>> >> >> >> >> you
>> >> >> >> >> suggested before, once the packages are generated, they will be
>> >> >> >> >> available in the packages/ directory, when chrooting in, Qi
>> can be
>> >> >> >> >> used
>> >> >> >> >> to install directly, for example. the core from
>> >> >> >> >> var/cache/qi/packages/cd1.
>> >> >> >> >>
>> >> >> >> >
>> >> >> >> > As I know, now qi has an `--outdir` command line option. It
>> can't
>> >> give
>> >> >> >> > you
>> >> >> >> > several cd's by stripping size but you can easily move all
>> packages
>> >> >> >> > from
>> >> >> >> > the 00-core.order to folder  var/cache/qi/packages/cd1
>> >> >> >> >
>> >> >> >>
>> >> >> >> Yes, and measure its size to see if it fits in 700 MB.  Other
>> series
>> >> >> >> can
>> >> >> >> make up the "cd2" and so on...
>> >> >> >>
>> >> >> >
>> >> >> > It seems to me that qi mustn't do it. It seems to me that it must
>> do
>> >> >> > something wrapper....
>> >> >>
>> >> >> Yes, from the 'build-for-release' script.
>> >> >>
>> >> >
>> >> > I understood that cdN must depend only on packages from it or other
>> >> > cd's
>> >> > with number lesser than N.
>> >> > Does the order files follow this idea?
>> >>
>> >> Right now the 'build-for-release' script proceed with all the order
>> >> files.  What I have in mind at the moment, is to process for example
>> >> "00-core.order" which would compose a CD (CD1), and see in the
>> >> following
>> >> orders, e.g "01-sound", "02-dict" and maybe "03-xorg" could compose
>> >> CD2...
>> >>
>> >
>> > OK. Now I want to know the difference between bootstrap stages and
>> > build-for-release.
>> > I can't understand. Why aren't they a one object?
>> >
>>
>> I won't connect the whole bootstrap process including the
>> 'build-for-release' process to the bootstrap, because currently the
>> release procedure is intended to be carried out by building absolutely
>> everything one by one with no parallel jobs for the compiler, then
>> moving the temporary system so that there is nothing stuck in the PATH
>> (to make sure), finally building the final system again with parallel
>> jobs to speed up the compilation and guarantee the circular
>> dependencies.  A procedure that takes many, many hours depending on
>> the
>> type of machine you have.
>>
>
> With no parallel jobs? Why?

To ensure the build in the first phase.  In the second phase with
parallel jobs (inside of the final system) I don't see it as risky.


I don't see building with parallel jobs as risky at all. What can happen
with parallel jobs in the first phase?

> This is more for the maintainer who considers this as the last step
>> ready to be released to the free software public.
>>
>
> I understand it but these scripts looks like duplicates in some
> places...
>

"Duplicate in some places" sounds misleading, and may imply something it
is not to other people.

I understand that you and I are not native English speakers, so it is
difficult to understand each other.  I assume you are new to Dragora (or
maybe you are not) but to avoid confusion, you may want to do everything
in one step, as some distributions do, which perform a cross-processing
in one automated step.

Yes, I am new to Dragora but I don't want to do everything in one step.

While this has its practical and end result, it
is something that I do not consider reliable.
 
 
Hence the development of Dragora 3 is built in several stages.

I agree with this decision

>>
>> >> >>>
>> >> >> >> >> Apart from this, my proposal is to create a rootfs, which you
>> >> unpack
>> >> >> >> >> directly, which is more direct and faster than having to
>> install
>> >> >> >> >> packages one by one via Qi.
>> >> >> >> >>
>> >> >> >> >
>> >> >> >> >  It is something like `darkcrusade` compiler collection. I also
>> >> think
>> >> >> >> > that
>> >> >> >> > the rootfs can be a template for different image types (iso,
>> qcow
>> >> ...)
>> >> >> >> > so I
>> >> >> >> > thought that the rootfs and the core image is one entity.
>> >> >> >> >
>> >> >> >> > It seems to me that it will be good to have an
>> xx-bootstrap.order
>> >> that
>> >> >> >> > contains minimal system that you can run...
>> >> >> >>
>> >> >> >> For a minimal system it is needed to identify, mark or
>> sub-categorize
>> >> >> >> the most essential packages required to run the system.
>> >> >> >>
>> >> >> >
>> >> >> > What do you propose?
>> >> >>
>> >> >> Well... the category of a package can be renamed to "essential", or
>> >> >> the
>> >> >> essential packages can be declared in a new order file.
>> >> >>
>> >> >> Alternatively for a minimal system you can make a new stage
>> containing
>> >> >> the C library, the kernel, perl, graft and qi, the init system, and
>> >> >> maybe other packages like busybox plus utilities to make network or
>> >> >> internet available.
>> >> >>
>> >> >
>> >> > I almost like this idea :)
>> >> >
>> >> > I think that it is already almost done because the stage 1 produces a
>> >> > temporary
>> >> > system that looks like the core system or the rootfs. I think that
>> >> > creation
>> >> > of a new stage
>> >> > is a bit unnecessary action. I think that we really need to make a new
>> >> > `essential`
>> >> > order file (maybe with a new category with the same name) that
>> contains
>> >> > only
>> >> > packages from the stage 1. It will permit not to change scheme of
>> >> > system
>> >> > building
>> >> > but it will give possibility to get rootfs or the core system.
>> >> > What do you think about it?
>> >>
>> >> Stage 1 provides software to build more software, for a minimal system
>> >> we don't need this, but a small system that can install the rest (or
>> >> the
>> >> desired available) binary packages, local or remote would be great.
>> >>
>> >> About a new order file for essential packages, sounds good.  I think
>> >> it
>> >> would also be a good idea to rename the category of those (super)
>> >> essential packages.  To identify them, to be able to search them more
>> >> easily, where maybe also the installer can offer such packages,
>> >> installation.
>> >>
>> >
>> > 00-essential.order, 01-core.order, 02-sound.order ...
>> >
>> > Move these packages to category essential: "The C library, the kernel,
>> > perl, graft and qi, the init system, and maybe other packages like
>> > busybox plus utilities to make network or internet available."
>> >
>> > Do you think about something like this?
>>
>> Well, essential packages for building are one thing, and essential
>> packages that make the system run are another.  In this case, we would
>> have to differentiate them.
>>
>
> Yes, it would be good. I looked at category kernel and think that we
> can't
> move to
> the hypothetical essential category because it will break kernel
> category...

Breaking the category name is not severe, it would result in a number of
common packages under the same category, since after all, things like
the kernel image, the init software are what a system needs to run.

 
I am not absolutely sure but moving other packages won't bring problems but the kernel category will look strange because it will contain only buildtree-generic and kmod packages. Am I correct?

> I want to know why package can have only one category?

In principle I did not think of a second category because I did not
consider it necessary, also to avoid making the package name even
longer.  Making it possible to have a second or more categories is going
to involve not only a bit of more code in Qi, but also in other tools
that want to know if the package have a first, second, ... category.

> In the case of the alternative in a separate stage, this does leave it
>> on the side of the bootstrap process rather than the temporary system
>> to
>> build the final system. In a hypothetical stage 3, this builds a
>> minimal
>> running system which gives the possibility to install the packages for
>> the matching architecture.
>>
>
> I think that if it must be a new stage, it must have number 2.
>
> Bootstrap process now:
> stage 0 produces cross-compiler
> stage 1 produces temporary system
> The user enters to the system and manually installs packages
> stage 2 produces iso images
>
> It would be much better to have this scheme:
> stage 0 produces cross-compiler
> stage 1 produces temporary system
>
> stage 2 produces minimal system by using qi from temporary system (not
> replacing temporary system)

Perhaps this is what you refer to as a duplicate?.  Note that the user
can build under a different host than the one he/she builds the system
for, so if you intend to invoke Qi/Graft (which is on the temporary
system) it may not work (relying on the temporary Perl).  To avoid this,
we should add e.g. Graft - assuming the user has Perl installed on his
system to the list of requirements; but it already induces the user to
have to do it even if our list gives instructions on how to do it under
Debian, Graft is not something that is available in the Debian
repositories.

> The user enters to the system produced at stage 2 and manually installs
> other packages
> stage 3 produces iso images
>
> I want to say that if we produces minimal system, full system must be a
> superset of the minimal...

Yes, but it does not have to be the same.  In the new stage you would
build the minimum required so that the user can install more local
packages or get them remotely (if possible).  There is no need for a
temporary system to exist, nor will it be necessary to process or build
anything, such as the temporary system from the stage 1.  For example:

     ./bootstrap -a i586 --stage 0 && ./bootstrap -a i586 --stage M

In this example, the cross compiler is built for Intel 586+, and then
the Minimal system is built for the same target, which also produces its
corresponding ISO or running images.  From there you can load the system
image and use the binary packages provided by Dragora (that match the
current architecture) to install the rest.

 

reply via email to

[Prev in Thread] Current Thread [Next in Thread]