Re: [lmi] Directories being created with wrong permissions (was: Getting rid of the rest of Boost)

[Vadim Zeitlin]

On Tue, 5 Oct 2021 11:11:33 +0000 Greg Chicares <gchicares@sbcglobal.net> wrote:

GC> On 10/4/21 11:18 PM, Vadim Zeitlin wrote:
GC> > On Mon, 4 Oct 2021 23:07:25 +0000 Greg Chicares <gchicares@sbcglobal.net> 
wrote:
GC> > 
GC> > GC> On 10/3/21 10:13 PM, Vadim Zeitlin wrote:
GC> [...]
GC> > GC> > I've also fixed the problem with
GC> > GC> > the bad ownership of /opt/lmi/local directory, which somehow ended 
up being
GC> > GC> > owned by root, breaking the MSW cross-build using the official 
makefiles as
GC> > GC> > it couldn't create gcc-x86_64-pc-linux-gnu subdirectory there.
GC> 
GC> Having so recently rerun './install_msw.sh' in a way that first calls
GC> 'make eviscerate' and then rebuilds everything, I thought I'd search
GC> its output log for a clue. I found this:
GC> 
GC> + [ ! -x /opt/lmi/src/lmi/third_party/libxml2/configure ]
GC> + mkdir --parents /opt/lmi/local/gcc_x86_64-pc-linux-gnu/xml-ad_hoc/libxml2
GC> + cd /opt/lmi/local/gcc_x86_64-pc-linux-gnu/xml-ad_hoc/libxml2
GC> + eval echo $libxml2_options
GC> 
GC> where 'mkdir --parents /opt/lmi/local/gcc_...' is the first command
GC> in that log that creates any '/opt/lmi/local/' directory.

 Thanks for this clue, I couldn't find where this directory was created,
and this is not really surprising because this command wasn't even executed
in the builds I was looking at: I use caching for third-party libraries to
avoid uselessly rebuilding them during each CI build, and so it (and other
directories and files created during libxml2 build) is being restored from
cache. And this is almost surely where things go wrong, because it seems
that the code run by the other "actions" used in my workflow runs as root,
even though our own build steps run as a normal non-root user.

 IOW, this almost certainly shows that there is nothing wrong with lmi
build itself.

GC> That command seems to be the likeliest culprit. 'install_xml_libraries.sh'
GC> uses autotools, which might somehow know that the real user is root,
GC> even though you use 'setpriv'.

 I can try running it using the same setpriv command locally, but I'm all
but sure that it's the cache "action" which is the culprit and not this
script itself.


GC> Do CI builds use the 'lmi_setup*.sh' apparatus?

 No, for several reasons: first of all, I don't really know how to use
them. Second, I think that using them requires creating a chroot and I
didn't really want to do it: using a Docker container is not really
different, but much faster because it is cached by GitHub Actions. If you
look at the build at https://github.com/let-me-illustrate/lmi/runs/3784875057
(there is nothing special about it, it's just the last one at the time of
this writing), you can see that setting up the container takes just 4
seconds. Third, I wanted to use the same CI workflow for both "official"
and autotools-based builds and I can share most of the logic between them
without using these scripts, but probably couldn't if I did use them (but I
could be wrong about this). There are other reasons to: these scripts are
probably incompatible with caching (which is very useful) and I'm not sure
if they're compatible with using ccache (which I haven't put in place yet,
but really ought to because it should significantly speed up the builds
without any real cost).

 However if you'd like the CI build to test these scripts, this could be
done, of course. In the worst case, we'd just have 2 different workflows,
one for them and another for the autotools builds (I'd like to keep them if
only because clang is only tested when using autotools currently). Please
let me know if you think it's worth it.

GC> That may indeed be an idiosyncratic problem specific to the CI
GC> environment, in which case an idiosyncratic solution is fine.

 To reiterate, I'm almost sure that the problem is indeed idiosyncratic.

GC> My concern is that it may be a general problem. I know that there
GC> is some permissions problem on our GNU/Linux server, and that it
GC> has something to do with autotoolized libraries such as libxml2.
GC> I need to solve that problem anyway; perhaps this is related.

 This isn't very useful, but I've never seen any such problems when
building locally.

 Regards,
VZ

pgp_OJis6rZri.pgp
Description: PGP signature