Re: [PATCH]: ls: add --user-format option for user defined format

[Ondřej Vašík]

Jim Meyering wrote:
> Back when I thought it might be worthwhile, I would have required a
> find-compatible format string.  One thing find has, but that stat lacks,
> is a complete range of date/time-formatting options.  i.e., you can
> print the date-only portion of the atime:
> 
>     $ find / -maxdepth 0 -printf '%p %AF\n'
>     / 2008-10-04
> 
> or print both date and time with a "T" between them:
> 
>     $ find / -maxdepth 0 -printf '%p %AFT%AT\n'
>     / 2008-10-04T04:02:02.0000000000

You could do that easily as you have ls
--time-style="+<timestyleformat>" for specifying timestamp... 

So just ls --time-style="+%F" --user-format=<format> 
and     ls --time-style="+%FT%T" --user-format=<format>

It has limitation of one user-defined time-style per format, but I can't
imagine use case of having more than one timestamp per entry (maybe to
have timestamp of last content modification and year of inode change,
but this one example could be done by column-width reduction to 4)

> Have you considered adding SELinux-related format directives to GNU find?

It's not only about those SELinux-related things. But I'm ok with doing
some extends to find -printf later.

Let's do some summary, feel free to add/comment items if you have
something not mentioned here:

What patched ls --user-format can and upstream find -printf not:
1) colored files by LS_COLORS or automatically if requested
2) could be used simply as ls alias for normal users - as they could
still specify additional ls options like time-style, quoting-style,
sorting style, blocksize, units and other things without modifying
format string
3) SELinux context and it's parts
4) ACL/SELinux special char ('+'/'.')
5) Displaying only parts of access rights mode string
6) Automated column width computation (in find -printf you have to
hardcode the column width in %N.NX syntax yourself, otherwise there is
no defined human-readable column structure.)
7) Automated handling of recent/nonrecent date/time structure like is
done in common "ls -l" listing
8) Printing file size in human readable mode
9) Printing total blocksize/size in bytes of the filesets
10) Printing filename with destination symlink if exists
   (name -> symlink format)
 
I guess points #1, #2, #6 are the most important things, as #1 and #2
makes the output more user friendly and #6 generally readable by human
without additional processing or reserving more than necessary column
width. Some things mentioned here could/should be added to find -printf
quite easily, #2, #6 and #9 difficult/impossible without huge
performance impact (and is total nonsense to add them to find, should be
handled by ls and justify the fact that ls --user-format as NOT
duplicating of find -printf functionality).

What upstream find -printf can and patched ls --user-format not:
1) Specifying subsections of displayed time more than once
2) More than one entry per line (no new line after entry by default)
3) Displaying depth (I guess not necessary at the ls)
4) \c in format (flush output immediately)
5) Device number on which file exists
6) File system type

Missing things #1, #2 could be easily added/changed (#2 oneliner and
will extend possible format strings, #1 means additional parsing or
sharing with find -printf (maybe via new gnulib module)). Point #3 is
not the case of coreutils. I'm not sure about usefulness of #4. Things
mentioned as #5 and #6 can't be printed by current ls, in general it
could be added.

For me:
ls is utility which should be able to list content of the directory(ies)
(and file informations) in HUMAN READABLE form - so within columns, with
colors, with additional user-modifiable enhancements. This new
--user-format should allow the user to create simple UNIVERSAL alias for
just viewing directory content in requested form without additional
processing.

find is utility which with -printf option should be able to list content
of selected set of files in form suitable for FURTHER PROCESSING. So you
could use e.g. sort, wc, awk and other tools on it to make it better
readable for humans or you can store the output for further user. Some
things which could be EASILY handled by ls could not be solved in find,
as it uses completely different approach.

find in comparision with ls much more powerful, but also comparatively
harder to use.  ls is light-weight, more handy for casual use.

Possible solutions:
1) To improve --user-format patch to comply completely with printf
 format options, to reduce the code by factorization of some parts
 to gnulib, to simplify main print_user_format function by making
 small subfunctions. Some sections of code might be worth of sharing
 with find -printf (e.g. time/date parsing definitions)

 Will take quite long of work/time which could be done later when
 using solution #2...

2) To make --user-format similar to find -printf - to comply with its
 syntax (but not extend some redundant sections - no need to duplicate
 time-style option functionality), to add some things to find -printf
 and to reduce code duplicities/amount caused by current ls
 --user-format patch. Additionally some options could be added to ls
 --user-format to choose e.g. displaying of total blocksize at the
 beginning of the output.
 I'm sure that I could reduce addition to the 4550 lines ls.c code
 to ~450 instead of 600 at the moment, which might be more acceptable.
 Some sections of code might be worth of sharing with find -printf.

 Will take considerable lower amount of time - as the functionality
 already works in ls --user-format patch, parser will be unified with
 the most of find -printf things, factorization could be done later.

3) To add some words about possibility of making user-defined output
 of file list by using find -printf to ls --format section and to
 improve find -printf support to support filetype colored filenames,
 SELinux/ACL character, SELinux context(and parts), human readable
 filesize and maybe other things which should be easy to add there.

 Will take lowest amount of time, but the functionality of ls will
 be not extended, some advantages thrown away. I don't like that
 "solution" as for me that patch is NOT duplicating find -printf
 functionality - it is offering additional value to ls usability.

I would prefer solution #2. Just want to note, that there is already
one completely different user-format style - in stat utility. So it
could be discussed whether to follow coreutils stat format style or
findutils find -printf format style more. Changing style for existing
utility is no-way, so we could not modify stat user-format I guess.

Complete implementation of find -printf options seems to be duplicate
work for me - e.g. time/style format options in find are not completely
documented in find man pages and basically you could do the same with
--time-style options (both using strftime() format), so there is already
one option to handle time-style and no need to duplicate it in the ls
format - as usually you need only one time-style for whole file entry.
But the rest should comply.
 
I tried to not affect the rest of code for non_user-format, so the patch
should have no performance impact for the rest. Benefit is that many
things are shared with the common ls formats, so it is consistent with
ls features (find output could differ - e.g. file types, size output for
devices (I guess it would be impossible to get output similar to ls -l
from "find -printf" when using on /dev )).

IMHO is that new long option for ls only additional value for ls and
generally useful thing... find -printf could reduce column width, but
could not compute automatic width, so if you would like to have general
alias without loss of data/wasting space, you can't. 

Generally find -printf use case is NOT the same as ls --user-format (or
printf, if renaming will be required).

Additionally, ls has simpler syntax to print directory content - e.g.
instead of
find . -mindepth 1 -maxdepth 1 \! -name .\* -printf <my format>
, you can just (for most of formats) use
ls --user-format=<my format>

I looked why the previous patches/requests for that ls option were
rejected, they generally do not have documentation, affects
unnecessarily general parts of the ls code and contain only basic subset
of possible format options. This patch covers all the ls functionality
(if not, please let me know what is missing), has minimal impact on the
rest of code and it is documented. Therefore please consider that long
option for ls again.

Greetings,
         Ondřej Vašík

signature.asc
Description: Toto je digitálně podepsaná část zprávy