Gnuastro 0.19.109 released

[Mohammad Akhlaghi]

Dear all,

The next stable version of Gnuastro is finally(!) ready for release. So I am
very happy to share the test release 0.19.109. It includes 109 Git 
commits by 10
people and feedback from 17 people in the 25 weeks since 0.19. For the 
list of
people see [1]. For the full list of changes since version 0.19, please 
see [2]
below.

Hopefully in a week or two the 20th version of Gnuastro will be
released. Therefore it would be great if you could try building and 
running this
tarball. If you see any unexpected warnings or crashes in your normal usage,
please report it so we can fix them for a smooth stable release.

Here is the compressed source and the GPG detached signature for this
release. To uncompress Lzip tarballs, see [3]. To check the validity of the
tarballs using the GPG detached signature (*.sig) see [4]. See [5] for 
the list
of software used to bootstrap this tarball.

  https://alpha.gnu.org/gnu/gnuastro/gnuastro-0.19.109-0c81c.tar.lz
  https://alpha.gnu.org/gnu/gnuastro/gnuastro-0.19.109-0c81c.tar.lz.sig

Here are the SHA1 and SHA256 checksums (other ways to check if the 
tarball you
download is what we distributed). Just note that the SHA256 checksum is 
base64
encoded, instead of the hexadecimal encoding that most checksum tools 
default to
(see [6] on how to generate it).

  4c1762a1d25779f4d0abd309f316a7eda69675b5  gnuastro-0.19.109-0c81c.tar.lz
  StqUDLyKdqqArED86YZ4II1rE/h29ChryIf5HYVUEKs= 
gnuastro-0.19.109-0c81c.tar.lz

If any of Gnuastro's programs or libraries are useful in your work, 
please cite
_and_ acknowledge them. For citation and acknowledgment guidelines, run the
relevant programs with a `--cite' option (it can be different for different
programs, so run it for all the programs you use). Citations _and_
acknowledgments are vital for the continued work on Gnuastro, so please 
don't
forget to support us by doing so.

Best wishes,
Mohammad

--
Staff Researcher
Centro de Estudios de Física del Cosmos de Aragón (CEFCA),
Plaza San Juan 1, Planta 2, Teruel 44001, Spain










[1] Thanks to everyone who has contributed!

The following people contributed changes to this release:

  Elham Saremi (16)
  Faezeh Bidjarchian (6)
  Giulia Golini (1)
  Mohammad Akhlaghi (54)
  Pedram Ashofteh-Ardakani (1)
  Raul Infante-Sainz (6)
  Samane Raji (1)
  Sepideh Eskandarlou (21)
  Zahra Sharbaf (1)
  Zohreh Ghaffari (2)

The following people provided comments, suggestions or found bugs:

  Alejandro Serrano Borlaff
  Bharat Bhandari
  Elham Saremi
  Faezeh Bidjarchian
  Fernando Buitrago
  Giulia Golini
  Leslie Hunt
  Martin Kuemmel
  Michael Stein
  Nafiseh Sedighi
  Paola Dimauro
  Samane Raji
  Sepideh Eskandarlou
  Siyang He
  Teet Kuutma
  Zahra Sharbaf
  Zohreh Ghaffari










[2] Updates since version 0.19:

   New program:
   - astscript-zeropoint: this script is used for calibrating the pixel
     values in one image (finding its "zero point") based on a reference
     catalog or (any number of) reference image(s). The script allows for
     checking many apertures in the same command to let you choose the
     best; and will operate in parallel where necessary. The documentation
     of this script contains two tutorials on how to optimally use it and
     interpret its results; please read and run them once before using it.
     This script was written as a team effort lead by Sepideh Eskandarlou
     and including (in alphabetical order): Elham Saremi, Giulia Golini,
     Raul Infante-Sainz, Samane Raji, Zahra Sharbaf, Zohreh Ghaffari.

   Arithmetic
   --writeall: Write all datasets on the stack as separate HDUs in the
     output; this is useful in debugging incomplete Arithmetic commands.
   - New operators (also available in Table).
     - isnotblank: same as 'isblank not', but slightly more efficient.
       This was suggested by Sepideh Eskandarlou.
     - swap: swap the top two datasets on the stack of operands.
     - index: return dataset of same size, with pixel values that are
       integers starting from 0 and incrementing one by one. This operator
       doesn't pop the top dataset from the stack, it only adds one (to
       avoid extra RAM usage in most usage scenarios).
     - counter: similar to 'index', but integers start with 1.
     - indexonly: similar to 'index', but pops the top stack dataset.
     - counteronly: similar to 'counter', but pops the top stack dataset.
     - constant: replace all elements of input image or column with the
       given constant value (for example to set all pixels in an image to
       NaN or zero), or to give identifiers to each source catalog when
       adding rows from different catalogs, see the documentation for
       complete examples.
     - counts-to-nanomaggy: convert counts to Nanomaggy (with fixed
       zeropoint of 22.5, used as the pixel units of many surveys like
       SDSS). This was suggested by Giulia Golini.
     - nanomaggy-to-counts: convert nanomaggy to counts. This was suggested
       by Giulia Golini.
     - box-vertices-on-sphere: calculate the RA,Dec of vertices of the four
       vertices of a rectangle from its center and width along RA and
       Dec. This takes into account the curved nature of the coordinate
       system. Added after discussion with Martin Kuemmel.
     - number-neighbors: Return the number of non-zero neighbors of each
       non-zero pixel in a binary image.
   - New operators (only in the Arithmetic program):
     - interpolate-meanngb: interpolate blank values with mean of the
       requested number of nearest neighbors.
   - Alternative (shorter) names for existing operators, added after
     discussion with Samane Raji.
     - u8:  same as 'uint8'   (to convert to unsigned 8-bit integers).
     - i8:  same as 'int8'    (to convert to signed 8-bit integers).
     - u16: same as 'uint16'  (to convert to unsigned 16-bit integers).
     - i16: same as 'int16'   (to convert to signed 16-bit integers).
     - u32: same as 'uint32'  (to convert to unsigned 32-bit integers).
     - i32: same as 'int32'   (to convert to signed 32-bit integers).
     - u64: same as 'uint64'  (to convert to unsigned 64-bit integers).
     - i64: same as 'int64'   (to convert to signed 64-bit integers).
     - f32: same as 'float32' (to convert to 32-bit floating point).
     - f64: same as 'float64' (to convert to 64-bit floating point).

   Crop:
   --append: if the output file already exists, append the cropped image
     HDU to the already existing HDUs of the file. Without this option, any
     existing HDUs in the output file will be removed (default behavior).
   --metaname: Specify the name of the cropped output HDU (value to the
     'EXTNAME' keyword in FITS).

   MakeCatalog:
   - Book: with the increasing number of possible measurements the
     "MakeCatalog measurements" section of the Gnuastro book has been
     broken into separate sub-section and contextually similar measurements
     are grouped separartely there.
   --sigclip-mean-sb: surface brightness (over one pixel's area in
     arcsec^2) of the sigma-clipped mean of the values. This is useful in
     scenarios where you want to compare surface brightness values
     independent of the object's area, and by removing outliers (for
     example in a radial profile). See the description of this option in
     the book for more. This was added after a discussion by Ignacio
     Trujillo and Zahra Sharbaf.
   --sigclip-mean-sb-err: the error in '--sigclip-mean-sb' but using the
     sigma-clipped standard deviation of the values as the error, not the
     standard deviation image.
   --sigclip-std-sb: the surface brightness (over one pixel's area in
     arcsec^2) of the sigma-clipped standard deviation of the values. This
     can be used to find the reliable surface brightness of a radial
     profile for example.

   NoiseChisel:
   --outliernumngb: the number of neighboring tiles to reject those that
     have passed (the mean-median quantile difference criteria) because of
     being on the wings of bright stars/galaxies. Until now, this number
     was the same as the number given to '--interpnumngb', but it is
     confusing and can cause problems (because interpolation is done after
     outlier rejection). From this version, '--interpnumngb' is only in
     charge of the number of neighboring tiles for interpolation outliers
     are defined based on the number of tiles given to this
     option. Therefore, if you are not detecting the wings of large
     galaxies, THE BEST solution is most-probably to increase
     '--outliernumngb'. This was done after a discussion with Elham Saremi.

   Statistics:
   --outliernumngb: see description of same option in NoiseChisel.

   Table:
   - Vector columns with multiple values per column are now supported. The
     following features have been added to help working on vector columns:
     --Book: a new "Vector columns" section has been added under the Table
       program which descibes the core concepts and usage examples of the
       options below.
     --information: also identifies vector columns (a column with more than
       one value), by placing an '(N)' after the type. Where 'N' is the
       number of elements within the vector.
     --tovector: merge multiple single-valued columns into a new vector
       column.
     --fromvector: extract identified elements/tokens of a vector column to
       separate single-valued columns.
     --keepvectfin: do not delete the input columns to '--tovector' and
       '--fromvector'.
   --txteasy: (or '-Y') when output is a plain-text file or just gets
     printed on standard output (terminal), all floating point columns are
     printed in fixed point notation (as in '123.456') instead of the
     default exponential notation (as in '1.23456e+02'). For 32-bit
     floating points, use a precision of 3 digits and for 64-bit floating
     points use a precision of 6 digits. This can be useful for human
     readability, but be careful with some scenarios (for example
     '1.23e-120', which will show only as '0.0'!). For more, see the
     changes in Table in this version.

   - New column arithmetic operator:
     - sorted-to-interval: return two columns from a single (sorted) input,
       containing the minimum and maximum values of an interval. The
       intervals are defined such that the intervals of every row fully
       cover the full range of values in the given column. This can be
       useful in scenarios where you need to build the inputs of the
       '--customtable' feature in MakeProfiles (to build a 2D image a
       custom profile).

   astscript-radial-profile:
   --precision: sample the radial profile at precisions less than one
     pixel. This is useful when you need to sample the profile within the
     central few pixels more accurately than integer pixels. See the
     documentation of this option in the book for a complete explanation
     with examples.

   Library:
   - GAL_ARITHMETIC_OP_SWAP: swap the top two operands.
   - GAL_ARITHMETIC_OP_INDEX: An index (counting from 0) for every element.
   - GAL_ARITHMETIC_OP_INDEXONLY: Similar to 'GAL_ARITHMETIC_OP_INDEX'.
   - GAL_ARITHMETIC_OP_COUNTER: A counter (counting from 1) for every 
element.
   - GAL_ARITHMETIC_OP_COUNTERONLY: Similar to 'GAL_ARITHMETIC_OP_COUNTER'.
   - GAL_ARITHMETIC_OP_COUNTS_TO_NANOMAGGY: convert counts to nanomaggy.
   - GAL_ARITHMETIC_OP_NANOMAGGY_TO_COUNTS: convert nanomaggy to counts.
   - GAL_ARITHMETIC_OP_BOX_VERTICES_ON_SPHERE: calculate the coordinates of
     vertices of a rectable on a sphere from its center and width/height.
   - gal_binary_number_neighbors: num. non-zero neighbors of non-zero 
pixels.
   - gal_blank_flag_not: binary dataset with 1 for those input pixels that
     were blank.
   - gal_data_alloc_empty: Allocate an empty dataset with a given number of
     dimensions.
   - gal_list_f64_to_data: convert list of float64s to a 'gal_data_t'
     dataset with the requested type.
   - gal_list_data_remove: Remove the given dataset from the given list.
   - gal_list_data_select_by_id: find/select a dataset from a list of
     datasets using an identification string (either counter or name).
   - gal_permutation_apply_onlydim0: When we have a 2D input, apply
     permutation for all the elements of each row (along dimension-0 in C).
   - gal_statistics_has_negative: see if input has a negative value.
   - gal_table_col_vector_extract: extract the given elements of a vector
     column into separate columns.
   - gal_table_cols_to_vector: merge multiple columns into a vector column.
   - gal_units_counts_to_nanomaggy: Convert counts to nanomaggy.
   - gal_units_nanomaggy_to_counts: Convert nanomaggy to counts.
   - gal_wcs_box_vertices_from_center: calculate the coordinates of
     vertices of a rectable on a sphere from its center and width/height.

** Removed features

  MakeCatalog
  - Removed the short option format for the following measurements
    (columns). This was done because the short options affect the sorting
    in the output of '--help', making it hard to find the options
    visually. Because most measurements do not have a short format, we have
    seen that the short format is rarely used. Only the most commonly used
    measurements keep their short format, and only when they don't affect
    the alphabetic sorting of the output of '--help':
       --area                     (short option was 'a')
       --axis-ratio               (short option was 'Q')
       --magnitude-error          (short option was 'G')
       --num-clumps               (short option was 'c')
       --position-angle           (short option was 'p')
       --semi-major               (short option was 'A')
       --semi-minor               (short option was 'B')
       --sn                       (short option was 'n')
       --sum                      (short option was 'b')
       --upperlimit-mag           (short option was 'u')


** Changed features

  Configuration:
  --with-python: this has replaced the old '--without-python' option. The
    Python extension features in the Gnuastro library are no longer built
    by default. To have them, you need to explicitly configure Gnuastro
    with this option. This was necessary because Python has many
    peculiarities that were causing problems for users that don't need it
    in Gnuastro. These features are only necessary for those who compile
    pyGnuastro from source, most future users of pyGnuastro will be
    installing pre-built binaries it through services like PyPI, so they
    won't be needing it either.

  MakeCatalog:
  - "Sum" used instead of "brightness"
    --sum: new name for the old '--brightness' column. "Brightness" has a
      specific meaning in astronomy/physics and has units of
      energy/time. However, astronomical images aren't in this unit, they
      are usually in counts, or can have an arbitray units (like
      nanomaggy). The new name is derived from what this column does (sum
      of pixel values) to avoid confusions.
    --sum-error: new name for '--brightnesserr'.
    --clumps-sum: new name for '--clumpbrightness'.
    --sum-no-river: new name for '--brightnessnoriver'.
  --checkuplim: now returns the center of each label's tile, not the first
    pixel (the bottom-left pixel position of the tile). The old behavior to
    return the first pixel was un-intuitive, would require several manual
    steps (to measure the minimum and maximum positions of each tile,
    measure the width along each dimension and add it), and could lead to
    confusing results when ignored. See the updated documentation of this
    option for more on custom usage in special situations.
  - Using a hyphen to help in readability and usability:
       Old name                         New name
       ========                         ========
       --areaarcsec2                    --area-arcsec2
       --areamaxv                       --area-max-val
       --areaminv                       --area-min-val
       --areaxy                         --area-xy
       --axisratio                      --axis-ratio
       --clumpsarea                     --clumps-area
       --clumpsgeow1                    --clumps-geo-w1
       --clumpsgeow2                    --clumps-geo-w2
       --clumpsgeow3                    --clumps-geo-w3
       --clumpsgeox                     --clumps-geo-x
       --clumpsgeoy                     --clumps-geo-y
       --clumpsgeoz                     --clumps-geo-z
       --clumpsmagnitude                --clumps-magnitude
       --clumpsw1                       --clumps-w1
       --clumpsw2                       --clumps-w2
       --clumpsw3                       --clumps-w3
       --clumpsx                        --clumps-x
       --clumpsy                        --clumps-y
       --clumpsz                        --clumps-z
       --fracmax                        --frac-max
       --fracmaxarea1                   --frac-max1-area
       --fracmaxarea2                   --frac-max2-area
       --fracmaxradius1                 --frac-max1-radius
       --fracmaxradius2                 --frac-max2-radius
       --fracmaxsum1                    --frac-max1-sum
       --fracmaxsum2                    --frac-max2-sum
       --geoarea                        --geo-area
       --geoareaxy                      --geo-area-xy
       --geoaxisratio                   --geo-axis-ratio
       --geopositionangle               --geo-position-angle
       --geosemimajor                   --geo-semi-major
       --geosemiminor                   --geo-semi-minor
       --geow1                          --geo-w1
       --geow2                          --geo-w2
       --geow3                          --geo-w3
       --geox                           --geo-x
       --geoy                           --geo-y
       --geoz                           --geo-z
       --halfmaxarea                    --half-max-area
       --halfmaxradius                  --half-max-radius
       --halfmaxsb                      --half-max-sb
       --halfmaxsum                     --half-max-sum
       --halfsumarea                    --half-sum-area
       --halfsumradius                  --half-sum-radius
       --halfsumsb                      --half-sum-sb
       --hostobjid                      --host-obj-id
       --idinhostobj                    --id-in-host-obj
       --magnitudeerr                   --magnitude-error
       --maxvx                          --max-val-x
       --maxvy                          --max-val-y
       --maxvz                          --max-val-z
       --maxx                           --max-x
       --maxy                           --max-y
       --maxz                           --max-z
       --minvx                          --min-val-x
       --minvy                          --min-val-y
       --minvz                          --min-val-z
       --minx                           --min-x
       --miny                           --min-y
       --minz                           --min-z
       --numclumps                      --num-clumps
       --objid                          --obj-id
       --positionangle                  --position-angle
       --riverave                       --river-mean
       --rivernum                       --river-number
       --sberror                        --sb-error
       --semimajor                      --semi-major
       --semiminor                      --semi-minor
       --sigclip-mean-sb-err            --sigclip-mean-sb-delta
       --skystd                         --sky-std
       --surfacebrightness              --sb
       --upperlimitmag                  --upperlimit-mag
       --upperlimitonesigma             --upperlimit-onesigma
       --upperlimitquantile             --upperlimit-quantile
       --upperlimitsb                   --upperlimit-sb
       --upperlimitsigma                --upperlimit-sigma
       --upperlimitskew                 --upperlimit-skew
       --weightarea                     --weight-area

  MakeNoise:
  --bgnotmag: new name for the old '--bgisbrightness' option. See the
    description of changed '--sum' in MakeCatalog (above) for more.

  MakeProfiles:
  --mcolissum: new name for the old '--mcolisbrightness' option. See the
    description of changed '--sum' in MakeCatalog (above) for more.
  - Units of the output FITS image (value to 'BUNIT' keyword) is now
    "counts", until now, it was "brightness". See the description of
    changed '--sum' in MakeCatalog (above) for more.

  Table:
  - To avoid potential loss of information in floating point columns, when
    printing the columns to standard output (in the terminal) or saving in
    a plain-text file, the default floating point formats are now printed
    in exponential (scientific) notation (e.g., 1.234e-5). Until now the
    default notation was fixed-point notation (e.g., 0.00001234). A new
    sub-section called "Printing floating point numbers" has been added to
    the book (under the "Table" section) to clarify this important point.
  -A: new short format for --txtf64format. The '-d' short format was
   conflicting with the short option name for '--descending'.

  astscript-psf-select-stars:
  - Now uses the Gaia DR3 dataset by default (until now it was using eDR3).

  astscript-psf-stamp:
  astscript-psf-unite:
  astscript-radial-profile:
  --axis-ratio: new name for old '--axisratio'
  --position-angle: new name for old '--positionangle'

  Library:
  - gal_blank_remove_rows: new 'onlydim0' argument to ignore vector columns
    when checking for blanks.
  - gal_txt_write: new 'tab0_img1' argument. Until now, this function would
    distinguish between images and tables using the dimensions of the
    input. But with the addition of vector columns in tables (that have 2
    dimensions) this argument becomes necessary.

** Bugs fixed
  bug #63266: Table ignores a value of 0 given to '--txtf32precision' or
              '--txtf32precision=0' (happens when floating point columns
              need to be rounded to integers). Reported by Sepideh
              Eskandarlou.
  bug #63270: Table aborts when called with '--descending' with complaint
              on '--txtf64format' (that was added in Gnuastro
              0.19). Reported by Giulia Golini.
  bug #63285: Table not printing anything in rows of integer columns with
              value of zero.
  bug #63340: Statistics printing zero for values smaller than 1e-6 and
              1e-14 for 32-bit or 64-bit floats respectively. Reported by
              Elham Saremi.
  bug #63345: WCS coordinate change not accounting for new equinox (only
              relevant for equatorial outputs). Reported by Alejandro
              Serrano Borlaff.
  bug #63399: Table's '--noblank' or '--noblankend' crashing when input
              has no rows. Reported by Elham Saremi.
  bug #63485: Oversampling in radial profile has incorrect center. Reported
              by Nafiseh Sedighi.
  bug #63543: Programs delete input when it has same name as output.
              Reported by Faezeh Bidjarchian.
  bug #63969: make distcheck error due to temporary files of
              astscript-psf-select-stars. Reported by Siyang He.










[3] Lzip has better compression ratio and archival features compared to the
`.gz' or `.xz' formats. Therefore Gnuastro's alpha/test releases are only in
this format, but for historical reasons we also include `.gz' tarballs 
in the
official releases. If you don't have Lzip (you can check with `lzip 
--version'
command), download and install it from its webpage:

  https://www.nongnu.org/lzip/lzip.html

If Lzip is present and you use GNU Tar, then the single command below should
uncompress and un-pack the tarball:

  $ tar xf gnuastro-0.19.109-0c81c.tar.lz

If the command above doesn't work, you have to un-compress and un-pack 
it with
two separate commands (or use a pipe to feed the output of the first 
into the
second: `lzip -cd gnuastro-0.19.109-0c81c.tar.lz | tar -xf -'):

  $ lzip -d gnuastro-0.19.109-0c81c.tar.lz
  $ tar xf gnuastro-0.19.109-0c81c.tar










[4] Use a .sig file to verify that the corresponding file (without the .sig
suffix) is intact.  First, be sure to download both the .sig file and the
corresponding tarball.  Then, run a command like this:

  gpg --verify gnuastro-0.19.109-0c81c.tar.lz.sig

If that command fails because you don't have the required public key, or 
that
public key has expired, try the following commands to update or refresh 
it, and
then rerun the 'gpg --verify' command.

  gpg --keyserver hkp://pgp.rediris.es --recv-keys 71E899012D174B66










[5] This tarball was bootstrapped (created) with the tools below. Note 
that you
don't need these to build Gnuastro from the tarball, these are the tools 
that
were used to make the tarball itself. They are only mentioned here to be 
able to
reproduce/recreate this tarball later.

  Texinfo 7.0.3
  Autoconf 2.71
  Automake 1.16.5
  Help2man 1.49.3
  ImageMagick 7.1.1-7
  Gnulib v0.1-6102-ge7d121249b
  Autoconf archives v2022.09.03-4-gda89908

The dependencies to build Gnuastro from this tarball on your system
are described here:

  https://www.gnu.org/s/gnuastro/manual/html_node/Dependencies.html










[6] To get the base64 SHA256 checksum, you can use the command below 
(assuming
you already have the 'openssl' command-line program).

  cat gnuastro-0.19.109-0c81c.tar.lz \
      | openssl dgst -binary -sha256 | openssl base64 -A