[aspell-devel] Encoding-related installation error for nb and is dictionaries on Mac

[Andrew Janke]

Hi, Aspell developers,

I'm running into an issue when installing the nb (Norwegian) or is
(Icelandic) dictionaries on macOS 10.13.4. It appears that some of the
files in the tarballs have file names with non-Unicode characters in
them, and I suspect that since I'm on a Unicode-based filesystem, it
can't create those files. This causes tar and `make install` to error out.

I downloaded the dictionaries from these URLs:

https://ftp.gnu.org/gnu/aspell/dict/is/aspell-is-0.51.1-0.tar.bz2
https://ftpmirror.gnu.org/aspell/dict/nb/aspell-nb-0.50.1-0.tar.bz2

When I attempt to install them, tar errors out, complaining it can't
create a file. Since I see the placeholder for "invalid Unicode
character" in the error message, I assume that's because the file name
is not valid Unicode, and it can't create a file with that name on my
system. Then `make install` errors out when it tries to copy that file
in to place.

The installation mostly succeeds: the main dictionary and most aliases
end up being installed, but it's missing one of the aliases. That's
mostly fine. The real problem is in a package manager context: some
users are attempting to install these dictionaries via the Mac Homebrew
package manager. Homebrew notices the `tar` error, doesn't realize it
can be ignored, and aborts the installation.

Here's the console output from my installation attempts:


[dicts] $ tar xf aspell-nb-0.50.1-0.tar.bz2
aspell-nb-0.50.1-0/bokm�l.alias: Can't create
'aspell-nb-0.50.1-0/bokm�l.alias'
tar: Error exit delayed from previous errors.
[dicts] $ cd aspell-nb-0.50.1-0
[aspell-nb-0.50.1-0] $ ./configure --vars ASPELL=/usr/local/bin/aspell
PREZIP=/usr/local/bin/prezip
Finding Dictionary file location ...
/usr/local/Cellar/aspell/0.60.6.1_1/lib/aspell-0.60
Finding Data file location ...
/usr/local/Cellar/aspell/0.60.6.1_1/lib/aspell-0.60
[aspell-nb-0.50.1-0] $ make install
/usr/local/bin/word-list-compress d < nb.cwl | /usr/local/bin/aspell 
--lang=nb create master ./nb.rws
mkdir -p /usr/local/Cellar/aspell/0.60.6.1_1/lib/aspell-0.60/
cp nb.rws bokmal.alias bokm�l.alias nb.multi no.alias norsk.alias
/usr/local/Cellar/aspell/0.60.6.1_1/lib/aspell-0.60/
cp: bokm�l.alias: No such file or directory
make: *** [install] Error 1


[dicts] $ tar xf aspell-is-0.51.1-0.tar.bz2
aspell-is-0.51.1-0/�slenska.alias: Can't create
'aspell-is-0.51.1-0/�slenska.alias'
tar: Error exit delayed from previous errors.
[dicts] $ cd aspell-is-0.51.1-0
[aspell-is-0.51.1-0] $ ./configure --vars ASPELL=/usr/local/bin/aspell
PREZIP=/usr/local/bin/prezip
Finding Dictionary file location ...
/usr/local/Cellar/aspell/0.60.6.1_1/lib/aspell-0.60
Finding Data file location ...
/usr/local/Cellar/aspell/0.60.6.1_1/lib/aspell-0.60
[aspell-is-0.51.1-0] $ make install
/usr/local/bin/word-list-compress d < is.cwl | /usr/local/bin/aspell 
--lang=is create master ./is.rws
mkdir -p /usr/local/Cellar/aspell/0.60.6.1_1/lib/aspell-0.60/
cp is.rws icelandic.alias is.multi �slenska.alias
/usr/local/Cellar/aspell/0.60.6.1_1/lib/aspell-0.60/
cp: �slenska.alias: No such file or directory
make: *** [install] Error 1


[dicts] $ tar --version
bsdtar 2.8.3 - libarchive 2.8.3


Bug reports from the Homebrew side, if you're interested:
https://github.com/Homebrew/homebrew-core/issues/20108
https://github.com/Homebrew/homebrew-core/issues/28074


Any advice on how to get this to work smoothly? What encoding are these
files/filenames in? How does the aspell dictionary system handle
different character encodings?

Thanks for your attention.

Cheers,
Andrew Janke