[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
branch master updated: Encode file names in Parser and Text Converter
|
From: |
Patrice Dumas |
|
Subject: |
branch master updated: Encode file names in Parser and Text Converter |
|
Date: |
Sat, 05 Mar 2022 17:06:42 -0500 |
This is an automated email from the git hooks/post-receive script.
pertusus pushed a commit to branch master
in repository texinfo.
The following commit(s) were added to refs/heads/master by this push:
new 941870cf4e Encode file names in Parser and Text Converter
941870cf4e is described below
commit 941870cf4e808a35b2568b6031ceedd076d84463
Author: Patrice Dumas <pertusus@free.fr>
AuthorDate: Sat Mar 5 23:04:43 2022 +0100
Encode file names in Parser and Text Converter
* tp/Texinfo/Common.pm (%default_parser_common_customization),
tp/Texinfo/ParserNonXS.pm (parse_texi_file),
tp/Texinfo/XS/parsetexi/Parsetexi.pm (parser, parse_texi_file):
add DATA_INPUT_ENCODING_NAME as a Parser configuration variable
and use it to decode the file name in error message for not
opened input file.
* tp/Texinfo/Convert/Converter.pm (encoded_output_file_name),
tp/Texinfo/Convert/Text.pm (converter, output, set_conf),
tp/Texinfo/Convert/Utils.pm (encoded_output_file_name):
move encoded_output_file_name from Texinfo::Convert::Converter
to Texinfo::Convert::Utils, but keep a wrapper to be able
to call the function from converters in object oriented style.
in Texinfo::Convert::Text, call set_output_encodings,
encoded_output_file_name, decode command line strings and encode
file names and output file contents. Define set_conf() for
Texinfo::Common::set_output_encodings() call.
---
ChangeLog | 22 +++++++
tp/TODO | 38 +++++++++----
tp/Texinfo/Common.pm | 1 +
tp/Texinfo/Convert/Converter.pm | 18 ++----
tp/Texinfo/Convert/Text.pm | 63 +++++++++++++++++----
tp/Texinfo/Convert/Utils.pm | 20 +++++++
tp/Texinfo/ParserNonXS.pm | 9 ++-
tp/Texinfo/XS/parsetexi/Parsetexi.pm | 18 ++++--
tp/tests/formatting/Makefile.am | 2 +-
tp/tests/formatting/list-of-tests | 10 +++-
.../tests/formatting/os\303\251_utf8.texi" | 2 +-
.../os\303\251_utf8_no_setfilename.texi" | 2 -
.../non_ascii_command_line/Chapteur.html | 2 +-
.../res_parser/non_ascii_command_line/index.html | 4 +-
.../os\303\251-texinfo.texi" | 2 +-
.../non_ascii_command_line/os\303\251.2" | 6 --
.../non_ascii_command_line/os\303\251_utf8.1" | 0
.../non_ascii_command_line/os\303\251_utf8.2" | 6 ++
.../os\303\251_utf8_abt.html" | 0
.../os\303\251_utf8_no_setfilename.1" | 0
.../os\303\251_utf8_no_setfilename.2" | 2 +
.../os\303\251_utf8_no_setfilename.txt" | 18 ++++++
.../res_parser/non_ascii_test_epub/os\303\251.2" | 5 --
.../non_ascii_test_epub/os\303\251_utf8.1" | 0
.../non_ascii_test_epub/os\303\251_utf8.2" | 5 ++
.../EPUB/images/an_\303\257mage.png" | Bin
.../EPUB/os\303\251_utf8.opf" | 6 +-
.../EPUB/xhtml/nav_toc.xhtml" | 4 +-
.../EPUB/xhtml/os\303\251_utf8.xhtml" | 0
.../META-INF/container.xml" | 2 +-
.../os\303\251_utf8_epub_package/mimetype" | 0
.../non_ascii_test_rawtext/os\303\251_utf8.1" | 0
.../non_ascii_test_rawtext/os\303\251_utf8.2" | 2 +
.../non_ascii_test_rawtext/os\303\251_utf8.txt" | 19 +++++++
tp/tests/layout/list-of-tests | 2 +-
...atting_non_ascii_no_setfilename_test_rawtext.sh | 19 +++++++
.../formatting_non_ascii_test_rawtext.sh | 19 +++++++
37 files changed, 257 insertions(+), 71 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index b0e1689c64..a92e3be173 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -7,6 +7,28 @@
* doc/info-stnd.texi (Invoking Info) <-R>: Comment out text about
GROFF_SGR as it is hardcoded now.
+2022-03-05 Patrice Dumas <pertusus@free.fr>
+
+ Encode file names in Parser and Text Converter
+
+ * tp/Texinfo/Common.pm (%default_parser_common_customization),
+ tp/Texinfo/ParserNonXS.pm (parse_texi_file),
+ tp/Texinfo/XS/parsetexi/Parsetexi.pm (parser, parse_texi_file):
+ add DATA_INPUT_ENCODING_NAME as a Parser configuration variable
+ and use it to decode the file name in error message for not
+ opened input file.
+
+ * tp/Texinfo/Convert/Converter.pm (encoded_output_file_name),
+ tp/Texinfo/Convert/Text.pm (converter, output, set_conf),
+ tp/Texinfo/Convert/Utils.pm (encoded_output_file_name):
+ move encoded_output_file_name from Texinfo::Convert::Converter
+ to Texinfo::Convert::Utils, but keep a wrapper to be able
+ to call the function from converters in object oriented style.
+ in Texinfo::Convert::Text, call set_output_encodings,
+ encoded_output_file_name, decode command line strings and encode
+ file names and output file contents. Define set_conf() for
+ Texinfo::Common::set_output_encodings() call.
+
2022-03-05 Patrice Dumas <pertusus@free.fr>
Pass created directory charcter string for error messages
diff --git a/tp/TODO b/tp/TODO
index e5094621cd..4eb7e14e85 100644
--- a/tp/TODO
+++ b/tp/TODO
@@ -29,7 +29,8 @@ bytes. To check that they can never be upgraded + document
MACRO_EXPAND
INTERNAL_LINKS
-Tests in non utf8 locales
+
+Tests in non utf8 locale (in practice tested in latin1 locale)
Tests with correct or acceptable results
@@ -55,11 +56,11 @@ non ascii output directory, utf8 manual
./texi2any.pl -o encodé/ ./t/input_files/simplest.texi
test of non ascii included file name in utf8 locale is already in formatting:
-formatting/osé.texi:@include included_akçentêd.texi
-./texi2any.pl --force -I tests/ tests/formatting/os*.texi
+formatting/osé_utf8.texi:@include included_akçentêd.texi
+./texi2any.pl --force -I tests/ tests/formatting/os*_utf8.texi
The file name is utf-8 encoded in messages, which is expected as we do not
decode/encode file names from the command line for messages
-osé.texi:15: warning: undefined flag: vùr
+osé_utf8.texi:15: warning: undefined flag: vùr
t/80include.t cpp_line_latin1
CPP directive with non ascii characters, latin1 manual
@@ -83,12 +84,29 @@ The file names and file contents looks right, though, with
latin1 only
encoded characters.
epub for utf8 encoded manual in latin1 locale
-./texi2any.pl --force -I tests/ --init init/epub3.pm tests/formatting/os*.texi
+./texi2any.pl --force -I tests/ --init init/epub3.pm
tests/formatting/os*_utf8.texi
epub for latin1 encoded manual in latin1 locale
cp tests/tex_html/tex_encode_latin1.texi tex_encodé_latin1.texi
./texi2any.pl --init init/epub3.pm tex_encodé_latin1.texi
+./texi2any.pl --force -I tests/ -c TEXINFO_OUTPUT_FORMAT=rawtext
tests/formatting/os*_utf8.texi
+output file name is in latin1, but the encoding inside is utf8 consistent
+with the document encoding.
+
+./texi2any.pl --force -I tests/ -c TEXINFO_OUTPUT_FORMAT=rawtext
tests/formatting/os*_utf8_no_setfilename.texi
+output file name is utf8 because the utf8 encoded input file name
+is decoded using the locale latin1 encoding keeping the 8bit characters
+from the utf8 encoding, and the encoding inside is utf8
+consistent with the document encoding.
+
+./texi2any.pl --force -I tests/ -o encodé/raw.txt -c
TEXINFO_OUTPUT_FORMAT=rawtext tests/formatting/os*_utf8.texi
+encodé/raw.txt encoded in latin1, and the encoding inside is utf8
+consistent with the document encoding.
+
+./texi2any.pl --force -I tests/ -c TEXINFO_OUTPUT_FORMAT=rawtext -c
'SUBDIR=subdîr' tests/formatting/os*_utf8.texi
+subdîr/osé_utf8.txt encoded in latin1, and the encoding inside is utf8
+consistent with the document encoding.
Tests with incorrect results, though not bugs
@@ -126,19 +144,17 @@ mixing latin1 and utf8.
Tests in utf8 locales. The archive epub file is not tested in the automated
tests.
epub for utf8 encoded manual in utf8 locale
-./texi2any.pl --force -I tests/ --init init/epub3.pm tests/formatting/os*.texi
+./texi2any.pl --force -I tests/ --init init/epub3.pm
tests/formatting/osé_utf8.texi
+Two following tests not important enough to have regression test
+./texi2any.pl --force -I tests/ -o encodé/raw.txt -c
TEXINFO_OUTPUT_FORMAT=rawtext tests/formatting/os*_utf8.texi
+./texi2any.pl --force -I tests/ -c TEXINFO_OUTPUT_FORMAT=rawtext -c
'SUBDIR=subdîr' tests/formatting/os*_utf8.texi
Test more interesting in non utf8 locale
Add tests even if not as interesting in UTF8 locale as in non UTF8?
./texi2any.pl --set TEXINFO_OUTPUT_FORMAT=debugtree --set USE_NODES=0 -o
résultat/encodé.txt ./t/input_files/simplest_no_node_section.texi
-
-Texinfo/Convert/Text.pm output()
-checks decoded/encoded and fix. Need to verify input available information
-
-
Associated code to check, requires bytes in input both for directory and
file name and return bytes
locate_init_file
diff --git a/tp/Texinfo/Common.pm b/tp/Texinfo/Common.pm
index c3479ef573..3be5934289 100644
--- a/tp/Texinfo/Common.pm
+++ b/tp/Texinfo/Common.pm
@@ -111,6 +111,7 @@ my %default_parser_common_customization = (
'DOC_ENCODING_FOR_INPUT_FILE_NAME' => 1, # if true use the document
encoding for input
# file names encoding, else use
# LOCALE_INPUT_FILE_NAME_ENCODING
+ 'DATA_INPUT_ENCODING_NAME' => undef, # encoding of command line strings
);
# Customization variables obeyed only by the parser, and the default values.
diff --git a/tp/Texinfo/Convert/Converter.pm b/tp/Texinfo/Convert/Converter.pm
index df9d68d701..0b5d79f66d 100644
--- a/tp/Texinfo/Convert/Converter.pm
+++ b/tp/Texinfo/Convert/Converter.pm
@@ -229,8 +229,7 @@ sub converter(;$)
}
}
- Texinfo::Common::set_output_encodings($converter,
- $converter->{'parser_info'});
+ Texinfo::Common::set_output_encodings($converter,
$converter->{'parser_info'});
# turn the array to a hash for speed. Not sure it really matters for such
# a small array.
@@ -1051,22 +1050,13 @@ sub encoded_input_file_name($$)
}
# TODO document
+# A wrapper around Texinfo::Utils::encoded_output_file_name() that
+# can be called in converters through an objet oriented syntax.
sub encoded_output_file_name($$)
{
my $self = shift;
my $file_name = shift;
-
- if ($self->get_conf('DOC_ENCODING_FOR_OUTPUT_FILE_NAME')) {
- my $document_encoding;
- $document_encoding = $self->{'parser_info'}->{'input_perl_encoding'}
- if ($self->{'parser_info'}
- and defined($self->{'parser_info'}->{'input_perl_encoding'}));
- return Texinfo::Common::encode_file_name($self, $file_name,
- $document_encoding);
- } else {
- return Texinfo::Common::encode_file_name($self, $file_name,
- $self->get_conf('LOCALE_OUTPUT_FILE_NAME_ENCODING'));
- }
+ return Texinfo::Convert::Utils::encoded_output_file_name($self, $file_name);
}
sub txt_image_text($$$)
diff --git a/tp/Texinfo/Convert/Text.pm b/tp/Texinfo/Convert/Text.pm
index 065030878d..a73785a9f6 100644
--- a/tp/Texinfo/Convert/Text.pm
+++ b/tp/Texinfo/Convert/Text.pm
@@ -26,6 +26,7 @@ use File::Basename;
use Data::Dumper;
use Carp qw(cluck carp);
+use Encode qw(decode);
use Texinfo::Common;
use Texinfo::Convert::Unicode;
@@ -690,6 +691,9 @@ sub converter($)
}
}
+ Texinfo::Common::set_output_encodings($converter,
$converter->{'parser_info'})
+ if ($converter->{'parser_info'});
+
bless $converter;
return $converter;
}
@@ -719,9 +723,13 @@ sub output($$)
#print STDERR "OUTPUT\n";
my $input_basename;
if (defined($self->{'parser_info'}->{'input_file_name'})) {
+ my $input_file_name = $self->{'parser_info'}->{'input_file_name'};
+ my $encoding = $self->{'DATA_INPUT_ENCODING_NAME'};
+ if (defined($encoding)) {
+ $input_file_name = decode($encoding, $input_file_name);
+ }
my ($directories, $suffix);
- ($input_basename, $directories, $suffix)
- = fileparse($self->{'parser_info'}->{'input_file_name'});
+ ($input_basename, $directories, $suffix) = fileparse($input_file_name);
} else {
# This could happen if called on a piece of texinfo
$input_basename = '';
@@ -748,11 +756,14 @@ sub output($$)
}
if (defined($self->{'SUBDIR'})) {
my $destination_directory = File::Spec->canonpath($self->{'SUBDIR'});
- if (! -d $destination_directory) {
- if (!mkdir($destination_directory, oct(755))) {
- #sprintf(__(
- # "could not create directory `%s': %s"),
- # $destination_directory, $!));
+ my ($encoded_destination_directory, $destination_directory_encoding)
+ = Texinfo::Convert::Utils::encoded_output_file_name($self,
+ $destination_directory);
+ if (! -d $encoded_destination_directory) {
+ if (!mkdir($encoded_destination_directory, oct(755))) {
+ warn sprintf(__(
+ "could not create directory `%s': %s"),
+ $destination_directory, $!)."\n";
return undef;
}
}
@@ -760,14 +771,36 @@ sub output($$)
}
} else {
$outfile = $self->{'OUTFILE'};
+ my ($output_basename, $output_directories, $output_suffix)
+ = fileparse($outfile);
+ if (defined($output_directories) and $output_directories ne './'
+ and $output_directories ne '.' and $output_directories ne '') {
+ my ($encoded_output_directories, $output_directories_encoding)
+ = Texinfo::Convert::Utils::encoded_output_file_name($self,
+ $output_directories);
+ if (! -d $encoded_output_directories) {
+ if (!mkdir($encoded_output_directories, oct(755))) {
+ warn sprintf(__(
+ "could not create directory `%s': %s"),
+ $output_directories, $!)."\n";
+ }
+ }
+ }
}
my $fh;
$self->{'output_files'} = {};
+ my ($encoded_outfile, $outfile_encoding);
if (defined($outfile)) {
+ ($encoded_outfile, $outfile_encoding)
+ = Texinfo::Convert::Utils::encoded_output_file_name($self, $outfile);
$fh = Texinfo::Common::output_files_open_out(
$self->{'output_files'}, $self,
- $outfile);
- return undef if (!$fh);
+ $encoded_outfile);
+ if (!$fh) {
+ warn sprintf(__("could not open %s for writing: %s"),
+ $outfile, $!)."\n";
+ return undef;
+ }
}
# mostly relevant for 'enabled_encoding', other options should be the same.
my %options = copy_options_for_convert_text($self);
@@ -781,7 +814,7 @@ sub output($$)
if ($fh) {
print $fh $result;
Texinfo::Common::output_files_register_closed(
- $self->{'output_files'}, $outfile);
+ $self->{'output_files'}, $encoded_outfile);
return undef if (!close($fh));
$result = '';
}
@@ -796,6 +829,16 @@ sub get_conf($$)
return $self->{$key};
}
+# used in Texinfo::Common::set_output_encodings
+sub set_conf($$$)
+{
+ my $self = shift;
+ my $conf = shift;
+ my $value = shift;
+
+ $self->{$conf} = $value;
+}
+
sub errors()
{
return undef;
diff --git a/tp/Texinfo/Convert/Utils.pm b/tp/Texinfo/Convert/Utils.pm
index e2f9916a69..9f4fdf0b0c 100644
--- a/tp/Texinfo/Convert/Utils.pm
+++ b/tp/Texinfo/Convert/Utils.pm
@@ -308,6 +308,26 @@ sub numbered_heading($$$;$)
return $result;
}
+# TODO document
+# this requires a converter argument
+sub encoded_output_file_name($$)
+{
+ my $self = shift;
+ my $file_name = shift;
+
+ if ($self->get_conf('DOC_ENCODING_FOR_OUTPUT_FILE_NAME')) {
+ my $document_encoding;
+ $document_encoding = $self->{'parser_info'}->{'input_perl_encoding'}
+ if ($self->{'parser_info'}
+ and defined($self->{'parser_info'}->{'input_perl_encoding'}));
+ return Texinfo::Common::encode_file_name($self, $file_name,
+ $document_encoding);
+ } else {
+ return Texinfo::Common::encode_file_name($self, $file_name,
+ $self->get_conf('LOCALE_OUTPUT_FILE_NAME_ENCODING'));
+ }
+}
+
# this requires a converter argument. It is defined here, in order
# to hide from the caller the 'translated_commands' converter key
# that is set by Texinfo::Convert::Converter. This is especially
diff --git a/tp/Texinfo/ParserNonXS.pm b/tp/Texinfo/ParserNonXS.pm
index 3920bd076e..1ba9fbe290 100644
--- a/tp/Texinfo/ParserNonXS.pm
+++ b/tp/Texinfo/ParserNonXS.pm
@@ -1012,16 +1012,21 @@ sub _open_in {
}
# parse a texi file
-# $INPUT_FILE_PATH is the name of the parsed file should be a binary string.
+# $INPUT_FILE_PATH is the name of the parsed file and should be a binary
string.
sub parse_texi_file($$)
{
my ($self, $input_file_path) = @_;
my $filehandle = do { local *FH };
if (!_open_in($self, $filehandle, $input_file_path)) {
+ my $input_file_name = $input_file_path;
+ my $encoding = $self->get_conf('DATA_INPUT_ENCODING_NAME');
+ if (defined($encoding)) {
+ $input_file_name = decode($encoding, $input_file_path);
+ }
$self->{'registrar'}->document_error($self,
sprintf(__("could not open %s: %s"),
- $input_file_path, $!));
+ $input_file_name, $!));
return undef;
}
diff --git a/tp/Texinfo/XS/parsetexi/Parsetexi.pm
b/tp/Texinfo/XS/parsetexi/Parsetexi.pm
index a0c7477f18..10bb7d1156 100644
--- a/tp/Texinfo/XS/parsetexi/Parsetexi.pm
+++ b/tp/Texinfo/XS/parsetexi/Parsetexi.pm
@@ -44,6 +44,7 @@ use strict;
use warnings;
use Storable qw(dclone); # standard in 5.007003
+use Encode qw(decode);
use Texinfo::Common;
use Texinfo::Report;
@@ -137,8 +138,8 @@ sub parser (;$$)
}
} elsif ($key eq 'accept_internalvalue') {
set_accept_internalvalue();
- } elsif ($key eq 'registrar') {
- # no action needed
+ } elsif ($key eq 'registrar' or $key eq 'DATA_INPUT_ENCODING_NAME') {
+ # no action needed, only used in perl code
} else {
warn "ignoring parser configuration value \"$key\"\n";
}
@@ -234,14 +235,19 @@ use File::Basename; # for fileparse
sub parse_texi_file ($$)
{
my $self = shift;
- my $file_name = shift;
+ my $input_file_path = shift;
my $tree_stream;
- my $status = parse_file ($file_name);
+ my $status = parse_file ($input_file_path);
if ($status) {
my ($registrar, $configuration_information) = _get_error_registrar($self);
+ my $input_file_name = $input_file_path;
+ my $encoding = $self->get_conf('DATA_INPUT_ENCODING_NAME');
+ if (defined($encoding)) {
+ $input_file_name = decode($encoding, $input_file_path);
+ }
$registrar->document_error($configuration_information,
- sprintf(__("could not open %s: %s"), $file_name, $!));
+ sprintf(__("could not open %s: %s"), $input_file_name, $!));
return undef;
}
@@ -256,7 +262,7 @@ sub parse_texi_file ($$)
############################################################
- my ($basename, $directories, $suffix) = fileparse($file_name);
+ my ($basename, $directories, $suffix) = fileparse($input_file_path);
$self->{'info'}->{'input_file_name'} = $basename;
$self->{'info'}->{'input_directory'} = $directories;
diff --git a/tp/tests/formatting/Makefile.am b/tp/tests/formatting/Makefile.am
index d5cd06aad9..2d44ab7285 100644
--- a/tp/tests/formatting/Makefile.am
+++ b/tp/tests/formatting/Makefile.am
@@ -3,7 +3,7 @@ EXTRA_DIST = \
ignore_and_comments.texi split_nocopying.texi \
inc_file.texi test_need.texi \
lightweight_markups.texi japanese_long_name.texi \
- osé.texi \
+ osé_utf8.texi osé_utf8_no_setfilename.texi \
çss.css cêss.css file.css an_ïmage.png list-of-tests res_parser
DISTCLEANFILES = tests.log tests.out
diff --git a/tp/tests/formatting/list-of-tests
b/tp/tests/formatting/list-of-tests
index e93f8459d6..ff96179427 100644
--- a/tp/tests/formatting/list-of-tests
+++ b/tp/tests/formatting/list-of-tests
@@ -19,8 +19,14 @@ cpp_lines ../../t/input_files/cpp_lines.texi
# ./texi2any.pl --footnote-style=bâd
# ./texi2any.pl --paragraph-indent=ïndent
# check non ascii command line arguments and css files
-non_ascii_command_line osé.texi --html --split=Mekanïk
--document-language=Destruktïw -c 'Kommandöh vâl' -D TÛT -D 'vùr ké' -U ôndef
-c 'FORMAT_MENU mînù' --macro-expand=@OUT_DIR@osé-texinfo.texi
--internal-links=@OUT_DIR@intérnal.txt --css-include çss.css --css-include
cêss.css --css-ref=rëf --css-ref=öref
+non_ascii_command_line osé_utf8.texi --html --split=Mekanïk
--document-language=Destruktïw -c 'Kommandöh vâl' -D TÛT -D 'vùr ké' -U ôndef
-c 'FORMAT_MENU mînù' --macro-expand=@OUT_DIR@osé-texinfo.texi
--internal-links=@OUT_DIR@intérnal.txt --css-include çss.css --css-include
cêss.css --css-ref=rëf --css-ref=öref
# test for the copying of image with non ascii characters for epub
-non_ascii_test_epub osé.texi --init epub3.pm -c 'EPUB_CREATE_CONTAINER 0'
+non_ascii_test_epub osé_utf8.texi --init epub3.pm -c 'EPUB_CREATE_CONTAINER 0'
+
+# check that the output is right when based on @setfilename
+non_ascii_test_rawtext osé_utf8.texi -c TEXINFO_OUTPUT_FORMAT=rawtext
+
+# check that the output is right when based on input file name
+non_ascii_no_setfilename_test_rawtext osé_utf8_no_setfilename.texi -c
TEXINFO_OUTPUT_FORMAT=rawtext
diff --git "a/tp/tests/formatting/os\303\251.texi"
"b/tp/tests/formatting/os\303\251_utf8.texi"
similarity index 92%
copy from "tp/tests/formatting/os\303\251.texi"
copy to "tp/tests/formatting/os\303\251_utf8.texi"
index 10141774bc..21c57ffc74 100644
--- "a/tp/tests/formatting/os\303\251.texi"
+++ "b/tp/tests/formatting/os\303\251_utf8.texi"
@@ -1,6 +1,6 @@
\input texinfo.tex
-@setfilename osé.info
+@setfilename osé_utf8.info
@node Top
@top Tôp
diff --git "a/tp/tests/formatting/os\303\251.texi"
"b/tp/tests/formatting/os\303\251_utf8_no_setfilename.texi"
similarity index 93%
rename from "tp/tests/formatting/os\303\251.texi"
rename to "tp/tests/formatting/os\303\251_utf8_no_setfilename.texi"
index 10141774bc..8b368f4782 100644
--- "a/tp/tests/formatting/os\303\251.texi"
+++ "b/tp/tests/formatting/os\303\251_utf8_no_setfilename.texi"
@@ -1,7 +1,5 @@
\input texinfo.tex
-@setfilename osé.info
-
@node Top
@top Tôp
diff --git
a/tp/tests/formatting/res_parser/non_ascii_command_line/Chapteur.html
b/tp/tests/formatting/res_parser/non_ascii_command_line/Chapteur.html
index 28445f9b5e..25c44935bf 100644
--- a/tp/tests/formatting/res_parser/non_ascii_command_line/Chapteur.html
+++ b/tp/tests/formatting/res_parser/non_ascii_command_line/Chapteur.html
@@ -53,7 +53,7 @@ ul.mark-néni {list-style-type: "vàça"}
<td valign="middle" align="left">[<a href="index.html" title="Cover (top) of
document">Top</a>]</td>
<td valign="middle" align="left">[Contents]</td>
<td valign="middle" align="left">[Index]</td>
-<td valign="middle" align="left">[<a href="osé_abt.html#SEC_About"
title="About (help)"> ? </a>]</td>
+<td valign="middle" align="left">[<a href="osé_utf8_abt.html#SEC_About"
title="About (help)"> ? </a>]</td>
</tr></table>
<hr>
<h1 class="chapter" id="Chapteur">1 Chapteùr</h1>
diff --git a/tp/tests/formatting/res_parser/non_ascii_command_line/index.html
b/tp/tests/formatting/res_parser/non_ascii_command_line/index.html
index 687a5a7831..34092e1514 100644
--- a/tp/tests/formatting/res_parser/non_ascii_command_line/index.html
+++ b/tp/tests/formatting/res_parser/non_ascii_command_line/index.html
@@ -47,7 +47,7 @@ ul.mark-néni {list-style-type: "vàça"}
<td valign="middle" align="left"> </td>
<td valign="middle" align="left">[Contents]</td>
<td valign="middle" align="left">[Index]</td>
-<td valign="middle" align="left">[<a href="osé_abt.html#SEC_About"
title="About (help)"> ? </a>]</td>
+<td valign="middle" align="left">[<a href="osé_utf8_abt.html#SEC_About"
title="About (help)"> ? </a>]</td>
</tr></table>
<hr>
<h1 class="top" id="Top-1">Tôp</h1>
@@ -60,7 +60,7 @@ ul.mark-néni {list-style-type: "vàça"}
<td valign="middle" align="left"> </td>
<td valign="middle" align="left">[Contents]</td>
<td valign="middle" align="left">[Index]</td>
-<td valign="middle" align="left">[<a href="osé_abt.html#SEC_About"
title="About (help)"> ? </a>]</td>
+<td valign="middle" align="left">[<a href="osé_utf8_abt.html#SEC_About"
title="About (help)"> ? </a>]</td>
</tr></table>
<p>
<span class="program-in-footer">This document was generated on <em
class="emph">a sunny day</em> using <a class="uref"
href="http://www.gnu.org/software/texinfo/";><em
class="emph">texi2any</em></a>.</span>
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251-texinfo.texi"
"b/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251-texinfo.texi"
index e4ad1dc5aa..a95a3ac74e 100644
---
"a/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251-texinfo.texi"
+++
"b/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251-texinfo.texi"
@@ -1,6 +1,6 @@
\input texinfo.tex
-@setfilename osé.info
+@setfilename osé_utf8.info
@node Top
@top Tôp
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251.2"
"b/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251.2"
deleted file mode 100644
index eeddb28fa3..0000000000
--- "a/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251.2"
+++ /dev/null
@@ -1,6 +0,0 @@
-texi2any: warning: Mekanïk is not a valid split possibility
-texi2any: warning: Destruktïw is not a valid language code
-texi2any: warning: unknown variable from command line: Kommandöh
-osé.texi:23: @include: could not find not_existïng.téxi
-osé.texi:21: warning: @image file `dîrectory/imàge' (for HTML) not found,
using `dîrectory/imàge.êxt'
-osé.texi:27: @verbatiminclude: could not find vi_not_existïng.téxi
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251.1"
"b/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251_utf8.1"
similarity index 100%
copy from "tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251.1"
copy to
"tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251_utf8.1"
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251_utf8.2"
"b/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251_utf8.2"
new file mode 100644
index 0000000000..f58f4f8afc
--- /dev/null
+++ "b/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251_utf8.2"
@@ -0,0 +1,6 @@
+texi2any: warning: Mekanïk is not a valid split possibility
+texi2any: warning: Destruktïw is not a valid language code
+texi2any: warning: unknown variable from command line: Kommandöh
+osé_utf8.texi:23: @include: could not find not_existïng.téxi
+osé_utf8.texi:21: warning: @image file `dîrectory/imàge' (for HTML) not found,
using `dîrectory/imàge.êxt'
+osé_utf8.texi:27: @verbatiminclude: could not find vi_not_existïng.téxi
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251_abt.html"
"b/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251_utf8_abt.html"
similarity index 100%
rename from
"tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251_abt.html"
rename to
"tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251_utf8_abt.html"
diff --git "a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251.1"
"b/tp/tests/formatting/res_parser/non_ascii_no_setfilename_test_rawtext/os\303\251_utf8_no_setfilename.1"
similarity index 100%
rename from "tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251.1"
rename to
"tp/tests/formatting/res_parser/non_ascii_no_setfilename_test_rawtext/os\303\251_utf8_no_setfilename.1"
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_no_setfilename_test_rawtext/os\303\251_utf8_no_setfilename.2"
"b/tp/tests/formatting/res_parser/non_ascii_no_setfilename_test_rawtext/os\303\251_utf8_no_setfilename.2"
new file mode 100644
index 0000000000..af63de4432
--- /dev/null
+++
"b/tp/tests/formatting/res_parser/non_ascii_no_setfilename_test_rawtext/os\303\251_utf8_no_setfilename.2"
@@ -0,0 +1,2 @@
+osé_utf8_no_setfilename.texi:13: warning: undefined flag: vùr
+osé_utf8_no_setfilename.texi:21: @include: could not find not_existïng.téxi
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_no_setfilename_test_rawtext/os\303\251_utf8_no_setfilename.txt"
"b/tp/tests/formatting/res_parser/non_ascii_no_setfilename_test_rawtext/os\303\251_utf8_no_setfilename.txt"
new file mode 100644
index 0000000000..65f722725e
--- /dev/null
+++
"b/tp/tests/formatting/res_parser/non_ascii_no_setfilename_test_rawtext/os\303\251_utf8_no_setfilename.txt"
@@ -0,0 +1,18 @@
+Tôp
+***
+
+1 Chapteùr
+**********
+
+
+value vùr .
+
+In included téxt.
+
+an_ïmage
+
+dîrectory/imàge
+
+
+In included téxt.
+
diff --git "a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251.2"
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251.2"
deleted file mode 100644
index a4e41d842b..0000000000
--- "a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251.2"
+++ /dev/null
@@ -1,5 +0,0 @@
-osé.texi:15: warning: undefined flag: vùr
-osé.texi:23: @include: could not find not_existïng.téxi
-osé.texi:21: warning: @image file `dîrectory/imàge' (for HTML) not found,
using `dîrectory/imàge.êxt'
-texi2any: @image file `dîrectory/imàge' can not be copied
-osé.texi:27: @verbatiminclude: could not find vi_not_existïng.téxi
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251.1"
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8.1"
similarity index 100%
copy from "tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251.1"
copy to "tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8.1"
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8.2"
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8.2"
new file mode 100644
index 0000000000..74bcf20373
--- /dev/null
+++ "b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8.2"
@@ -0,0 +1,5 @@
+osé_utf8.texi:15: warning: undefined flag: vùr
+osé_utf8.texi:23: @include: could not find not_existïng.téxi
+osé_utf8.texi:21: warning: @image file `dîrectory/imàge' (for HTML) not found,
using `dîrectory/imàge.êxt'
+texi2any: @image file `dîrectory/imàge' can not be copied
+osé_utf8.texi:27: @verbatiminclude: could not find vi_not_existïng.téxi
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/images/an_\303\257mage.png"
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/images/an_\303\257mage.png"
similarity index 100%
rename from
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/images/an_\303\257mage.png"
rename to
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/images/an_\303\257mage.png"
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/os\303\251.opf"
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/os\303\251_utf8.opf"
similarity index 84%
rename from
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/os\303\251.opf"
rename to
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/os\303\251_utf8.opf"
index 661241a187..6ac8460e68 100644
---
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/os\303\251.opf"
+++
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/os\303\251_utf8.opf"
@@ -1,13 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<package xmlns="http://www.idpf.org/2007/opf"; version="3.2"
unique-identifier="texi-uid">
<metadata xmlns:dc="http://purl.org/dc/elements/1.1/";>
- <dc:identifier id="texi-uid">texinfo:osé</dc:identifier>
+ <dc:identifier id="texi-uid">texinfo:osé_utf8</dc:identifier>
<dc:title>Tôp</dc:title>
</metadata>
<manifest>
<item id="nav" properties="nav" media-type="application/xhtml+xml"
href="xhtml/nav_toc.xhtml"/>
- <item id="unit1" media-type="application/xhtml+xml"
href="xhtml/osé.xhtml"/>
- <item id="unit2" media-type="application/xhtml+xml"
href="xhtml/osé.xhtml"/>
+ <item id="unit1" media-type="application/xhtml+xml"
href="xhtml/osé_utf8.xhtml"/>
+ <item id="unit2" media-type="application/xhtml+xml"
href="xhtml/osé_utf8.xhtml"/>
<item id="image1" media-type=" image/png" href="images/an_ïmage.png"/>
</manifest>
<spine>
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/xhtml/nav_toc.xhtml"
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/xhtml/nav_toc.xhtml"
similarity index 72%
rename from
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/xhtml/nav_toc.xhtml"
rename to
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/xhtml/nav_toc.xhtml"
index 23a7c47783..ef634137e1 100644
---
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/xhtml/nav_toc.xhtml"
+++
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/xhtml/nav_toc.xhtml"
@@ -9,8 +9,8 @@
<h1>Table of contents</h1>
<ol>
- <li><a href="osé.xhtml#Top">Tôp</a></li>
- <li><a href="osé.xhtml#Chapt_00eaur">1 Chapteùr</a></li>
+ <li><a href="osé_utf8.xhtml#Top">Tôp</a></li>
+ <li><a href="osé_utf8.xhtml#Chapt_00eaur">1 Chapteùr</a></li>
</ol>
</nav>
</body>
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/xhtml/os\303\251.xhtml"
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/xhtml/os\303\251_utf8.xhtml"
similarity index 100%
rename from
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/EPUB/xhtml/os\303\251.xhtml"
rename to
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/EPUB/xhtml/os\303\251_utf8.xhtml"
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/META-INF/container.xml"
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/META-INF/container.xml"
similarity index 81%
rename from
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/META-INF/container.xml"
rename to
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/META-INF/container.xml"
index 623c34a8f0..6640a792e1 100644
---
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/META-INF/container.xml"
+++
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/META-INF/container.xml"
@@ -1,7 +1,7 @@
<?xml version="1.0"?>
<container version="1.0"
xmlns="urn:oasis:names:tc:opendocument:xmlns:container">
<rootfiles>
- <rootfile full-path="EPUB/osé.opf"
+ <rootfile full-path="EPUB/osé_utf8.opf"
media-type="application/oebps-package+xml" />
</rootfiles>
</container>
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/mimetype"
"b/tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/mimetype"
similarity index 100%
rename from
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_epub_package/mimetype"
rename to
"tp/tests/formatting/res_parser/non_ascii_test_epub/os\303\251_utf8_epub_package/mimetype"
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251.1"
"b/tp/tests/formatting/res_parser/non_ascii_test_rawtext/os\303\251_utf8.1"
similarity index 100%
rename from "tp/tests/formatting/res_parser/non_ascii_command_line/os\303\251.1"
rename to
"tp/tests/formatting/res_parser/non_ascii_test_rawtext/os\303\251_utf8.1"
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_test_rawtext/os\303\251_utf8.2"
"b/tp/tests/formatting/res_parser/non_ascii_test_rawtext/os\303\251_utf8.2"
new file mode 100644
index 0000000000..a4d36147e2
--- /dev/null
+++ "b/tp/tests/formatting/res_parser/non_ascii_test_rawtext/os\303\251_utf8.2"
@@ -0,0 +1,2 @@
+osé_utf8.texi:15: warning: undefined flag: vùr
+osé_utf8.texi:23: @include: could not find not_existïng.téxi
diff --git
"a/tp/tests/formatting/res_parser/non_ascii_test_rawtext/os\303\251_utf8.txt"
"b/tp/tests/formatting/res_parser/non_ascii_test_rawtext/os\303\251_utf8.txt"
new file mode 100644
index 0000000000..0a440d1743
--- /dev/null
+++
"b/tp/tests/formatting/res_parser/non_ascii_test_rawtext/os\303\251_utf8.txt"
@@ -0,0 +1,19 @@
+
+Tôp
+***
+
+1 Chapteùr
+**********
+
+
+value vùr .
+
+In included téxt.
+
+an_ïmage
+
+dîrectory/imàge
+
+
+In included téxt.
+
diff --git a/tp/tests/layout/list-of-tests b/tp/tests/layout/list-of-tests
index 7938c42ebc..20ddce41ff 100644
--- a/tp/tests/layout/list-of-tests
+++ b/tp/tests/layout/list-of-tests
@@ -32,7 +32,7 @@ formatting_html32 formatting.texi --init html32.pm
formatting_regions formatting_regions.texi
formatting_numerical_entities formatting.texi -c 'USE_NUMERIC_ENTITY 1'
formatting_enable_encoding formatting.texi --enable-encoding
-formatting_xhtml formatting.texi -c DOCTYPE='<?xml version="1.0"
encoding="UTF-8"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd";>' -c 'USE_XML_SYNTAX 1' -c
'NO_CUSTOM_HTML_ATTRIBUTE 1' -c 'HTML_ROOT_ELEMENT_ATTRIBUTES
xmlns="http://www.w3.org/1999/xhtml";'
+formatting_xhtml formatting.texi -c DOCTYPE='<?xml version="1.0"
encoding="UTF-8"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd";>' -c 'USE_XML_SYNTAX 1' -c
'NO_CUSTOM_HTML_ATTRIBUTE 1' -c 'HTML_ROOT_ELEMENT_ATTRIBUTES
xmlns="http://www.w3.org/1999/xhtml";'
formatting_exotic formatting.texi --split section --no-header
--no-number-sections -c 'TOC_LINKS 1' -c 'DEF_TABLE 1' -c
'XREF_USE_NODE_NAME_ARG 1' --footnote-style=end --css-ref
http://www.environnement.ens.fr/perso/dumas/background-color.css
--internal-links=/dev/null -c 'USE_TITLEPAGE_FOR_TITLE 0'
# use of the doctype is to be able to use W3C old validator, it
# could be removed when validation can be done differently
diff --git
a/tp/tests/test_scripts/formatting_non_ascii_no_setfilename_test_rawtext.sh
b/tp/tests/test_scripts/formatting_non_ascii_no_setfilename_test_rawtext.sh
new file mode 100755
index 0000000000..452ca6eeab
--- /dev/null
+++ b/tp/tests/test_scripts/formatting_non_ascii_no_setfilename_test_rawtext.sh
@@ -0,0 +1,19 @@
+#! /bin/sh
+# This file generated by maintain/regenerate_cmd_tests.sh
+
+if test z"$srcdir" = "z"; then
+ srcdir=.
+fi
+
+one_test_logs_dir=test_log
+
+
+dir=formatting
+name='non_ascii_no_setfilename_test_rawtext'
+mkdir -p $dir
+
+"$srcdir"/run_parser_all.sh -dir $dir $name
+exit_status=$?
+cat $dir/$one_test_logs_dir/$name.log
+exit $exit_status
+
diff --git a/tp/tests/test_scripts/formatting_non_ascii_test_rawtext.sh
b/tp/tests/test_scripts/formatting_non_ascii_test_rawtext.sh
new file mode 100755
index 0000000000..271a74e1cd
--- /dev/null
+++ b/tp/tests/test_scripts/formatting_non_ascii_test_rawtext.sh
@@ -0,0 +1,19 @@
+#! /bin/sh
+# This file generated by maintain/regenerate_cmd_tests.sh
+
+if test z"$srcdir" = "z"; then
+ srcdir=.
+fi
+
+one_test_logs_dir=test_log
+
+
+dir=formatting
+name='non_ascii_test_rawtext'
+mkdir -p $dir
+
+"$srcdir"/run_parser_all.sh -dir $dir $name
+exit_status=$?
+cat $dir/$one_test_logs_dir/$name.log
+exit $exit_status
+
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- branch master updated: Encode file names in Parser and Text Converter,
Patrice Dumas <=