Changes/additions to input character set handling:

 - After discussion with Simon, change the default input charset back to ASCII,
   rather than trying to work it out from the locale, for the sake of promoting
   .but file portability.

 - Add a new command-line option "--input-charset=csname", which overrides the
   ASCII default for all input files (since there's no other way to use a
   non-ASCII-compatible input file).

 - Output a warning if -Cinput-charset:foo is specified that it has no effect.

 - Update the docs to match all this. Also try to clarify some other things in
   this area that caught me out.

[originally from svn r5332]

3 files changed, 32 insertions, 5 deletions

diff --git a/doc/input.but b/doc/input.but
index f0097c5..7d5eb97 100644
--- a/doc/input.but
+++ b/doc/input.but
@@ -1338,9 +1338,9 @@ For example, \c{iso-8859-1}, \c{iso8859-1} and \c{iso_8859-1} are
 all recognised, \c{GB2312} and \c{EUC-CN} both work, and so on.
 
 This directive takes effect immediately after the \c{\\cfg} command.
-All text after that in the file is expected to be in the new
-character set. You can even change character set several times
-within a file if you really want to.
+All text after that until the end of the input file is expected to be
+in the new character set. You can even change character set several
+times within a file if you really want to. 
 
 When Halibut reads the input file, everything you type will be
 converted into \i{Unicode} from the character set you specify here,
@@ -1403,8 +1403,10 @@ The \i{default settings} for the above options are:
 \c \cfg{appendix}{Appendix}
 \c \cfg{input-charset}{ASCII}
 
-(The default settings for \cw{\\cfg\{quotes\}} are backend-specific;
-see \k{output}.)
+The default for \cw{\\cfg\{input-charset\}} can be changed with the
+\cw{--input-charset} option; see \k{running-options}. The default
+settings for \cw{\\cfg\{quotes\}} are backend-specific; see
+\k{output}.
 
 \H{input-macro} Defining \i{macros}
 
diff --git a/doc/manpage.but b/doc/manpage.but
index 94b42d6..5889e59 100644
--- a/doc/manpage.but
+++ b/doc/manpage.but
@@ -106,6 +106,11 @@ Halibut. Using this directive is exactly equivalent to appending an
 extra input file to the command line which contains the directive
 \cw{\\cfg\{}\e{word}\cw{\}\{}\e{word}\cw{\}\{}\e{word}...\cw{\}}.
 
+\dt \cw{--input-charset}\cw{=}\e{charset}
+
+\dd Changes the assumed character set for input files from the
+default of ASCII.
+
 \dt \cw{--precise}
 
 \dd Makes Halibut report the column number as well as the line
diff --git a/doc/running.but b/doc/running.but
index 37e36f3..56790ec 100644
--- a/doc/running.but
+++ b/doc/running.but
@@ -163,6 +163,10 @@ might have to type four backslashes on your shell command line. This
 is not part of Halibut's own behaviour, and it cannot do anything
 about it.)
 
+Configuration directives created in this way take effect after all
+other input has been processed. (In most cases, this has the effect of
+overriding any other instances of the directive in the input.)
+
 }
 
 The options which set the output file names actually work by
@@ -178,6 +182,22 @@ directives.)
 
 In addition to these, there are also a few other options:
 
+\dt \i\cw{--input-charset}\cw{=}\e{charset}
+
+\dd Changes the default assumed character set for all input files from
+ASCII to something else. (\cw{-Cinput-charset} cannot be used for
+this, as \cw{-C} directives are processed after all other input, so
+wouldn't affect any files.)
+
+\lcont{
+
+Any \cw{\\cfg\{input-charset\}} directives within input files override
+this option.
+
+See \k{input-config} for more information about the input character set.
+
+}
+
 \dt \i\cw{--help}
 
 \dd Print a brief help message and exit immediately. (Don't confuse