diff options
Diffstat (limited to 'upstream/archlinux/man1/perllocale.1perl')
| -rw-r--r-- | upstream/archlinux/man1/perllocale.1perl | 1764 |
1 files changed, 1764 insertions, 0 deletions
diff --git a/upstream/archlinux/man1/perllocale.1perl b/upstream/archlinux/man1/perllocale.1perl new file mode 100644 index 00000000..835b04d5 --- /dev/null +++ b/upstream/archlinux/man1/perllocale.1perl @@ -0,0 +1,1764 @@ +.\" -*- mode: troff; coding: utf-8 -*- +.\" Automatically generated by Pod::Man 5.01 (Pod::Simple 3.43) +.\" +.\" Standard preamble: +.\" ======================================================================== +.de Sp \" Vertical space (when we can't use .PP) +.if t .sp .5v +.if n .sp +.. +.de Vb \" Begin verbatim text +.ft CW +.nf +.ne \\$1 +.. +.de Ve \" End verbatim text +.ft R +.fi +.. +.\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>. +.ie n \{\ +. ds C` "" +. ds C' "" +'br\} +.el\{\ +. ds C` +. ds C' +'br\} +.\" +.\" Escape single quotes in literal strings from groff's Unicode transform. +.ie \n(.g .ds Aq \(aq +.el .ds Aq ' +.\" +.\" If the F register is >0, we'll generate index entries on stderr for +.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index +.\" entries marked with X<> in POD. Of course, you'll have to process the +.\" output yourself in some meaningful fashion. +.\" +.\" Avoid warning from groff about undefined register 'F'. +.de IX +.. +.nr rF 0 +.if \n(.g .if rF .nr rF 1 +.if (\n(rF:(\n(.g==0)) \{\ +. if \nF \{\ +. de IX +. tm Index:\\$1\t\\n%\t"\\$2" +.. +. if !\nF==2 \{\ +. nr % 0 +. nr F 2 +. \} +. \} +.\} +.rr rF +.\" ======================================================================== +.\" +.IX Title "PERLLOCALE 1perl" +.TH PERLLOCALE 1perl 2024-02-11 "perl v5.38.2" "Perl Programmers Reference Guide" +.\" For nroff, turn off justification. Always turn off hyphenation; it makes +.\" way too many mistakes in technical documents. +.if n .ad l +.nh +.SH NAME +perllocale \- Perl locale handling (internationalization and localization) +.SH DESCRIPTION +.IX Header "DESCRIPTION" +In the beginning there was ASCII, the "American Standard Code for +Information Interchange", which works quite well for Americans with +their English alphabet and dollar-denominated currency. But it doesn't +work so well even for other English speakers, who may use different +currencies, such as the pound sterling (as the symbol for that currency +is not in ASCII); and it's hopelessly inadequate for many of the +thousands of the world's other languages. +.PP +To address these deficiencies, the concept of locales was invented +(formally the ISO C, XPG4, POSIX 1.c "locale system"). And applications +were and are being written that use the locale mechanism. The process of +making such an application take account of its users' preferences in +these kinds of matters is called \fBinternationalization\fR (often +abbreviated as \fBi18n\fR); telling such an application about a particular +set of preferences is known as \fBlocalization\fR (\fBl10n\fR). +.PP +Perl has been extended to support certain types of locales available in +the locale system. This is controlled per application by using one +pragma, one function call, and several environment variables. +.PP +Perl supports single-byte locales that are supersets of ASCII, such as +the ISO 8859 ones, and one multi-byte-type locale, UTF\-8 ones, described +in the next paragraph. Perl doesn't support any other multi-byte +locales, such as the ones for East Asian languages. +.PP +Unfortunately, there are quite a few deficiencies with the design (and +often, the implementations) of locales. Unicode was invented (see +perlunitut for an introduction to that) in part to address these +design deficiencies, and nowadays, there is a series of "UTF\-8 +locales", based on Unicode. These are locales whose character set is +Unicode, encoded in UTF\-8. Starting in v5.20, Perl fully supports +UTF\-8 locales, except for sorting and string comparisons like \f(CW\*(C`lt\*(C'\fR and +\&\f(CW\*(C`ge\*(C'\fR. Starting in v5.26, Perl can handle these reasonably as well, +depending on the platform's implementation. However, for earlier +releases or for better control, use Unicode::Collate. There are +actually two slightly different types of UTF\-8 locales: one for Turkic +languages and one for everything else. +.PP +Starting in Perl v5.30, Perl detects Turkic locales by their +behaviour, and seamlessly handles both types; previously only the +non-Turkic one was supported. The name of the locale is ignored, if +your system has a \f(CW\*(C`tr_TR.UTF\-8\*(C'\fR locale and it doesn't behave like a +Turkic locale, perl will treat it like a non-Turkic locale. +.PP +Perl continues to support the old non UTF\-8 locales as well. There are +currently no UTF\-8 locales for EBCDIC platforms. +.PP +(Unicode is also creating \f(CW\*(C`CLDR\*(C'\fR, the "Common Locale Data Repository", +<http://cldr.unicode.org/> which includes more types of information than +are available in the POSIX locale system. At the time of this writing, +there was no CPAN module that provides access to this XML-encoded data. +However, it is possible to compute the POSIX locale data from them, and +earlier CLDR versions had these already extracted for you as UTF\-8 locales +<http://unicode.org/Public/cldr/2.0.1/>.) +.SH "WHAT IS A LOCALE" +.IX Header "WHAT IS A LOCALE" +A locale is a set of data that describes various aspects of how various +communities in the world categorize their world. These categories are +broken down into the following types (some of which include a brief +note here): +.ie n .IP "Category ""LC_NUMERIC"": Numeric formatting" 4 +.el .IP "Category \f(CWLC_NUMERIC\fR: Numeric formatting" 4 +.IX Item "Category LC_NUMERIC: Numeric formatting" +This indicates how numbers should be formatted for human readability, +for example the character used as the decimal point. +.ie n .IP "Category ""LC_MONETARY"": Formatting of monetary amounts" 4 +.el .IP "Category \f(CWLC_MONETARY\fR: Formatting of monetary amounts" 4 +.IX Item "Category LC_MONETARY: Formatting of monetary amounts" + +.ie n .IP "Category ""LC_TIME"": Date/Time formatting" 4 +.el .IP "Category \f(CWLC_TIME\fR: Date/Time formatting" 4 +.IX Item "Category LC_TIME: Date/Time formatting" + +.ie n .IP "Category ""LC_MESSAGES"": Error and other messages" 4 +.el .IP "Category \f(CWLC_MESSAGES\fR: Error and other messages" 4 +.IX Item "Category LC_MESSAGES: Error and other messages" +This is used by Perl itself only for accessing operating system error +messages via \f(CW$!\fR and \f(CW$^E\fR. +.ie n .IP "Category ""LC_COLLATE"": Collation" 4 +.el .IP "Category \f(CWLC_COLLATE\fR: Collation" 4 +.IX Item "Category LC_COLLATE: Collation" +This indicates the ordering of letters for comparison and sorting. +In Latin alphabets, for example, "b", generally follows "a". +.ie n .IP "Category ""LC_CTYPE"": Character Types" 4 +.el .IP "Category \f(CWLC_CTYPE\fR: Character Types" 4 +.IX Item "Category LC_CTYPE: Character Types" +This indicates, for example if a character is an uppercase letter. +.IP "Other categories" 4 +.IX Item "Other categories" +Some platforms have other categories, dealing with such things as +measurement units and paper sizes. None of these are used directly by +Perl, but outside operations that Perl interacts with may use +these. See "Not within the scope of "use locale"" below. +.PP +More details on the categories used by Perl are given below in "LOCALE +CATEGORIES". +.PP +Together, these categories go a long way towards being able to customize +a single program to run in many different locations. But there are +deficiencies, so keep reading. +.SH "PREPARING TO USE LOCALES" +.IX Header "PREPARING TO USE LOCALES" +Perl itself (outside the POSIX module) will not use locales unless +specifically requested to (but +again note that Perl may interact with code that does use them). Even +if there is such a request, \fBall\fR of the following must be true +for it to work properly: +.IP \(bu 4 +\&\fBYour operating system must support the locale system\fR. If it does, +you should find that the \f(CWsetlocale()\fR function is a documented part of +its C library. +.IP \(bu 4 +\&\fBDefinitions for locales that you use must be installed\fR. You, or +your system administrator, must make sure that this is the case. The +available locales, the location in which they are kept, and the manner +in which they are installed all vary from system to system. Some systems +provide only a few, hard-wired locales and do not allow more to be +added. Others allow you to add "canned" locales provided by the system +supplier. Still others allow you or the system administrator to define +and add arbitrary locales. (You may have to ask your supplier to +provide canned locales that are not delivered with your operating +system.) Read your system documentation for further illumination. +.IP \(bu 4 +\&\fBPerl must believe that the locale system is supported\fR. If it does, +\&\f(CW\*(C`perl \-V:d_setlocale\*(C'\fR will say that the value for \f(CW\*(C`d_setlocale\*(C'\fR is +\&\f(CW\*(C`define\*(C'\fR. +.PP +If you want a Perl application to process and present your data +according to a particular locale, the application code should include +the \f(CW\*(C`use\ locale\*(C'\fR pragma (see "The "use locale" pragma") where +appropriate, and \fBat least one\fR of the following must be true: +.IP 1. 4 +\&\fBThe locale-determining environment variables (see "ENVIRONMENT") +must be correctly set up\fR at the time the application is started, either +by yourself or by whomever set up your system account; or +.IP 2. 4 +\&\fBThe application must set its own locale\fR using the method described in +"The setlocale function". +.SH "USING LOCALES" +.IX Header "USING LOCALES" +.ie n .SS "The ""use locale"" pragma" +.el .SS "The \f(CW""use locale""\fP pragma" +.IX Subsection "The ""use locale"" pragma" +Starting in Perl 5.28, this pragma may be used in +multi-threaded applications on systems that have thread-safe +locale ability. Some caveats apply, see "Multi-threaded" below. On +systems without this capability, or in earlier Perls, do NOT use this +pragma in scripts that have multiple threads active. The +locale in these cases is not local to a single thread. Another thread +may change the locale at any time, which could cause at a minimum that a +given thread is operating in a locale it isn't expecting to be in. On +some platforms, segfaults can also occur. The locale change need not be +explicit; some operations cause perl itself to change the locale. You +are vulnerable simply by having done a \f(CW"use\ locale"\fR. +.PP +By default, Perl itself (outside the POSIX module) +ignores the current locale. The \f(CW\*(C`use\ locale\*(C'\fR +pragma tells Perl to use the current locale for some operations. +Starting in v5.16, there are optional parameters to this pragma, +described below, which restrict which operations are affected by it. +.PP +The current locale is set at execution time by +\&\fBsetlocale()\fR described below. If that function +hasn't yet been called in the course of the program's execution, the +current locale is that which was determined by the "ENVIRONMENT" in +effect at the start of the program. +If there is no valid environment, the current locale is whatever the +system default has been set to. On POSIX systems, it is likely, but +not necessarily, the "C" locale. On Windows, the default is set via the +computer's \f(CW\*(C`Control\ Panel\->Regional\ and\ Language\ Options\*(C'\fR (or its +current equivalent). +.PP +The operations that are affected by locale are: +.ie n .IP "\fBNot within the scope of \fR\fB""use locale""\fR" 4 +.el .IP "\fBNot within the scope of \fR\f(CB""use locale""\fR" 4 +.IX Item "Not within the scope of ""use locale""" +Only certain operations (all originating outside Perl) should be +affected, as follows: +.RS 4 +.IP \(bu 4 +The current locale is used when going outside of Perl with +operations like \fBsystem()\fR or +qx//, if those operations are +locale-sensitive. +.IP \(bu 4 +Also Perl gives access to various C library functions through the +POSIX module. Some of those functions are always affected by the +current locale. For example, \f(CWPOSIX::strftime()\fR uses \f(CW\*(C`LC_TIME\*(C'\fR; +\&\f(CWPOSIX::strtod()\fR uses \f(CW\*(C`LC_NUMERIC\*(C'\fR; \f(CWPOSIX::strcoll()\fR and +\&\f(CWPOSIX::strxfrm()\fR use \f(CW\*(C`LC_COLLATE\*(C'\fR. All such functions +will behave according to the current underlying locale, even if that +locale isn't exposed to Perl space. +.Sp +This applies as well to I18N::Langinfo. +.IP \(bu 4 +XS modules for all categories but \f(CW\*(C`LC_NUMERIC\*(C'\fR get the underlying +locale, and hence any C library functions they call will use that +underlying locale. For more discussion, see "CAVEATS" in perlxs. +.RE +.RS 4 +.Sp +Note that all C programs (including the perl interpreter, which is +written in C) always have an underlying locale. That locale is the "C" +locale unless changed by a call to \fBsetlocale()\fR. When Perl starts up, it changes the underlying locale to the +one which is indicated by the "ENVIRONMENT". When using the POSIX +module or writing XS code, it is important to keep in mind that the +underlying locale may be something other than "C", even if the program +hasn't explicitly changed it. +.Sp + +.RE +.ie n .IP "\fBLingering effects of \fR\fB""use\ locale""\fR" 4 +.el .IP "\fBLingering effects of \fR\f(CBuse\ locale\fR" 4 +.IX Item "Lingering effects of use locale" +Certain Perl operations that are set-up within the scope of a +\&\f(CW\*(C`use locale\*(C'\fR retain that effect even outside the scope. +These include: +.RS 4 +.IP \(bu 4 +The output format of a \fBwrite()\fR is determined by an +earlier format declaration ("format" in perlfunc), so whether or not the +output is affected by locale is determined by if the \f(CWformat()\fR is +within the scope of a \f(CW\*(C`use locale\*(C'\fR, not whether the \f(CWwrite()\fR +is. +.IP \(bu 4 +Regular expression patterns can be compiled using +qr// with actual +matching deferred to later. Again, it is whether or not the compilation +was done within the scope of \f(CW\*(C`use locale\*(C'\fR that determines the match +behavior, not if the matches are done within such a scope or not. +.RE +.RS 4 +.Sp + +.RE +.ie n .IP "\fBUnder \fR\fB""""use locale"";""\fR" 4 +.el .IP "\fBUnder \fR\f(CB""use locale"";\fR" 4 +.IX Item "Under ""use locale"";" +.RS 4 +.PD 0 +.IP \(bu 4 +.PD +All the above operations +.IP \(bu 4 +\&\fBFormat declarations\fR ("format" in perlfunc) and hence any subsequent +\&\f(CWwrite()\fRs use \f(CW\*(C`LC_NUMERIC\*(C'\fR. +.IP \(bu 4 +\&\fBstringification and output\fR use \f(CW\*(C`LC_NUMERIC\*(C'\fR. +These include the results of +\&\f(CWprint()\fR, +\&\f(CWprintf()\fR, +\&\f(CWsay()\fR, +and +\&\f(CWsprintf()\fR. +.IP \(bu 4 +\&\fBThe comparison operators\fR (\f(CW\*(C`lt\*(C'\fR, \f(CW\*(C`le\*(C'\fR, \f(CW\*(C`cmp\*(C'\fR, \f(CW\*(C`ge\*(C'\fR, and \f(CW\*(C`gt\*(C'\fR) use +\&\f(CW\*(C`LC_COLLATE\*(C'\fR. \f(CWsort()\fR is also affected if used without an +explicit comparison function, because it uses \f(CW\*(C`cmp\*(C'\fR by default. +.Sp +\&\fBNote:\fR \f(CW\*(C`eq\*(C'\fR and \f(CW\*(C`ne\*(C'\fR are unaffected by locale: they always +perform a char-by-char comparison of their scalar operands. What's +more, if \f(CW\*(C`cmp\*(C'\fR finds that its operands are equal according to the +collation sequence specified by the current locale, it goes on to +perform a char-by-char comparison, and only returns \fI0\fR (equal) if the +operands are char-for-char identical. If you really want to know whether +two strings\-\-which \f(CW\*(C`eq\*(C'\fR and \f(CW\*(C`cmp\*(C'\fR may consider different\-\-are equal +as far as collation in the locale is concerned, see the discussion in +"Category \f(CW\*(C`LC_COLLATE\*(C'\fR: Collation". +.IP \(bu 4 +\&\fBRegular expressions and case-modification functions\fR (\f(CWuc()\fR, \f(CWlc()\fR, +\&\f(CWucfirst()\fR, and \f(CWlcfirst()\fR) use \f(CW\*(C`LC_CTYPE\*(C'\fR +.IP \(bu 4 +\&\fBThe variables \fR\f(CB$!\fR (and its synonyms \f(CW$ERRNO\fR and +\&\f(CW$OS_ERROR\fR) \fBand\fR \f(CW$^E\fR> (and its synonym +\&\f(CW$EXTENDED_OS_ERROR\fR) when used as strings use \f(CW\*(C`LC_MESSAGES\*(C'\fR. +.RE +.RS 4 +.RE +.PP +The default behavior is restored with the \f(CW\*(C`no\ locale\*(C'\fR pragma, or +upon reaching the end of the block enclosing \f(CW\*(C`use locale\*(C'\fR. +Note that \f(CW\*(C`use locale\*(C'\fR calls may be +nested, and that what is in effect within an inner scope will revert to +the outer scope's rules at the end of the inner scope. +.PP +The string result of any operation that uses locale +information is tainted (if your perl supports taint checking), +as it is possible for a locale to be untrustworthy. +See "SECURITY". +.PP +Starting in Perl v5.16 in a very limited way, and more generally in +v5.22, you can restrict which category or categories are enabled by this +particular instance of the pragma by adding parameters to it. For +example, +.PP +.Vb 1 +\& use locale qw(:ctype :numeric); +.Ve +.PP +enables locale awareness within its scope of only those operations +(listed above) that are affected by \f(CW\*(C`LC_CTYPE\*(C'\fR and \f(CW\*(C`LC_NUMERIC\*(C'\fR. +.PP +The possible categories are: \f(CW\*(C`:collate\*(C'\fR, \f(CW\*(C`:ctype\*(C'\fR, \f(CW\*(C`:messages\*(C'\fR, +\&\f(CW\*(C`:monetary\*(C'\fR, \f(CW\*(C`:numeric\*(C'\fR, \f(CW\*(C`:time\*(C'\fR, and the pseudo category +\&\f(CW\*(C`:characters\*(C'\fR (described below). +.PP +Thus you can say +.PP +.Vb 1 +\& use locale \*(Aq:messages\*(Aq; +.Ve +.PP +and only \f(CW$!\fR and \f(CW$^E\fR +will be locale aware. Everything else is unaffected. +.PP +Since Perl doesn't currently do anything with the \f(CW\*(C`LC_MONETARY\*(C'\fR +category, specifying \f(CW\*(C`:monetary\*(C'\fR does effectively nothing. Some +systems have other categories, such as \f(CW\*(C`LC_PAPER\*(C'\fR, but Perl +also doesn't do anything with them, and there is no way to specify +them in this pragma's arguments. +.PP +You can also easily say to use all categories but one, by either, for +example, +.PP +.Vb 2 +\& use locale \*(Aq:!ctype\*(Aq; +\& use locale \*(Aq:not_ctype\*(Aq; +.Ve +.PP +both of which mean to enable locale awareness of all categories but +\&\f(CW\*(C`LC_CTYPE\*(C'\fR. Only one category argument may be specified in a +\&\f(CW\*(C`use\ locale\*(C'\fR if it is of the negated form. +.PP +Prior to v5.22 only one form of the pragma with arguments is available: +.PP +.Vb 1 +\& use locale \*(Aq:not_characters\*(Aq; +.Ve +.PP +(and you have to say \f(CW\*(C`not_\*(C'\fR; you can't use the bang \f(CW\*(C`!\*(C'\fR form). This +pseudo category is a shorthand for specifying both \f(CW\*(C`:collate\*(C'\fR and +\&\f(CW\*(C`:ctype\*(C'\fR. Hence, in the negated form, it is nearly the same thing as +saying +.PP +.Vb 1 +\& use locale qw(:messages :monetary :numeric :time); +.Ve +.PP +We use the term "nearly", because \f(CW\*(C`:not_characters\*(C'\fR also turns on +\&\f(CW\*(C`use\ feature\ \*(Aqunicode_strings\*(Aq\*(C'\fR within its scope. This form is +less useful in v5.20 and later, and is described fully in +"Unicode and UTF\-8", but briefly, it tells Perl to not use the +character portions of the locale definition, that is the \f(CW\*(C`LC_CTYPE\*(C'\fR and +\&\f(CW\*(C`LC_COLLATE\*(C'\fR categories. Instead it will use the native character set +(extended by Unicode). When using this parameter, you are responsible +for getting the external character set translated into the +native/Unicode one (which it already will be if it is one of the +increasingly popular UTF\-8 locales). There are convenient ways of doing +this, as described in "Unicode and UTF\-8". +.SS "The setlocale function" +.IX Subsection "The setlocale function" +WARNING! Prior to Perl 5.28 or on a system that does not support +thread-safe locale operations, do NOT use this function in a +thread. The locale will change in all other threads at the +same time, and should your thread get paused by the operating system, +and another started, that thread will not have the locale it is +expecting. On some platforms, there can be a race leading to segfaults +if two threads call this function nearly simultaneously. This warning +does not apply on unthreaded builds, or on perls where +\&\f(CW\*(C`${^SAFE_LOCALES}\*(C'\fR exists and is non-zero; namely Perl 5.28 and later +unthreaded or compiled to be locale-thread-safe. On z/OS systems, this +function becomes a no-op once any thread is started. Thus, on that +system, you can set up the locale before creating any threads, and that +locale will be the one in effect for the entire program. +.PP +Otherwise, you can switch locales as often as you wish at run time with +the \f(CWPOSIX::setlocale()\fR function: +.PP +.Vb 6 +\& # Import locale\-handling tool set from POSIX module. +\& # This example uses: setlocale \-\- the function call +\& # LC_CTYPE \-\- explained below +\& # (Showing the testing for success/failure of operations is +\& # omitted in these examples to avoid distracting from the main +\& # point) +\& +\& use POSIX qw(locale_h); +\& use locale; +\& my $old_locale; +\& +\& # query and save the old locale +\& $old_locale = setlocale(LC_CTYPE); +\& +\& setlocale(LC_CTYPE, "fr_CA.ISO8859\-1"); +\& # LC_CTYPE now in locale "French, Canada, codeset ISO 8859\-1" +\& +\& setlocale(LC_CTYPE, ""); +\& # LC_CTYPE now reset to the default defined by the +\& # LC_ALL/LC_CTYPE/LANG environment variables, or to the system +\& # default. See below for documentation. +\& +\& # restore the old locale +\& setlocale(LC_CTYPE, $old_locale); +.Ve +.PP +The first argument of \f(CWsetlocale()\fR gives the \fBcategory\fR, the second the +\&\fBlocale\fR. The category tells in what aspect of data processing you +want to apply locale-specific rules. Category names are discussed in +"LOCALE CATEGORIES" and "ENVIRONMENT". The locale is the name of a +collection of customization information corresponding to a particular +combination of language, country or territory, and codeset. Read on for +hints on the naming of locales: not all systems name locales as in the +example. +.PP +If no second argument is provided and the category is something other +than \f(CW\*(C`LC_ALL\*(C'\fR, the function returns a string naming the current locale +for the category. You can use this value as the second argument in a +subsequent call to \f(CWsetlocale()\fR, \fBbut\fR on some platforms the string +is opaque, not something that most people would be able to decipher as +to what locale it means. +.PP +If no second argument is provided and the category is \f(CW\*(C`LC_ALL\*(C'\fR, the +result is implementation-dependent. It may be a string of +concatenated locale names (separator also implementation-dependent) +or a single locale name. Please consult your \fBsetlocale\fR\|(3) man page for +details. +.PP +If a second argument is given and it corresponds to a valid locale, +the locale for the category is set to that value, and the function +returns the now-current locale value. You can then use this in yet +another call to \f(CWsetlocale()\fR. (In some implementations, the return +value may sometimes differ from the value you gave as the second +argument\-\-think of it as an alias for the value you gave.) +.PP +As the example shows, if the second argument is an empty string, the +category's locale is returned to the default specified by the +corresponding environment variables. Generally, this results in a +return to the default that was in force when Perl started up: changes +to the environment made by the application after startup may or may not +be noticed, depending on your system's C library. +.PP +Note that when a form of \f(CW\*(C`use locale\*(C'\fR that doesn't include all +categories is specified, Perl ignores the excluded categories. +.PP +If \f(CWsetlocale()\fR fails for some reason (for example, an attempt to set +to a locale unknown to the system), the locale for the category is not +changed, and the function returns \f(CW\*(C`undef\*(C'\fR. +.PP +Starting in Perl 5.28, on multi-threaded perls compiled on systems that +implement POSIX 2008 thread-safe locale operations, this function +doesn't actually call the system \f(CW\*(C`setlocale\*(C'\fR. Instead those +thread-safe operations are used to emulate the \f(CW\*(C`setlocale\*(C'\fR function, +but in a thread-safe manner. +.PP +You can force the thread-safe locale operations to always be used (if +available) by recompiling perl with +.PP +.Vb 1 +\& \-Accflags=\*(Aq\-DUSE_THREAD_SAFE_LOCALE\*(Aq +.Ve +.PP +added to your call to \fIConfigure\fR. +.PP +For further information about the categories, consult \fBsetlocale\fR\|(3). +.SS "Multi-threaded operation" +.IX Subsection "Multi-threaded operation" +Beginning in Perl 5.28, multi-threaded locale operation is supported on +systems that implement either the POSIX 2008 or Windows-specific +thread-safe locale operations. Many modern systems, such as various +Unix variants and Darwin do have this. +.PP +You can tell if using locales is safe on your system by looking at the +read-only boolean variable \f(CW\*(C`${^SAFE_LOCALES}\*(C'\fR. The value is 1 if the +perl is not threaded, or if it is using thread-safe locale operations. +.PP +Thread-safe operations are supported in Windows starting in Visual Studio +2005, and in systems compatible with POSIX 2008. Some platforms claim +to support POSIX 2008, but have buggy implementations, so that the hints +files for compiling to run on them turn off attempting to use +thread-safety. \f(CW\*(C`${^SAFE_LOCALES}\*(C'\fR will be 0 on them. +.PP +Be aware that writing a multi-threaded application will not be portable +to a platform which lacks the native thread-safe locale support. On +systems that do have it, you automatically get this behavior for +threaded perls, without having to do anything. If for some reason, you +don't want to use this capability (perhaps the POSIX 2008 support is +buggy on your system), you can manually compile Perl to use the old +non-thread-safe implementation by passing the argument +\&\f(CW\*(C`\-Accflags=\*(Aq\-DNO_THREAD_SAFE_LOCALE\*(Aq\*(C'\fR to \fIConfigure\fR. +Except on Windows, this will continue to use certain of the POSIX 2008 +functions in some situations. If these are buggy, you can pass the +following to \fIConfigure\fR instead or additionally: +\&\f(CW\*(C`\-Accflags=\*(Aq\-DNO_POSIX_2008_LOCALE\*(Aq\*(C'\fR. This will also keep the code +from using thread-safe locales. +\&\f(CW\*(C`${^SAFE_LOCALES}\*(C'\fR will be 0 on systems that turn off the thread-safe +operations. +.PP +Normally on unthreaded builds, the traditional \f(CWsetlocale()\fR is used +and not the thread-safe locale functions. You can force the use of these +on systems that have them by adding the +\&\f(CW\*(C`\-Accflags=\*(Aq\-DUSE_THREAD_SAFE_LOCALE\*(Aq\*(C'\fR to \fIConfigure\fR. +.PP +The initial program is started up using the locale specified from the +environment, as currently, described in "ENVIRONMENT". All newly +created threads start with \f(CW\*(C`LC_ALL\*(C'\fR set to \f(CW"C"\fR. Each thread may +use \f(CWPOSIX::setlocale()\fR to query or switch its locale at any time, +without affecting any other thread. All locale-dependent operations +automatically use their thread's locale. +.PP +This should be completely transparent to any applications written +entirely in Perl (minus a few rarely encountered caveats given in the +"Multi-threaded" section). Information for XS module writers is given +in "Locale-aware XS code" in perlxs. +.SS "Finding locales" +.IX Subsection "Finding locales" +For locales available in your system, consult also \fBsetlocale\fR\|(3) to +see whether it leads to the list of available locales (search for the +\&\fISEE ALSO\fR section). If that fails, try the following command lines: +.PP +.Vb 1 +\& locale \-a +\& +\& nlsinfo +\& +\& ls /usr/lib/nls/loc +\& +\& ls /usr/lib/locale +\& +\& ls /usr/lib/nls +\& +\& ls /usr/share/locale +.Ve +.PP +and see whether they list something resembling these +.PP +.Vb 7 +\& en_US.ISO8859\-1 de_DE.ISO8859\-1 ru_RU.ISO8859\-5 +\& en_US.iso88591 de_DE.iso88591 ru_RU.iso88595 +\& en_US de_DE ru_RU +\& en de ru +\& english german russian +\& english.iso88591 german.iso88591 russian.iso88595 +\& english.roman8 russian.koi8r +.Ve +.PP +Sadly, even though the calling interface for \f(CWsetlocale()\fR has been +standardized, names of locales and the directories where the +configuration resides have not been. The basic form of the name is +\&\fIlanguage_territory\fR\fB.\fR\fIcodeset\fR, but the latter parts after +\&\fIlanguage\fR are not always present. The \fIlanguage\fR and \fIcountry\fR +are usually from the standards \fBISO 3166\fR and \fBISO 639\fR, the +two-letter abbreviations for the countries and the languages of the +world, respectively. The \fIcodeset\fR part often mentions some \fBISO +8859\fR character set, the Latin codesets. For example, \f(CW\*(C`ISO 8859\-1\*(C'\fR +is the so-called "Western European codeset" that can be used to encode +most Western European languages adequately. Again, there are several +ways to write even the name of that one standard. Lamentably. +.PP +Two special locales are worth particular mention: "C" and "POSIX". +Currently these are effectively the same locale: the difference is +mainly that the first one is defined by the C standard, the second by +the POSIX standard. They define the \fBdefault locale\fR in which +every program starts in the absence of locale information in its +environment. (The \fIdefault\fR default locale, if you will.) Its language +is (American) English and its character codeset ASCII or, rarely, a +superset thereof (such as the "DEC Multinational Character Set +(DEC-MCS)"). \fBWarning\fR. The C locale delivered by some vendors +may not actually exactly match what the C standard calls for. So +beware. +.PP +\&\fBNOTE\fR: Not all systems have the "POSIX" locale (not all systems are +POSIX-conformant), so use "C" when you need explicitly to specify this +default locale. +.SS "LOCALE PROBLEMS" +.IX Subsection "LOCALE PROBLEMS" +You may encounter the following warning message at Perl startup: +.PP +.Vb 6 +\& perl: warning: Setting locale failed. +\& perl: warning: Please check that your locale settings: +\& LC_ALL = "En_US", +\& LANG = (unset) +\& are supported and installed on your system. +\& perl: warning: Falling back to the standard locale ("C"). +.Ve +.PP +This means that your locale settings had \f(CW\*(C`LC_ALL\*(C'\fR set to "En_US" and +LANG exists but has no value. Perl tried to believe you but could not. +Instead, Perl gave up and fell back to the "C" locale, the default locale +that is supposed to work no matter what. (On Windows, it first tries +falling back to the system default locale.) This usually means your +locale settings were wrong, they mention locales your system has never +heard of, or the locale installation in your system has problems (for +example, some system files are broken or missing). There are quick and +temporary fixes to these problems, as well as more thorough and lasting +fixes. +.SS "Testing for broken locales" +.IX Subsection "Testing for broken locales" +If you are building Perl from source, the Perl test suite file +\&\fIlib/locale.t\fR can be used to test the locales on your system. +Setting the environment variable \f(CW\*(C`PERL_DEBUG_FULL_TEST\*(C'\fR to 1 +will cause it to output detailed results. For example, on Linux, you +could say +.PP +.Vb 1 +\& PERL_DEBUG_FULL_TEST=1 ./perl \-T \-Ilib lib/locale.t > locale.log 2>&1 +.Ve +.PP +Besides many other tests, it will test every locale it finds on your +system to see if they conform to the POSIX standard. If any have +errors, it will include a summary near the end of the output of which +locales passed all its tests, and which failed, and why. +.SS "Temporarily fixing locale problems" +.IX Subsection "Temporarily fixing locale problems" +The two quickest fixes are either to render Perl silent about any +locale inconsistencies or to run Perl under the default locale "C". +.PP +Perl's moaning about locale problems can be silenced by setting the +environment variable \f(CW\*(C`PERL_BADLANG\*(C'\fR to "0" or "". +This method really just sweeps the problem under the carpet: you tell +Perl to shut up even when Perl sees that something is wrong. Do not +be surprised if later something locale-dependent misbehaves. +.PP +Perl can be run under the "C" locale by setting the environment +variable \f(CW\*(C`LC_ALL\*(C'\fR to "C". This method is perhaps a bit more civilized +than the \f(CW\*(C`PERL_BADLANG\*(C'\fR approach, but setting \f(CW\*(C`LC_ALL\*(C'\fR (or +other locale variables) may affect other programs as well, not just +Perl. In particular, external programs run from within Perl will see +these changes. If you make the new settings permanent (read on), all +programs you run see the changes. See "ENVIRONMENT" for +the full list of relevant environment variables and "USING LOCALES" +for their effects in Perl. Effects in other programs are +easily deducible. For example, the variable \f(CW\*(C`LC_COLLATE\*(C'\fR may well affect +your \fBsort\fR program (or whatever the program that arranges "records" +alphabetically in your system is called). +.PP +You can test out changing these variables temporarily, and if the +new settings seem to help, put those settings into your shell startup +files. Consult your local documentation for the exact details. For +Bourne-like shells (\fBsh\fR, \fBksh\fR, \fBbash\fR, \fBzsh\fR): +.PP +.Vb 2 +\& LC_ALL=en_US.ISO8859\-1 +\& export LC_ALL +.Ve +.PP +This assumes that we saw the locale "en_US.ISO8859\-1" using the commands +discussed above. We decided to try that instead of the above faulty +locale "En_US"\-\-and in Cshish shells (\fBcsh\fR, \fBtcsh\fR) +.PP +.Vb 1 +\& setenv LC_ALL en_US.ISO8859\-1 +.Ve +.PP +or if you have the "env" application you can do (in any shell) +.PP +.Vb 1 +\& env LC_ALL=en_US.ISO8859\-1 perl ... +.Ve +.PP +If you do not know what shell you have, consult your local +helpdesk or the equivalent. +.SS "Permanently fixing locale problems" +.IX Subsection "Permanently fixing locale problems" +The slower but superior fixes are when you may be able to yourself +fix the misconfiguration of your own environment variables. The +mis(sing)configuration of the whole system's locales usually requires +the help of your friendly system administrator. +.PP +First, see earlier in this document about "Finding locales". That tells +how to find which locales are really supported\-\-and more importantly, +installed\-\-on your system. In our example error message, environment +variables affecting the locale are listed in the order of decreasing +importance (and unset variables do not matter). Therefore, having +LC_ALL set to "En_US" must have been the bad choice, as shown by the +error message. First try fixing locale settings listed first. +.PP +Second, if using the listed commands you see something \fBexactly\fR +(prefix matches do not count and case usually counts) like "En_US" +without the quotes, then you should be okay because you are using a +locale name that should be installed and available in your system. +In this case, see "Permanently fixing your system's locale configuration". +.SS "Permanently fixing your system's locale configuration" +.IX Subsection "Permanently fixing your system's locale configuration" +This is when you see something like: +.PP +.Vb 4 +\& perl: warning: Please check that your locale settings: +\& LC_ALL = "En_US", +\& LANG = (unset) +\& are supported and installed on your system. +.Ve +.PP +but then cannot see that "En_US" listed by the above-mentioned +commands. You may see things like "en_US.ISO8859\-1", but that isn't +the same. In this case, try running under a locale +that you can list and which somehow matches what you tried. The +rules for matching locale names are a bit vague because +standardization is weak in this area. See again the +"Finding locales" about general rules. +.SS "Fixing system locale configuration" +.IX Subsection "Fixing system locale configuration" +Contact a system administrator (preferably your own) and report the exact +error message you get, and ask them to read this same documentation you +are now reading. They should be able to check whether there is something +wrong with the locale configuration of the system. The "Finding locales" +section is unfortunately a bit vague about the exact commands and places +because these things are not that standardized. +.SS "The localeconv function" +.IX Subsection "The localeconv function" +The \f(CWPOSIX::localeconv()\fR function allows you to get particulars of the +locale-dependent numeric formatting information specified by the current +underlying \f(CW\*(C`LC_NUMERIC\*(C'\fR and \f(CW\*(C`LC_MONETARY\*(C'\fR locales (regardless of +whether called from within the scope of \f(CW\*(C`use\ locale\*(C'\fR or not). (If +you just want the name of +the current locale for a particular category, use \f(CWPOSIX::setlocale()\fR +with a single parameter\-\-see "The setlocale function".) +.PP +.Vb 1 +\& use POSIX qw(locale_h); +\& +\& # Get a reference to a hash of locale\-dependent info +\& $locale_values = localeconv(); +\& +\& # Output sorted list of the values +\& for (sort keys %$locale_values) { +\& printf "%\-20s = %s\en", $_, $locale_values\->{$_} +\& } +.Ve +.PP +\&\f(CWlocaleconv()\fR takes no arguments, and returns \fBa reference to\fR a hash. +The keys of this hash are variable names for formatting, such as +\&\f(CW\*(C`decimal_point\*(C'\fR and \f(CW\*(C`thousands_sep\*(C'\fR. The values are the +corresponding, er, values. See "localeconv" in POSIX for a longer +example listing the categories an implementation might be expected to +provide; some provide more and others fewer. You don't need an +explicit \f(CW\*(C`use locale\*(C'\fR, because \f(CWlocaleconv()\fR always observes the +current locale. +.PP +Here's a simple-minded example program that rewrites its command-line +parameters as integers correctly formatted in the current locale: +.PP +.Vb 1 +\& use POSIX qw(locale_h); +\& +\& # Get some of locale\*(Aqs numeric formatting parameters +\& my ($thousands_sep, $grouping) = +\& @{localeconv()}{\*(Aqthousands_sep\*(Aq, \*(Aqgrouping\*(Aq}; +\& +\& # Apply defaults if values are missing +\& $thousands_sep = \*(Aq,\*(Aq unless $thousands_sep; +\& +\& # grouping and mon_grouping are packed lists +\& # of small integers (characters) telling the +\& # grouping (thousand_seps and mon_thousand_seps +\& # being the group dividers) of numbers and +\& # monetary quantities. The integers\*(Aq meanings: +\& # 255 means no more grouping, 0 means repeat +\& # the previous grouping, 1\-254 means use that +\& # as the current grouping. Grouping goes from +\& # right to left (low to high digits). In the +\& # below we cheat slightly by never using anything +\& # else than the first grouping (whatever that is). +\& if ($grouping) { +\& @grouping = unpack("C*", $grouping); +\& } else { +\& @grouping = (3); +\& } +\& +\& # Format command line params for current locale +\& for (@ARGV) { +\& $_ = int; # Chop non\-integer part +\& 1 while +\& s/(\ed)(\ed{$grouping[0]}($|$thousands_sep))/$1$thousands_sep$2/; +\& print "$_"; +\& } +\& print "\en"; +.Ve +.PP +Note that if the platform doesn't have \f(CW\*(C`LC_NUMERIC\*(C'\fR and/or +\&\f(CW\*(C`LC_MONETARY\*(C'\fR available or enabled, the corresponding elements of the +hash will be missing. +.SS I18N::Langinfo +.IX Subsection "I18N::Langinfo" +Another interface for querying locale-dependent information is the +\&\f(CWI18N::Langinfo::langinfo()\fR function. +.PP +The following example will import the \f(CWlanginfo()\fR function itself and +three constants to be used as arguments to \f(CWlanginfo()\fR: a constant for +the abbreviated first day of the week (the numbering starts from +Sunday = 1) and two more constants for the affirmative and negative +answers for a yes/no question in the current locale. +.PP +.Vb 1 +\& use I18N::Langinfo qw(langinfo ABDAY_1 YESSTR NOSTR); +\& +\& my ($abday_1, $yesstr, $nostr) +\& = map { langinfo } qw(ABDAY_1 YESSTR NOSTR); +\& +\& print "$abday_1? [$yesstr/$nostr] "; +.Ve +.PP +In other words, in the "C" (or English) locale the above will probably +print something like: +.PP +.Vb 1 +\& Sun? [yes/no] +.Ve +.PP +See I18N::Langinfo for more information. +.SH "LOCALE CATEGORIES" +.IX Header "LOCALE CATEGORIES" +The following subsections describe basic locale categories. Beyond these, +some combination categories allow manipulation of more than one +basic category at a time. See "ENVIRONMENT" for a discussion of these. +.ie n .SS "Category ""LC_COLLATE"": Collation: Text Comparisons and Sorting" +.el .SS "Category \f(CWLC_COLLATE\fP: Collation: Text Comparisons and Sorting" +.IX Subsection "Category LC_COLLATE: Collation: Text Comparisons and Sorting" +In the scope of a \f(CW\*(C`use\ locale\*(C'\fR form that includes collation, Perl +looks to the \f(CW\*(C`LC_COLLATE\*(C'\fR +environment variable to determine the application's notions on collation +(ordering) of characters. For example, "b" follows "a" in Latin +alphabets, but where do "á" and "å" belong? And while +"color" follows "chocolate" in English, what about in traditional Spanish? +.PP +The following collations all make sense and you may meet any of them +if you \f(CW"use locale"\fR. +.PP +.Vb 4 +\& A B C D E a b c d e +\& A a B b C c D d E e +\& a A b B c C d D e E +\& a b c d e A B C D E +.Ve +.PP +Here is a code snippet to tell what "word" +characters are in the current locale, in that locale's order: +.PP +.Vb 2 +\& use locale; +\& print +(sort grep /\ew/, map { chr } 0..255), "\en"; +.Ve +.PP +Compare this with the characters that you see and their order if you +state explicitly that the locale should be ignored: +.PP +.Vb 2 +\& no locale; +\& print +(sort grep /\ew/, map { chr } 0..255), "\en"; +.Ve +.PP +This machine-native collation (which is what you get unless \f(CW\*(C`use\ locale\*(C'\fR has appeared earlier in the same block) must be used for +sorting raw binary data, whereas the locale-dependent collation of the +first example is useful for natural text. +.PP +As noted in "USING LOCALES", \f(CW\*(C`cmp\*(C'\fR compares according to the current +collation locale when \f(CW\*(C`use locale\*(C'\fR is in effect, but falls back to a +char-by-char comparison for strings that the locale says are equal. You +can use \f(CWPOSIX::strcoll()\fR if you don't want this fall-back: +.PP +.Vb 3 +\& use POSIX qw(strcoll); +\& $equal_in_locale = +\& !strcoll("space and case ignored", "SpaceAndCaseIgnored"); +.Ve +.PP +\&\f(CW$equal_in_locale\fR will be true if the collation locale specifies a +dictionary-like ordering that ignores space characters completely and +which folds case. +.PP +Perl uses the platform's C library collation functions \f(CWstrcoll()\fR and +\&\f(CWstrxfrm()\fR. That means you get whatever they give. On some +platforms, these functions work well on UTF\-8 locales, giving +a reasonable default collation for the code points that are important in +that locale. (And if they aren't working well, the problem may only be +that the locale definition is deficient, so can be fixed by using a +better definition file. Unicode's definitions (see "Freely available +locale definitions") provide reasonable UTF\-8 locale collation +definitions.) Starting in Perl v5.26, Perl's use of these functions has +been made more seamless. This may be sufficient for your needs. For +more control, and to make sure strings containing any code point (not +just the ones important in the locale) collate properly, the +Unicode::Collate module is suggested. +.PP +In non\-UTF\-8 locales (hence single byte), code points above 0xFF are +technically invalid. But if present, again starting in v5.26, they will +collate to the same position as the highest valid code point does. This +generally gives good results, but the collation order may be skewed if +the valid code point gets special treatment when it forms particular +sequences with other characters as defined by the locale. +When two strings collate identically, the code point order is used as a +tie breaker. +.PP +If Perl detects that there are problems with the locale collation order, +it reverts to using non-locale collation rules for that locale. +.PP +If you have a single string that you want to check for "equality in +locale" against several others, you might think you could gain a little +efficiency by using \f(CWPOSIX::strxfrm()\fR in conjunction with \f(CW\*(C`eq\*(C'\fR: +.PP +.Vb 8 +\& use POSIX qw(strxfrm); +\& $xfrm_string = strxfrm("Mixed\-case string"); +\& print "locale collation ignores spaces\en" +\& if $xfrm_string eq strxfrm("Mixed\-casestring"); +\& print "locale collation ignores hyphens\en" +\& if $xfrm_string eq strxfrm("Mixedcase string"); +\& print "locale collation ignores case\en" +\& if $xfrm_string eq strxfrm("mixed\-case string"); +.Ve +.PP +\&\f(CWstrxfrm()\fR takes a string and maps it into a transformed string for use +in char-by-char comparisons against other transformed strings during +collation. "Under the hood", locale-affected Perl comparison operators +call \f(CWstrxfrm()\fR for both operands, then do a char-by-char +comparison of the transformed strings. By calling \f(CWstrxfrm()\fR explicitly +and using a non locale-affected comparison, the example attempts to save +a couple of transformations. But in fact, it doesn't save anything: Perl +magic (see "Magic Variables" in perlguts) creates the transformed version of a +string the first time it's needed in a comparison, then keeps this version around +in case it's needed again. An example rewritten the easy way with +\&\f(CW\*(C`cmp\*(C'\fR runs just about as fast. It also copes with null characters +embedded in strings; if you call \f(CWstrxfrm()\fR directly, it treats the first +null it finds as a terminator. Don't expect the transformed strings +it produces to be portable across systems\-\-or even from one revision +of your operating system to the next. In short, don't call \f(CWstrxfrm()\fR +directly: let Perl do it for you. +.PP +Note: \f(CW\*(C`use locale\*(C'\fR isn't shown in some of these examples because it isn't +needed: \f(CWstrcoll()\fR and \f(CWstrxfrm()\fR are POSIX functions +which use the standard system-supplied \f(CW\*(C`libc\*(C'\fR functions that +always obey the current \f(CW\*(C`LC_COLLATE\*(C'\fR locale. +.ie n .SS "Category ""LC_CTYPE"": Character Types" +.el .SS "Category \f(CWLC_CTYPE\fP: Character Types" +.IX Subsection "Category LC_CTYPE: Character Types" +In the scope of a \f(CW\*(C`use\ locale\*(C'\fR form that includes \f(CW\*(C`LC_CTYPE\*(C'\fR, Perl +obeys the \f(CW\*(C`LC_CTYPE\*(C'\fR locale +setting. This controls the application's notion of which characters are +alphabetic, numeric, punctuation, \fIetc\fR. This affects Perl's \f(CW\*(C`\ew\*(C'\fR +regular expression metanotation, +which stands for alphanumeric characters\-\-that is, alphabetic, +numeric, and the platform's native underscore. +(Consult perlre for more information about +regular expressions.) Thanks to \f(CW\*(C`LC_CTYPE\*(C'\fR, depending on your locale +setting, characters like "æ", "ð", "ß", and +"ø" may be understood as \f(CW\*(C`\ew\*(C'\fR characters. +It also affects things like \f(CW\*(C`\es\*(C'\fR, \f(CW\*(C`\eD\*(C'\fR, and the POSIX character +classes, like \f(CW\*(C`[[:graph:]]\*(C'\fR. (See perlrecharclass for more +information on all these.) +.PP +The \f(CW\*(C`LC_CTYPE\*(C'\fR locale also provides the map used in transliterating +characters between lower and uppercase. This affects the case-mapping +functions\-\-\f(CWfc()\fR, \f(CWlc()\fR, \f(CWlcfirst()\fR, \f(CWuc()\fR, and \f(CWucfirst()\fR; +case-mapping +interpolation with \f(CW\*(C`\eF\*(C'\fR, \f(CW\*(C`\el\*(C'\fR, \f(CW\*(C`\eL\*(C'\fR, \f(CW\*(C`\eu\*(C'\fR, or \f(CW\*(C`\eU\*(C'\fR in double-quoted +strings and \f(CW\*(C`s///\*(C'\fR substitutions; and case-insensitive regular expression +pattern matching using the \f(CW\*(C`i\*(C'\fR modifier. +.PP +Starting in v5.20, Perl supports UTF\-8 locales for \f(CW\*(C`LC_CTYPE\*(C'\fR, but +otherwise Perl only supports single-byte locales, such as the ISO 8859 +series. This means that wide character locales, for example for Asian +languages, are not well-supported. Use of these locales may cause core +dumps. If the platform has the capability for Perl to detect such a +locale, starting in Perl v5.22, Perl will warn, default +enabled, using the \f(CW\*(C`locale\*(C'\fR warning +category, whenever such a locale is switched into. The UTF\-8 locale +support is actually a +superset of POSIX locales, because it is really full Unicode behavior +as if no \f(CW\*(C`LC_CTYPE\*(C'\fR locale were in effect at all (except for tainting; +see "SECURITY"). POSIX locales, even UTF\-8 ones, +are lacking certain concepts in Unicode, such as the idea that changing +the case of a character could expand to be more than one character. +Perl in a UTF\-8 locale, will give you that expansion. Prior to v5.20, +Perl treated a UTF\-8 locale on some platforms like an ISO 8859\-1 one, +with some restrictions, and on other platforms more like the "C" locale. +For releases v5.16 and v5.18, \f(CW\*(C`use\ locale\ \*(Aqnot_characters\*(C'\fR could be +used as a workaround for this (see "Unicode and UTF\-8"). +.PP +Note that there are quite a few things that are unaffected by the +current locale. Any literal character is the native character for the +given platform. Hence 'A' means the character at code point 65 on ASCII +platforms, and 193 on EBCDIC. That may or may not be an 'A' in the +current locale, if that locale even has an 'A'. +Similarly, all the escape sequences for particular characters, +\&\f(CW\*(C`\en\*(C'\fR for example, always mean the platform's native one. This means, +for example, that \f(CW\*(C`\eN\*(C'\fR in regular expressions (every character +but new-line) works on the platform character set. +.PP +Starting in v5.22, Perl will by default warn when switching into a +locale that redefines any ASCII printable character (plus \f(CW\*(C`\et\*(C'\fR and +\&\f(CW\*(C`\en\*(C'\fR) into a different class than expected. This is likely to +happen on modern locales only on EBCDIC platforms, where, for example, +a CCSID 0037 locale on a CCSID 1047 machine moves \f(CW"["\fR, but it can +happen on ASCII platforms with the ISO 646 and other +7\-bit locales that are essentially obsolete. Things may still work, +depending on what features of Perl are used by the program. For +example, in the example from above where \f(CW"|"\fR becomes a \f(CW\*(C`\ew\*(C'\fR, and +there are no regular expressions where this matters, the program may +still work properly. The warning lists all the characters that +it can determine could be adversely affected. +.PP +\&\fBNote:\fR A broken or malicious \f(CW\*(C`LC_CTYPE\*(C'\fR locale definition may result +in clearly ineligible characters being considered to be alphanumeric by +your application. For strict matching of (mundane) ASCII letters and +digits\-\-for example, in command strings\-\-locale\-aware applications +should use \f(CW\*(C`\ew\*(C'\fR with the \f(CW\*(C`/a\*(C'\fR regular expression modifier. See "SECURITY". +.ie n .SS "Category ""LC_NUMERIC"": Numeric Formatting" +.el .SS "Category \f(CWLC_NUMERIC\fP: Numeric Formatting" +.IX Subsection "Category LC_NUMERIC: Numeric Formatting" +After a proper \f(CWPOSIX::setlocale()\fR call, and within the scope +of a \f(CW\*(C`use locale\*(C'\fR form that includes numerics, Perl obeys the +\&\f(CW\*(C`LC_NUMERIC\*(C'\fR locale information, which controls an application's idea +of how numbers should be formatted for human readability. +In most implementations the only effect is to +change the character used for the decimal point\-\-perhaps from "." to ",". +The functions aren't aware of such niceties as thousands separation and +so on. (See "The localeconv function" if you care about these things.) +.PP +.Vb 2 +\& use POSIX qw(strtod setlocale LC_NUMERIC); +\& use locale; +\& +\& setlocale LC_NUMERIC, ""; +\& +\& $n = 5/2; # Assign numeric 2.5 to $n +\& +\& $a = " $n"; # Locale\-dependent conversion to string +\& +\& print "half five is $n\en"; # Locale\-dependent output +\& +\& printf "half five is %g\en", $n; # Locale\-dependent output +\& +\& print "DECIMAL POINT IS COMMA\en" +\& if $n == (strtod("2,5"))[0]; # Locale\-dependent conversion +.Ve +.PP +See also I18N::Langinfo and \f(CW\*(C`RADIXCHAR\*(C'\fR. +.ie n .SS "Category ""LC_MONETARY"": Formatting of monetary amounts" +.el .SS "Category \f(CWLC_MONETARY\fP: Formatting of monetary amounts" +.IX Subsection "Category LC_MONETARY: Formatting of monetary amounts" +The C standard defines the \f(CW\*(C`LC_MONETARY\*(C'\fR category, but not a function +that is affected by its contents. (Those with experience of standards +committees will recognize that the working group decided to punt on the +issue.) Consequently, Perl essentially takes no notice of it. If you +really want to use \f(CW\*(C`LC_MONETARY\*(C'\fR, you can query its contents\-\-see +"The localeconv function"\-\-and use the information that it returns in your +application's own formatting of currency amounts. However, you may well +find that the information, voluminous and complex though it may be, still +does not quite meet your requirements: currency formatting is a hard nut +to crack. +.PP +See also I18N::Langinfo and \f(CW\*(C`CRNCYSTR\*(C'\fR. +.ie n .SS "Category ""LC_TIME"": Respresentation of time" +.el .SS "Category \f(CWLC_TIME\fP: Respresentation of time" +.IX Subsection "Category LC_TIME: Respresentation of time" +Output produced by \f(CWPOSIX::strftime()\fR, which builds a formatted +human-readable date/time string, is affected by the current \f(CW\*(C`LC_TIME\*(C'\fR +locale. Thus, in a French locale, the output produced by the \f(CW%B\fR +format element (full month name) for the first month of the year would +be "janvier". Here's how to get a list of long month names in the +current locale: +.PP +.Vb 5 +\& use POSIX qw(strftime); +\& for (0..11) { +\& $long_month_name[$_] = +\& strftime("%B", 0, 0, 0, 1, $_, 96); +\& } +.Ve +.PP +Note: \f(CW\*(C`use locale\*(C'\fR isn't needed in this example: \f(CWstrftime()\fR is a POSIX +function which uses the standard system-supplied \f(CW\*(C`libc\*(C'\fR function that +always obeys the current \f(CW\*(C`LC_TIME\*(C'\fR locale. +.PP +See also I18N::Langinfo and \f(CW\*(C`ABDAY_1\*(C'\fR..\f(CW\*(C`ABDAY_7\*(C'\fR, \f(CW\*(C`DAY_1\*(C'\fR..\f(CW\*(C`DAY_7\*(C'\fR, +\&\f(CW\*(C`ABMON_1\*(C'\fR..\f(CW\*(C`ABMON_12\*(C'\fR, and \f(CW\*(C`ABMON_1\*(C'\fR..\f(CW\*(C`ABMON_12\*(C'\fR. +.SS "Other categories" +.IX Subsection "Other categories" +The remaining locale categories are not currently used by Perl itself. +But again note that things Perl interacts with may use these, including +extensions outside the standard Perl distribution, and by the +operating system and its utilities. Note especially that the string +value of \f(CW$!\fR and the error messages given by external utilities may +be changed by \f(CW\*(C`LC_MESSAGES\*(C'\fR. If you want to have portable error +codes, use \f(CW\*(C`%!\*(C'\fR. See Errno. +.SH SECURITY +.IX Header "SECURITY" +Although the main discussion of Perl security issues can be found in +perlsec, a discussion of Perl's locale handling would be incomplete +if it did not draw your attention to locale-dependent security issues. +Locales\-\-particularly on systems that allow unprivileged users to +build their own locales\-\-are untrustworthy. A malicious (or just plain +broken) locale can make a locale-aware application give unexpected +results. Here are a few possibilities: +.IP \(bu 4 +Regular expression checks for safe file names or mail addresses using +\&\f(CW\*(C`\ew\*(C'\fR may be spoofed by an \f(CW\*(C`LC_CTYPE\*(C'\fR locale that claims that +characters such as \f(CW">"\fR and \f(CW"|"\fR are alphanumeric. +.IP \(bu 4 +String interpolation with case-mapping, as in, say, \f(CW$dest = +"C:\eU$name.$ext"\fR, may produce dangerous results if a bogus \f(CW\*(C`LC_CTYPE\*(C'\fR +case-mapping table is in effect. +.IP \(bu 4 +A sneaky \f(CW\*(C`LC_COLLATE\*(C'\fR locale could result in the names of students with +"D" grades appearing ahead of those with "A"s. +.IP \(bu 4 +An application that takes the trouble to use information in +\&\f(CW\*(C`LC_MONETARY\*(C'\fR may format debits as if they were credits and vice versa +if that locale has been subverted. Or it might make payments in US +dollars instead of Hong Kong dollars. +.IP \(bu 4 +The date and day names in dates formatted by \f(CWstrftime()\fR could be +manipulated to advantage by a malicious user able to subvert the +\&\f(CW\*(C`LC_DATE\*(C'\fR locale. ("Look\-\-it says I wasn't in the building on +Sunday.") +.PP +Such dangers are not peculiar to the locale system: any aspect of an +application's environment which may be modified maliciously presents +similar challenges. Similarly, they are not specific to Perl: any +programming language that allows you to write programs that take +account of their environment exposes you to these issues. +.PP +Perl cannot protect you from all possibilities shown in the +examples\-\-there is no substitute for your own vigilance\-\-but, when +\&\f(CW\*(C`use locale\*(C'\fR is in effect, Perl uses the tainting mechanism (see +perlsec) to mark string results that become locale-dependent, and +which may be untrustworthy in consequence. +.PP +Note that it is possible to compile Perl without taint support, +in which case all taint features silently do nothing. +.PP +Here is a summary of the tainting behavior of operators and functions +that may be affected by the locale: +.IP \(bu 4 +\&\fBComparison operators\fR (\f(CW\*(C`lt\*(C'\fR, \f(CW\*(C`le\*(C'\fR, \f(CW\*(C`ge\*(C'\fR, \f(CW\*(C`gt\*(C'\fR and \f(CW\*(C`cmp\*(C'\fR): +.Sp +Scalar true/false (or less/equal/greater) result is never tainted. +.IP \(bu 4 +\&\fBCase-mapping interpolation\fR (with \f(CW\*(C`\el\*(C'\fR, \f(CW\*(C`\eL\*(C'\fR, \f(CW\*(C`\eu\*(C'\fR, \f(CW\*(C`\eU\*(C'\fR, or \f(CW\*(C`\eF\*(C'\fR) +.Sp +The result string containing interpolated material is tainted if +a \f(CW\*(C`use locale\*(C'\fR form that includes \f(CW\*(C`LC_CTYPE\*(C'\fR is in effect. +.IP \(bu 4 +\&\fBMatching operator\fR (\f(CW\*(C`m//\*(C'\fR): +.Sp +Scalar true/false result never tainted. +.Sp +All subpatterns, either delivered as a list-context result or as \f(CW$1\fR +\&\fIetc\fR., are tainted if a \f(CW\*(C`use locale\*(C'\fR form that includes +\&\f(CW\*(C`LC_CTYPE\*(C'\fR is in effect, and the subpattern +regular expression contains a locale-dependent construct. These +constructs include \f(CW\*(C`\ew\*(C'\fR (to match an alphanumeric character), \f(CW\*(C`\eW\*(C'\fR +(non-alphanumeric character), \f(CW\*(C`\eb\*(C'\fR and \f(CW\*(C`\eB\*(C'\fR (word-boundary and +non-boundardy, which depend on what \f(CW\*(C`\ew\*(C'\fR and \f(CW\*(C`\eW\*(C'\fR match), \f(CW\*(C`\es\*(C'\fR +(whitespace character), \f(CW\*(C`\eS\*(C'\fR (non whitespace character), \f(CW\*(C`\ed\*(C'\fR and +\&\f(CW\*(C`\eD\*(C'\fR (digits and non-digits), and the POSIX character classes, such as +\&\f(CW\*(C`[:alpha:]\*(C'\fR (see "POSIX Character Classes" in perlrecharclass). +.Sp +Tainting is also likely if the pattern is to be matched +case-insensitively (via \f(CW\*(C`/i\*(C'\fR). The exception is if all the code points +to be matched this way are above 255 and do not have folds under Unicode +rules to below 256. Tainting is not done for these because Perl +only uses Unicode rules for such code points, and those rules are the +same no matter what the current locale. +.Sp +The matched-pattern variables, \f(CW$&\fR, \f(CW\*(C`$\`\*(C'\fR (pre-match), \f(CW\*(C`$\*(Aq\*(C'\fR +(post-match), and \f(CW$+\fR (last match) also are tainted. +.IP \(bu 4 +\&\fBSubstitution operator\fR (\f(CW\*(C`s///\*(C'\fR): +.Sp +Has the same behavior as the match operator. Also, the left +operand of \f(CW\*(C`=~\*(C'\fR becomes tainted when a \f(CW\*(C`use locale\*(C'\fR +form that includes \f(CW\*(C`LC_CTYPE\*(C'\fR is in effect, if modified as +a result of a substitution based on a regular +expression match involving any of the things mentioned in the previous +item, or of case-mapping, such as \f(CW\*(C`\el\*(C'\fR, \f(CW\*(C`\eL\*(C'\fR,\f(CW\*(C`\eu\*(C'\fR, \f(CW\*(C`\eU\*(C'\fR, or \f(CW\*(C`\eF\*(C'\fR. +.IP \(bu 4 +\&\fBOutput formatting functions\fR (\f(CWprintf()\fR and \f(CWwrite()\fR): +.Sp +Results are never tainted because otherwise even output from print, +for example \f(CWprint(1/7)\fR, should be tainted if \f(CW\*(C`use locale\*(C'\fR is in +effect. +.IP \(bu 4 +\&\fBCase-mapping functions\fR (\f(CWlc()\fR, \f(CWlcfirst()\fR, \f(CWuc()\fR, \f(CWucfirst()\fR): +.Sp +Results are tainted if a \f(CW\*(C`use locale\*(C'\fR form that includes \f(CW\*(C`LC_CTYPE\*(C'\fR is +in effect. +.IP \(bu 4 +\&\fBPOSIX locale-dependent functions\fR (\f(CWlocaleconv()\fR, \f(CWstrcoll()\fR, +\&\f(CWstrftime()\fR, \f(CWstrxfrm()\fR): +.Sp +Results are never tainted. +.PP +Three examples illustrate locale-dependent tainting. +The first program, which ignores its locale, won't run: a value taken +directly from the command line may not be used to name an output file +when taint checks are enabled. +.PP +.Vb 2 +\& #/usr/local/bin/perl \-T +\& # Run with taint checking +\& +\& # Command line sanity check omitted... +\& $tainted_output_file = shift; +\& +\& open(F, ">$tainted_output_file") +\& or warn "Open of $tainted_output_file failed: $!\en"; +.Ve +.PP +The program can be made to run by "laundering" the tainted value through +a regular expression: the second example\-\-which still ignores locale +information\-\-runs, creating the file named on its command line +if it can. +.PP +.Vb 1 +\& #/usr/local/bin/perl \-T +\& +\& $tainted_output_file = shift; +\& $tainted_output_file =~ m%[\ew/]+%; +\& $untainted_output_file = $&; +\& +\& open(F, ">$untainted_output_file") +\& or warn "Open of $untainted_output_file failed: $!\en"; +.Ve +.PP +Compare this with a similar but locale-aware program: +.PP +.Vb 1 +\& #/usr/local/bin/perl \-T +\& +\& $tainted_output_file = shift; +\& use locale; +\& $tainted_output_file =~ m%[\ew/]+%; +\& $localized_output_file = $&; +\& +\& open(F, ">$localized_output_file") +\& or warn "Open of $localized_output_file failed: $!\en"; +.Ve +.PP +This third program fails to run because \f(CW$&\fR is tainted: it is the result +of a match involving \f(CW\*(C`\ew\*(C'\fR while \f(CW\*(C`use locale\*(C'\fR is in effect. +.SH ENVIRONMENT +.IX Header "ENVIRONMENT" +.IP PERL_SKIP_LOCALE_INIT 12 +.IX Item "PERL_SKIP_LOCALE_INIT" +This environment variable, available starting in Perl v5.20, if set +(to any value), tells Perl to not use the rest of the +environment variables to initialize with. Instead, Perl uses whatever +the current locale settings are. This is particularly useful in +embedded environments, see +"Using embedded Perl with POSIX locales" in perlembed. +.IP PERL_BADLANG 12 +.IX Item "PERL_BADLANG" +A string that can suppress Perl's warning about failed locale settings +at startup. Failure can occur if the locale support in the operating +system is lacking (broken) in some way\-\-or if you mistyped the name of +a locale when you set up your environment. If this environment +variable is absent, or has a value other than "0" or "", Perl will +complain about locale setting failures. +.Sp +\&\fBNOTE\fR: \f(CW\*(C`PERL_BADLANG\*(C'\fR only gives you a way to hide the warning message. +The message tells about some problem in your system's locale support, +and you should investigate what the problem is. +.PP +The following environment variables are not specific to Perl: They are +part of the standardized (ISO C, XPG4, POSIX 1.c) \f(CWsetlocale()\fR method +for controlling an application's opinion on data. Windows is non-POSIX, +but Perl arranges for the following to work as described anyway. +If the locale given by an environment variable is not valid, Perl tries +the next lower one in priority. If none are valid, on Windows, the +system default locale is then tried. If all else fails, the \f(CW"C"\fR +locale is used. If even that doesn't work, something is badly broken, +but Perl tries to forge ahead with whatever the locale settings might +be. +.ie n .IP """LC_ALL""" 12 +.el .IP \f(CWLC_ALL\fR 12 +.IX Item "LC_ALL" +\&\f(CW\*(C`LC_ALL\*(C'\fR is the "override-all" locale environment variable. If +set, it overrides all the rest of the locale environment variables. +.ie n .IP """LANGUAGE""" 12 +.el .IP \f(CWLANGUAGE\fR 12 +.IX Item "LANGUAGE" +\&\fBNOTE\fR: \f(CW\*(C`LANGUAGE\*(C'\fR is a GNU extension, it affects you only if you +are using the GNU libc. This is the case if you are using e.g. Linux. +If you are using "commercial" Unixes you are most probably \fInot\fR +using GNU libc and you can ignore \f(CW\*(C`LANGUAGE\*(C'\fR. +.Sp +However, in the case you are using \f(CW\*(C`LANGUAGE\*(C'\fR: it affects the +language of informational, warning, and error messages output by +commands (in other words, it's like \f(CW\*(C`LC_MESSAGES\*(C'\fR) but it has higher +priority than \f(CW\*(C`LC_ALL\*(C'\fR. Moreover, it's not a single value but +instead a "path" (":"\-separated list) of \fIlanguages\fR (not locales). +See the GNU \f(CW\*(C`gettext\*(C'\fR library documentation for more information. +.ie n .IP """LC_CTYPE""" 12 +.el .IP \f(CWLC_CTYPE\fR 12 +.IX Item "LC_CTYPE" +In the absence of \f(CW\*(C`LC_ALL\*(C'\fR, \f(CW\*(C`LC_CTYPE\*(C'\fR chooses the character type +locale. In the absence of both \f(CW\*(C`LC_ALL\*(C'\fR and \f(CW\*(C`LC_CTYPE\*(C'\fR, \f(CW\*(C`LANG\*(C'\fR +chooses the character type locale. +.ie n .IP """LC_COLLATE""" 12 +.el .IP \f(CWLC_COLLATE\fR 12 +.IX Item "LC_COLLATE" +In the absence of \f(CW\*(C`LC_ALL\*(C'\fR, \f(CW\*(C`LC_COLLATE\*(C'\fR chooses the collation +(sorting) locale. In the absence of both \f(CW\*(C`LC_ALL\*(C'\fR and \f(CW\*(C`LC_COLLATE\*(C'\fR, +\&\f(CW\*(C`LANG\*(C'\fR chooses the collation locale. +.ie n .IP """LC_MONETARY""" 12 +.el .IP \f(CWLC_MONETARY\fR 12 +.IX Item "LC_MONETARY" +In the absence of \f(CW\*(C`LC_ALL\*(C'\fR, \f(CW\*(C`LC_MONETARY\*(C'\fR chooses the monetary +formatting locale. In the absence of both \f(CW\*(C`LC_ALL\*(C'\fR and \f(CW\*(C`LC_MONETARY\*(C'\fR, +\&\f(CW\*(C`LANG\*(C'\fR chooses the monetary formatting locale. +.ie n .IP """LC_NUMERIC""" 12 +.el .IP \f(CWLC_NUMERIC\fR 12 +.IX Item "LC_NUMERIC" +In the absence of \f(CW\*(C`LC_ALL\*(C'\fR, \f(CW\*(C`LC_NUMERIC\*(C'\fR chooses the numeric format +locale. In the absence of both \f(CW\*(C`LC_ALL\*(C'\fR and \f(CW\*(C`LC_NUMERIC\*(C'\fR, \f(CW\*(C`LANG\*(C'\fR +chooses the numeric format. +.ie n .IP """LC_TIME""" 12 +.el .IP \f(CWLC_TIME\fR 12 +.IX Item "LC_TIME" +In the absence of \f(CW\*(C`LC_ALL\*(C'\fR, \f(CW\*(C`LC_TIME\*(C'\fR chooses the date and time +formatting locale. In the absence of both \f(CW\*(C`LC_ALL\*(C'\fR and \f(CW\*(C`LC_TIME\*(C'\fR, +\&\f(CW\*(C`LANG\*(C'\fR chooses the date and time formatting locale. +.ie n .IP """LANG""" 12 +.el .IP \f(CWLANG\fR 12 +.IX Item "LANG" +\&\f(CW\*(C`LANG\*(C'\fR is the "catch-all" locale environment variable. If it is set, it +is used as the last resort after the overall \f(CW\*(C`LC_ALL\*(C'\fR and the +category-specific \f(CW\*(C`LC_\fR\f(CIfoo\fR\f(CW\*(C'\fR. +.SS Examples +.IX Subsection "Examples" +The \f(CW\*(C`LC_NUMERIC\*(C'\fR controls the numeric output: +.PP +.Vb 4 +\& use locale; +\& use POSIX qw(locale_h); # Imports setlocale() and the LC_ constants. +\& setlocale(LC_NUMERIC, "fr_FR") or die "Pardon"; +\& printf "%g\en", 1.23; # If the "fr_FR" succeeded, probably shows 1,23. +.Ve +.PP +and also how strings are parsed by \f(CWPOSIX::strtod()\fR as numbers: +.PP +.Vb 5 +\& use locale; +\& use POSIX qw(locale_h strtod); +\& setlocale(LC_NUMERIC, "de_DE") or die "Entschuldigung"; +\& my $x = strtod("2,34") + 5; +\& print $x, "\en"; # Probably shows 7,34. +.Ve +.SH NOTES +.IX Header "NOTES" +.ie n .SS "String ""eval"" and ""LC_NUMERIC""" +.el .SS "String \f(CWeval\fP and \f(CWLC_NUMERIC\fP" +.IX Subsection "String eval and LC_NUMERIC" +A string eval parses its expression as standard +Perl. It is therefore expecting the decimal point to be a dot. If +\&\f(CW\*(C`LC_NUMERIC\*(C'\fR is set to have this be a comma instead, the parsing will +be confused, perhaps silently. +.PP +.Vb 6 +\& use locale; +\& use POSIX qw(locale_h); +\& setlocale(LC_NUMERIC, "fr_FR") or die "Pardon"; +\& my $a = 1.2; +\& print eval "$a + 1.5"; +\& print "\en"; +.Ve +.PP +prints \f(CW\*(C`13,5\*(C'\fR. This is because in that locale, the comma is the +decimal point character. The \f(CW\*(C`eval\*(C'\fR thus expands to: +.PP +.Vb 1 +\& eval "1,2 + 1.5" +.Ve +.PP +and the result is not what you likely expected. No warnings are +generated. If you do string \f(CW\*(C`eval\*(C'\fR's within the scope of +\&\f(CW\*(C`use\ locale\*(C'\fR, you should instead change the \f(CW\*(C`eval\*(C'\fR line to do +something like: +.PP +.Vb 1 +\& print eval "no locale; $a + 1.5"; +.Ve +.PP +This prints \f(CW2.7\fR. +.PP +You could also exclude \f(CW\*(C`LC_NUMERIC\*(C'\fR, if you don't need it, by +.PP +.Vb 1 +\& use locale \*(Aq:!numeric\*(Aq; +.Ve +.SS "Backward compatibility" +.IX Subsection "Backward compatibility" +Versions of Perl prior to 5.004 \fBmostly\fR ignored locale information, +generally behaving as if something similar to the \f(CW"C"\fR locale were +always in force, even if the program environment suggested otherwise +(see "The setlocale function"). By default, Perl still behaves this +way for backward compatibility. If you want a Perl application to pay +attention to locale information, you \fBmust\fR use the \f(CW\*(C`use\ locale\*(C'\fR +pragma (see "The "use locale" pragma") or, in the unlikely event +that you want to do so for just pattern matching, the +\&\f(CW\*(C`/l\*(C'\fR regular expression modifier (see "Character set +modifiers" in perlre) to instruct it to do so. +.PP +Versions of Perl from 5.002 to 5.003 did use the \f(CW\*(C`LC_CTYPE\*(C'\fR +information if available; that is, \f(CW\*(C`\ew\*(C'\fR did understand what +were the letters according to the locale environment variables. +The problem was that the user had no control over the feature: +if the C library supported locales, Perl used them. +.SS "I18N:Collate obsolete" +.IX Subsection "I18N:Collate obsolete" +In versions of Perl prior to 5.004, per-locale collation was possible +using the \f(CW\*(C`I18N::Collate\*(C'\fR library module. This module is now mildly +obsolete and should be avoided in new applications. The \f(CW\*(C`LC_COLLATE\*(C'\fR +functionality is now integrated into the Perl core language: One can +use locale-specific scalar data completely normally with \f(CW\*(C`use locale\*(C'\fR, +so there is no longer any need to juggle with the scalar references of +\&\f(CW\*(C`I18N::Collate\*(C'\fR. +.SS "Sort speed and memory use impacts" +.IX Subsection "Sort speed and memory use impacts" +Comparing and sorting by locale is usually slower than the default +sorting; slow-downs of two to four times have been observed. It will +also consume more memory: once a Perl scalar variable has participated +in any string comparison or sorting operation obeying the locale +collation rules, it will take 3\-15 times more memory than before. (The +exact multiplier depends on the string's contents, the operating system +and the locale.) These downsides are dictated more by the operating +system's implementation of the locale system than by Perl. +.SS "Freely available locale definitions" +.IX Subsection "Freely available locale definitions" +The Unicode CLDR project extracts the POSIX portion of many of its +locales, available at +.PP +.Vb 1 +\& https://unicode.org/Public/cldr/2.0.1/ +.Ve +.PP +(Newer versions of CLDR require you to compute the POSIX data yourself. +See <http://unicode.org/Public/cldr/latest/>.) +.PP +There is a large collection of locale definitions at: +.PP +.Vb 1 +\& http://std.dkuug.dk/i18n/WG15\-collection/locales/ +.Ve +.PP +You should be aware that it is +unsupported, and is not claimed to be fit for any purpose. If your +system allows installation of arbitrary locales, you may find the +definitions useful as they are, or as a basis for the development of +your own locales. +.SS "I18n and l10n" +.IX Subsection "I18n and l10n" +"Internationalization" is often abbreviated as \fBi18n\fR because its first +and last letters are separated by eighteen others. (You may guess why +the internalin ... internaliti ... i18n tends to get abbreviated.) In +the same way, "localization" is often abbreviated to \fBl10n\fR. +.SS "An imperfect standard" +.IX Subsection "An imperfect standard" +Internationalization, as defined in the C and POSIX standards, can be +criticized as incomplete and ungainly. They also have a tendency, like +standards groups, to divide the world into nations, when we all know +that the world can equally well be divided into bankers, bikers, gamers, +and so on. +.SH "Unicode and UTF\-8" +.IX Header "Unicode and UTF-8" +The support of Unicode is new starting from Perl version v5.6, and more fully +implemented in versions v5.8 and later. See perluniintro. +.PP +Starting in Perl v5.20, UTF\-8 locales are supported in Perl, except +\&\f(CW\*(C`LC_COLLATE\*(C'\fR is only partially supported; collation support is improved +in Perl v5.26 to a level that may be sufficient for your needs +(see "Category \f(CW\*(C`LC_COLLATE\*(C'\fR: Collation: Text Comparisons and Sorting"). +.PP +If you have Perl v5.16 or v5.18 and can't upgrade, you can use +.PP +.Vb 1 +\& use locale \*(Aq:not_characters\*(Aq; +.Ve +.PP +When this form of the pragma is used, only the non-character portions of +locales are used by Perl, for example \f(CW\*(C`LC_NUMERIC\*(C'\fR. Perl assumes that +you have translated all the characters it is to operate on into Unicode +(actually the platform's native character set (ASCII or EBCDIC) plus +Unicode). For data in files, this can conveniently be done by also +specifying +.PP +.Vb 1 +\& use open \*(Aq:locale\*(Aq; +.Ve +.PP +This pragma arranges for all inputs from files to be translated into +Unicode from the current locale as specified in the environment (see +"ENVIRONMENT"), and all outputs to files to be translated back +into the locale. (See open). On a per-filehandle basis, you can +instead use the PerlIO::locale module, or the Encode::Locale +module, both available from CPAN. The latter module also has methods to +ease the handling of \f(CW\*(C`ARGV\*(C'\fR and environment variables, and can be used +on individual strings. If you know that all your locales will be +UTF\-8, as many are these days, you can use the +\&\fB\-C\fR command line switch. +.PP +This form of the pragma allows essentially seamless handling of locales +with Unicode. The collation order will be by Unicode code point order. +Unicode::Collate can be used to get Unicode rules collation. +.PP +All the modules and switches just described can be used in v5.20 with +just plain \f(CW\*(C`use locale\*(C'\fR, and, should the input locales not be UTF\-8, +you'll get the less than ideal behavior, described below, that you get +with pre\-v5.16 Perls, or when you use the locale pragma without the +\&\f(CW\*(C`:not_characters\*(C'\fR parameter in v5.16 and v5.18. If you are using +exclusively UTF\-8 locales in v5.20 and higher, the rest of this section +does not apply to you. +.PP +There are two cases, multi-byte and single-byte locales. First +multi-byte: +.PP +The only multi-byte (or wide character) locale that Perl is ever likely +to support is UTF\-8. This is due to the difficulty of implementation, +the fact that high quality UTF\-8 locales are now published for every +area of the world (<https://unicode.org/Public/cldr/2.0.1/> for +ones that are already set-up, but from an earlier version; +<https://unicode.org/Public/cldr/latest/> for the most up-to-date, but +you have to extract the POSIX information yourself), and +failing all that, you can use the Encode module to translate to/from +your locale. So, you'll have to do one of those things if you're using +one of these locales, such as Big5 or Shift JIS. For UTF\-8 locales, in +Perls (pre v5.20) that don't have full UTF\-8 locale support, they may +work reasonably well (depending on your C library implementation) +simply because both +they and Perl store characters that take up multiple bytes the same way. +However, some, if not most, C library implementations may not process +the characters in the upper half of the Latin\-1 range (128 \- 255) +properly under \f(CW\*(C`LC_CTYPE\*(C'\fR. To see if a character is a particular type +under a locale, Perl uses the functions like \f(CWisalnum()\fR. Your C +library may not work for UTF\-8 locales with those functions, instead +only working under the newer wide library functions like \f(CWiswalnum()\fR, +which Perl does not use. +These multi-byte locales are treated like single-byte locales, and will +have the restrictions described below. Starting in Perl v5.22 a warning +message is raised when Perl detects a multi-byte locale that it doesn't +fully support. +.PP +For single-byte locales, +Perl generally takes the tack to use locale rules on code points that can fit +in a single byte, and Unicode rules for those that can't (though this +isn't uniformly applied, see the note at the end of this section). This +prevents many problems in locales that aren't UTF\-8. Suppose the locale +is ISO8859\-7, Greek. The character at 0xD7 there is a capital Chi. But +in the ISO8859\-1 locale, Latin1, it is a multiplication sign. The POSIX +regular expression character class \f(CW\*(C`[[:alpha:]]\*(C'\fR will magically match +0xD7 in the Greek locale but not in the Latin one. +.PP +However, there are places where this breaks down. Certain Perl constructs are +for Unicode only, such as \f(CW\*(C`\ep{Alpha}\*(C'\fR. They assume that 0xD7 always has its +Unicode meaning (or the equivalent on EBCDIC platforms). Since Latin1 is a +subset of Unicode and 0xD7 is the multiplication sign in both Latin1 and +Unicode, \f(CW\*(C`\ep{Alpha}\*(C'\fR will never match it, regardless of locale. A similar +issue occurs with \f(CW\*(C`\eN{...}\*(C'\fR. Prior to v5.20, it is therefore a bad +idea to use \f(CW\*(C`\ep{}\*(C'\fR or +\&\f(CW\*(C`\eN{}\*(C'\fR under plain \f(CW\*(C`use locale\*(C'\fR\-\-\fIunless\fR you can guarantee that the +locale will be ISO8859\-1. Use POSIX character classes instead. +.PP +Another problem with this approach is that operations that cross the +single byte/multiple byte boundary are not well-defined, and so are +disallowed. (This boundary is between the codepoints at 255/256.) +For example, lower casing LATIN CAPITAL LETTER Y WITH DIAERESIS (U+0178) +should return LATIN SMALL LETTER Y WITH DIAERESIS (U+00FF). But in the +Greek locale, for example, there is no character at 0xFF, and Perl +has no way of knowing what the character at 0xFF is really supposed to +represent. Thus it disallows the operation. In this mode, the +lowercase of U+0178 is itself. +.PP +The same problems ensue if you enable automatic UTF\-8\-ification of your +standard file handles, default \f(CWopen()\fR layer, and \f(CW@ARGV\fR on non\-ISO8859\-1, +non\-UTF\-8 locales (by using either the \fB\-C\fR command line switch or the +\&\f(CW\*(C`PERL_UNICODE\*(C'\fR environment variable; see +perlrun). +Things are read in as UTF\-8, which would normally imply a Unicode +interpretation, but the presence of a locale causes them to be interpreted +in that locale instead. For example, a 0xD7 code point in the Unicode +input, which should mean the multiplication sign, won't be interpreted by +Perl that way under the Greek locale. This is not a problem +\&\fIprovided\fR you make certain that all locales will always and only be either +an ISO8859\-1, or, if you don't have a deficient C library, a UTF\-8 locale. +.PP +Still another problem is that this approach can lead to two code +points meaning the same character. Thus in a Greek locale, both U+03A7 +and U+00D7 are GREEK CAPITAL LETTER CHI. +.PP +Because of all these problems, starting in v5.22, Perl will raise a +warning if a multi-byte (hence Unicode) code point is used when a +single-byte locale is in effect. (Although it doesn't check for this if +doing so would unreasonably slow execution down.) +.PP +Vendor locales are notoriously buggy, and it is difficult for Perl to test +its locale-handling code because this interacts with code that Perl has no +control over; therefore the locale-handling code in Perl may be buggy as +well. (However, the Unicode-supplied locales should be better, and +there is a feed back mechanism to correct any problems. See +"Freely available locale definitions".) +.PP +If you have Perl v5.16, the problems mentioned above go away if you use +the \f(CW\*(C`:not_characters\*(C'\fR parameter to the locale pragma (except for vendor +bugs in the non-character portions). If you don't have v5.16, and you +\&\fIdo\fR have locales that work, using them may be worthwhile for certain +specific purposes, as long as you keep in mind the gotchas already +mentioned. For example, if the collation for your locales works, it +runs faster under locales than under Unicode::Collate; and you gain +access to such things as the local currency symbol and the names of the +months and days of the week. (But to hammer home the point, in v5.16, +you get this access without the downsides of locales by using the +\&\f(CW\*(C`:not_characters\*(C'\fR form of the pragma.) +.PP +Note: The policy of using locale rules for code points that can fit in a +byte, and Unicode rules for those that can't is not uniformly applied. +Pre\-v5.12, it was somewhat haphazard; in v5.12 it was applied fairly +consistently to regular expression matching except for bracketed +character classes; in v5.14 it was extended to all regex matches; and in +v5.16 to the casing operations such as \f(CW\*(C`\eL\*(C'\fR and \f(CWuc()\fR. For +collation, in all releases so far, the system's \f(CWstrxfrm()\fR function is +called, and whatever it does is what you get. Starting in v5.26, various +bugs are fixed with the way perl uses this function. +.SH BUGS +.IX Header "BUGS" +.ie n .SS "Collation of strings containing embedded ""NUL"" characters" +.el .SS "Collation of strings containing embedded \f(CWNUL\fP characters" +.IX Subsection "Collation of strings containing embedded NUL characters" +\&\f(CW\*(C`NUL\*(C'\fR characters will sort the same as the lowest collating control +character does, or to \f(CW"\e001"\fR in the unlikely event that there are no +control characters at all in the locale. In cases where the strings +don't contain this non\-\f(CW\*(C`NUL\*(C'\fR control, the results will be correct, and +in many locales, this control, whatever it might be, will rarely be +encountered. But there are cases where a \f(CW\*(C`NUL\*(C'\fR should sort before this +control, but doesn't. If two strings do collate identically, the one +containing the \f(CW\*(C`NUL\*(C'\fR will sort to earlier. Prior to 5.26, there were +more bugs. +.SS Multi-threaded +.IX Subsection "Multi-threaded" +XS code or C\-language libraries called from it that use the system +\&\f(CWsetlocale(3)\fR function (except on Windows) likely will not work +from a multi-threaded application without changes. See +"Locale-aware XS code" in perlxs. +.PP +An XS module that is locale-dependent could have been written under the +assumption that it will never be called in a multi-threaded environment, +and so uses other non-locale constructs that aren't multi-thread-safe. +See "Thread-aware system interfaces" in perlxs. +.PP +POSIX does not define a way to get the name of the current per-thread +locale. Some systems, such as Darwin and FreeBSD do implement a +function, \fBquerylocale\fR\|(3) to do this. On non-Windows systems without +it, such as Linux, there are some additional caveats: +.IP \(bu 4 +An embedded perl needs to be started up while the global locale is in +effect. See "Using embedded Perl with POSIX locales" in perlembed. +.IP \(bu 4 +It becomes more important for perl to know about all the possible +locale categories on the platform, even if they aren't apparently used +in your program. Perl knows all of the Linux ones. If your platform +has others, you can submit an issue at +<https://github.com/Perl/perl5/issues> for +inclusion of it in the next release. In the meantime, it is possible to +edit the Perl source to teach it about the category, and then recompile. +Search for instances of, say, \f(CW\*(C`LC_PAPER\*(C'\fR in the source, and use that as +a template to add the omitted one. +.IP \(bu 4 +It is possible, though hard to do, to call \f(CW\*(C`POSIX::setlocale\*(C'\fR with a +locale that it doesn't recognize as syntactically legal, but actually is +legal on that system. This should happen only with embedded perls, or +if you hand-craft a locale name yourself. +.SS "Broken systems" +.IX Subsection "Broken systems" +In certain systems, the operating system's locale support +is broken and cannot be fixed or used by Perl. Such deficiencies can +and will result in mysterious hangs and/or Perl core dumps when +\&\f(CW\*(C`use locale\*(C'\fR is in effect. When confronted with such a system, +please report in excruciating detail to +<<https://github.com/Perl/perl5/issues>>, and +also contact your vendor: bug fixes may exist for these problems +in your operating system. Sometimes such bug fixes are called an +operating system upgrade. If you have the source for Perl, include in +the bug report the output of the test described above in "Testing +for broken locales". +.SH "SEE ALSO" +.IX Header "SEE ALSO" +I18N::Langinfo, perluniintro, perlunicode, open, +"localeconv" in POSIX, +"setlocale" in POSIX, "strcoll" in POSIX, "strftime" in POSIX, +"strtod" in POSIX, "strxfrm" in POSIX. +.PP +For special considerations when Perl is embedded in a C program, +see "Using embedded Perl with POSIX locales" in perlembed. +.SH HISTORY +.IX Header "HISTORY" +Jarkko Hietaniemi's original \fIperli18n.pod\fR heavily hacked by Dominic +Dunlop, assisted by the perl5\-porters. Prose worked over a bit by +Tom Christiansen, and now maintained by Perl 5 porters. |