This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: awk gsub problem


On 09/19/2010 02:33 PM, Lee wrote:
If LANG is "en_US" or "en_US.utf8", then the regular expression "[a-z]"
does *not* correspond anymore to the ASCII codes.  Rather it corresponds
to something like "[aAbBcCdD...zZ]", independent of the actual character
encoding ISO-8859-1 or UTF-8.

In glibc, [a-z] gets translated according to locale collation order. If A collates before a, then it maps to [aBbCc..Zz], if A collates after a, then it maps to [aAbB...yYz] (notice that in either case, one of the two capital letters is omitted, so it is NOT the same as all 26 letters in both cases).


This has been a MUCH complained-about feature of glibc, which has in turn been copied by bash, awk, grep, etc.

Note that POSIX explicitly states that [a-z] has unspecified results in any locale except C. So the glibc behavior is permitted, but so is the traditional behavior of just the 26 lowercase letters.

If you can convince the glibc folks that [a-z] should have the traditional behavior, more power to you.

http://lists.gnu.org/archive/html/bug-grep/2010-09/msg00030.html

--
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]