Commit graph

556 commits

Author SHA1 Message Date
Pedro F. Giffuni
0716c0ff7a minor perf enhancement for UTF-8
Reduce some duplicate code.

Reference:
https://www.illumos.org/issues/628

Obtained from:	Illumos
MFC after:	1 week
2014-07-04 22:39:39 +00:00
Pedro F. Giffuni
0f5132cd25 citrus: Avoid invalid code points.
From the OpenBSD log:
The UTF-8 decoder should not accept byte sequences which decode to unicode
code positions U+D800 to U+DFFF (UTF-16 surrogates), U+FFFE, and U+FFFF.

http://www.cl.cam.ac.uk/~mgk25/unicode.html#utf-8
http://unicode.org/faq/utf_bom.html#utf8-4

Reported by:	Stefan Sperling
Obtained from:	OpenBSD
MFC after:	5 days
2014-05-01 01:42:48 +00:00
Pedro F. Giffuni
97ecaa8907 citrus: Avoid invalid code points.
From the OpenBSD log:
The UTF-8 decoder should not accept byte sequences which decode to unicode
code positions U+D800 to U+DFFF (UTF-16 surrogates), U+FFFE, and U+FFFF.

http://www.cl.cam.ac.uk/~mgk25/unicode.html#utf-8
http://unicode.org/faq/utf_bom.html#utf8-4

Reported by:	Stefan Sperling
Obtained from:	OpenBSD
MFC after:	5 days
2014-04-29 15:25:57 +00:00
David Chisnall
8d07b7deff Fix an issue where the locale and rune locale could become out of sync,
causing mb* functions (and similar) to be called with the wrong data
(possibly a null pointer, causing a crash).

PR:		standards/188036
MFC after:	1 week
2014-04-02 11:10:46 +00:00
Marcel Moolenaar
8876613dc5 Replace use of ${.CURDIR} by ${LIBC_SRCTOP} and define ${LIBC_SRCTOP}
if not already defined. This allows building libc from outside of
lib/libc using a reach-over makefile.

A typical use-case is to build a standard ILP32 version and a COMPAT32
version in a single iteration by building the COMPAT32 version using a
reach-over makefile.

Obtained from:	Juniper Networks, Inc.
2014-03-04 02:19:39 +00:00
Peter Wemm
19dfd82d81 Replace the #define for "iconv" so it is for the function name instead of
a macro with parameters.  Remove a __DECONST hack and add consts instead
for gnu libiconv API compatability.  This makes it work with things like
devel/boost-libs that expects to use "iconv" as though it were a pointer.
2013-07-03 07:03:19 +00:00
Ed Schouten
49111f0092 Add libiconv based versions of *c16*() and *c32*().
I initially thought wchar_t was locale independent, but this seems to be
only the case on Linux. This means that we cannot depend on the *wc*()
routines to implement *c16*() and *c32*(). Instead, use the Citrus
libiconv that is part of libc.

I'll see if there is anything I can do to make the existing functions
somewhat useful in case the system is built without libiconv in the
nearby future. If not, I'll simply remove the broken implementations.

Reviewed by:	jilles, gabor
2013-06-03 17:17:56 +00:00
Ed Schouten
50c77c6e8b Add <uchar.h>.
The <uchar.h> header, part of C11, adds a small number of utility
functions for 16/32-bit "universal" characters, which may or may not be
UTF-16/32. As our wchar_t is already ISO 10646, simply add light-weight
wrappers around wcrtomb() and mbrtowc().

While there, also add (non-yet-standard) _l functions, similar to the
ones we already have for the other locale-dependent functions.

Reviewed by:	theraven
2013-05-21 19:59:37 +00:00
Sergey Kandaurov
424b842a5d Document that the return type is different from 1003.1-2008.
MFC after:	1 week
2013-05-04 17:21:44 +00:00
Sergey Kandaurov
4f79ce7b4c mdoc: missing comma in .Dd macro. 2013-05-04 17:06:47 +00:00
Sergey Kandaurov
67ff590b9e Also, add a missing period. 2013-05-03 13:27:13 +00:00
Sergey Kandaurov
cc7a693fa7 Remove an extra comma. 2013-05-03 12:45:45 +00:00
Sergey Kandaurov
88bbbe88d7 Remove the STANDARDS section.
querylocale is not part of IEEE Std 1003.1-2008.

MFC after:	3 days
2013-05-03 12:42:43 +00:00
Jilles Tjoelker
86bca8fb51 btowc(3), isblank(3): Correct prototypes for _l variants.
MFC after:	1 week
2013-03-27 21:31:40 +00:00
Jilles Tjoelker
07588bf421 libc: Make various internal file descriptors close-on-exec.
These are obtained via fopen().
2012-12-11 22:52:56 +00:00
Brooks Davis
1652751915 Improve style(9) compliance of function declarations. 2012-12-10 17:34:33 +00:00
Jilles Tjoelker
05eb11cbc4 libc: Use O_CLOEXEC for various internal file descriptors.
This fixes a race condition where another thread may fork() before CLOEXEC
is set, unintentionally passing the descriptor to the child process.

This commit only adds O_CLOEXEC flags to open() or openat() calls where no
fcntl(fd, F_SETFD, FD_CLOEXEC) follows. The separate fcntl() call still
leaves a race window so it should be fixed later.
2012-09-29 11:54:34 +00:00
Joel Dahl
76c9beff1b Remove trailing whitespace. 2012-07-30 21:02:44 +00:00
Isabell Long
ea91541917 Add more locale-specific functions to the relevant man pages and Makefile:
- lib/libc/locale/islower.3
  - lib/libc/locale/ispunct.3
  - lib/libc/locale/nl_langinfo.3
  - lib/libc/locale/isgraph.3
  - lib/libc/locale/isspace.3

Reviewed by:	bz
Approved by:	theraven
MFC after:	5 days
2012-07-30 20:56:19 +00:00
Joel Dahl
a18f885152 Start manpage with Dd macro and also remove a trailing whitespace
while here.
2012-07-26 14:46:19 +00:00
Isabell Long
b3f18e536a Add a new man page containing details of new locale-specific functions for
wctype.h, iswalnum_l(3).  Add it and its functions to the Makefile.

Reviewed by:	gavin, jilles
Approved by:	theraven
MFC after:	5 days
2012-07-25 22:17:44 +00:00
David Chisnall
bb4317bf3c Restore the __collate_load_error global that was accidentally removed in the
xlocale refactoring.

MFC after:	1 week
2012-07-06 20:16:22 +00:00
Jilles Tjoelker
57ca132fc8 ctype_l(3): Note that not all these functions are specified by POSIX.1-2008.
The function isascii_l() is not in POSIX even though isascii() is,
probably because isascii() is marked as obsolete.  The other functions,
like digittoint_l() and ishexnumber_l(), are FreeBSD-specific just like
their non-_l versions.
2012-07-01 22:18:20 +00:00
David Chisnall
1967332863 Fix a leak when setting the global character locale to "C" from something else.
Reported by:	mm
2012-06-11 14:02:02 +00:00
Glen Barber
a3fb6da9ba General mdoc(7) and typo fixes.
PR:		167734
Submitted by:	Nobuyuki Koganemaru (kogane!jp.freebsd.org)
MFC after:	3 days
2012-05-11 20:06:46 +00:00
Dimitry Andric
83c476cd09 Fix copy/paste error in lib/libc/locale/toupper.c.
Submitted by:	Kohji Okuno <okuno.kohji@jp.panasonic.com>
2012-05-10 20:03:34 +00:00
David Chisnall
6ed2e496fd Fix some incorrect symbol versions.
Reported by:	das
2012-04-22 18:51:38 +00:00
Joel Dahl
26c9eb4a77 mdoc: fix function type. 2012-04-07 09:26:21 +00:00
Joel Dahl
748611c9c0 mdoc: fix column names, indentation, column separation within each row, and
quotation. Also make sure we have the same amount of columns in each row as
the number of columns we specify in the head arguments.

Reviewed by:	brueffer
2012-04-07 09:05:30 +00:00
Eitan Adler
50d675f7a9 Remove trailing whitespace per mdoc lint warning
Disussed with:	gavin
No objection from:	doc
Approved by:	joel
MFC after:	3 days
2012-03-29 05:02:12 +00:00
Joel Dahl
6ae6870fe8 mdoc: remove unknown macro. 2012-03-26 19:06:59 +00:00
Joel Dahl
5b40a50e9a mdoc: remove empty lines. 2012-03-26 15:30:28 +00:00
Joel Dahl
edf5bf9919 mdoc: document title should be all caps. 2012-03-25 19:34:05 +00:00
Joel Dahl
12afe06c06 Make sure sections are sorted into conventional order. 2012-03-25 16:00:56 +00:00
Joel Dahl
41949a1ed5 Remove superfluous paragraph macro. 2012-03-25 12:13:24 +00:00
David Chisnall
5a674eeb8f Make __get_locale() static inline, not just inline, so when compiling libc in
debug mode (without optimisations) it doesn't generate a linker failure.

Approved by:	dim (mentor)
2012-03-19 11:47:37 +00:00
David Chisnall
c889dd01b9 First set of xlocale man pages. More to follow...
Approved by:	dim (mentor)
2012-03-13 20:02:41 +00:00
David Chisnall
4d7815ecc4 More xlocale cleanups.
Approved by:	dim (mentor)
2012-03-13 14:14:13 +00:00
Dimitry Andric
cc09dfac61 Fix build of libc.so after r232620. This caused a duplicate definition
of __getCurrentRuneLocale().

Pointy hat to:	me
2012-03-06 21:56:30 +00:00
David Chisnall
7dfd88318b Remove some duplicated copyright notices.
Approved by:	dim (mentor)
2012-03-06 12:53:44 +00:00
David Chisnall
a8ed63bb3d Reapply 227753 (xlocale cleanup), plus some fixes so that it passes build
universe with gcc.

Approved by:	dim (mentor)
2012-03-04 15:31:13 +00:00
Dimitry Andric
b74cf6dcf1 Revert r231673 and r231682 for now, until we can run a full make
universe with them.  Sorry for the breakage.

Pointy hat to:	     me and brooks
2012-02-14 21:48:46 +00:00
David Chisnall
7780c181c5 Fix a misplaced __NO_TLS locations, and change a GNUism to a C11ism for
consistency.

Approved by:	brooks (mentor)
2012-02-14 14:24:37 +00:00
David Chisnall
82dd5016bd Cleanup of xlocale:
- Address performance regressions encountered by das@ by caching per-thread
  data in TLS where available.
- Add a __NO_TLS flag to cdefs.h to indicate where not available.
- Reorganise the xlocale.h definitions into xlocale/*.h so that they can be
  included from multiple places.
- Export the POSIX2008 subset of xlocale when POSIX2008 says it should be
  exported, independently of whether xlocale.h is included.
- Fix the bug where programs using ctype functions always assumed ASCII unless
  recompiled.
- Fix some style(9) violations.

Reviewed by:	brooks (mentor)
Approved by:	dim (mentor)
2012-02-14 12:03:23 +00:00
Xin LI
945aab9099 wctob() returns EOF and not WEOF.
Noticed by:	Zhihao Yuan <lichray gmail com>
MFC after:	1 week
2012-02-14 02:03:17 +00:00
Jilles Tjoelker
571eea83bb libc: Eliminate some relative relocations in setlocale(). 2011-12-27 23:28:01 +00:00
Jilles Tjoelker
54ad8a62ed libc: Eliminate 13 relative relocations in wctype(). 2011-12-05 00:00:47 +00:00
David E. O'Brien
8449399687 Split sentences at period boundaries. 2011-12-02 11:55:09 +00:00
David Chisnall
2013459487 Fix a crash when trying to duplicate a locale that contains some implicit C locale components.
Reported by:	Michael Butler
Approved by:	dim (mentor)
2011-11-22 14:11:42 +00:00
David Chisnall
3c87aa1d3d Implement xlocale APIs from Darwin, mainly for use by libc++. This adds a
load of _l suffixed versions of various standard library functions that use
the global locale, making them take an explicit locale parameter.  Also
adds support for per-thread locales.  This work was funded by the FreeBSD
Foundation.

Please test any code you have that uses the C standard locale functions!

Reviewed by:    das (gdtoa changes)
Approved by:    dim (mentor)
2011-11-20 14:45:42 +00:00