postgresql/contrib/intarray
David Rowley 54f82c4aae Fix selectivity estimation integer overflow in contrib/intarray
This fixes a poorly written integer comparison function which was
performing subtraction in an attempt to return a negative value when
a < b and a positive value when a > b, and 0 when the values were equal.
Unfortunately that didn't always work correctly due to two's complement
having the INT_MIN 1 further from zero than INT_MAX.  This could result
in an overflow and cause the comparison function to return an incorrect
result, which would result in the binary search failing to find the
value being searched for.

This could cause poor selectivity estimates when the statistics stored
the value of INT_MAX (2147483647) and the value being searched for was
large enough to result in the binary search doing a comparison with that
INT_MAX value.

Author: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAEoWx2ng1Ot5LoKbVU-Dh---dFTUZWJRH8wv2chBu29fnNDMaQ@mail.gmail.com
Backpatch-through: 14
2026-01-04 20:34:01 +13:00
..
bench Pre-beta mechanical code beautification. 2023-05-19 17:24:48 -04:00
data Fix integer-overflow problem in intarray's g_int_decompress(). 2024-01-07 15:19:50 -05:00
expected Fix GIN's shimTriConsistentFn to not corrupt its input. 2025-04-12 12:27:46 -04:00
sql Fix GIN's shimTriConsistentFn to not corrupt its input. 2025-04-12 12:27:46 -04:00
.gitignore Support "make check" in contrib 2011-04-25 22:27:11 +03:00
_int.h Fix typos in comments 2023-05-02 12:23:08 +09:00
_int_bool.c Convert contrib/intarray's bqarr_in() to report errors softly 2022-12-28 10:00:12 -05:00
_int_gin.c Make the order of the header file includes consistent in contrib modules. 2019-10-24 08:05:34 +05:30
_int_gist.c Fix integer-overflow problem in intarray's g_int_decompress(). 2024-01-07 15:19:50 -05:00
_int_op.c Clean up newlines following left parentheses 2020-01-30 13:42:14 -03:00
_int_selfuncs.c Fix selectivity estimation integer overflow in contrib/intarray 2026-01-04 20:34:01 +13:00
_int_tool.c Remove useless casts to (void *) in arguments of some system functions 2023-02-07 06:57:59 +01:00
_intbig_gist.c Remove unnecessary pfree() in g_intbig_compress(). 2023-07-13 13:08:08 -04:00
intarray--1.0--1.1.sql Add selectivity estimation functions for intarray operators. 2015-07-21 20:59:24 +03:00
intarray--1.1--1.2.sql Make contrib modules' installation scripts more secure. 2020-08-10 10:44:42 -04:00
intarray--1.2--1.3.sql Implement operator class parameters 2020-03-30 19:17:23 +03:00
intarray--1.2.sql Update extensions with GIN/GIST support for parallel query. 2016-06-14 13:34:37 -04:00
intarray--1.3--1.4.sql Remove <@ from contrib/intarray's GiST operator classes. 2020-08-08 17:26:29 -04:00
intarray--1.4--1.5.sql Remove deprecated containment operators for contrib types. 2021-03-05 10:45:41 -05:00
intarray.control Remove deprecated containment operators for contrib types. 2021-03-05 10:45:41 -05:00
Makefile Remove deprecated containment operators for contrib types. 2021-03-05 10:45:41 -05:00
meson.build Update copyright for 2023 2023-01-02 15:00:37 -05:00