postgresql/contrib/intarray
David Rowley 07c1c6ec51 Fix selectivity estimation integer overflow in contrib/intarray
This fixes a poorly written integer comparison function which was
performing subtraction in an attempt to return a negative value when
a < b and a positive value when a > b, and 0 when the values were equal.
Unfortunately that didn't always work correctly due to two's complement
having the INT_MIN 1 further from zero than INT_MAX.  This could result
in an overflow and cause the comparison function to return an incorrect
result, which would result in the binary search failing to find the
value being searched for.

This could cause poor selectivity estimates when the statistics stored
the value of INT_MAX (2147483647) and the value being searched for was
large enough to result in the binary search doing a comparison with that
INT_MAX value.

Author: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAEoWx2ng1Ot5LoKbVU-Dh---dFTUZWJRH8wv2chBu29fnNDMaQ@mail.gmail.com
Backpatch-through: 14
2026-01-04 20:33:14 +13:00
..
bench Update copyright for 2025 2025-01-01 11:21:55 -05:00
data Fix integer-overflow problem in intarray's g_int_decompress(). 2024-01-07 15:19:50 -05:00
expected Fix GIN's shimTriConsistentFn to not corrupt its input. 2025-04-12 12:28:02 -04:00
sql Fix GIN's shimTriConsistentFn to not corrupt its input. 2025-04-12 12:28:02 -04:00
.gitignore Support "make check" in contrib 2011-04-25 22:27:11 +03:00
_int.h Specialize intarray sorting 2025-02-18 11:04:55 +07:00
_int_bool.c Remove useless casts to (void *) 2024-11-28 08:27:20 +01:00
_int_gin.c Make the order of the header file includes consistent in contrib modules. 2019-10-24 08:05:34 +05:30
_int_gist.c Fix integer-overflow problem in intarray's g_int_decompress(). 2024-01-07 15:19:50 -05:00
_int_op.c Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
_int_selfuncs.c Fix selectivity estimation integer overflow in contrib/intarray 2026-01-04 20:33:14 +13:00
_int_tool.c Specialize intarray sorting 2025-02-18 11:04:55 +07:00
_intbig_gist.c Use new overflow-safe integer comparison functions. 2024-02-16 14:05:36 -06:00
intarray--1.0--1.1.sql Add selectivity estimation functions for intarray operators. 2015-07-21 20:59:24 +03:00
intarray--1.1--1.2.sql Make contrib modules' installation scripts more secure. 2020-08-10 10:44:42 -04:00
intarray--1.2--1.3.sql Implement operator class parameters 2020-03-30 19:17:23 +03:00
intarray--1.2.sql Update extensions with GIN/GIST support for parallel query. 2016-06-14 13:34:37 -04:00
intarray--1.3--1.4.sql Remove <@ from contrib/intarray's GiST operator classes. 2020-08-08 17:26:29 -04:00
intarray--1.4--1.5.sql Remove deprecated containment operators for contrib types. 2021-03-05 10:45:41 -05:00
intarray.control Remove deprecated containment operators for contrib types. 2021-03-05 10:45:41 -05:00
Makefile Remove deprecated containment operators for contrib types. 2021-03-05 10:45:41 -05:00
meson.build Update copyright for 2025 2025-01-01 11:21:55 -05:00