postgresql/contrib/btree_gist/sql/float8.sql
Tom Lane d569ccd408 btree_gist: fix NaN handling in float4/float8 opclasses.
The float4 and float8 btree_gist opclasses compared keys with raw C
operators (==, <, >).  IEEE 754 makes every comparison involving NaN
false, so GiST disagreed with the regular float comparison operators
and with the btree opclass, which uses float[4|8]_cmp_internal()
(so that all NaNs are equal and NaN sorts after every non-NaN value).

In addition, the penalty and distance functions were not careful
about NaNs, and the penalty functions could also misbehave for IEEE
infinities.  Wrong answers from the penalty functions would probably
do no more than make the index non-optimal, but the distance mistakes
were visible from SQL.

To fix, make the comparison functions rely on the same NaN-aware
comparison functions the core code uses, and rewrite the penalty
and distance functions to follow the rules that NaNs are equal
but maximally far away from non-NaNs.  The penalty_num() code was
formerly shared between integral and float cases, but I chose to make
two copies so that the integral cases are not saddled with the extra
logic for NaNs and infinities/overflows.  I also rewrote it as static
inline functions instead of an unreadable and uncommented macro.

The float penalty functions were previously unreached by the
regression tests, so add new test cases to exercise them.

There's no on-disk format change, but users who have NaN entries
in a btree_gist index would be well advised to reindex it.

Bug: #19501
Bug: #19524
Reported-by: Man Zeng <zengman@halodbtech.com>
Reported-by: Yuelin Wang <3020001251@tju.edu.cn>
Author: Bill Kim <billkimjh@gmail.com>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19501-3bff3bbc97f1e7c9@postgresql.org
Discussion: https://postgr.es/m/19524-9559d302c8455664@postgresql.org
Discussion: https://postgr.es/m/CAMQXxcgbtD2LXfX0tpgvOizxP-XxrCHV2ZDy4By_TZnJMsxXWQ@mail.gmail.com
Backpatch-through: 14
2026-07-01 13:27:22 -04:00

54 lines
1.6 KiB
SQL

-- float8 check
CREATE TABLE float8tmp (a float8);
\copy float8tmp from 'data/float8.data'
SET enable_seqscan=on;
SELECT count(*) FROM float8tmp WHERE a < -1890.0;
SELECT count(*) FROM float8tmp WHERE a <= -1890.0;
SELECT count(*) FROM float8tmp WHERE a = -1890.0;
SELECT count(*) FROM float8tmp WHERE a >= -1890.0;
SELECT count(*) FROM float8tmp WHERE a > -1890.0;
SELECT a, a <-> '-1890.0' FROM float8tmp ORDER BY a <-> '-1890.0' LIMIT 3;
CREATE INDEX float8idx ON float8tmp USING gist ( a );
SET enable_seqscan=off;
SELECT count(*) FROM float8tmp WHERE a < -1890.0::float8;
SELECT count(*) FROM float8tmp WHERE a <= -1890.0::float8;
SELECT count(*) FROM float8tmp WHERE a = -1890.0::float8;
SELECT count(*) FROM float8tmp WHERE a >= -1890.0::float8;
SELECT count(*) FROM float8tmp WHERE a > -1890.0::float8;
EXPLAIN (COSTS OFF)
SELECT a, a <-> '-1890.0' FROM float8tmp ORDER BY a <-> '-1890.0' LIMIT 3;
SELECT a, a <-> '-1890.0' FROM float8tmp ORDER BY a <-> '-1890.0' LIMIT 3;
-- EXCLUDE constraint must block a duplicate NaN, same as it does for finite
-- values.
CREATE TABLE float8excl (a float8, EXCLUDE USING gist (a WITH =));
INSERT INTO float8excl VALUES ('NaN'::float8);
INSERT INTO float8excl VALUES ('NaN'::float8); -- expect: violates EXCLUDE
SELECT count(*) FROM float8excl;
-- Test double-column index
CREATE INDEX float8idx2 ON float8tmp USING gist ( a, abs(a) );
EXPLAIN (COSTS OFF)
SELECT count(*) FROM float8tmp WHERE abs(a) = 1890.0::float8;
SELECT count(*) FROM float8tmp WHERE abs(a) = 1890.0::float8;
RESET enable_seqscan;
RESET enable_indexscan;
RESET enable_bitmapscan;