postgresql/contrib/pg_visibility/sql/pg_visibility.sql

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

197 lines
6.4 KiB
MySQL
Raw Normal View History

CREATE EXTENSION pg_visibility;
Combine visibilitymap_set() cases in lazy_scan_prune() lazy_scan_prune() previously had two separate cases that called visibilitymap_set() after pruning and freezing. These branches were nearly identical except that one attempted to avoid dirtying the heap buffer. However, that situation can never occur — the heap buffer cannot be clean at that point (and we would hit an assertion if it were). In lazy_scan_prune(), when we change a previously all-visible page to all-frozen and the page was recorded as all-visible in the visibility map by find_next_unskippable_block(), the heap buffer will always be dirty. Either we have just frozen a tuple and already dirtied the buffer, or the buffer was modified between find_next_unskippable_block() and heap_page_prune_and_freeze() and then pruned in heap_page_prune_and_freeze(). Additionally, XLogRegisterBuffer() asserts that the buffer is dirty, so attempting to add a clean heap buffer to the WAL chain would assert out anyway. Since the “clean heap buffer with already set VM” case is impossible, the two visibilitymap_set() branches in lazy_scan_prune() can be merged. Doing so makes the intent clearer and emphasizes that the heap buffer must always be marked dirty before being added to the WAL chain. This commit also adds a test case for vacuuming when no heap modifications are required. Currently this ensures that the heap buffer is marked dirty before it is added to the WAL chain, but if we later remove the heap buffer from the VM-set WAL chain or pass it with the REGBUF_NO_CHANGES flag, this test would guard that behavior. Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Srinath Reddy Sadipiralla <srinath2133@gmail.com> Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru> Discussion: https://postgr.es/m/5CEAA162-67B1-44DA-B60D-8B65717E8B05%40gmail.com Discussion: https://postgr.es/m/flat/CAAKRu_ZWx5gCbeCf7PWCv8p5%3D%3Db7EEws0VD2wksDxpXCvCyHvQ%40mail.gmail.com
2026-01-26 15:57:51 -05:00
CREATE EXTENSION pageinspect;
Skip WAL for new relfilenodes, under wal_level=minimal. Until now, only selected bulk operations (e.g. COPY) did this. If a given relfilenode received both a WAL-skipping COPY and a WAL-logged operation (e.g. INSERT), recovery could lose tuples from the COPY. See src/backend/access/transam/README section "Skipping WAL for New RelFileNode" for the new coding rules. Maintainers of table access methods should examine that section. To maintain data durability, just before commit, we choose between an fsync of the relfilenode and copying its contents to WAL. A new GUC, wal_skip_threshold, guides that choice. If this change slows a workload that creates small, permanent relfilenodes under wal_level=minimal, try adjusting wal_skip_threshold. Users setting a timeout on COMMIT may need to adjust that timeout, and log_min_duration_statement analysis will reflect time consumption moving to COMMIT from commands like COPY. Internally, this requires a reliable determination of whether RollbackAndReleaseCurrentSubTransaction() would unlink a relation's current relfilenode. Introduce rd_firstRelfilenodeSubid. Amend the specification of rd_createSubid such that the field is zero when a new rel has an old rd_node. Make relcache.c retain entries for certain dropped relations until end of transaction. Bump XLOG_PAGE_MAGIC, since this introduces XLOG_GIST_ASSIGN_LSN. Future servers accept older WAL, so this bump is discretionary. Kyotaro Horiguchi, reviewed (in earlier, similar versions) by Robert Haas. Heikki Linnakangas and Michael Paquier implemented earlier designs that materially clarified the problem. Reviewed, in earlier designs, by Andrew Dunstan, Andres Freund, Alvaro Herrera, Tom Lane, Fujii Masao, and Simon Riggs. Reported by Martijn van Oosterhout. Discussion: https://postgr.es/m/20150702220524.GA9392@svana.org
2020-04-04 15:25:34 -04:00
--
-- recently-dropped table
--
\set VERBOSITY sqlstate
BEGIN;
CREATE TABLE droppedtest (c int);
SELECT 'droppedtest'::regclass::oid AS oid \gset
SAVEPOINT q; DROP TABLE droppedtest; RELEASE q;
SAVEPOINT q; SELECT * FROM pg_visibility_map(:oid); ROLLBACK TO q;
-- ERROR: could not open relation with OID 16xxx
SAVEPOINT q; SELECT 1; ROLLBACK TO q;
SAVEPOINT q; SELECT 1; ROLLBACK TO q;
SELECT pg_relation_size(:oid), pg_relation_filepath(:oid),
has_table_privilege(:oid, 'SELECT');
SELECT * FROM pg_visibility_map(:oid);
-- ERROR: could not open relation with OID 16xxx
ROLLBACK;
\set VERBOSITY default
--
-- check that using the module's functions with unsupported relations will fail
--
-- partitioned tables (the parent ones) don't have visibility maps
create table test_partitioned (a int) partition by list (a);
-- these should all fail
select pg_visibility('test_partitioned', 0);
select pg_visibility_map('test_partitioned');
select pg_visibility_map_summary('test_partitioned');
select pg_check_frozen('test_partitioned');
select pg_truncate_visibility_map('test_partitioned');
create table test_partition partition of test_partitioned for values in (1);
create index test_index on test_partition (a);
-- indexes do not, so these all fail
select pg_visibility('test_index', 0);
select pg_visibility_map('test_index');
select pg_visibility_map_summary('test_index');
select pg_check_frozen('test_index');
select pg_truncate_visibility_map('test_index');
create view test_view as select 1;
-- views do not have VMs, so these all fail
select pg_visibility('test_view', 0);
select pg_visibility_map('test_view');
select pg_visibility_map_summary('test_view');
select pg_check_frozen('test_view');
select pg_truncate_visibility_map('test_view');
create sequence test_sequence;
-- sequences do not have VMs, so these all fail
select pg_visibility('test_sequence', 0);
select pg_visibility_map('test_sequence');
select pg_visibility_map_summary('test_sequence');
select pg_check_frozen('test_sequence');
select pg_truncate_visibility_map('test_sequence');
create foreign data wrapper dummy;
create server dummy_server foreign data wrapper dummy;
create foreign table test_foreign_table () server dummy_server;
-- foreign tables do not have VMs, so these all fail
select pg_visibility('test_foreign_table', 0);
select pg_visibility_map('test_foreign_table');
select pg_visibility_map_summary('test_foreign_table');
select pg_check_frozen('test_foreign_table');
select pg_truncate_visibility_map('test_foreign_table');
-- check some of the allowed relkinds
create table regular_table (a int, b text);
alter table regular_table alter column b set storage external;
insert into regular_table values (1, repeat('one', 1000)), (2, repeat('two', 1000));
vacuum (disable_page_skipping) regular_table;
select count(*) > 0 from pg_visibility('regular_table');
select count(*) > 0 from pg_visibility((select reltoastrelid from pg_class where relname = 'regular_table'));
truncate regular_table;
select count(*) > 0 from pg_visibility('regular_table');
select count(*) > 0 from pg_visibility((select reltoastrelid from pg_class where relname = 'regular_table'));
create materialized view matview_visibility_test as select * from regular_table;
vacuum (disable_page_skipping) matview_visibility_test;
select count(*) > 0 from pg_visibility('matview_visibility_test');
insert into regular_table values (1), (2);
refresh materialized view matview_visibility_test;
select count(*) > 0 from pg_visibility('matview_visibility_test');
-- regular tables which are part of a partition *do* have visibility maps
insert into test_partition values (1);
vacuum (disable_page_skipping) test_partition;
select count(*) > 0 from pg_visibility('test_partition', 0);
select count(*) > 0 from pg_visibility_map('test_partition');
select count(*) > 0 from pg_visibility_map_summary('test_partition');
select * from pg_check_frozen('test_partition'); -- hopefully none
select pg_truncate_visibility_map('test_partition');
Combine visibilitymap_set() cases in lazy_scan_prune() lazy_scan_prune() previously had two separate cases that called visibilitymap_set() after pruning and freezing. These branches were nearly identical except that one attempted to avoid dirtying the heap buffer. However, that situation can never occur — the heap buffer cannot be clean at that point (and we would hit an assertion if it were). In lazy_scan_prune(), when we change a previously all-visible page to all-frozen and the page was recorded as all-visible in the visibility map by find_next_unskippable_block(), the heap buffer will always be dirty. Either we have just frozen a tuple and already dirtied the buffer, or the buffer was modified between find_next_unskippable_block() and heap_page_prune_and_freeze() and then pruned in heap_page_prune_and_freeze(). Additionally, XLogRegisterBuffer() asserts that the buffer is dirty, so attempting to add a clean heap buffer to the WAL chain would assert out anyway. Since the “clean heap buffer with already set VM” case is impossible, the two visibilitymap_set() branches in lazy_scan_prune() can be merged. Doing so makes the intent clearer and emphasizes that the heap buffer must always be marked dirty before being added to the WAL chain. This commit also adds a test case for vacuuming when no heap modifications are required. Currently this ensures that the heap buffer is marked dirty before it is added to the WAL chain, but if we later remove the heap buffer from the VM-set WAL chain or pass it with the REGBUF_NO_CHANGES flag, this test would guard that behavior. Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Srinath Reddy Sadipiralla <srinath2133@gmail.com> Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru> Discussion: https://postgr.es/m/5CEAA162-67B1-44DA-B60D-8B65717E8B05%40gmail.com Discussion: https://postgr.es/m/flat/CAAKRu_ZWx5gCbeCf7PWCv8p5%3D%3Db7EEws0VD2wksDxpXCvCyHvQ%40mail.gmail.com
2026-01-26 15:57:51 -05:00
-- test the case where vacuum phase I does not need to modify the heap buffer
-- and only needs to set the VM
create temp table test_vac_unmodified_heap(a int);
Combine visibilitymap_set() cases in lazy_scan_prune() lazy_scan_prune() previously had two separate cases that called visibilitymap_set() after pruning and freezing. These branches were nearly identical except that one attempted to avoid dirtying the heap buffer. However, that situation can never occur — the heap buffer cannot be clean at that point (and we would hit an assertion if it were). In lazy_scan_prune(), when we change a previously all-visible page to all-frozen and the page was recorded as all-visible in the visibility map by find_next_unskippable_block(), the heap buffer will always be dirty. Either we have just frozen a tuple and already dirtied the buffer, or the buffer was modified between find_next_unskippable_block() and heap_page_prune_and_freeze() and then pruned in heap_page_prune_and_freeze(). Additionally, XLogRegisterBuffer() asserts that the buffer is dirty, so attempting to add a clean heap buffer to the WAL chain would assert out anyway. Since the “clean heap buffer with already set VM” case is impossible, the two visibilitymap_set() branches in lazy_scan_prune() can be merged. Doing so makes the intent clearer and emphasizes that the heap buffer must always be marked dirty before being added to the WAL chain. This commit also adds a test case for vacuuming when no heap modifications are required. Currently this ensures that the heap buffer is marked dirty before it is added to the WAL chain, but if we later remove the heap buffer from the VM-set WAL chain or pass it with the REGBUF_NO_CHANGES flag, this test would guard that behavior. Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Srinath Reddy Sadipiralla <srinath2133@gmail.com> Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru> Discussion: https://postgr.es/m/5CEAA162-67B1-44DA-B60D-8B65717E8B05%40gmail.com Discussion: https://postgr.es/m/flat/CAAKRu_ZWx5gCbeCf7PWCv8p5%3D%3Db7EEws0VD2wksDxpXCvCyHvQ%40mail.gmail.com
2026-01-26 15:57:51 -05:00
insert into test_vac_unmodified_heap values (1);
vacuum (freeze) test_vac_unmodified_heap;
select pg_visibility_map_summary('test_vac_unmodified_heap');
-- the checkpoint cleans the buffer dirtied by freezing the sole tuple
checkpoint;
-- truncating the VM ensures that the next vacuum will need to set it
select pg_truncate_visibility_map('test_vac_unmodified_heap');
select pg_visibility_map_summary('test_vac_unmodified_heap');
-- though the VM is truncated, the heap page-level visibility hint,
-- PD_ALL_VISIBLE should still be set
SELECT (flags & x'0004'::int) <> 0
FROM page_header(get_raw_page('test_vac_unmodified_heap', 0));
-- vacuum sets the VM
vacuum test_vac_unmodified_heap;
select pg_visibility_map_summary('test_vac_unmodified_heap');
-- test copy freeze
create table copyfreeze (a int, b char(1500));
-- load all rows via COPY FREEZE and ensure that all pages are set all-visible
-- and all-frozen.
begin;
truncate copyfreeze;
copy copyfreeze from stdin freeze;
1 '1'
2 '2'
3 '3'
4 '4'
5 '5'
6 '6'
\.
commit;
select * from pg_visibility_map('copyfreeze');
select * from pg_check_frozen('copyfreeze');
-- load half the rows via regular COPY and rest via COPY FREEZE. The pages
-- which are touched by regular COPY must not be set all-visible/all-frozen. On
-- the other hand, pages allocated by COPY FREEZE should be marked
-- all-frozen/all-visible.
begin;
truncate copyfreeze;
copy copyfreeze from stdin;
1 '1'
2 '2'
3 '3'
4 '4'
5 '5'
6 '6'
\.
copy copyfreeze from stdin freeze;
7 '7'
8 '8'
9 '9'
10 '10'
11 '11'
12 '12'
\.
commit;
select * from pg_visibility_map('copyfreeze');
select * from pg_check_frozen('copyfreeze');
-- Try a mix of regular COPY and COPY FREEZE.
begin;
truncate copyfreeze;
copy copyfreeze from stdin freeze;
1 '1'
2 '2'
3 '3'
4 '4'
5 '5'
\.
copy copyfreeze from stdin;
6 '6'
\.
copy copyfreeze from stdin freeze;
7 '7'
8 '8'
9 '9'
10 '10'
11 '11'
12 '12'
\.
commit;
select * from pg_visibility_map('copyfreeze');
select * from pg_check_frozen('copyfreeze');
-- cleanup
drop table test_partitioned;
drop view test_view;
drop sequence test_sequence;
drop foreign table test_foreign_table;
drop server dummy_server;
drop foreign data wrapper dummy;
drop materialized view matview_visibility_test;
drop table regular_table;
drop table copyfreeze;