postgresql/src/test/regress/sql/subscription.sql

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

293 lines
11 KiB
MySQL
Raw Normal View History

--
-- SUBSCRIPTION
--
CREATE ROLE regress_subscription_user LOGIN SUPERUSER;
CREATE ROLE regress_subscription_user2;
CREATE ROLE regress_subscription_user_dummy LOGIN NOSUPERUSER;
SET SESSION AUTHORIZATION 'regress_subscription_user';
-- fail - no publications
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub CONNECTION 'foo';
-- fail - no connection
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub PUBLICATION foo;
-- fail - cannot do CREATE SUBSCRIPTION CREATE SLOT inside transaction block
BEGIN;
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub CONNECTION 'testconn' PUBLICATION testpub WITH (create_slot);
COMMIT;
-- fail - invalid connection string
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub CONNECTION 'testconn' PUBLICATION testpub;
-- fail - duplicate publications
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION foo, testpub, foo WITH (connect = false);
-- ok
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false);
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
COMMENT ON SUBSCRIPTION regress_testsub IS 'test subscription';
SELECT obj_description(s.oid, 'pg_subscription') FROM pg_subscription s;
-- Check if the subscription stats are created and stats_reset is updated
-- by pg_stat_reset_subscription_stats().
SELECT subname, stats_reset IS NULL stats_reset_is_null FROM pg_stat_subscription_stats WHERE subname = 'regress_testsub';
SELECT pg_stat_reset_subscription_stats(oid) FROM pg_subscription WHERE subname = 'regress_testsub';
SELECT subname, stats_reset IS NULL stats_reset_is_null FROM pg_stat_subscription_stats WHERE subname = 'regress_testsub';
-- Reset the stats again and check if the new reset_stats is updated.
SELECT stats_reset as prev_stats_reset FROM pg_stat_subscription_stats WHERE subname = 'regress_testsub' \gset
SELECT pg_stat_reset_subscription_stats(oid) FROM pg_subscription WHERE subname = 'regress_testsub';
SELECT :'prev_stats_reset' < stats_reset FROM pg_stat_subscription_stats WHERE subname = 'regress_testsub';
-- fail - name already exists
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false);
-- fail - must be superuser
SET SESSION AUTHORIZATION 'regress_subscription_user2';
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub2 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION foo WITH (connect = false);
SET SESSION AUTHORIZATION 'regress_subscription_user';
-- fail - invalid option combinations
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub2 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, copy_data = true);
CREATE SUBSCRIPTION regress_testsub2 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, enabled = true);
CREATE SUBSCRIPTION regress_testsub2 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, create_slot = true);
CREATE SUBSCRIPTION regress_testsub2 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (slot_name = NONE, enabled = true);
CREATE SUBSCRIPTION regress_testsub2 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (slot_name = NONE, enabled = false, create_slot = true);
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub2 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (slot_name = NONE);
CREATE SUBSCRIPTION regress_testsub2 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (slot_name = NONE, enabled = false);
CREATE SUBSCRIPTION regress_testsub2 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (slot_name = NONE, create_slot = false);
-- ok - with slot_name = NONE
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
CREATE SUBSCRIPTION regress_testsub3 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (slot_name = NONE, connect = false);
-- fail
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub3 ENABLE;
ALTER SUBSCRIPTION regress_testsub3 REFRESH PUBLICATION;
2022-07-20 23:17:38 -04:00
-- fail - origin must be either none or any
CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (slot_name = NONE, connect = false, origin = foo);
-- now it works
CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (slot_name = NONE, connect = false, origin = none);
\dRs+ regress_testsub4
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
DROP SUBSCRIPTION regress_testsub3;
2022-07-20 23:17:38 -04:00
DROP SUBSCRIPTION regress_testsub4;
-- fail, connection string does not parse
CREATE SUBSCRIPTION regress_testsub5 CONNECTION 'i_dont_exist=param' PUBLICATION testpub;
-- fail, connection string parses, but doesn't work (and does so without
-- connecting, so this is reliable and safe)
CREATE SUBSCRIPTION regress_testsub5 CONNECTION 'port=-1' PUBLICATION testpub;
-- fail - invalid connection string during ALTER
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
\dRs+
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
ALTER SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist2';
ALTER SUBSCRIPTION regress_testsub SET (slot_name = 'newname');
-- fail
ALTER SUBSCRIPTION regress_testsub SET (slot_name = '');
-- fail
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_doesnotexist CONNECTION 'dbname=regress_doesnotexist2';
ALTER SUBSCRIPTION regress_testsub SET (create_slot = false);
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
-- ok - with lsn = NONE
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
-- fail
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
\dRs+
BEGIN;
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub ENABLE;
\dRs
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub DISABLE;
\dRs
COMMIT;
-- fail - must be owner of subscription
SET ROLE regress_subscription_user_dummy;
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub RENAME TO regress_testsub_dummy;
RESET ROLE;
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub RENAME TO regress_testsub_foo;
ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = local);
ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
\dRs+
-- rename back to keep the rest simple
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub_foo RENAME TO regress_testsub;
-- fail - new owner must be superuser
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub OWNER TO regress_subscription_user2;
ALTER ROLE regress_subscription_user2 SUPERUSER;
-- now it works
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub OWNER TO regress_subscription_user2;
-- fail - cannot do DROP SUBSCRIPTION inside transaction block with slot name
BEGIN;
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
DROP SUBSCRIPTION regress_testsub;
COMMIT;
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
-- now it works
BEGIN;
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
DROP SUBSCRIPTION regress_testsub;
COMMIT;
Fix regression tests to use only global names beginning with "regress_". In commit 18555b132 we tentatively established a rule that regression tests should use names containing "regression" for databases, and names starting with "regress_" for all other globally-visible object names, so as to circumscribe the side-effects that "make installcheck" could have on an existing installation. However, no enforcement mechanism was created, so it's unsurprising that some new violations have crept in since then. In fact, a whole new *category* of violations has crept in, to wit we now also have globally-visible subscription and replication origin names, and "make installcheck" could very easily clobber user-created objects of those types. So it's past time to do something about this. This commit sanitizes the tests enough that they will pass (i.e. not generate any visible warnings) with the enforcement mechanism I'll add in the next commit. There are some TAP tests that still trigger the warnings, but the warnings do not cause test failure. Since these tests do not actually run against a pre-existing installation, there's no need to worry whether they could conflict with user-created objects. The problem with rolenames.sql testing special role names like "user" is still there, and is dealt with only very cosmetically in this patch (by hiding the warnings :-(). What we actually need to do to be safe is to take that test script out of "make installcheck" altogether, but that seems like material for a separate patch. Discussion: https://postgr.es/m/16638.1468620817@sss.pgh.pa.us
2019-06-29 11:09:03 -04:00
DROP SUBSCRIPTION IF EXISTS regress_testsub;
DROP SUBSCRIPTION regress_testsub; -- fail
-- fail - binary must be boolean
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, binary = foo);
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, binary = true);
\dRs+
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
DROP SUBSCRIPTION regress_testsub;
Perform apply of large transactions by parallel workers. Currently, for large transactions, the publisher sends the data in multiple streams (changes divided into chunks depending upon logical_decoding_work_mem), and then on the subscriber-side, the apply worker writes the changes into temporary files and once it receives the commit, it reads from those files and applies the entire transaction. To improve the performance of such transactions, we can instead allow them to be applied via parallel workers. In this approach, we assign a new parallel apply worker (if available) as soon as the xact's first stream is received and the leader apply worker will send changes to this new worker via shared memory. The parallel apply worker will directly apply the change instead of writing it to temporary files. However, if the leader apply worker times out while attempting to send a message to the parallel apply worker, it will switch to "partial serialize" mode - in this mode, the leader serializes all remaining changes to a file and notifies the parallel apply workers to read and apply them at the end of the transaction. We use a non-blocking way to send the messages from the leader apply worker to the parallel apply to avoid deadlocks. We keep this parallel apply assigned till the transaction commit is received and also wait for the worker to finish at commit. This preserves commit ordering and avoid writing to and reading from files in most cases. We still need to spill if there is no worker available. This patch also extends the SUBSCRIPTION 'streaming' parameter so that the user can control whether to apply the streaming transaction in a parallel apply worker or spill the change to disk. The user can set the streaming parameter to 'on/off', or 'parallel'. The parameter value 'parallel' means the streaming will be applied via a parallel apply worker, if available. The parameter value 'on' means the streaming transaction will be spilled to disk. The default value is 'off' (same as current behaviour). In addition, the patch extends the logical replication STREAM_ABORT message so that abort_lsn and abort_time can also be sent which can be used to update the replication origin in parallel apply worker when the streaming transaction is aborted. Because this message extension is needed to support parallel streaming, parallel streaming is not supported for publications on servers < PG16. Author: Hou Zhijie, Wang wei, Amit Kapila with design inputs from Sawada Masahiko Reviewed-by: Sawada Masahiko, Peter Smith, Dilip Kumar, Shi yu, Kuroda Hayato, Shveta Mallik Discussion: https://postgr.es/m/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
2023-01-08 20:30:39 -05:00
-- fail - streaming must be boolean or 'parallel'
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = foo);
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = true);
\dRs+
Perform apply of large transactions by parallel workers. Currently, for large transactions, the publisher sends the data in multiple streams (changes divided into chunks depending upon logical_decoding_work_mem), and then on the subscriber-side, the apply worker writes the changes into temporary files and once it receives the commit, it reads from those files and applies the entire transaction. To improve the performance of such transactions, we can instead allow them to be applied via parallel workers. In this approach, we assign a new parallel apply worker (if available) as soon as the xact's first stream is received and the leader apply worker will send changes to this new worker via shared memory. The parallel apply worker will directly apply the change instead of writing it to temporary files. However, if the leader apply worker times out while attempting to send a message to the parallel apply worker, it will switch to "partial serialize" mode - in this mode, the leader serializes all remaining changes to a file and notifies the parallel apply workers to read and apply them at the end of the transaction. We use a non-blocking way to send the messages from the leader apply worker to the parallel apply to avoid deadlocks. We keep this parallel apply assigned till the transaction commit is received and also wait for the worker to finish at commit. This preserves commit ordering and avoid writing to and reading from files in most cases. We still need to spill if there is no worker available. This patch also extends the SUBSCRIPTION 'streaming' parameter so that the user can control whether to apply the streaming transaction in a parallel apply worker or spill the change to disk. The user can set the streaming parameter to 'on/off', or 'parallel'. The parameter value 'parallel' means the streaming will be applied via a parallel apply worker, if available. The parameter value 'on' means the streaming transaction will be spilled to disk. The default value is 'off' (same as current behaviour). In addition, the patch extends the logical replication STREAM_ABORT message so that abort_lsn and abort_time can also be sent which can be used to update the replication origin in parallel apply worker when the streaming transaction is aborted. Because this message extension is needed to support parallel streaming, parallel streaming is not supported for publications on servers < PG16. Author: Hou Zhijie, Wang wei, Amit Kapila with design inputs from Sawada Masahiko Reviewed-by: Sawada Masahiko, Peter Smith, Dilip Kumar, Shi yu, Kuroda Hayato, Shveta Mallik Discussion: https://postgr.es/m/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
2023-01-08 20:30:39 -05:00
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
-- fail - publication already exists
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub WITH (refresh = false);
-- fail - publication used more than once
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub1 WITH (refresh = false);
-- ok - add two publications into subscription
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
-- fail - publications already exist
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
-- fail - publication used more than once
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub1 WITH (refresh = false);
-- fail - all publications are deleted
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub, testpub1, testpub2 WITH (refresh = false);
-- fail - publication does not exist in subscription
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub3 WITH (refresh = false);
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
DROP SUBSCRIPTION regress_testsub;
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION mypub
WITH (connect = false, create_slot = false, copy_data = false);
ALTER SUBSCRIPTION regress_testsub ENABLE;
Allow multiple xacts during table sync in logical replication. For the initial table data synchronization in logical replication, we use a single transaction to copy the entire table and then synchronize the position in the stream with the main apply worker. There are multiple downsides of this approach: (a) We have to perform the entire copy operation again if there is any error (network breakdown, error in the database operation, etc.) while we synchronize the WAL position between tablesync worker and apply worker; this will be onerous especially for large copies, (b) Using a single transaction in the synchronization-phase (where we can receive WAL from multiple transactions) will have the risk of exceeding the CID limit, (c) The slot will hold the WAL till the entire sync is complete because we never commit till the end. This patch solves all the above downsides by allowing multiple transactions during the tablesync phase. The initial copy is done in a single transaction and after that, we commit each transaction as we receive. To allow recovery after any error or crash, we use a permanent slot and origin to track the progress. The slot and origin will be removed once we finish the synchronization of the table. We also remove slot and origin of tablesync workers if the user performs DROP SUBSCRIPTION .. or ALTER SUBSCRIPTION .. REFERESH and some of the table syncs are still not finished. The commands ALTER SUBSCRIPTION ... REFRESH PUBLICATION and ALTER SUBSCRIPTION ... SET PUBLICATION ... with refresh option as true cannot be executed inside a transaction block because they can now drop the slots for which we have no provision to rollback. This will also open up the path for logical replication of 2PC transactions on the subscriber side. Previously, we can't do that because of the requirement of maintaining a single transaction in tablesync workers. Bump catalog version due to change of state in the catalog (pg_subscription_rel). Author: Peter Smith, Amit Kapila, and Takamichi Osumi Reviewed-by: Ajin Cherian, Petr Jelinek, Hou Zhijie and Amit Kapila Discussion: https://postgr.es/m/CAA4eK1KHJxaZS-fod-0fey=0tq3=Gkn4ho=8N4-5HWiCfu0H1A@mail.gmail.com
2021-02-11 21:11:51 -05:00
-- fail - ALTER SUBSCRIPTION with refresh is not allowed in a transaction
-- block or function
BEGIN;
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION mypub WITH (refresh = true);
END;
BEGIN;
ALTER SUBSCRIPTION regress_testsub REFRESH PUBLICATION;
END;
CREATE FUNCTION func() RETURNS VOID AS
$$ ALTER SUBSCRIPTION regress_testsub SET PUBLICATION mypub WITH (refresh = true) $$ LANGUAGE SQL;
SELECT func();
ALTER SUBSCRIPTION regress_testsub DISABLE;
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
DROP FUNCTION func;
Add support for prepared transactions to built-in logical replication. To add support for streaming transactions at prepare time into the built-in logical replication, we need to do the following things: * Modify the output plugin (pgoutput) to implement the new two-phase API callbacks, by leveraging the extended replication protocol. * Modify the replication apply worker, to properly handle two-phase transactions by replaying them on prepare. * Add a new SUBSCRIPTION option "two_phase" to allow users to enable two-phase transactions. We enable the two_phase once the initial data sync is over. We however must explicitly disable replication of two-phase transactions during replication slot creation, even if the plugin supports it. We don't need to replicate the changes accumulated during this phase, and moreover, we don't have a replication connection open so we don't know where to send the data anyway. The streaming option is not allowed with this new two_phase option. This can be done as a separate patch. We don't allow to toggle two_phase option of a subscription because it can lead to an inconsistent replica. For the same reason, we don't allow to refresh the publication once the two_phase is enabled for a subscription unless copy_data option is false. Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow Tested-By: Haiying Tang Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-13 22:03:50 -04:00
-- fail - two_phase must be boolean
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, two_phase = foo);
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, two_phase = true);
\dRs+
--fail - alter of two_phase option not supported.
ALTER SUBSCRIPTION regress_testsub SET (two_phase = false);
-- but can alter streaming when two_phase enabled
Add support for prepared transactions to built-in logical replication. To add support for streaming transactions at prepare time into the built-in logical replication, we need to do the following things: * Modify the output plugin (pgoutput) to implement the new two-phase API callbacks, by leveraging the extended replication protocol. * Modify the replication apply worker, to properly handle two-phase transactions by replaying them on prepare. * Add a new SUBSCRIPTION option "two_phase" to allow users to enable two-phase transactions. We enable the two_phase once the initial data sync is over. We however must explicitly disable replication of two-phase transactions during replication slot creation, even if the plugin supports it. We don't need to replicate the changes accumulated during this phase, and moreover, we don't have a replication connection open so we don't know where to send the data anyway. The streaming option is not allowed with this new two_phase option. This can be done as a separate patch. We don't allow to toggle two_phase option of a subscription because it can lead to an inconsistent replica. For the same reason, we don't allow to refresh the publication once the two_phase is enabled for a subscription unless copy_data option is false. Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow Tested-By: Haiying Tang Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-13 22:03:50 -04:00
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
Add support for prepared transactions to built-in logical replication. To add support for streaming transactions at prepare time into the built-in logical replication, we need to do the following things: * Modify the output plugin (pgoutput) to implement the new two-phase API callbacks, by leveraging the extended replication protocol. * Modify the replication apply worker, to properly handle two-phase transactions by replaying them on prepare. * Add a new SUBSCRIPTION option "two_phase" to allow users to enable two-phase transactions. We enable the two_phase once the initial data sync is over. We however must explicitly disable replication of two-phase transactions during replication slot creation, even if the plugin supports it. We don't need to replicate the changes accumulated during this phase, and moreover, we don't have a replication connection open so we don't know where to send the data anyway. The streaming option is not allowed with this new two_phase option. This can be done as a separate patch. We don't allow to toggle two_phase option of a subscription because it can lead to an inconsistent replica. For the same reason, we don't allow to refresh the publication once the two_phase is enabled for a subscription unless copy_data option is false. Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow Tested-By: Haiying Tang Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-13 22:03:50 -04:00
DROP SUBSCRIPTION regress_testsub;
-- two_phase and streaming are compatible.
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = true, two_phase = true);
Add support for prepared transactions to built-in logical replication. To add support for streaming transactions at prepare time into the built-in logical replication, we need to do the following things: * Modify the output plugin (pgoutput) to implement the new two-phase API callbacks, by leveraging the extended replication protocol. * Modify the replication apply worker, to properly handle two-phase transactions by replaying them on prepare. * Add a new SUBSCRIPTION option "two_phase" to allow users to enable two-phase transactions. We enable the two_phase once the initial data sync is over. We however must explicitly disable replication of two-phase transactions during replication slot creation, even if the plugin supports it. We don't need to replicate the changes accumulated during this phase, and moreover, we don't have a replication connection open so we don't know where to send the data anyway. The streaming option is not allowed with this new two_phase option. This can be done as a separate patch. We don't allow to toggle two_phase option of a subscription because it can lead to an inconsistent replica. For the same reason, we don't allow to refresh the publication once the two_phase is enabled for a subscription unless copy_data option is false. Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow Tested-By: Haiying Tang Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-13 22:03:50 -04:00
\dRs+
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
Add support for prepared transactions to built-in logical replication. To add support for streaming transactions at prepare time into the built-in logical replication, we need to do the following things: * Modify the output plugin (pgoutput) to implement the new two-phase API callbacks, by leveraging the extended replication protocol. * Modify the replication apply worker, to properly handle two-phase transactions by replaying them on prepare. * Add a new SUBSCRIPTION option "two_phase" to allow users to enable two-phase transactions. We enable the two_phase once the initial data sync is over. We however must explicitly disable replication of two-phase transactions during replication slot creation, even if the plugin supports it. We don't need to replicate the changes accumulated during this phase, and moreover, we don't have a replication connection open so we don't know where to send the data anyway. The streaming option is not allowed with this new two_phase option. This can be done as a separate patch. We don't allow to toggle two_phase option of a subscription because it can lead to an inconsistent replica. For the same reason, we don't allow to refresh the publication once the two_phase is enabled for a subscription unless copy_data option is false. Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow Tested-By: Haiying Tang Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-13 22:03:50 -04:00
-- fail - disable_on_error must be boolean
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, disable_on_error = foo);
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, disable_on_error = false);
\dRs+
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_subscription_user;
DROP ROLE regress_subscription_user2;
DROP ROLE regress_subscription_user_dummy;