upstream/kubernetes
1
0
Fork 0
mirror of https://github.com/kubernetes/kubernetes.git synced 2026-05-25 02:33:36 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
kubernetes/test/integration/dualstack/dualstack_test.go

1818 lines
70 KiB
Go
Raw Normal View History

dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
/*
Copyright 2020 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package dualstack
import (
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
"encoding/json"
Fix Go vet errors for master golang Co-authored-by: Rajalakshmi-Girish <rajalakshmi.girish1@ibm.com> Co-authored-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
2024-09-12 08:45:22 -04:00
"errors"
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
"fmt"
"reflect"
"strings"
"testing"
"time"
Use canonical json-patch v4 import The canonical import for json-patch v4 is gopkg.in/evanphx/json-patch.v4 (see https://github.com/evanphx/json-patch/blob/master/README.md#get-it for reference). Using the v4-specific path should also reduce the risk of unwanted v5 upgrade attempts, because they won't be offered as automated upgrades by dependency upgrade management tools, and they won't happen through indirect dependencies (see https://github.com/kubernetes/kubernetes/pull/120327 for context). Signed-off-by: Stephen Kitt <skitt@redhat.com>
2024-02-16 07:57:24 -05:00
jsonpatch "gopkg.in/evanphx/json-patch.v4"
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
v1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
"k8s.io/apimachinery/pkg/types"
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
"k8s.io/apimachinery/pkg/util/intstr"
"k8s.io/apimachinery/pkg/util/wait"
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
"k8s.io/client-go/kubernetes"
kubeapiservertesting "k8s.io/kubernetes/cmd/kube-apiserver/app/testing"
"k8s.io/kubernetes/pkg/features"
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
"k8s.io/kubernetes/test/integration/framework"
Pass Context to StartTestServer Signed-off-by: Kante Yin <kerthcet@gmail.com>
2023-03-31 00:19:26 -04:00
"k8s.io/kubernetes/test/utils/ktesting"
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
netutils "k8s.io/utils/net"
)
// TestCreateServiceSingleStackIPv4 test the Service dualstackness in an IPv4 SingleStack cluster
func TestCreateServiceSingleStackIPv4(t *testing.T) {
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
for _, enableMultiServiceCIDR := range []bool{false, true} {
for _, disableAllocatorDualWrite := range []bool{false, true} {
Fix DisableAllocatorDualWrite dependency on MultiCIDRServiceAllocator
2025-10-14 20:22:19 -04:00
if !enableMultiServiceCIDR && disableAllocatorDualWrite {
// Inavlid configuration: DisableAllocatorDualWrite depends on MultiServiceCIDR
continue
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
t.Run(fmt.Sprintf("MultiServiceCIDR=%v DisableAllocatorDualWrite=%v", enableMultiServiceCIDR, disableAllocatorDualWrite), func(t *testing.T) {
// Create an IPv4 single stack control-plane
tCtx := ktesting.Init(t)
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags := []string{
"--service-cluster-ip-range=10.0.0.0/16",
"--advertise-address=10.1.1.1",
"--disable-admission-plugins=ServiceAccount",
fmt.Sprintf("--feature-gates=%s=%v,%s=%v", features.MultiCIDRServiceAllocator, enableMultiServiceCIDR, features.DisableAllocatorDualWrite, disableAllocatorDualWrite),
}
lock DisableAllocatorDualWrite
2025-08-29 07:43:56 -04:00
if !enableMultiServiceCIDR || !disableAllocatorDualWrite {
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags = append(flags, "--emulated-version=1.33")
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags,
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions)
defer s.TearDownFn()
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
// Wait until the default "kubernetes" service is created.
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
var testcases = []struct {
name string
serviceType v1.ServiceType
clusterIPs []string
ipFamilies []v1.IPFamily
ipFamilyPolicy v1.IPFamilyPolicy
expectedIPFamilies []v1.IPFamily
expectError bool
}{
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: nil,
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: false,
},
{
name: "Type ClusterIP - Client Allocated IP - Default IP Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{"10.0.0.16"},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: nil,
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: nil,
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: true,
},
}
for i, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("svc-test-%d", i), // use different services for each test
},
Spec: v1.ServiceSpec{
Type: tc.serviceType,
ClusterIPs: tc.clusterIPs,
IPFamilies: tc.ipFamilies,
Ports: []v1.ServicePort{
{
Port: 443,
TargetPort: intstr.FromInt32(443),
},
},
},
}
if len(tc.ipFamilyPolicy) > 0 {
svc.Spec.IPFamilyPolicy = &tc.ipFamilyPolicy
}
if len(tc.clusterIPs) > 0 {
svc.Spec.ClusterIP = tc.clusterIPs[0]
}
// create the service
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
if (err != nil) != tc.expectError {
t.Errorf("Test failed expected result: %v received %v ", tc.expectError, err)
}
// if no error was expected validate the service otherwise return
if err != nil {
return
}
// validate the service was created correctly if it was not expected to fail
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
if err != nil {
t.Errorf("Unexpected error to get the service %s %v", svc.Name, err)
}
if err := validateServiceAndClusterIPFamily(svc, tc.expectedIPFamilies); err != nil {
t.Errorf("Unexpected error validating the service %s\n%+v\n%v", svc.Name, svc, err)
}
})
}
})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
}
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
}
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
// TestCreateServiceSingleStackIPv6 test the Service dualstackness in an IPv6 only DualStack cluster
func TestCreateServiceSingleStackIPv6(t *testing.T) {
for _, enableMultiServiceCIDR := range []bool{false, true} {
for _, disableAllocatorDualWrite := range []bool{false, true} {
Fix DisableAllocatorDualWrite dependency on MultiCIDRServiceAllocator
2025-10-14 20:22:19 -04:00
if !enableMultiServiceCIDR && disableAllocatorDualWrite {
// Inavlid configuration: DisableAllocatorDualWrite depends on MultiServiceCIDR
continue
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
t.Run(fmt.Sprintf("MultiServiceCIDR=%v DisableAllocatorDualWrite=%v", enableMultiServiceCIDR, disableAllocatorDualWrite), func(t *testing.T) {
// Create an IPv6 only control-plane
tCtx := ktesting.Init(t)
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags := []string{
"--service-cluster-ip-range=2001:db8:1::/108",
"--advertise-address=2001:db8::10",
"--disable-admission-plugins=ServiceAccount",
fmt.Sprintf("--feature-gates=%s=%v,%s=%v", features.MultiCIDRServiceAllocator, enableMultiServiceCIDR, features.DisableAllocatorDualWrite, disableAllocatorDualWrite),
}
lock DisableAllocatorDualWrite
2025-08-29 07:43:56 -04:00
if !enableMultiServiceCIDR || !disableAllocatorDualWrite {
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags = append(flags, "--emulated-version=1.33")
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags,
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions)
defer s.TearDownFn()
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
// Wait until the default "kubernetes" service is created.
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
var testcases = []struct {
name string
serviceType v1.ServiceType
clusterIPs []string
ipFamilies []v1.IPFamily
expectedIPFamilies []v1.IPFamily
ipFamilyPolicy v1.IPFamilyPolicy
expectError bool
}{
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: true,
},
}
for i, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("svc-test-%d", i), // use different services for each test
},
Spec: v1.ServiceSpec{
Type: tc.serviceType,
ClusterIPs: tc.clusterIPs,
IPFamilies: tc.ipFamilies,
IPFamilyPolicy: &tc.ipFamilyPolicy,
Ports: []v1.ServicePort{
{
Name: fmt.Sprintf("port-test-%d", i),
Port: 443,
TargetPort: intstr.IntOrString{IntVal: 443},
Protocol: "TCP",
},
},
},
}
// create the service
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
if (err != nil) != tc.expectError {
t.Errorf("Test failed expected result: %v received %v ", tc.expectError, err)
}
// if no error was expected validate the service otherwise return
if err != nil {
return
}
// validate the service was created correctly if it was not expected to fail
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
if err != nil {
t.Errorf("Unexpected error to get the service %s %v", svc.Name, err)
}
if err := validateServiceAndClusterIPFamily(svc, tc.expectedIPFamilies); err != nil {
t.Errorf("Unexpected error validating the service %s %v", svc.Name, err)
}
})
}
})
}
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
}
}
// TestCreateServiceDualStackIPv4IPv6 test the Service dualstackness in a IPv4IPv6 DualStack cluster
func TestCreateServiceDualStackIPv4IPv6(t *testing.T) {
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
for _, enableMultiServiceCIDR := range []bool{false, true} {
for _, disableAllocatorDualWrite := range []bool{false, true} {
Fix DisableAllocatorDualWrite dependency on MultiCIDRServiceAllocator
2025-10-14 20:22:19 -04:00
if !enableMultiServiceCIDR && disableAllocatorDualWrite {
// Inavlid configuration: DisableAllocatorDualWrite depends on MultiServiceCIDR
continue
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
t.Run(fmt.Sprintf("MultiServiceCIDR=%v DisableAllocatorDualWrite=%v", enableMultiServiceCIDR, disableAllocatorDualWrite), func(t *testing.T) {
// Create an IPv4IPv6 dual stack control-plane
tCtx := ktesting.Init(t)
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags := []string{
"--service-cluster-ip-range=10.0.0.0/16,2001:db8:1::/108",
"--advertise-address=10.0.0.1",
"--disable-admission-plugins=ServiceAccount",
fmt.Sprintf("--feature-gates=%s=%v,%s=%v", features.MultiCIDRServiceAllocator, enableMultiServiceCIDR, features.DisableAllocatorDualWrite, disableAllocatorDualWrite),
}
lock DisableAllocatorDualWrite
2025-08-29 07:43:56 -04:00
if !enableMultiServiceCIDR || !disableAllocatorDualWrite {
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags = append(flags, "--emulated-version=1.33")
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags,
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions)
defer s.TearDownFn()
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
// Wait until the default "kubernetes" service is created.
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
var testcases = []struct {
name string
serviceType v1.ServiceType
clusterIPs []string
ipFamilies []v1.IPFamily
expectedIPFamilies []v1.IPFamily
ipFamilyPolicy v1.IPFamilyPolicy
expectError bool
}{
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: false,
},
{
name: "Type ClusterIP - Client Allocated IP - IPv4 Family",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{"10.0.0.16"},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectError: false,
},
{
name: "Type ClusterIP - Client Allocated IP - IPv6 Family",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{"2001:db8:1::16"},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol},
expectError: false,
},
{
name: "Type ClusterIP - Client Allocated IP - IPv4 IPv6 Family ",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{"10.0.0.17", "2001:db8:1::17"},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Client Allocated IP - IPv4 IPv6 Family ",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{"10.0.0.17", "2001:db8:1::17"},
ipFamilies: nil,
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectError: false,
},
{
name: "Type ClusterIP - Client Allocated IP - IPv4 IPv6 Family ",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{"10.0.0.18", "2001:db8:1::18"},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: true,
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
},
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
}
for i, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("svc-test-%d", i), // use different services for each test
},
Spec: v1.ServiceSpec{
Type: tc.serviceType,
ClusterIPs: tc.clusterIPs,
IPFamilies: tc.ipFamilies,
Ports: []v1.ServicePort{
{
Port: 443,
TargetPort: intstr.FromInt32(443),
},
},
},
}
if len(tc.ipFamilyPolicy) > 0 {
svc.Spec.IPFamilyPolicy = &tc.ipFamilyPolicy
}
if len(tc.clusterIPs) > 0 {
svc.Spec.ClusterIP = tc.clusterIPs[0]
}
// create a service
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
if (err != nil) != tc.expectError {
t.Errorf("Test failed expected result: %v received %v ", tc.expectError, err)
}
// if no error was expected validate the service otherwise return
if err != nil {
return
}
// validate the service was created correctly if it was not expected to fail
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
if err != nil {
t.Errorf("Unexpected error to get the service %s %v", svc.Name, err)
}
if err := validateServiceAndClusterIPFamily(svc, tc.expectedIPFamilies); err != nil {
t.Errorf("Unexpected error validating the service %s %v", svc.Name, err)
}
})
}
})
}
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
}
}
// TestCreateServiceDualStackIPv6IPv4 test the Service dualstackness in a IPv6IPv4 DualStack cluster
func TestCreateServiceDualStackIPv6IPv4(t *testing.T) {
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
for _, enableMultiServiceCIDR := range []bool{false, true} {
for _, disableAllocatorDualWrite := range []bool{false, true} {
Fix DisableAllocatorDualWrite dependency on MultiCIDRServiceAllocator
2025-10-14 20:22:19 -04:00
if !enableMultiServiceCIDR && disableAllocatorDualWrite {
// Inavlid configuration: DisableAllocatorDualWrite depends on MultiServiceCIDR
continue
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
t.Run(fmt.Sprintf("MultiServiceCIDR=%v DisableAllocatorDualWrite=%v", enableMultiServiceCIDR, disableAllocatorDualWrite), func(t *testing.T) {
// Create an IPv6IPv4 dual stack control-plane
tCtx := ktesting.Init(t)
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags := []string{
"--service-cluster-ip-range=2001:db8:1::/108,10.0.0.0/16",
"--advertise-address=2001:db8::10",
"--disable-admission-plugins=ServiceAccount",
fmt.Sprintf("--feature-gates=%s=%v,%s=%v", features.MultiCIDRServiceAllocator, enableMultiServiceCIDR, features.DisableAllocatorDualWrite, disableAllocatorDualWrite),
}
lock DisableAllocatorDualWrite
2025-08-29 07:43:56 -04:00
if !enableMultiServiceCIDR || !disableAllocatorDualWrite {
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags = append(flags, "--emulated-version=1.33")
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
integration test for compatibility version set emulation version for the test that require to disable the locked feature gate.
2025-05-02 04:52:37 -04:00
flags,
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions)
defer s.TearDownFn()
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
// Wait until the default "kubernetes" service is created.
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
// verify client is working
if err := wait.PollImmediate(5*time.Second, 2*time.Minute, func() (bool, error) {
_, err := client.CoreV1().Endpoints("default").Get(tCtx, "kubernetes", metav1.GetOptions{})
if err != nil {
t.Logf("error fetching endpoints: %v", err)
return false, nil
}
return true, nil
}); err != nil {
t.Errorf("server without enabled endpoints failed to register: %v", err)
}
var testcases = []struct {
name string
serviceType v1.ServiceType
clusterIPs []string
ipFamilies []v1.IPFamily
expectedIPFamilies []v1.IPFamily
ipFamilyPolicy v1.IPFamilyPolicy
expectError bool
}{
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - Default IP Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: nil,
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv4 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
},
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
{
name: "Type ClusterIP - Server Allocated IP - IPv4 IPv6 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Single Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicySingleStack,
expectError: true,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Prefer Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyPreferDualStack,
expectError: false,
},
{
name: "Type ClusterIP - Server Allocated IP - IPv6 IPv4 Family - Policy Required Dual Stack",
serviceType: v1.ServiceTypeClusterIP,
clusterIPs: []string{},
ipFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
expectedIPFamilies: []v1.IPFamily{v1.IPv6Protocol, v1.IPv4Protocol},
ipFamilyPolicy: v1.IPFamilyPolicyRequireDualStack,
expectError: false,
},
}
for i, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("svc-test-%d", i), // use different services for each test
},
Spec: v1.ServiceSpec{
Type: tc.serviceType,
ClusterIPs: tc.clusterIPs,
IPFamilies: tc.ipFamilies,
IPFamilyPolicy: &tc.ipFamilyPolicy,
Ports: []v1.ServicePort{
{
Port: 443,
TargetPort: intstr.FromInt32(443),
},
},
},
}
// create a service
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
if (err != nil) != tc.expectError {
t.Errorf("Test failed expected result: %v received %v ", tc.expectError, err)
}
// if no error was expected validate the service otherwise return
if err != nil {
return
}
// validate the service was created correctly if it was not expected to fail
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
if err != nil {
t.Errorf("Unexpected error to get the service %s %v", svc.Name, err)
}
if err := validateServiceAndClusterIPFamily(svc, tc.expectedIPFamilies); err != nil {
t.Errorf("Unexpected error validating the service %s %v", svc.Name, err)
}
})
}
})
}
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
}
}
// TestUpgradeDowngrade tests upgrading and downgrading a service from/to dual-stack
func TestUpgradeDowngrade(t *testing.T) {
// Create an IPv4IPv6 dual stack control-plane
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
tCtx := ktesting.Init(t)
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
[]string{
cluster ip allocator should check first on the legacy allocators Kubernetes clusters allow to define an IPv6 range of /108 for IPv6 despite the old allocators will only use the first /112 of that range. The new allocators does not have this limitation, so they can allocate IPs on the whole space, the problem happens on upgrades from clusters that were already using this size, since the new allocators by default will try to allocate addresses that works for both new and old allocatos to allow safe upgrades. The new allocators, when configured to keep compatibility with the old allocators, must try first to allocate an IP that is compatible with the old allocators and only fall back to the new behavior if it is not possible.
2025-01-24 05:37:43 -05:00
"--service-cluster-ip-range=10.0.0.0/16,2001:db8:1::/108",
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
"--advertise-address=10.0.0.1",
"--disable-admission-plugins=ServiceAccount",
},
etcdOptions)
defer s.TearDownFn()
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
// Wait until the default "kubernetes" service is created.
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
upgradeServiceName := "svc-upgrade"
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: upgradeServiceName,
},
Spec: v1.ServiceSpec{
Type: v1.ServiceTypeClusterIP,
Ports: []v1.ServicePort{
{
Port: 443,
test: replace intstr.FromInt with intstr.FromInt32 This touches cases where FromInt() is used on numeric constants, or values which are already int32s, or int variables which are defined close by and can be changed to int32s with little impact. Signed-off-by: Stephen Kitt <skitt@redhat.com>
2023-03-14 11:17:48 -04:00
TargetPort: intstr.FromInt32(443),
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
},
},
},
}
// create a service
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error while creating service:%v", err)
}
// validate the service was created correctly if it was not expected to fail
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
if err := validateServiceAndClusterIPFamily(svc, []v1.IPFamily{v1.IPv4Protocol} /* default cluster config */); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svc.Name, err)
}
// upgrade it
requireDualStack := v1.IPFamilyPolicyRequireDualStack
svc.Spec.IPFamilyPolicy = &requireDualStack
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
upgraded, err := client.CoreV1().Services(metav1.NamespaceDefault).Update(tCtx, svc, metav1.UpdateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error upgrading service to dual stack. %v", err)
}
if err := validateServiceAndClusterIPFamily(upgraded, []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol} /* +1 family */); err != nil {
t.Fatalf("Unexpected error validating the service(after upgrade) %s %v", svc.Name, err)
}
// downgrade it
singleStack := v1.IPFamilyPolicySingleStack
upgraded.Spec.IPFamilyPolicy = &singleStack
upgraded.Spec.ClusterIPs = upgraded.Spec.ClusterIPs[0:1]
upgraded.Spec.IPFamilies = upgraded.Spec.IPFamilies[0:1]
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
downgraded, err := client.CoreV1().Services(metav1.NamespaceDefault).Update(tCtx, upgraded, metav1.UpdateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error downgrading service to single stack. %v", err)
}
if err := validateServiceAndClusterIPFamily(downgraded, []v1.IPFamily{v1.IPv4Protocol} /* -1 family */); err != nil {
t.Fatalf("unexpected error validating the service(after downgrade) %s %v", svc.Name, err)
}
// run test again this time without removing secondary IPFamily or ClusterIP
downgraded.Spec.IPFamilyPolicy = &requireDualStack
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
upgradedAgain, err := client.CoreV1().Services(metav1.NamespaceDefault).Update(tCtx, downgraded, metav1.UpdateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error upgrading service to dual stack. %v", err)
}
if err := validateServiceAndClusterIPFamily(upgradedAgain, []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol} /* +1 family */); err != nil {
t.Fatalf("Unexpected error validating the service(after upgrade) %s %v", svc.Name, err)
}
upgradedAgain.Spec.IPFamilyPolicy = &singleStack
// api-server automatically removes the secondary ClusterIP and IPFamily
// when a servie is downgraded.
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
downgradedAgain, err := client.CoreV1().Services(metav1.NamespaceDefault).Update(tCtx, upgradedAgain, metav1.UpdateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error downgrading service to single stack. %v", err)
}
if err := validateServiceAndClusterIPFamily(downgradedAgain, []v1.IPFamily{v1.IPv4Protocol} /* -1 family */); err != nil {
t.Fatalf("unexpected error validating the service(after downgrade) %s %v", svc.Name, err)
}
}
// TestConvertToFromExternalName tests the compatibility with old clients that
// may not clear ClusterIPs
func TestConvertToFromExternalName(t *testing.T) {
// Create an IPv4IPv6 dual stack control-plane
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
tCtx := ktesting.Init(t)
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
[]string{
cluster ip allocator should check first on the legacy allocators Kubernetes clusters allow to define an IPv6 range of /108 for IPv6 despite the old allocators will only use the first /112 of that range. The new allocators does not have this limitation, so they can allocate IPs on the whole space, the problem happens on upgrades from clusters that were already using this size, since the new allocators by default will try to allocate addresses that works for both new and old allocatos to allow safe upgrades. The new allocators, when configured to keep compatibility with the old allocators, must try first to allocate an IP that is compatible with the old allocators and only fall back to the new behavior if it is not possible.
2025-01-24 05:37:43 -05:00
"--service-cluster-ip-range=10.0.0.0/16,2001:db8:1::/108",
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
"--advertise-address=10.0.0.1",
"--disable-admission-plugins=ServiceAccount",
},
etcdOptions)
defer s.TearDownFn()
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
// Wait until the default "kubernetes" service is created.
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
serviceName := "svc-ext-name"
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: serviceName,
},
Spec: v1.ServiceSpec{
Type: v1.ServiceTypeClusterIP,
Ports: []v1.ServicePort{
{
Port: 443,
test: replace intstr.FromInt with intstr.FromInt32 This touches cases where FromInt() is used on numeric constants, or values which are already int32s, or int variables which are defined close by and can be changed to int32s with little impact. Signed-off-by: Stephen Kitt <skitt@redhat.com>
2023-03-14 11:17:48 -04:00
TargetPort: intstr.FromInt32(443),
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
},
},
},
}
// create a service
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error while creating service:%v", err)
}
// validate the service was created correctly if it was not expected to fail
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
if err := validateServiceAndClusterIPFamily(svc, []v1.IPFamily{v1.IPv4Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svc.Name, err)
}
// convert to ExternalName
svc.Spec.Type = v1.ServiceTypeExternalName
svc.Spec.ClusterIP = "" // not clearing ClusterIPs
svc.Spec.ExternalName = "something.somewhere"
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
externalNameSvc, err := client.CoreV1().Services(metav1.NamespaceDefault).Update(tCtx, svc, metav1.UpdateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error converting service to external name. %v", err)
}
if len(externalNameSvc.Spec.ClusterIPs) > 0 || len(externalNameSvc.Spec.ClusterIP) > 0 || len(externalNameSvc.Spec.IPFamilies) > 0 {
t.Fatalf("unpexpected externalname service with ClusterIPs %v or ClusterIP %v or IPFamilies %v", externalNameSvc.Spec.ClusterIPs, externalNameSvc.Spec.ClusterIP, externalNameSvc.Spec.IPFamilies)
}
// convert to a ClusterIP service
externalNameSvc.Spec.Type = v1.ServiceTypeClusterIP
externalNameSvc.Spec.ExternalName = ""
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
clusterIPSvc, err := client.CoreV1().Services(metav1.NamespaceDefault).Update(tCtx, externalNameSvc, metav1.UpdateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error converting service to ClusterIP. %v", err)
}
if err := validateServiceAndClusterIPFamily(clusterIPSvc, []v1.IPFamily{v1.IPv4Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svc.Name, err)
}
}
// TestPreferDualStack preferDualstack on create and update
func TestPreferDualStack(t *testing.T) {
// Create an IPv4IPv6 dual stack control-plane
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
tCtx := ktesting.Init(t)
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
[]string{
cluster ip allocator should check first on the legacy allocators Kubernetes clusters allow to define an IPv6 range of /108 for IPv6 despite the old allocators will only use the first /112 of that range. The new allocators does not have this limitation, so they can allocate IPs on the whole space, the problem happens on upgrades from clusters that were already using this size, since the new allocators by default will try to allocate addresses that works for both new and old allocatos to allow safe upgrades. The new allocators, when configured to keep compatibility with the old allocators, must try first to allocate an IP that is compatible with the old allocators and only fall back to the new behavior if it is not possible.
2025-01-24 05:37:43 -05:00
"--service-cluster-ip-range=10.0.0.0/16,2001:db8:1::/108",
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
"--advertise-address=10.0.0.1",
"--disable-admission-plugins=ServiceAccount",
},
etcdOptions)
defer s.TearDownFn()
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
// Wait until the default "kubernetes" service is created.
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
preferDualStack := v1.IPFamilyPolicyPreferDualStack
serviceName := "svc-upgrade"
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: serviceName,
},
Spec: v1.ServiceSpec{
Type: v1.ServiceTypeClusterIP,
IPFamilyPolicy: &preferDualStack,
Ports: []v1.ServicePort{
{
Port: 443,
test: replace intstr.FromInt with intstr.FromInt32 This touches cases where FromInt() is used on numeric constants, or values which are already int32s, or int variables which are defined close by and can be changed to int32s with little impact. Signed-off-by: Stephen Kitt <skitt@redhat.com>
2023-03-14 11:17:48 -04:00
TargetPort: intstr.FromInt32(443),
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
},
},
},
}
// create a service
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error while creating service:%v", err)
}
// validate the service was created correctly if it was not expected to fail
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
if err := validateServiceAndClusterIPFamily(svc, []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svc.Name, err)
}
// update it
svc.Spec.Selector = map[string]string{"foo": "bar"}
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
upgraded, err := client.CoreV1().Services(metav1.NamespaceDefault).Update(tCtx, svc, metav1.UpdateOptions{})
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
if err != nil {
t.Fatalf("unexpected error upgrading service to dual stack. %v", err)
}
if err := validateServiceAndClusterIPFamily(upgraded, []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service(after upgrade) %s %v", svc.Name, err)
}
}
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
type labelsForMergePatch struct {
Labels map[string]string `json:"lables,omitempty"`
}
// tests an update service while dualstack flag is off
func TestServiceUpdate(t *testing.T) {
// Create an IPv4 single stack control-plane
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
tCtx := ktesting.Init(t)
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
[]string{
"--service-cluster-ip-range=10.0.0.0/16",
"--advertise-address=10.0.0.1",
"--disable-admission-plugins=ServiceAccount",
},
etcdOptions)
defer s.TearDownFn()
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
// Wait until the default "kubernetes" service is created.
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
serviceName := "test-service"
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: serviceName,
},
Spec: v1.ServiceSpec{
Type: v1.ServiceTypeClusterIP,
Ports: []v1.ServicePort{
{
Port: 443,
test: replace intstr.FromInt with intstr.FromInt32 This touches cases where FromInt() is used on numeric constants, or values which are already int32s, or int variables which are defined close by and can be changed to int32s with little impact. Signed-off-by: Stephen Kitt <skitt@redhat.com>
2023-03-14 11:17:48 -04:00
TargetPort: intstr.FromInt32(443),
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
},
},
},
}
// create the service
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
// if no error was expected validate the service otherwise return
if err != nil {
t.Errorf("unexpected error creating service:%v", err)
return
}
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
if err != nil {
t.Errorf("Unexpected error to get the service %s %v", svc.Name, err)
}
// update using put
svc.Labels = map[string]string{"x": "y"}
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Update(tCtx, svc, metav1.UpdateOptions{})
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
if err != nil {
t.Errorf("Unexpected error updating the service %s %v", svc.Name, err)
}
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
// update using StrategicMergePatchType
labels := labelsForMergePatch{
Labels: map[string]string{"foo": "bar"},
}
patchBytes, err := json.Marshal(&labels)
if err != nil {
t.Fatalf("failed to json.Marshal labels: %v", err)
}
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Patch(tCtx, svc.Name, types.StrategicMergePatchType, patchBytes, metav1.PatchOptions{})
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
if err != nil {
t.Fatalf("unexpected error patching service using strategic merge patch. %v", err)
}
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
current, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
// update using json patch
toUpdate := current.DeepCopy()
currentJSON, err := json.Marshal(current)
if err != nil {
t.Fatalf("unexpected error marshal current service. %v", err)
}
toUpdate.Labels = map[string]string{"alpha": "bravo"}
toUpdateJSON, err := json.Marshal(toUpdate)
if err != nil {
t.Fatalf("unexpected error marshal toupdate service. %v", err)
}
patchBytes, err = jsonpatch.CreateMergePatch(currentJSON, toUpdateJSON)
if err != nil {
t.Fatalf("unexpected error creating json patch. %v", err)
}
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Patch(tCtx, svc.Name, types.MergePatchType, patchBytes, metav1.PatchOptions{})
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
if err != nil {
t.Fatalf("unexpected error patching service using merge patch. %v", err)
}
// validate the service was created correctly if it was not expected to fail
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
add tests that update services while gate is off
2020-10-29 14:03:30 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
}
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
// validateServiceAndClusterIPFamily checks that the service has the expected IPFamilies
func validateServiceAndClusterIPFamily(svc *v1.Service, expectedIPFamilies []v1.IPFamily) error {
// create a slice for the errors
var errstrings []string
if svc.Spec.IPFamilies == nil {
return fmt.Errorf("service ip family nil for service %s/%s", svc.Namespace, svc.Name)
}
if !reflect.DeepEqual(svc.Spec.IPFamilies, expectedIPFamilies) {
return fmt.Errorf("ip families mismatch for service: %s/%s, expected: %s, actual: %s", svc.Namespace, svc.Name, expectedIPFamilies, svc.Spec.IPFamilies)
}
if len(svc.Spec.ClusterIPs) == 0 {
return fmt.Errorf("svc %s is invalid it does not have ClusterIP", svc.Name)
}
// not headless
if len(svc.Spec.ClusterIPs) > 0 && svc.Spec.ClusterIPs[0] != v1.ClusterIPNone {
if len(svc.Spec.ClusterIPs) != len(svc.Spec.IPFamilies) {
return fmt.Errorf("svc %v is invalid len(ClusterIPs:%v) != len(IPFamilies:%v)", svc.Name, svc.Spec.ClusterIPs, svc.Spec.IPFamilies)
}
}
for j, ip := range svc.Spec.ClusterIPs {
// we should never be here
if ip == v1.ClusterIPNone && len(svc.Spec.ClusterIPs) > 1 {
errstrings = append(errstrings, fmt.Sprintf("Error validating Service: %s, None is used with +1 clusterIPs (%v)", svc.Name, svc.Spec.ClusterIPs))
}
if ip == v1.ClusterIPNone {
break // the service is headless. the rest of family check is pointless
}
// the clusterIP assigned should have the same IPFamily requested
if netutils.IsIPv6String(ip) != (expectedIPFamilies[j] == v1.IPv6Protocol) {
errstrings = append(errstrings, fmt.Sprintf("got unexpected service ip %s, should belong to %s ip family", ip, expectedIPFamilies[j]))
}
}
if len(errstrings) > 0 {
errstrings = append(errstrings, fmt.Sprintf("Error validating Service: %s, ClusterIPs: %v Expected IPFamilies %v", svc.Name, svc.Spec.ClusterIPs, expectedIPFamilies))
Fix Go vet errors for master golang Co-authored-by: Rajalakshmi-Girish <rajalakshmi.girish1@ibm.com> Co-authored-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
2024-09-12 08:45:22 -04:00
return errors.New(strings.Join(errstrings, "\n"))
dual stack services (#91824) * api: structure change * api: defaulting, conversion, and validation * [FIX] validation: auto remove second ip/family when service changes to SingleStack * [FIX] api: defaulting, conversion, and validation * api-server: clusterIPs alloc, printers, storage and strategy * [FIX] clusterIPs default on read * alloc: auto remove second ip/family when service changes to SingleStack * api-server: repair loop handling for clusterIPs * api-server: force kubernetes default service into single stack * api-server: tie dualstack feature flag with endpoint feature flag * controller-manager: feature flag, endpoint, and endpointSlice controllers handling multi family service * [FIX] controller-manager: feature flag, endpoint, and endpointSlicecontrollers handling multi family service * kube-proxy: feature-flag, utils, proxier, and meta proxier * [FIX] kubeproxy: call both proxier at the same time * kubenet: remove forced pod IP sorting * kubectl: modify describe to include ClusterIPs, IPFamilies, and IPFamilyPolicy * e2e: fix tests that depends on IPFamily field AND add dual stack tests * e2e: fix expected error message for ClusterIP immutability * add integration tests for dualstack the third phase of dual stack is a very complex change in the API, basically it introduces Dual Stack services. Main changes are: - It pluralizes the Service IPFamily field to IPFamilies, and removes the singular field. - It introduces a new field IPFamilyPolicyType that can take 3 values to express the "dual-stack(mad)ness" of the cluster: SingleStack, PreferDualStack and RequireDualStack - It pluralizes ClusterIP to ClusterIPs. The goal is to add coverage to the services API operations, taking into account the 6 different modes a cluster can have: - single stack: IP4 or IPv6 (as of today) - dual stack: IPv4 only, IPv6 only, IPv4 - IPv6, IPv6 - IPv4 * [FIX] add integration tests for dualstack * generated data * generated files Co-authored-by: Antonio Ojea <aojea@redhat.com>
2020-10-26 16:15:59 -04:00
}
return nil
}
integeration tests
2021-06-22 13:40:09 -04:00
func TestUpgradeServicePreferToDualStack(t *testing.T) {
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
sharedEtcd := framework.SharedEtcd()
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
tCtx := ktesting.Init(t)
Pass Context to StartTestServer Signed-off-by: Kante Yin <kerthcet@gmail.com>
2023-03-31 00:19:26 -04:00
Allow single-to-dual-stack reconfiguration for ServiceCIDR This change modifies the validation logic for ServiceCIDR updates (`ValidateServiceCIDRUpdate`) to specifically permit upgrading a single-stack ServiceCIDR (either IPv4 or IPv6) to a dual-stack configuration. This reconfiguration path is considered safe because it only involves adding a new CIDR range without altering the existing primary CIDR. This ensures that existing Service IP allocations are not disrupted. Other modifications, such as: - Downgrading from dual-stack to single-stack - Reordering CIDRs in a dual-stack configuration - Changing the primary CIDR during a single-to-dual-stack reconfiguration remain disallowed by the validation. These operations carry a higher risk of breaking existing Services or cluster networking configurations. Preventing these updates automatically encourages administrators to perform such changes manually after carefully assessing the potential impact on their specific cluster environment. The validation errors and controller logs provide guidance when such disallowed changes are attempted. Change-Id: I41dc09dfddb05f277925da2262f8114d6accbd1d
2025-04-11 11:37:28 -04:00
// Create an IPv4 only control-plane
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
[]string{
"--service-cluster-ip-range=192.168.0.0/24",
"--advertise-address=10.0.0.1",
"--disable-admission-plugins=ServiceAccount",
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
},
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
sharedEtcd)
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
integeration tests
2021-06-22 13:40:09 -04:00
// Wait until the default "kubernetes" service is created.
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
preferDualStack := v1.IPFamilyPolicyPreferDualStack
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-prefer-dual",
},
Spec: v1.ServiceSpec{
Type: v1.ServiceTypeClusterIP,
ClusterIPs: nil,
IPFamilies: nil,
IPFamilyPolicy: &preferDualStack,
Ports: []v1.ServicePort{
{
Name: "svc-port-1",
Port: 443,
TargetPort: intstr.IntOrString{IntVal: 443},
Protocol: "TCP",
},
},
},
}
Allow single-to-dual-stack reconfiguration for ServiceCIDR This change modifies the validation logic for ServiceCIDR updates (`ValidateServiceCIDRUpdate`) to specifically permit upgrading a single-stack ServiceCIDR (either IPv4 or IPv6) to a dual-stack configuration. This reconfiguration path is considered safe because it only involves adding a new CIDR range without altering the existing primary CIDR. This ensures that existing Service IP allocations are not disrupted. Other modifications, such as: - Downgrading from dual-stack to single-stack - Reordering CIDRs in a dual-stack configuration - Changing the primary CIDR during a single-to-dual-stack reconfiguration remain disallowed by the validation. These operations carry a higher risk of breaking existing Services or cluster networking configurations. Preventing these updates automatically encourages administrators to perform such changes manually after carefully assessing the potential impact on their specific cluster environment. The validation errors and controller logs provide guidance when such disallowed changes are attempted. Change-Id: I41dc09dfddb05f277925da2262f8114d6accbd1d
2025-04-11 11:37:28 -04:00
// create a copy of the service so we can test creating it again after reconfiguring the control plane
svcDual := svc.DeepCopy()
svcDual.Name = "svc-dual"
integeration tests
2021-06-22 13:40:09 -04:00
// create the service
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
// validate the service was created correctly if it was not expected to fail
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
if err := validateServiceAndClusterIPFamily(svc, []v1.IPFamily{v1.IPv4Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svc.Name, err)
}
// reconfigure the apiserver to be dual-stack
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
s.TearDownFn()
integeration tests
2021-06-22 13:40:09 -04:00
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
s = kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
[]string{
cluster ip allocator should check first on the legacy allocators Kubernetes clusters allow to define an IPv6 range of /108 for IPv6 despite the old allocators will only use the first /112 of that range. The new allocators does not have this limitation, so they can allocate IPs on the whole space, the problem happens on upgrades from clusters that were already using this size, since the new allocators by default will try to allocate addresses that works for both new and old allocatos to allow safe upgrades. The new allocators, when configured to keep compatibility with the old allocators, must try first to allocate an IP that is compatible with the old allocators and only fall back to the new behavior if it is not possible.
2025-01-24 05:37:43 -05:00
"--service-cluster-ip-range=192.168.0.0/24,2001:db8:1::/108",
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
"--disable-admission-plugins=ServiceAccount",
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
},
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
sharedEtcd)
defer s.TearDownFn()
client, err = kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
integeration tests
2021-06-22 13:40:09 -04:00
// Wait until the default "kubernetes" service is created.
if err = wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
// validate the service was created correctly if it was not expected to fail
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
// service should remain single stack
if err = validateServiceAndClusterIPFamily(svc, []v1.IPFamily{v1.IPv4Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svc.Name, err)
}
Allow single-to-dual-stack reconfiguration for ServiceCIDR This change modifies the validation logic for ServiceCIDR updates (`ValidateServiceCIDRUpdate`) to specifically permit upgrading a single-stack ServiceCIDR (either IPv4 or IPv6) to a dual-stack configuration. This reconfiguration path is considered safe because it only involves adding a new CIDR range without altering the existing primary CIDR. This ensures that existing Service IP allocations are not disrupted. Other modifications, such as: - Downgrading from dual-stack to single-stack - Reordering CIDRs in a dual-stack configuration - Changing the primary CIDR during a single-to-dual-stack reconfiguration remain disallowed by the validation. These operations carry a higher risk of breaking existing Services or cluster networking configurations. Preventing these updates automatically encourages administrators to perform such changes manually after carefully assessing the potential impact on their specific cluster environment. The validation errors and controller logs provide guidance when such disallowed changes are attempted. Change-Id: I41dc09dfddb05f277925da2262f8114d6accbd1d
2025-04-11 11:37:28 -04:00
// validate that new services created with prefer dual are now dual stack
// create the service
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svcDual, metav1.CreateOptions{})
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
// validate the service was created correctly
svcDual, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svcDual.Name, metav1.GetOptions{})
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svcDual.Name, err)
}
// service should be dual stack
if err = validateServiceAndClusterIPFamily(svcDual, []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svcDual.Name, err)
}
integeration tests
2021-06-22 13:40:09 -04:00
}
Allow single-to-dual-stack reconfiguration for ServiceCIDR This change modifies the validation logic for ServiceCIDR updates (`ValidateServiceCIDRUpdate`) to specifically permit upgrading a single-stack ServiceCIDR (either IPv4 or IPv6) to a dual-stack configuration. This reconfiguration path is considered safe because it only involves adding a new CIDR range without altering the existing primary CIDR. This ensures that existing Service IP allocations are not disrupted. Other modifications, such as: - Downgrading from dual-stack to single-stack - Reordering CIDRs in a dual-stack configuration - Changing the primary CIDR during a single-to-dual-stack reconfiguration remain disallowed by the validation. These operations carry a higher risk of breaking existing Services or cluster networking configurations. Preventing these updates automatically encourages administrators to perform such changes manually after carefully assessing the potential impact on their specific cluster environment. The validation errors and controller logs provide guidance when such disallowed changes are attempted. Change-Id: I41dc09dfddb05f277925da2262f8114d6accbd1d
2025-04-11 11:37:28 -04:00
func TestDowngradeServicePreferFromDualStack(t *testing.T) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
tCtx := ktesting.Init(t)
Pass Context to StartTestServer Signed-off-by: Kante Yin <kerthcet@gmail.com>
2023-03-31 00:19:26 -04:00
integeration tests
2021-06-22 13:40:09 -04:00
// Create a dual stack control-plane
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
[]string{
cluster ip allocator should check first on the legacy allocators Kubernetes clusters allow to define an IPv6 range of /108 for IPv6 despite the old allocators will only use the first /112 of that range. The new allocators does not have this limitation, so they can allocate IPs on the whole space, the problem happens on upgrades from clusters that were already using this size, since the new allocators by default will try to allocate addresses that works for both new and old allocatos to allow safe upgrades. The new allocators, when configured to keep compatibility with the old allocators, must try first to allocate an IP that is compatible with the old allocators and only fall back to the new behavior if it is not possible.
2025-01-24 05:37:43 -05:00
"--service-cluster-ip-range=192.168.0.0/24,2001:db8:1::/108",
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
"--advertise-address=10.0.0.1",
"--disable-admission-plugins=ServiceAccount",
},
etcdOptions)
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
move IPv6DualStack feature to stable. (#104691) * kube-proxy * endpoints controller * app: kube-controller-manager * app: cloud-controller-manager * kubelet * app: api-server * node utils + registry/strategy * api: validation (comment removal) * api:pod strategy (util pkg) * api: docs * core: integration testing * kubeadm: change feature gate to GA * service registry and rest stack * move feature to GA * generated
2021-09-24 19:30:22 -04:00
integeration tests
2021-06-22 13:40:09 -04:00
// Wait until the default "kubernetes" service is created.
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
preferDualStack := v1.IPFamilyPolicyPreferDualStack
svc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-prefer-dual01",
},
Spec: v1.ServiceSpec{
Type: v1.ServiceTypeClusterIP,
ClusterIPs: nil,
IPFamilies: nil,
IPFamilyPolicy: &preferDualStack,
Ports: []v1.ServicePort{
{
Name: "svc-port-1",
Port: 443,
TargetPort: intstr.IntOrString{IntVal: 443},
Protocol: "TCP",
},
},
},
}
Allow single-to-dual-stack reconfiguration for ServiceCIDR This change modifies the validation logic for ServiceCIDR updates (`ValidateServiceCIDRUpdate`) to specifically permit upgrading a single-stack ServiceCIDR (either IPv4 or IPv6) to a dual-stack configuration. This reconfiguration path is considered safe because it only involves adding a new CIDR range without altering the existing primary CIDR. This ensures that existing Service IP allocations are not disrupted. Other modifications, such as: - Downgrading from dual-stack to single-stack - Reordering CIDRs in a dual-stack configuration - Changing the primary CIDR during a single-to-dual-stack reconfiguration remain disallowed by the validation. These operations carry a higher risk of breaking existing Services or cluster networking configurations. Preventing these updates automatically encourages administrators to perform such changes manually after carefully assessing the potential impact on their specific cluster environment. The validation errors and controller logs provide guidance when such disallowed changes are attempted. Change-Id: I41dc09dfddb05f277925da2262f8114d6accbd1d
2025-04-11 11:37:28 -04:00
// create a copy of the service so we can test creating it again after reconfiguring the control plane
svcSingle := svc.DeepCopy()
svcSingle.Name = "svc-single"
integeration tests
2021-06-22 13:40:09 -04:00
// create the service
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svc, metav1.CreateOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
// validate the service was created correctly if it was not expected to fail
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
if err := validateServiceAndClusterIPFamily(svc, []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svc.Name, err)
}
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
// reconfigure the apiserver to be single stack
s.TearDownFn()
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
integeration tests
2021-06-22 13:40:09 -04:00
// reset secondary
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
s = kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
[]string{
"--service-cluster-ip-range=192.168.0.0/24",
"--advertise-address=10.0.0.1",
"--disable-admission-plugins=ServiceAccount",
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
},
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions)
defer s.TearDownFn()
integeration tests
2021-06-22 13:40:09 -04:00
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
client, err = kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
integeration tests
2021-06-22 13:40:09 -04:00
// Wait until the default "kubernetes" service is created.
if err = wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, "kubernetes", metav1.GetOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil && !apierrors.IsNotFound(err) {
return false, err
}
return !apierrors.IsNotFound(err), nil
}); err != nil {
t.Fatalf("creating kubernetes service timed out")
}
// validate the service is still there.
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
svc, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svc.Name, metav1.GetOptions{})
integeration tests
2021-06-22 13:40:09 -04:00
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svc.Name, err)
}
fix comment on dual-stack integration test
2021-06-25 17:24:49 -04:00
// service should remain dual stack
integeration tests
2021-06-22 13:40:09 -04:00
if err = validateServiceAndClusterIPFamily(svc, []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svc.Name, err)
}
Allow single-to-dual-stack reconfiguration for ServiceCIDR This change modifies the validation logic for ServiceCIDR updates (`ValidateServiceCIDRUpdate`) to specifically permit upgrading a single-stack ServiceCIDR (either IPv4 or IPv6) to a dual-stack configuration. This reconfiguration path is considered safe because it only involves adding a new CIDR range without altering the existing primary CIDR. This ensures that existing Service IP allocations are not disrupted. Other modifications, such as: - Downgrading from dual-stack to single-stack - Reordering CIDRs in a dual-stack configuration - Changing the primary CIDR during a single-to-dual-stack reconfiguration remain disallowed by the validation. These operations carry a higher risk of breaking existing Services or cluster networking configurations. Preventing these updates automatically encourages administrators to perform such changes manually after carefully assessing the potential impact on their specific cluster environment. The validation errors and controller logs provide guidance when such disallowed changes are attempted. Change-Id: I41dc09dfddb05f277925da2262f8114d6accbd1d
2025-04-11 11:37:28 -04:00
// validate that new services created with prefer dual are now single stack
// create the service
_, err = client.CoreV1().Services(metav1.NamespaceDefault).Create(tCtx, svcSingle, metav1.CreateOptions{})
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
// validate the service was created correctly
svcSingle, err = client.CoreV1().Services(metav1.NamespaceDefault).Get(tCtx, svcSingle.Name, metav1.GetOptions{})
if err != nil {
t.Fatalf("Unexpected error to get the service %s %v", svcSingle.Name, err)
}
// service should be single stack
if err = validateServiceAndClusterIPFamily(svcSingle, []v1.IPFamily{v1.IPv4Protocol}); err != nil {
t.Fatalf("Unexpected error validating the service %s %v", svcSingle.Name, err)
}
integeration tests
2021-06-22 13:40:09 -04:00
}
integration test
2021-08-20 17:13:03 -04:00
type serviceMergePatch struct {
Spec specMergePatch `json:"spec,omitempty"`
}
type specMergePatch struct {
Type v1.ServiceType `json:"type,omitempty"`
ExternalName string `json:"externalName,omitempty"`
}
// tests success when converting ClusterIP:Headless service to ExternalName
func Test_ServiceChangeTypeHeadlessToExternalNameWithPatch(t *testing.T) {
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
tCtx := ktesting.Init(t)
bugfix: initialize secondary range registry with the right value When MultiCIDRServiceAllocator feature is enabled, we added an additional feature gate DisableAllocatorDualWrite that allows to enable a mirror behavior on the old allocator to deal with problems during cluster upgrades. During the implementation the secondary range of the legacy allocator was initialized with the valuye of the primary range, hence, when a Service tried to allocate a new IP on the secondary range, it succeded in the new ip allocator but failed when it tried to allocate the same IP on the legacy allocator, since it has a different range. Expand the integration test that run over all the combinations of Service ClusterIP possibilities to run with all the possible combinations of the feature gates. The integration test need to change the way of starting the apiserver otherwise it will timeout.
2024-09-24 11:19:50 -04:00
etcdOptions := framework.SharedEtcd()
apiServerOptions := kubeapiservertesting.NewDefaultTestServerOptions()
s := kubeapiservertesting.StartTestServerOrDie(t,
apiServerOptions,
[]string{
"--service-cluster-ip-range=192.168.0.0/24",
"--advertise-address=10.0.0.1",
"--disable-admission-plugins=ServiceAccount",
},
etcdOptions)
defer s.TearDownFn()
client, err := kubernetes.NewForConfig(s.ClientConfig)
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
integration test
2021-08-20 17:13:03 -04:00
Clean shutdown of dualstack integration tests
2022-05-27 16:43:34 -04:00
ns := framework.CreateNamespaceOrDie(client, "test-service-allocate-node-ports", t)
defer framework.DeleteNamespaceOrDie(client, ns, t)
integration test
2021-08-20 17:13:03 -04:00
service := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "test-123",
},
Spec: v1.ServiceSpec{
Type: v1.ServiceTypeClusterIP,
ClusterIP: "None",
Selector: map[string]string{"foo": "bar"},
},
}
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
service, err = client.CoreV1().Services(ns.Name).Create(tCtx, service, metav1.CreateOptions{})
integration test
2021-08-20 17:13:03 -04:00
if err != nil {
t.Fatalf("Error creating test service: %v", err)
}
serviceMergePatch := serviceMergePatch{
Spec: specMergePatch{
Type: v1.ServiceTypeExternalName,
ExternalName: "foo.bar",
},
}
patchBytes, err := json.Marshal(&serviceMergePatch)
if err != nil {
t.Fatalf("failed to json.Marshal ports: %v", err)
}
test: use cancelation from ktesting The return type of ktesting.NewTestContext is now a TContext. Code which combined it WithCancel often didn't compile anymore (cannot overwrite ktesting.TContext with context.Context). This is a good thing because all of that code can be simplified to let ktesting handle the cancelation.
2023-12-25 13:40:56 -05:00
_, err = client.CoreV1().Services(ns.Name).Patch(tCtx, service.Name, types.StrategicMergePatchType, patchBytes, metav1.PatchOptions{})
integration test
2021-08-20 17:13:03 -04:00
if err != nil {
t.Fatalf("unexpected error patching service using strategic merge patch. %v", err)
}
}
Reference in a new issue Copy permalink