Advisory GHSA-pvhj-cqh2-rpq8 - High - Tenant Owners could grant themselves cross-tenant and cluster-scoped read access through clusterResources in a namespaced ProxySetting. Affects Capsule Proxy <= 0.14.1.
(Enterprise): Projectcapsule is now providing their releases on an immutable OCI registry, which allows users to verify the integrity of the images and provides a more secure way to distribute the images. Which is not possible on GHCR due to the fact that GHCR does not support immutability of images.
[OpenShift] Added documentation showing how to prevent CapsuleUsers from modifying OpenShift-managed namespace labels and annotations, including openshift.io/run-level, to help prevent SecurityContextConstraints bypasses. Read More

Breaking Changes ⚠️

Proxy: A namespaced ProxySetting can no longer grant clusterResources. Move existing grants to a GlobalProxySettings, which requires the ProxyClusterScoped feature gate.

Features ✨

Capsule: Resource policies for TenantResource and GlobalTenantResource add conditional application and control over resource adoption, protection, and cleanup.
Capsule: The rules API now offers more ways to configure and enforce namespace profiles:
- Workload Placement Mutation: Configure schedulers, node selectors, tolerations, affinity, and topology spread constraints on new Pods with merge or replace actions.
- Workload User Namespace Mutation: Request separate user namespaces for new Pods with security.hostUsers: false.
- Workload Read-only Root Filesystem Mutation: Set security.readOnlyRootFilesystem for selected regular, init, and newly added ephemeral containers.
- Workload Seccomp Mutation: Default or replace Pod-level seccomp profiles, including node-local profile paths.
- Workload AppArmor Mutation: Default or replace Pod-level AppArmor profiles, including named profiles loaded on nodes.
- Mutation Conditions: Apply individual mutation entries conditionally with CEL expressions that inspect the resource and admission request.
- Workload Placement Enforcement: Control which schedulers, node selectors, tolerations, affinity rules, and topology spread constraints workloads may use.
- Workload Type Enforcement: Allow, deny, or audit selected workload kinds, such as blocking DaemonSets in application namespaces.
- Pod Disruption Budget Enforcement: Require room for voluntary evictions, prevent overlapping PDBs, and control unhealthy Pod eviction. Checks cover PDB and workload changes, including controller scaling.
- Workload Seccomp Enforcement: Restrict effective seccomp profile types and local profile paths across selected Pods and containers.
- Workload AppArmor Enforcement: Restrict effective AppArmor profiles, including container overrides and named profiles loaded on nodes.
- NetworkPolicy Ingress and Egress CIDR Enforcement: Allow, deny, or audit IPv4 and IPv6 ingress and egress CIDR grants in selected namespaces, accounting for overlapping ranges and ipBlock.except.
- Enforcement Conditions: Apply an enforcement block only to resources and admission requests that match your CEL conditions.
- Additional PersistentVolume Access: Authorize restore handoffs and other explicit PVC bindings to PVs without a Tenant label, using namespace profiles, read-only label selectors, audiences, and CEL conditions on the referenced volume. Existing Tenant ownership labels remain authoritative.

Documentation 📚

We have added new documentation for a better experience. See the following topics:

Ecosystem 🌐

Dedicated Addons joining the capsule ecosystem

Change Windows: Change Windows helps organizations plan routine IT maintenance with fewer surprises and less manual coordination. It gives teams advance notice and can prevent conflicting changes while planned work is underway. It can also pause development environments overnight and bring them back in the morning, helping control infrastructure costs. These routines run automatically on your Kubernetes platform, so teams can spend less time coordinating maintenance and more time delivering services.
Resource Permits: Resource Permits helps platform teams offer services without handling every request manually or handing out broad administrator access. Teams request a predefined service, reviewers decide whether to approve it, and Kubernetes tracks the resources and their lifecycle. Requests can be made through the Kubernetes API, the kubectl plugin, or Headlamp.

Newly added documentation to integrate Capsule with other applications:

Roadmap 🗺️

In the upcoming releases we are planning to work on the following features:

Capsule: Porting more Properties to the Namespace Rule Approach.
Capsule: Adding triggers for Global/TenantResources.
Capsule: Adding healthChecks for Global/TenantResources.
Capsule: Adding Generic Implementation for Global/TenantResources.
Website: Improving the documentation with more examples and use-cases.

2 - Overview

Run multiple teams on a single Kubernetes cluster with strong isolation, self-service, and zero cluster sprawl.

Capsule is a Kubernetes Operator that turns a single cluster into a shared, multi-tenant platform. Teams get their own isolated space: a Tenant. Within their Tenant they own namespaces, resource budgets, and policies. Cluster administrators maintain full control, while teams work autonomously without stepping on each other.

No custom Kubernetes distribution. No extra tooling your users need to learn. Just plain Kubernetes, made shareable.

The Problem

Kubernetes namespaces provide a basic level of isolation, but they have no hierarchy. As soon as multiple teams or customers need to share a cluster, you face hard choices:

Isolation is all-or-nothing - there is no native way to group namespaces per team or enforce consistent policies across them.
Namespace sprawl - cluster admins become a bottleneck, manually creating and configuring every namespace.
Cluster sprawl - organizations spin up a separate cluster per team to achieve proper isolation, multiplying operational overhead across the board.

How Capsule Works

Capsule introduces the Tenant: a lightweight, cluster-scoped resource that groups one or more Kubernetes namespaces under a shared set of boundaries.

Everything defined on a Tenant is automatically inherited by all its namespaces:

RBAC bindings - roles are propagated to every namespace without manual setup.
Resource quotas & limits - CPU, memory, and storage budgets managed at the tenant level via Resource Pools or Custom Quotas.
Admission rules - allowed image registries, pull policies, security contexts, and more.
Templated resource distribution - using Replications, resources such as NetworkPolicies, ImagePullSecrets, and LimitRanges are automatically distributed into all namespaces a Tenant Owner creates, using Go templates for dynamic values like namespace name or tenant name.

capsule-workflow

Who Does What

Role	Responsibility
Cluster Admin	Installs Capsule, creates Tenants, sets resource budgets and policies. Never a bottleneck for day-to-day namespace work.
Tenant Owner	Creates and manages namespaces within their Tenant. Assigns access to team members. No cluster-level permissions needed.
Tenant User	Deploys workloads inside tenant namespaces, within the limits the owner has set.

This shift-left model means Tenant Owners handle day-to-day namespace operations themselves, freeing cluster admins from repetitive provisioning work.

Key Features

Tenants & Namespaces: Group namespaces into logical units per team, product, or customer. Policy inheritance is automatic.
Resource Management: Distribute CPU, memory, and storage budgets across namespaces with flexible claiming rather than fixed per-namespace quotas.
Replications: Propagate Kubernetes resources (Secrets, ConfigMaps, etc.) across tenant namespaces automatically.
Policy Rules: Enforce allowed registries, pull policies, and namespace metadata requirements on a per-tenant basis.
Capsule Proxy: Let users run kubectl get namespaces and see only their own, without granting cluster-wide LIST permissions. Also works for other cluster-wide requests, like kubectl get pods -A, or listing Persistent Volumes that are used by a Persistent Volume Claim inside the tenant.

Capsule Controller

The Capsule controller is a Kubernetes operator that continuously watches Tenant resources and reconciles the desired state across all namespaces that belong to a tenant. When a Tenant is created or updated, the controller automatically propagates the configured policies to every namespace in that tenant.

When a Tenant Owner creates a new namespace, the controller detects it and immediately applies all inherited policies. This means tenants are always in a consistent, compliant state, even as they grow.

capsule-operator

Get Started

3 - Quickstart 🚀

Create your first Capsule Tenant and start using it

This guide gets you from zero to a working multi-tenant cluster in minutes. You will install Capsule, create a Tenant as a cluster administrator, and then immediately switch to the tenant owner’s perspective to see what Capsule actually does.

Installation

Start a local Kubernetes cluster with KinD:

Get Here

# kind.yaml
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
nodes:
  - role: control-plane
  - role: worker
    extraPortMappings:
      - hostPort: 9001
        containerPort: 9001

kind create cluster --name capsule --config kind.yaml --wait=120s

The extra port mapping is for the Capsule Proxy, which lets tenant users issue kubectl get namespaces and see only their own. Install Capsule with the Proxy included:

helm upgrade --install capsule oci://ghcr.io/projectcapsule/charts/capsule --debug --create-namespace -n capsule-system --version 0.14.6 \
		--set 'proxy.enabled=true' \
		--set 'proxy.certManager.generateCertificates=false' \
		--set 'proxy.options.additionalSANs={localhost}' \
		--set 'proxy.options.generateCertificates=true' \
		--set "proxy.options.leaderElection=true" \
		--set "proxy.options.roleBindingReflector=true" \
		--set "proxy.service.type=NodePort" \
		--set "proxy.kind=DaemonSet" \
		--set "proxy.daemonset.hostNetwork=true" \
		--set "proxy.serviceMonitor.enabled=false" \
		--set "proxy.options.extraArgs={--feature-gates=ProxyClusterScoped=true}"  \
		--set 'crds.install=true' \
		--set 'certManager.generateCertificates=false' \
		--set 'tls.enableController=true' \
		--set 'tls.create=true'

Verify everything is running:

kubectl get pods -n capsule-system

NAME                                          READY   STATUS      RESTARTS      AGE
capsule-controller-manager-7584dc9546-l6tgl   1/1     Running     1 (21s ago)   29s
capsule-crds-vfq9k                            0/1     Completed   0             41s
capsule-post-install-2lm99                    0/1     Completed   0             28s
capsule-proxy-fjl5s                           0/1     Running     0             29s
capsule-proxy-certgen-5x7d6                   0/1     Completed   0             29s

For more installation options see the installation guide.

Create Your First Tenant

A Tenant groups one or more namespaces under a shared set of policies and limits. The cluster administrator creates and owns tenants. Users assigned as TenantOwner manage namespaces within them, without needing cluster-admin rights.

Apply the following Tenant as cluster admin:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  namespaceOptions:
    quota: 2
  forceTenantPrefix: true
  rules:
    - enforce:
        action: allow
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - "Namespace"
            labels:
              environment:
                required: true
                default: "dev"
                values:
                  - exact:
                      - dev
                      - test
                      - prod
    - namespaceSelector:
        matchLabels:
          environment: prod
      enforce:
        action: allow
        workloads:
          qosClasses:
            - Guaranteed
    - namespaceSelector:
        matchExpressions:
        - key: environment
          operator: NotIn
          values:
          - prod
      enforce:
        action: allow
        workloads:
          qosClasses:
            - BestEffort
            - Burstable
            - Guaranteed

What this configures:

owners: alice is the Tenant Owner and can create namespaces inside this tenant.
namespaceOptions.quota: 2: alice can create at most 2 namespaces. Read more
forceTenantPrefix: true: every namespace must start with solar-. Read more
rules: the environment label is required on every namespace, with dev as the default. Capsule enforces this at admission time. Read more
QoS rules: production namespaces (labeled environment=prod) only accept Guaranteed pods. Development and test namespaces accept any QoS class. Read more

Tenant Owners

Capsule only acts on requests from subjects it recognises as Capsule Users. The recommended way to register a user is to create a TenantOwner resource. The label projectcapsule.dev/tenant: "solar" binds it to the tenant automatically via aggregation:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantOwner
metadata:
  name: alice
  labels:
    projectcapsule.dev/tenant: "solar"
spec:
  kind: User
  name: "alice"

Capsule matches users by the groups they carry on every request. Creating a TenantOwner registers the subject automatically - no manual configuration needed. You can verify who is recognised at any time:

kubectl get capsuleconfiguration default -o jsonpath='{.status.users}' | jq

For the quickstart we use impersonation (--as-group projectcapsule.dev) which bypasses the need for a real certificate or token. In production, authentication depends on your cluster setup (X.509 certificates, OIDC tokens, etc.), use the Gangplank workflow to issue real kubeconfigs.

Verify the Tenant is active and alice is listed as an owner:

kubectl get tnt solar

NAME    STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR   READY   STATUS       AGE
solar   Active   2                 0                                 True    reconciled   10s

kubectl get tenant solar -o jsonpath='{.status.owners}' | jq

As a Tenant Owner

Now switch to alice’s perspective. Use impersonation to simulate her identity:

# All following commands run as alice
alias kubectl-alice='kubectl --as alice --as-group projectcapsule.dev'

Create a namespace

Try creating a namespace without the required prefix:

kubectl-alice create namespace development

Error from server (Forbidden): admission webhook "namespaces.mutating.projectcapsule.dev" denied the request: The Namespace name must start with 'solar-' when ForceTenantPrefix is enabled in the Tenant.

Capsule immediately enforces the naming rule. Try with the correct prefix:

kubectl-alice create namespace solar-development -o yaml

The namespace is created and Capsule automatically applies the default label environment=dev:

apiVersion: v1
kind: Namespace
metadata:
  labels:
    capsule.clastix.io/tenant: solar
    environment: dev
    kubernetes.io/metadata.name: solar-development
  name: solar-development

Enforce label constraints

The environment label can only be set to dev, test, or prod. Try to label the namespace with a value that is not allowed:

kubectl-alice label namespace solar-development environment=staging --overwrite

Error from server (Forbidden): admission webhook "namespaces.validating.projectcapsule.dev" denied the request: metadata label "staging" at metadata.labels["environment"] is not allowed by namespace rule: value did not match any allowed rule. Allowed metadata values: exact: dev, test, prod

Allowed values work fine:

kubectl-alice label namespace solar-development environment=test --overwrite

Namespace quota

Create a second namespace, this time explicitly as production:

kubectl-alice apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: solar-production
  labels:
    environment: prod
EOF

Attempting a third is denied:

kubectl-alice create namespace solar-staging

Error from server (Forbidden): admission webhook "namespaces.validating.projectcapsule.dev" denied the request: Cannot exceed Namespace quota: please, reach out to the system administrators

List with the Proxy

Without the Proxy, kubectl get namespaces -A returns Forbidden for non-admin users. Point alice’s kubeconfig to the Capsule Proxy to get a filtered view:

curl -s https://raw.githubusercontent.com/projectcapsule/capsule/main/hack/create-user.sh | bash -s -- alice solar projectcapsule.dev
KUBECONFIG=alice-solar.kubeconfig kubectl config set clusters.kind-capsule.certificate-authority-data $(kubectl -n capsule-system get secret capsule-proxy -o jsonpath='{.data.ca}')
KUBECONFIG=alice-solar.kubeconfig kubectl config set clusters.kind-capsule.server https://localhost:9001
export KUBECONFIG=alice-solar.kubeconfig

Now list namespaces; alice sees only hers:

kubectl get ns -A

NAME                STATUS   AGE
solar-development   Active   5m
solar-production    Active   2m

In production, automate kubeconfig distribution with Gangplank.

Going Further

Want to see more of what Capsule can do? The Going Further guide builds directly on this quickstart and covers Pod Security Standards enforcement, service type restrictions, permission bindings per environment, and automatic LimitRange distribution with GlobalTenantResource. None of it is required for a working setup, but it shows the full power of the platform.

Next Steps

You have seen the core of Capsule: a cluster administrator defines constraints, and tenant owners work freely within them without cluster-admin rights.

Topic	Link
Installation guide	Installation
Rules	Rules
Tenant resource replication	TenantResources
Cross-tenant replication	GlobalTenantResources
Resource Pools	Resource Pools
Custom Quotas	Custom Quotas
Capsule Proxy	Capsule Proxy
Day-2 Operations	Day-2 Operations

3.1 - Going Further

Optional deep-dive into Capsule rules, permissions, and resource distribution.

Optional

This page builds on the basic quickstart. It shows more of what Capsule can do once you have a working Tenant.

The examples below extend the solar Tenant from the quickstart. Apply each section on top of the existing Tenant with kubectl apply.

Pod Security Standards

Kubernetes Pod Security Standards (PSS) control what security contexts are allowed in a namespace. Capsule can enforce a PSS level per environment and even manage a label so users cannot override it.

Add the following rules to the solar Tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  namespaceOptions:
    quota: 2
  forceTenantPrefix: true
  rules:
    - enforce:
        action: allow
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - "Namespace"
            labels:
              environment:
                required: true
                default: "dev"
                values:
                  - exact:
                      - dev
                      - test
                      - prod

    # dev and test: allow restricted or baseline, default to restricted
    - namespaceSelector:
        matchExpressions:
        - key: environment
          operator: NotIn
          values:
          - prod
      enforce:
        action: allow
        workloads:
          qosClasses:
            - BestEffort
            - Burstable
            - Guaranteed
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - "Namespace"
            labels:
              pod-security.kubernetes.io/enforce:
                required: true
                default: "restricted"
                values:
                  - exact:
                      - restricted
                      - baseline

    # prod: lock pod-security to restricted, users cannot change it
    - namespaceSelector:
        matchLabels:
          environment: prod
      enforce:
        action: allow
        workloads:
          qosClasses:
            - Guaranteed
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - "Namespace"
            labels:
              pod-security.kubernetes.io/enforce:
                required: true
                managed: "restricted"

The key difference between values and managed:

values defines what a user is allowed to set.
managed means Capsule owns the value. It is applied automatically and any attempt to change it is silently corrected by the webhook.

See it in action

Create a production namespace and try to set a permissive PSS level:

kubectl-alice apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: solar-production
  labels:
    environment: prod
    pod-security.kubernetes.io/enforce: privileged
EOF

The namespace is created, but inspect it:

kubectl get namespace solar-production --show-labels

NAME               STATUS   LABELS
solar-production   Active   environment=prod   pod-security.kubernetes.io/enforce=restricted   ...

Capsule silently corrected privileged to restricted because the label is managed. No error, no friction - the policy is simply enforced.

Now try to change it after creation:

kubectl-alice label namespace solar-production pod-security.kubernetes.io/enforce=baseline --overwrite

Error from server (Forbidden): admission webhook "namespaces.validating.projectcapsule.dev" denied the request: metadata label "baseline" at metadata.labels["pod-security.kubernetes.io/enforce"] is not allowed by namespace rule: value did not match any allowed rule.

In a development namespace the user has more freedom. Create one and change the PSS to baseline:

kubectl-alice create namespace solar-development
kubectl-alice label namespace solar-development pod-security.kubernetes.io/enforce=baseline --overwrite

This succeeds because the dev rule lists both restricted and baseline as allowed values.

Service Restrictions

Prevent tenants from creating NodePort or LoadBalancer services, and optionally restrict ExternalName hostnames to a pattern:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  rules:
  - enforce:
      action: allow
      services:
        types:
          - ClusterIP
          - ExternalName
        externalNames:
          hostnames:
            - exp: ".*\\.solar\\.svc\\.company\\.com"

Any attempt to create a NodePort or LoadBalancer service is denied at admission. ExternalName services are allowed only if their hostname matches the pattern - in this case any subdomain of solar.svc.company.com.

Permission Bindings

Automatically distribute RoleBindings across Tenant namespaces based on namespace metadata. This example gives an operators group edit access in dev and test namespaces, and only view access in production:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  rules:
    - namespaceSelector:
        matchExpressions:
        - key: environment
          operator: NotIn
          values:
          - prod
      permissions:
        bindings:
          - clusterRoleName: 'edit'
            subjects:
              - kind: Group
<<<<<<< HEAD
<<<<<<< HEAD
                name: "tenant:{{ .tenant.metadata.name }}:operators"
=======
                name: "solar:operators"
>>>>>>> 040fb2af9f39a8b5da7f6bdd18c5e15c63687b80
=======
                name: "tenant:{{ .tenant.metadata.name }}:operators"
>>>>>>> 5298aace8a45da78c5806f42687fcf787e5cbf47

    - namespaceSelector:
        matchLabels:
          environment: prod
      permissions:
        bindings:
          - clusterRoleName: 'view'
            subjects:
              - kind: Group
                name: "solar:operators"

Capsule creates and maintains the RoleBindings automatically. When alice creates a new namespace with environment=dev, the edit binding is added without any manual step.

Resource Distribution with GlobalTenantResource

GlobalTenantResource lets a cluster administrator push resources into every namespace of selected Tenants automatically. A common use case is distributing LimitRange objects to cap resource consumption per container.

LimitRanges

The following example enforces different LimitRange defaults per environment:

Get Here

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: limitranges
spec:
  resyncPeriod: 60s
  resources:
    # bronze: no defaults, containers must declare their own
    - namespaceSelector:
        matchLabels:
          environment: dev
      rawItems:
        - apiVersion: v1
          kind: LimitRange
          metadata:
            name: service-level-bronze
          spec:
            limits:
              - type: Container

    # silver: default memory limits for test
    - namespaceSelector:
        matchLabels:
          environment: test
      rawItems:
        - apiVersion: v1
          kind: LimitRange
          metadata:
            name: service-level-silver
          spec:
            limits:
              - default:
                  memory: "256Mi"
                defaultRequest:
                  cpu: 128m
                  memory: "256Mi"
                type: Container

    # gold: enforced defaults for production
    - namespaceSelector:
        matchLabels:
          environment: prod
      rawItems:
        - apiVersion: v1
          kind: LimitRange
          metadata:
            name: service-level-gold
          spec:
            limits:
              - default:
                  cpu: 128m
                  memory: "256Mi"
                defaultRequest:
                  cpu: 128m
                  memory: "256Mi"
                type: Container

Apply this and Capsule will immediately create the appropriate LimitRange in every matching namespace. New namespaces pick it up within the resyncPeriod. Verify:

kubectl get limitrange -n solar-production

NAME                 CREATED AT
service-level-gold   2026-07-24T10:00:00Z

Networkpolicies

Distribute a NetworkPolicy to all Namespaces of a Tenant to enforce a certain network policy for all workloads within the Tenant/Namespace. The following NetworkPolicy is an attempt to achieve a default deny policy for all Namespaces of the Tenant but allow intra-namespace communication and allow communication between all Namespaces of the same Tenant. It also allows communication to system namespaces (eg. monitoring, ingress, etc.). Read More

Get Here

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: default-networkpolicies
spec:
  resyncPeriod: 60s
  resources:
    - rawItems:
        - apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: default-policy
          spec:
            # Apply to all pods in this namespace
            podSelector: {}
            policyTypes:
              - Ingress
              - Egress
            ingress:
              # Allow traffic from the same namespace (intra-namespace communication)
              - from:
                  - podSelector: {}

              # Allow traffic from all namespaces within the tenant
              - from:
                  - namespaceSelector:
                      matchLabels:
                        capsule.clastix.io/tenant: "{{tenant.name}}"

              # Allow ingress from other namespaces labeled (System Namespaces, eg. Monitoring, Ingress)
              - from:
                  - namespaceSelector:
                      matchLabels:
                        company.com/system: "true"

            egress:
              # Allow DNS to kube-dns service IP (might be different in your setup)
              - to:
                  - ipBlock:
                      cidr: 10.96.0.10/32
                ports:
                  - protocol: UDP
                    port: 53
                  - protocol: TCP
                    port: 53

              # Allow traffic to all namespaces within the tenant
              - to:
                  - namespaceSelector:
                      matchLabels:
                        capsule.clastix.io/tenant: "{{tenant.name}}"

Quota Management

Capsule also provides two dedicated quota mechanisms worth knowing about:

Resource Pools - a cluster-scoped pool of resources (CPU, memory, storage) that selected namespaces can draw from via ResourcePoolClaim objects. Useful when you want a shared budget across multiple tenants or namespaces, with claims that stack automatically. See Resource Pools.
Custom Quotas - a flexible quota system that sits between classic ResourceQuota and Resource Pools, allowing fine-grained per-tenant quota management with custom rules on all types of resources. See Custom Quotas.

Both are managed by cluster administrators and are independent of the Tenant rules shown on this page.

Full Tenant

Here we have two Tenants with different rules and permissions. The solar tenant is a production tenant with multiple application stages with strict rules and permissions, while the lunar tenant is a development tenant with more relaxed rules and permissions.

Get Here

# solar.yaml
---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  permissions:
    matchOwners:
    - matchLabels:
        team: platform
  owners:
  - name: alice
    kind: User
  - name: bob
    kind: User
  namespaceOptions:
    quota: 2
  forceTenantPrefix: true
  resourceQuotas:
    scope: Tenant
    items:
    - hard:
        limits.cpu: "8"
        limits.memory: 16Gi
        requests.cpu: "8"
        requests.memory: 16Gi
  rules:
    - audience:
        - kind: Custom
          name: "CapsuleUser"
      enforce:
        action: deny
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - "Namespace"
            labels:
              "openshift.io/.*":
                required: false
                values:
                  - exp: ".*"
    - enforce:
        action: allow
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - "Namespace"
            labels:
              environment:
                required: true
                default: "dev"
                values:
                  - exact:
                      - dev
                      - test
                      - prod
        services:
          types:
            - ClusterIP
            - ExternalName
          externalNames:
            hostnames:
              - exp: ".*\\.{{ .tenant.metadata.name }}\\.svc\\.company\\.com"
    - namespaceSelector:
        matchExpressions:
        - key: environment
          operator: NotIn
          values:
          - prod
      permissions:
        bindings:
          - clusterRoleName: 'edit'
            subjects:
              - kind: Group
                name: tenant:{{ .tenant.metadata.name }}:operators
      enforce:
        action: allow
        workloads:
          qosClasses:
            - Guaranteed
            - BestEffort
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - "Namespace"
            labels:
              pod-security.kubernetes.io/enforce:
                required: true
                default: "restricted"
                values:
                  - exact:
                      - restricted
                      - baseline
    - namespaceSelector:
        matchLabels:
          environment: prod
      permissions:
        bindings:
          - clusterRoleName: 'view'
            subjects:
              - kind: Group
                name: tenant:{{ .tenant.metadata.name }}:operators
      enforce:
        action: allow
        workloads:
          qosClasses:
            - Guaranteed
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - "Namespace"
            labels:
              pod-security.kubernetes.io/enforce:
                required: true
                managed: "restricted"

Once applied we can verify the Tenant with the following command:

kubectl get tnt solar

NAME    STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR   READY   STATUS       AGE
solar   Active   2                 0                                 True    reconciled   35s

Next Steps

Topic	Link
Installation guide	Installation
Rules	Rules
Tenant resource replication	TenantResources
Cross-tenant replication	GlobalTenantResources
Resource Pools	Resource Pools
Custom Quotas	Custom Quotas
Capsule Proxy	Capsule Proxy
Day-2 Operations	Day-2 Operations

4 - Operating

Get started with using tenancy on Kubernetes

4.1 - Setup

Setting up your environment for Capsule

4.1.1 - Installation

Installing the Capsule Controller

Requirements

Helm 3 is required when installing the Capsule Operator chart. Follow Helm’s official documentation for installing Helm on your operating system.
A Kubernetes cluster (v1.16+) with the following Admission Controllers enabled:
- PodNodeSelector
- LimitRanger
- ResourceQuota
- MutatingAdmissionWebhook
- ValidatingAdmissionWebhook
A Kubeconfig file accessing the Kubernetes cluster with cluster admin permissions.
Cert-Manager is required by default but can be disabled. It is used to manage the TLS certificates for the Capsule Admission Webhooks.

Installation

We officially only support the installation of Capsule using the Helm chart. The chart itself handles the installation/upgrade of the required CustomResourceDefinitions. The following Artifact Hub repositories are official:

Perform the following steps to install the Capsule operator:

Add repository:

 helm repo add projectcapsule https://projectcapsule.github.io/charts

Install Capsule:

 helm install capsule projectcapsule/capsule --version 0.14.6 -n capsule-system --create-namespace

or (OCI)

 helm install capsule oci://ghcr.io/projectcapsule/charts/capsule --version 0.14.6 -n capsule-system --create-namespace

Show the status:
```
 helm status capsule -n capsule-system
```

Upgrade the Chart

 helm upgrade capsule projectcapsule/capsule -n capsule-system

or (OCI)

 helm upgrade capsule oci://ghcr.io/projectcapsule/charts/capsule --version 0.14.6

Uninstall the Chart

 helm uninstall capsule -n capsule-system

Production

Here are some key considerations to keep in mind when installing Capsule. Also check out the Best Practices for more information.

Scalability

For large clusters you might need to consider adjusting values for the Capsule controller.

QPS/Burst

In order to handle a large number of tenants and resources, you may need to increase the QPS and Burst values for the Capsule controller. This avoids the controller being throttled by the Kubernetes API server (Client Rate limited). You can set the following values in the Helm chart:

manager:
  options:
    clientConnectionQPS: 400
    clientConnectionBurst: 200

Workers

Define the number of workers for the Capsule controller, which translates into the number of concurrent reconciles:

manager:
  options:
    workers: 4

Cache Synchronisation

The more resources you have in your cluster, the longer it will take for the Capsule controller to sync its cache. You can adjust the cache sync period to a higher value to reduce the load on the API server:

manager:
  options:
    cacheSyncTimeout: "10m"

Leader Election Timeout

In high pressure environments leader election may fail due to the default timeout values. You can adjust the leader election timeout values to avoid this issue:

E0707 08:38:18.319041       1 leaderelection.go:452] "Error retrieving lease lock"
  err="Get \"https://10.96.0.1:443/apis/coordination.k8s.io/v1/namespaces/capsule-
  system/leases/42c733ea.clastix.capsule.io?timeout=5s\": net/http: request canceled
  (Client.Timeout exceeded while awaiting headers)" lock="capsule-
  system/42c733ea.clastix.capsule.io"
  I0707 08:38:18.442700       1 leaderelection.go:299] "Failed to renew lease"

Tune leader election with manager.options.leaderElection.leaseDuration, manager.options.leaderElection.renewDeadline, and manager.options.leaderElection.retryPeriod. Increasing these values makes Capsule more tolerant of slow or overloaded Kubernetes API servers; for example, raising renewDeadline also raises the leader-election client request timeout because controller-runtime uses roughly half of that value. The tradeoff is slower failover: if the active controller really dies, standby replicas will wait longer before taking leadership. Keep the ordering valid: leaseDuration should be greater than renewDeadline, and renewDeadline should be greater than retryPeriod.

manager:
  options:
    leaderElection:
      leaseDuration: "60s"
      renewDeadline: "40s"
      retryPeriod: "5s"

Worst-case leader failover is slower, around 60s, if the active pod really dies. Keep manager.options.leaderElection.leaseDuration > manager.options.leaderElection.renewDeadline > manager.options.leaderElection.retryPeriod.

API Priority and Fairness (APF)

With APF enabled, the Capsule controller will be subject to the APF configuration of the cluster. If you are running a large cluster with many tenants, you may need to adjust the APF configuration to ensure that the Capsule controller has sufficient resources to operate effectively. For more information on APF, see Kubernetes API Priority and Fairness.

We provide a built-in APF configuration for the Capsule controller, which provides API priority for all resources managed by Capsule. This configuration is applied automatically when you install Capsule. To enable the built-in APF configuration, set the following value in the Helm chart:

# Manager Options
manager:
  apiPriorityAndFairness:
    # -- Change to `true` if you want to insulate the API calls made by Capsule admission controller activities.
    # This will help ensure Capsule stability in busy clusters.
    # Ref: https://kubernetes.io/docs/concepts/cluster-administration/flow-control/
    enabled: true

    # -- Only the first matching FlowSchema for a given request matters. If multiple FlowSchemas match a single inbound request, it will be assigned based on the one with the highest matchingPrecedence.
    # Ref: https://kubernetes.io/docs/concepts/cluster-administration/flow-control/#flowschema
    matchingPrecedence: 900

    # -- Priority level configuration.
    # The block is directly forwarded into the priorityLevelConfiguration, so you can use whatever specification you want.
    # ref: https://kubernetes.io/docs/concepts/cluster-administration/flow-control/#prioritylevelconfiguration
    priorityLevelConfigurationSpec:
      type: Limited
      limited:
          nominalConcurrencyShares: 100
          limitResponse:
            type: Queue
            queuing:
              queues: 64
              handSize: 6
              queueLengthLimit: 100

Strict RBAC

Attention

Ensure to first upgrade to version 0.13.0 of capsule before enabling strict mode. As it requires fields which are newly added with version 0.13.0.

By default, the Capsule controller runs with the ClusterRole cluster-admin, which provides full access to the cluster. This is because the controller must be able to grant RoleBindings on a per-namespace basis: Kubernetes prevents privilege escalation through RBAC, so a RoleBinding referencing a ClusterRole can only be created by a subject that either already holds all permissions of that ClusterRole or holds the bind verb on it. For production environments we recommend configuring stricter RBAC permissions for the Capsule controller by setting the following value in the Helm chart:

manager:
  rbac:
    strict: true

This replaces the cluster-admin binding with a ClusterRole that grants the controller only the permissions required for its own operation (manager.rbac.minimal is a deprecated alias for this option; both flags enable the same behavior). To satisfy the escalation prevention described above, this ClusterRole contains the narrow bind verb on the ClusterRoles listed in manager.rbac.bindableClusterRoles (default: admin) plus the ClusterRoles managed by Capsule itself (the manager.options.rbac provisioner, deleter, administrationClusterRoles and promotionClusterRoles), which are always bindable. With default values this renders as:

- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles"]
  resourceNames: ["admin", "capsule-namespace-deleter", "capsule-namespace-provisioner"]
  verbs: ["bind"]

Tenant owner RoleBindings for the default ClusterRoles therefore work out of the box, without the controller inheriting all of admin’s permissions.

Chart versions v0.13.11 and earlier

On older chart versions strict mode is only enabled via manager.rbac.minimal: true (manager.rbac.strict had no effect there), and the controller ClusterRole includes neither the bind rule above nor create on NetworkPolicies, which GlobalTenantResource propagation requires. Add both via Helm values:

manager:
  rbac:
    minimal: true
    clusterRole:
      extraResources:
        - apiGroups: ["rbac.authorization.k8s.io"]
          resources: ["clusterroles"]
          resourceNames: ["admin", "capsule-namespace-provisioner", "capsule-namespace-deleter"]
          verbs: ["bind"]
        - apiGroups: ["networking.k8s.io"]
          resources: ["networkpolicies"]
          verbs: ["get", "list", "watch", "create", "update", "patch", "delete", "deletecollection"]

Or aggregate the full admin ClusterRole to the controller — note that the controller then holds all of admin’s permissions:

kubectl label clusterrole admin projectcapsule.dev/aggregate-to-controller=true

Any further ClusterRole that is assigned to Tenant owners or used for additional RoleBindings needs the same treatment. This applies only to ClusterRoles that are not managed by Capsule (see Configuration). The narrowest option is adding it to manager.rbac.bindableClusterRoles — note that the list replaces the default, so keep admin in it:

manager:
  rbac:
    strict: true
    bindableClusterRoles:
      - admin
      - prometheus-servicemonitors-viewer

Alternatively, you can aggregate the permissions of further ClusterRoles into the controller. The ClusterRole for the controller aggregates all ClusterRoles carrying one of the following labels:

projectcapsule.dev/aggregate-to-controller: "true"
projectcapsule.dev/aggregate-to-controller-instance: {{ .Release.Name }}

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: prometheus-servicemonitors-viewer
  labels:
    projectcapsule.dev/aggregate-to-controller: "true"
rules:
...

This is broader than bind — the controller then actually holds these permissions. It is required whenever the controller must exercise the permissions itself, for example when replicating resources via GlobalTenantResources without impersonation.

You can also directly grant arbitrary additional permissions via Helm values:

manager:
  rbac:
    strict: true
    clusterRole:
      extraResources:
        - apiGroups: ["storage.k8s.io"]
          resources: ["storageclasses"]
          verbs: ["get", "list", "watch", "update", "patch"]

If you are missing permissions you will see an error status for the respective tenants reflecting

kubectl get tnt
NAME    STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR   READY   STATUS                                                                                                                                                                                                                                                                                                                                          AGE
green   Active                     2                                 False   cannot sync rolebindings items: rolebindings.rbac.authorization.k8s.io "capsule:managed:658936e7f2a30e35" is forbidden: user "system:serviceaccount:capsule-system:capsule" (groups=["system:serviceaccounts" "system:serviceaccounts:capsule-system" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:...   5s

Before you enable strict mode, you must implement the required permissions for your use case. Depending on which features you are using, you may need to take manual action, for example:

Admission Policies

While Capsule provides a robust framework for managing multi-tenancy in Kubernetes, it does not include built-in admission policies for enforcing specific security or operational standards for all possible aspects of a Kubernetes cluster. We provide additional policy recommendations here.

Certificate Management

By default, Capsule delegates its certificate management to cert-manager. This is the recommended way to manage the TLS certificates for Capsule. However, you can also use Capsule’s built-in TLS reconciler to manage the certificates. This is not recommended for production environments. To enable the TLS reconciler, use the following values:

certManager:
  generateCertificates: false
tls:
  enableController: true
  create: true

Webhooks

Capsule makes use of webhooks for admission control. Ensure that your cluster supports webhooks and that they are properly configured. The webhooks are automatically created by Capsule during installation. However, some of these webhooks will cause problems when Capsule is not running (this is especially problematic in single-node clusters). Here are the webhooks you need to watch out for.

Generally, we recommend using matchConditions for all webhooks to avoid problems when Capsule is not running. You should exclude your system-critical components from the Capsule webhooks. For namespaced resources (pods, services, etc.) the webhooks select only namespaces that are part of a Capsule Tenant. If your system-critical components are not part of a Capsule Tenant, they will not be affected by the webhooks. However, if you have system-critical components that are part of a Capsule Tenant, you should exclude them from the Capsule webhooks by using matchConditions as well, or add more specific namespaceSelectors/objectSelectors to exclude them. This can also improve performance.

Refer to the webhook values.

System-critical Components

It’s strongly advised to exclude system-critical components from the Capsule webhooks. This includes components like kube-system, kube-public, and kube-node-lease namespaces, as well as any other namespaces that are critical for the operation of your cluster. You can use matchConditions to exclude these namespaces from the Capsule webhooks.

webhooks:
  matchConditions:
    - name: exclude-control-plane
      expression: '!(request.userInfo.username in ["system:apiserver", "system:kube-controller-manager", "system:kube-scheduler"]) && !("system:nodes" in request.userInfo.groups) && !("system:serviceaccounts:kube-system" in request.userInfo.groups)'

Nodes

There is a webhook which catches interactions with the Node resource. This webhook is mainly relevant when you make use of Node metadata. In most other cases, it will only cause problems. By default, the webhook is disabled, but you can enable it by setting the following value:

webhooks:
  hooks:
    nodes:
      enabled: true

Or you could at least consider to set the failure policy to Ignore, if you don’t want to disrupt critical nodes:

webhooks:
  hooks:
    nodes:
      failurePolicy: Ignore

If you still want to use the feature, you could exclude the kube-system namespace (or any other namespace you want to exclude) from the webhook by setting the following value:

webhooks:
  hooks:
    nodes:
      matchConditions:
      - name: 'exclude-kubelet-requests'
        expression: '!("system:nodes" in request.userInfo.groups)'
      - name: 'exclude-kube-system'
        expression: '!("system:serviceaccounts:kube-system" in request.userInfo.groups)'

Namespaces

Namespaces are the most important resource in Capsule. The Namespace webhook is responsible for enforcing the Capsule Tenant boundaries. It is enabled by default and should not be disabled. However, you may change the matchConditions to exclude certain namespaces from the Capsule Tenant boundaries. For example, you can exclude the kube-system namespace by setting the following value:

webhooks:
  hooks:
    namespaces:
      matchConditions:
      - name: 'exclude-kube-system'
        expression: '!("system:serviceaccounts:kube-system" in request.userInfo.groups)'

Protected

By default resources with the following values are protected by a webhook to be changed by [Capsule Users]:

webhooks:
  hooks:
    managed:
      objectSelector:
        matchExpressions:
          - key: "projectcapsule.dev/created-by"
            operator: In
            values:
            - "controller"
            - "resources"
          - key: "projectcapsule.dev/managed-by"
            operator: In
            values:
            - "controller"

GitOps

There are no specific requirements for using Capsule with GitOps tools like ArgoCD or FluxCD. You can manage Capsule resources as you would with any other Kubernetes resource.

ArgoCD

Visit the ArgoCD Integration for more options to integrate Capsule with ArgoCD.

Manifests to get you started with ArgoCD. For ArgoCD you might need to skip the validation of the CapsuleConfiguration resources, otherwise there might be errors on the first install:

Information

The Validate=false option is required for the CapsuleConfiguration resource, because ArgoCD tries to validate the resource before the Capsule CRDs are installed via our CRD Lifecycle hook. Upstream Issue. This has mainly been observed in ArgoCD Applications using Service-Side Diff/Apply.

manager:
  options:
    annotations:
      argocd.argoproj.io/sync-options: "Validate=false,SkipDryRunOnMissingResource=true"

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: capsule
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: system
  source:
    repoURL: ghcr.io/projectcapsule/charts
    targetRevision: 0.14.6
    chart: capsule
    helm:
      valuesObject:
        crds:
          install: true
        manager:
          options:
            annotations:
              argocd.argoproj.io/sync-options: "Validate=false,SkipDryRunOnMissingResource=true"
            capsuleConfiguration: default
            ignoreUserGroups:
              - oidc:administators
            users:
              - kind: Group
                name: oidc:kubernetes-users
              - kind: Group
                name: system:serviceaccounts:tenants-system
        monitoring:
          dashboards:
            enabled: true
          serviceMonitor:
            enabled: true
            annotations:
              argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true

  destination:
    server: https://kubernetes.default.svc
    namespace: capsule-system

  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - ServerSideApply=true
    - CreateNamespace=true
    - PrunePropagationPolicy=foreground
    - PruneLast=true
    - RespectIgnoreDifferences=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m
---
apiVersion: v1
kind: Secret
metadata:
  name: capsule-repo
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  url: ghcr.io/projectcapsule/charts
  name: capsule
  project: system
  type: helm
  enableOCI: "true"

FluxCD

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: capsule
  namespace: flux-system
spec:
  serviceAccountName: kustomize-controller
  targetNamespace: "capsule-system"
  interval: 10m
  releaseName: "capsule"
  chart:
    spec:
      chart: capsule
      version: "0.14.6"
      sourceRef:
        kind: HelmRepository
        name: capsule
      interval: 24h
  install:
    createNamespace: true
  upgrade:
    remediation:
      remediateLastFailure: true
  driftDetection:
    mode: enabled
  values:
    crds:
      install: true
    manager:
      options:
        capsuleConfiguration: default
        ignoreUserGroups:
          - oidc:administators
        users:
          - kind: Group
            name: oidc:kubernetes-users
          - kind: Group
            name: system:serviceaccounts:tenants-system
    monitoring:
      dashboards:
        enabled: true
      serviceMonitor:
        enabled: true
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: capsule
  namespace: flux-system
spec:
  type: "oci"
  interval: 12h0m0s
  url: oci://ghcr.io/projectcapsule/charts

Security

See all available Artifacts

Signature

To verify artifacts you need to have cosign installed. This guide assumes you are using v2.x of cosign. All of the signatures are created using keyless signing. You can set the environment variable COSIGN_REPOSITORY to point to this repository. For example:

# Docker Image
export COSIGN_REPOSITORY=ghcr.io/projectcapsule/capsule

# Helm Chart
export COSIGN_REPOSITORY=ghcr.io/projectcapsule/charts/capsule

To verify the signature of the docker image, run the following command.

COSIGN_REPOSITORY=ghcr.io/projectcapsule/charts/capsule cosign verify ghcr.io/projectcapsule/capsule:<release_tag> \
  --certificate-identity-regexp="https://github.com/projectcapsule/capsule/.github/workflows/docker-publish.yml@refs/tags/*" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" | jq

To verify the signature of the helm image, run the following command.

COSIGN_REPOSITORY=ghcr.io/projectcapsule/charts/capsule cosign verify ghcr.io/projectcapsule/charts/capsule:<release_tag> \
  --certificate-identity-regexp="https://github.com/projectcapsule/capsule/.github/workflows/helm-publish.yml@refs/tags/*" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" | jq

Provenance

Capsule creates and attests to the provenance of its builds using the SLSA standard and meets the SLSA Level 3 specification. The attested provenance may be verified using the cosign tool.

Verify the provenance of the docker image.

cosign verify-attestation --type slsaprovenance \
  --certificate-identity-regexp="https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@refs/tags/*" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
  ghcr.io/projectcapsule/capsule:0.14.6 | jq .payload -r | base64 --decode | jq

cosign verify-attestation --type slsaprovenance \
  --certificate-identity-regexp="https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@refs/tags/*" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
  ghcr.io/projectcapsule/charts/capsule:0.14.6 | jq .payload -r | base64 --decode | jq

Software Bill of Materials (SBOM)

An SBOM (Software Bill of Materials) in CycloneDX JSON format is published for each release, including pre-releases. You can set the environment variable COSIGN_REPOSITORY to point to this repository. For example:

# Docker Image
export COSIGN_REPOSITORY=ghcr.io/projectcapsule/capsule

# Helm Chart
export COSIGN_REPOSITORY=ghcr.io/projectcapsule/charts/capsule

To inspect the SBOM of the docker image, run the following command.

COSIGN_REPOSITORY=ghcr.io/projectcapsule/capsule cosign download sbom ghcr.io/projectcapsule/capsule:0.14.6

To inspect the SBOM of the helm image, run the following command.

COSIGN_REPOSITORY=ghcr.io/projectcapsule/charts/capsule cosign download sbom ghcr.io/projectcapsule/charts/capsule:0.14.6

Compatibility

The Kubernetes compatibility is announced for each Release. Generally we are up to date with the latest upstream Kubernetes Version. Note that the Capsule project offers support only for the latest minor version of Kubernetes. Backwards compatibility with older versions of Kubernetes and OpenShift is offered by vendors.

4.1.2 - Configuration

Understand the Capsule configuration options and how to use them.

The configuration for the capsule controller is done via its dedicated configuration Custom Resource. You can explain the configuration options and how to use them:

CapsuleConfiguration

The configuration for Capsule is done via its dedicated configuration Custom Resource. You can explain the configuration options and how to use them:

kubectl explain capsuleConfiguration.spec

`administrators`

These entities are automatically owners for all existing tenants. Meaning they can add namespaces to any tenant. However they must be specific by using the capsule label for interacting with namespaces. Because if that label is not defined, it’s assumed that namespace interaction was not targeted towards a tenant and will therefore be ignored by capsule. May also be handy in GitOps scenarios where certain service accounts need to be able to manage namespaces for all tenants.

manager:
  options:
    administrators:
      - kind: User
        name: admin-user

`users`

manager:
  options:
    users:
      - kind: User
        name: owner-user
      - kind: Group
        name: projectcapsule.dev

`ignoreUserWithGroups`

Define groups which when found in the request of a user will be ignored by the Capsule. This might be useful if you have one group where all the users are in, but you want to separate administrators from normal users with additional groups.

manager:
  options:
    ignoreUserWithGroups:
      - company:org:administrators

`enableTLSReconciler`

Toggles the TLS reconciler, the controller that is able to generate CA and certificates for the webhooks when not using an already provided CA and certificate, or when these are managed externally with Vault, or cert-manager.

tls:
  enableController: true

`forceTenantPrefix`

Enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash. This is useful to avoid Namespace name collision in a public CaaS environment.

manager:
  options:
    forceTenantPrefix: true

`nodeMetadata`

Allows to set the forbidden metadata for the worker nodes that could be patched by a Tenant. This applies only if the Tenant has an active NodeSelector, and the Owner have right to patch their nodes.

manager:
  options:
    nodeMetadata:
      forbiddenLabels:
        denied:
          - "node-role.kubernetes.io/*"
        deniedRegex: ""
      forbiddenAnnotations:
        denied:
          - "node.alpha.kubernetes.io/*"
        deniedRegex: ""

`overrides`

Allows to set different name rather than the canonical one for the Capsule configuration objects, such as webhook secret or configurations.

`protectedNamespaceRegex`

Disallow creation of namespaces, whose name matches this regexp

manager:
  options:
    protectedNamespaceRegex: "^(kube|default|capsule|admin|system|com|org|local|localhost|io)$"

`allowServiceAccountPromotion`

ServiceAccounts within tenant namespaces can be promoted to owners of the given tenant this can be achieved by labeling the serviceaccount and then they are considered owners. This can only be done by other owners of the tenant. However ServiceAccounts which have been promoted to owner can not promote further serviceAccounts.

manager:
  options:
    allowServiceAccountPromotion: true

`cacheInvalidation`

The reconcile periode caches are invalidated. Invalidation is already attempted when resources change, however in certain scenarios it might be necessary to do out of order cache invalidations to ensure proper garbage collection of resources.

manager:
  options:
    cacheInvalidation: 0h30m0s

`rbac`

Define configurations for the RBAC which is being managed and applied by Capsule.

manager:
  options:
    rbac:
      # -- The ClusterRoles applied for Administrators
      administrationClusterRoles:
        - capsule-namespace-deleter

      # -- The ClusterRoles applied for ServiceAccounts which had owner Promotion
      promotionClusterRoles:
        - capsule-namespace-provisioner
        - capsule-namespace-deleter

      # -- Name for the ClusterRole required to grant Namespace Deletion permissions.
      deleter: capsule-namespace-deleter

      # -- Name for the ClusterRole required to grant Namespace Provision permissions.
      provisioner: capsule-namespace-provisioner

`impersonation`

Replications impersonate a ServiceAccount, falling back to the controller ServiceAccount when none is resolved. tenantDefaultServiceAccount is set to default for TenantResource; GlobalTenantResource has no default and requires both globalDefaultServiceAccount and globalDefaultServiceAccountNamespace. Within this configuration you can define properties such as the endpoint of the kube-apiserver and if service account promotion should be allowed for this client. By default the https://kubernetes.default.svc endpoint is used.

manager:
  options:
    impersonation:
      # Kubernetes API Endpoint to use for the operations
      endpoint: "https://capsule-proxy.capsule-system.svc:8081"

      # Toggles if TLS verification for the endpoint is performed or not
      skipTlsVerify: false

      # Key in the secret that holds the CA certificate (e.g., "ca.crt")
      caSecretKey: "ca.crt"

      # Name of the secret containing the CA certificate
      caSecretName: "capsule-proxy-tls"

      # Namespace where the CA certificate secret is located
      caSecretNamespace: "capsule-system"

      # Default ServiceAccount for global resources (GlobalTenantResource) [Cluster Scope]
      # When defined, users are required to use this ServiceAccount anywhere in the cluster
      # unless they explicitly provide their own. Once this is set, Capsule will add this ServiceAccount
      # for all GlobalTenantResources, if they don't already have a ServiceAccount defined.
      globalDefaultServiceAccount: "capsule-global-sa"

      # Namespace of the for the ServiceAccount provided by the globalDefaultServiceAccount property
      globalDefaultServiceAccountNamespace: "tenant-system"

      # Default ServiceAccount for tenant resources (TenantResource) [Namespaced Scope]
      # Applied to every TenantResource which doesn't define its own, in that resource's
      # own Namespace.
      tenantDefaultServiceAccount: "default"

`admission`

Configuration for the dynamic admission webhooks used by Capsule for mutating and validating requests. The settings are used from the static webhook configurations created during installation of Capsule and abstracted by the helm chart

manager:
  options:
    admission:
      mutating:
        client:
          caBundle: cert
          url: https://172.24.52.212:9443
        name: capsule-dynamic
      validating:
        client:
          caBundle: cert
          url: https://172.24.52.212:9443
        name: capsule-dynamic

Controller Options

Depending on the version of the Capsule Controller, the configuration options may vary. You can view the options for the latest version of the Capsule Controller or by executing the controller locally:

$ go run ./cmd/controller/ --zap-log-level 7 -h
      --cache-sync-timeout duration       The timeout used when waiting for controller cache synchronization. If unset or 0, the controller-runtime default is used.
      --client-connection-burst int32     Burst to use for interacting with kubernetes apiserver. (default 30)
      --client-connection-qps float32     QPS to use for interacting with kubernetes apiserver. (default 20)
      --configuration-name string         The CapsuleConfiguration resource name to use (default "default")
      --enable-http2                      If set, HTTP/2 will be enabled for the metrics and webhook servers
      --enable-leader-election            Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
      --enable-pprof                      Enables Pprof endpoint for profiling (not recommend in production)
      --metrics-addr string               The address the metric endpoint binds to. (default ":8080")
      --metrics-cert-key string           The name of the metrics server key file. (default "tls.key")
      --metrics-cert-name string          The name of the metrics server certificate file. (default "tls.crt")
      --metrics-cert-path string          The directory that contains the metrics server certificate.
      --metrics-secure                    If set, the metrics endpoint is served securely via HTTPS. Use --metrics-secure=false to use HTTP instead.
      --version                           Print the Capsule version and exit
      --webhook-cert-key string           The name of the webhook key file. (default "tls.key")
      --webhook-cert-name string          The name of the webhook certificate file. (default "tls.crt")
      --webhook-cert-path string          The directory that contains the webhook certificate. (default "/tmp/k8s-webhook-server/serving-certs")
      --webhook-port int                  The port the webhook server binds to. (default 9443)
      --workers int                       MaxConcurrentReconciles is the maximum number of concurrent Reconciles which can be run. (default 1)
      --zap-devel                         Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error)
      --zap-encoder encoder               Zap log encoding (one of 'json' or 'console')
      --zap-log-level level               Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', 'panic'or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
      --zap-stacktrace-level level        Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic').
      --zap-time-encoding time-encoding   Zap time encoding (one of 'epoch', 'millis', 'nano', 'iso8601', 'rfc3339' or 'rfc3339nano'). Defaults to 'epoch'.

Define additional options in the values.yaml when installing via Helm:

manager:
  extraArgs:
  - "--enable-leader-election=true"

4.1.3 - Authentication

Integrate Capsule with Authentication of your Kubernetes cluster

Capsule does not care about the authentication strategy used in the cluster and all the Kubernetes methods of authentication are supported. The only requirement to use Capsule is to assign tenant users to the group defined by userGroups option in the CapsuleConfiguration, which defaults to projectcapsule.dev.

OIDC

In the following guide, we’ll use Keycloak an Open Source Identity and Access Management server capable to authenticate users via OIDC and release JWT tokens as proof of authentication.

Configuring OIDC Server

Configure Keycloak as OIDC server:

Add a realm called caas, or use any existing realm instead
Add a group projectcapsule.dev
Add a user alice assigned to group projectcapsule.dev
Add an OIDC client called kubernetes (Public)
Add an OIDC client called kubernetes-auth (Confidential (Client Secret))

For the kubernetes client, create protocol mappers called groups and audience If everything is done correctly, now you should be able to authenticate in Keycloak and see user groups in JWT tokens. Use the following snippet to authenticate in Keycloak as alice user:

$ KEYCLOAK=sso.clastix.io
$ REALM=kubernetes-auth
$ OIDC_ISSUER=${KEYCLOAK}/realms/${REALM}

$ curl -k -s https://${OIDC_ISSUER}/protocol/openid-connect/token \
     -d grant_type=password \
     -d response_type=id_token \
     -d scope=openid \
     -d client_id=${OIDC_CLIENT_ID} \
     -d client_secret=${OIDC_CLIENT_SECRET} \
     -d username=${USERNAME} \
     -d password=${PASSWORD} | jq

The result will include an ACCESS_TOKEN, a REFRESH_TOKEN, and an ID_TOKEN. The access-token can generally be disregarded for Kubernetes. It would be used if the identity provider was managing roles and permissions for the users but that is done in Kubernetes itself with RBAC. The id-token is short lived while the refresh-token has longer expiration. The refresh-token is used to fetch a new id-token when the id-token expires.

{
   "access_token":"ACCESS_TOKEN",
   "refresh_token":"REFRESH_TOKEN",
   "id_token": "ID_TOKEN",
   "token_type":"bearer",
   "scope": "openid groups profile email"
}

To introspect the ID_TOKEN token run:

$ curl -k -s https://${OIDC_ISSUER}/protocol/openid-connect/introspect \
     -d token=${ID_TOKEN} \
     --user kubernetes-auth:${OIDC_CLIENT_SECRET} | jq

The result will be like the following:

{
  "exp": 1601323086,
  "iat": 1601322186,
  "aud": "kubernetes",
  "typ": "ID",
  "azp": "kubernetes",
  "preferred_username": "alice",
  "email_verified": false,
  "acr": "1",
  "groups": [
    "capsule.clastix.io"
  ],
  "client_id": "kubernetes",
  "username": "alice",
  "active": true
}

Configuring Kubernetes API Server

Configuring Kubernetes for OIDC Authentication requires adding several parameters to the API Server. Please, refer to the documentation for details and examples. Most likely, your kube-apiserver.yaml manifest will looks like the following:

Authentication Configuration (Recommended)

The configuration file approach allows you to configure multiple JWT authenticators, each with a unique issuer.url and issuer.discoveryURL. The configuration file even allows you to specify CEL expressions to map claims to user attributes, and to validate claims and user information.

apiVersion: apiserver.config.k8s.io/v1beta1
kind: AuthenticationConfiguration
jwt:
- issuer:
    url: https://${OIDC_ISSUER}
    audiences:
    - kubernetes
    - kubernetes-auth
    audienceMatchPolicy: MatchAny
  claimMappings:
    username:
      claim: 'email'
      prefix: ""
    groups:
      claim: 'groups'
      prefix: ""
  certificateAuthority: <PEM encoded CA certificates>

This file must be present and consistent across all kube-apiserver instances in the cluster. Add the following flag to the kube-apiserver manifest:

spec:
  containers:
  - command:
    - kube-apiserver
    ...
    - --authentication-configuration-file=/etc/kubernetes/authentication/authentication.yaml

spec:
  containers:
  - command:
    - kube-apiserver
    ...
    - --oidc-issuer-url=https://${OIDC_ISSUER}
    - --oidc-ca-file=/etc/kubernetes/oidc/ca.crt
    - --oidc-client-id=kubernetes
    - --oidc-username-claim=preferred_username
    - --oidc-groups-claim=groups
    - --oidc-username-prefix=-

KinD

As reference, here is an example of a KinD configuration for OIDC Authentication, which can be useful for local testing:

apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
nodes:
  - role: control-plane
    kubeadmConfigPatches:
     - |
       kind: ClusterConfiguration
       apiServer:
           extraArgs:
             oidc-issuer-url: https://${OIDC_ISSUER}
             oidc-username-claim: preferred_username
             oidc-client-id: kubernetes
             oidc-username-prefix: "keycloak:"
             oidc-groups-claim: groups
             oidc-groups-prefix: "keycloak:"
             enable-admission-plugins: PodNodeSelector

Configuring kubectl

There are two options to use kubectl with OIDC:

OIDC Authenticator
Use the –token option

Plugin

One way to use OIDC authentication is the use of a kubectl plugin. The Kubelogin Plugin for kubectl simplifies the process of obtaining an OIDC token and configuring kubectl to use it. Follow the link to obtain installation instructions.

kubectl oidc-login setup \
	--oidc-issuer-url=https://${OIDC_ISSUER} \
	--oidc-client-id=kubernetes-auth \
	--oidc-client-secret=${OIDC_CLIENT_SECRET}

Manual

To use the OIDC Authenticator, add an oidc user entry to your kubeconfig file:

$ kubectl config set-credentials oidc \
    --auth-provider=oidc \
    --auth-provider-arg=idp-issuer-url=https://${OIDC_ISSUER} \
    --auth-provider-arg=idp-certificate-authority=/path/to/ca.crt \
    --auth-provider-arg=client-id=kubernetes-auth \
    --auth-provider-arg=client-secret=${OIDC_CLIENT_SECRET} \
    --auth-provider-arg=refresh-token=${REFRESH_TOKEN} \
    --auth-provider-arg=id-token=${ID_TOKEN} \
    --auth-provider-arg=extra-scopes=groups

To use the --token option:

$ kubectl config set-credentials oidc --token=${ID_TOKEN}

Point the kubectl to the URL where the Kubernetes APIs Server is reachable:

$ kubectl config set-cluster mycluster \
    --server=https://kube.projectcapsule.io:6443 \
    --certificate-authority=~/.kube/ca.crt

If your APIs Server is reachable through the capsule-proxy, make sure to use the URL of the capsule-proxy.

Create a new context for the OIDC authenticated users:

$ kubectl config set-context alice-oidc@mycluster \
    --cluster=mycluster \
    --user=oidc

As user alice, you should be able to use kubectl to create some namespaces:

$ kubectl --context alice-oidc@mycluster create namespace wind-production
$ kubectl --context alice-oidc@mycluster create namespace wind-development
$ kubectl --context alice-oidc@mycluster create namespace water-marketing

Warning: once your ID_TOKEN expires, the kubectl OIDC Authenticator will attempt to refresh automatically your ID_TOKEN using the REFRESH_TOKEN. In case the OIDC uses a self signed CA certificate, make sure to specify it with the idp-certificate-authority option in your kubeconfig file, otherwise you’ll not able to refresh the tokens.

4.1.4 - OpenShift

How to install Capsule and the Capsule Proxy on OpenShift

Introduction

Capsule is a Kubernetes multi-tenancy operator that enables secure namespace-as-a-service in Kubernetes clusters. When combined with OpenShift’s robust security model, it provides an excellent platform for multi-tenant environments.

This guide demonstrates how to deploy Capsule and Capsule Proxy on OpenShift using the nonroot-v2 and restricted-v2 SecurityContextConstraint (SCC), ensuring tenant owners operate within OpenShift’s security boundaries.

Why Capsule on OpenShift

While OpenShift can already be configured for multi-tenancy (for example with Kyverno), Capsule takes it a step further and makes it easier to manage.

When people think of a multi-tenant Kubernetes cluster, they often expect one or two namespaces with few privileges. Capsule, however, is different. As a tenant owner, you can create as many namespaces as you want. RBAC is much easier because Capsule handles it, making it less error-prone. Resource quotas are not set per namespace but are spread across the whole tenant, simplifying management. Capsule Proxy also solves RBAC issues when listing cluster-wide resources. Furthermore, some operators can be installed inside a tenant by using the Capsule Proxy: add the service account as a tenant owner and set the KUBERNETES_SERVICE_HOST environment variable of the operator deployment to the Capsule Proxy URL. The operator then behaves as if it has cluster-admin access, while remaining fully confined to the tenant.

Prerequisites

Before starting, ensure you have:

OpenShift cluster with cluster-admin privileges
kubectl CLI configured
Helm 3.x installed
cert-manager installed

Limitations

The following limitations are known when using OpenShift with Capsule:

A tenant owner cannot create a namespace/project in the OpenShift GUI. This must be done with kubectl.
When copying the login token from the OpenShift GUI, the server address will always point to the Kubernetes API instead of the Capsule Proxy, so the --server flag has to be adjusted manually. Browser-based oc login --web against the Capsule Proxy works without this workaround when public paths are configured (see Capsule Proxy). An RFE has been filed with Red Hat to make this URL configurable (RFE-7592). If you have a support contract with Red Hat, consider opening a support request (SR) asking for this feature. The more requests there are, the higher the priority.

Capsule Installation

Remove the self-provisioners ClusterRoleBinding

By default, OpenShift includes a self-provisioner role and ClusterRoleBinding that allows all users to create namespaces. Capsule requires this to be removed. See the Red Hat documentation for details.

Remove the subjects from the ClusterRoleBinding:

kubectl patch clusterrolebinding.rbac self-provisioners -p '{"subjects": null}'

Also set autoupdate to false so the ClusterRoleBinding is not reverted by OpenShift.

kubectl patch clusterrolebinding.rbac self-provisioners -p '{ "metadata": { "annotations": { "rbac.authorization.kubernetes.io/autoupdate": "false" } } }'

Extend the admin role

This example extends the default Kubernetes admin role so tenant owners gain admin privileges on all namespaces within their tenant. The extension adds:

The finalizers required to create/edit resources managed by Capsule
The SCCs that tenant owners can use (in this example, restricted-v2 and nonroot-v2)

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: extend-admin-role
  labels:
    rbac.authorization.k8s.io/aggregate-to-admin: 'true'
rules:
  - verbs:
      - update
    apiGroups:
      - capsule.clastix.io
    resources:
      - '*/finalizers'
  - apiGroups:
      - security.openshift.io
    resources:
      - securitycontextconstraints
    resourceNames:
      - restricted-v2
      - nonroot-v2
    verbs:
      - 'use'

Helm Chart Values

The jobs that Capsule uses can be run with the restricted-v2 SCC, so their securityContext and podSecurityContext must be disabled. For Capsule itself, they are left enabled because Capsule runs as nonroot-v2, which is still a very secure SCC. Always set pullPolicy: Always on a multi-tenant cluster to ensure the intended images are used. The following chart values can be used:

  podSecurityContext:
    enabled: true
  securityContext:
    enabled: true
  jobs:
    podSecurityContext:
      enabled: false
    securityContext:
      enabled: false
    image:
      pullPolicy: Always
  manager:
    image:
      pullPolicy: Always

Deploy the Capsule Helm chart with (at least) these values.

etcd Encryption

If etcd encryption is enabled on your OpenShift cluster and you are using TenantResource or GlobalTenantResource to replicate ConfigMaps or Secrets, the Capsule replication webhook must be configured to skip requests from OpenShift’s storage version migrator. Without this exclusion, the migrator, which iterates over all Secrets and ConfigMaps to rotate the encryption key, will be blocked or cause reconciliation errors, because the webhook intercepts its requests.

Add the following matchCondition to the replication webhook via the Helm chart values:

webhooks:
  hooks:
    replications:
      matchConditions:
        - name: "exclude-privileged-users"
          expression: >
            !(
              request.userInfo.username in [
                "system:serviceaccount:openshift-kube-storage-version-migrator:kube-storage-version-migrator-sa"
              ]
            )

Example Tenant and TenantOwners

A minimal example tenant looks like the following:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  imagePullPolicies:
    - Always
  permissions:
    matchOwners:
      - matchLabels:
          team: devops
  priorityClasses:
    allowed:
      - openshift-user-critical
  rules:
    - audience:
        - kind: Custom
          name: "CapsuleUser"
      enforce:
        action: deny
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - "Namespace"
            labels:
              ".*openshift.io/.*":
                required: false
              "pod-security.kubernetes.io/enforce":
                required: false
            annotations:
              ".*openshift.io/.*":
                required: false
              "pod-security.kubernetes.io/enforce":
                required: false

Block openshift labels and annotations

Make sure that CapsuleUsers cannot modify OpenShift-managed labels and annotations. These metadata fields should be mutable only by OpenShift-managed components. For example, modifying the openshift.io/run-level label on a namespace can allow a user to bypass SecurityContextConstraints and create privileged workloads. Similarly, the openshift.io/sa.scc.uid-range annotation controls important SecurityContextConstraints configuration and must not be user-modifiable. The pod-security.kubernetes.io/enforce label should also be protected because it is managed by OpenShift based on the SCCs. See the Red Hat documentation for more information.

Combined with a TenantOwner resource to grant access to the tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantOwner
metadata:
  labels:
    team: devops
  name: devops
spec:
  kind: Group
  name: "oidc:org:devops:a"

More information about tenants and tenant owners can be found in the chapter Tenants.

Capsule Proxy

For Capsule Proxy, all (pod)SecurityContexts can be disabled. By disabling these, the proxy and its jobs run under the nonroot-v2 SCC. This example also enables the ProxyAllNamespaced feature, which is one of the Proxy’s most powerful capabilities. The following helm values can be used as a template:

  global:
    jobs:
      kubectl:
        securityContext:
          enabled: false
  securityContext:
    enabled: false
  podSecurityContext:
    enabled: false
  options:
    generateCertificates: false #set to false, since we are using cert-manager in .Values.certManager.generateCertificates
    enableSSL: true
    extraArgs:
      - '--feature-gates=ProxyAllNamespaced=true'
  image:
    pullPolicy: Always
  webhooks:
    enabled: true
  certManager:
    generateCertificates: true
  ingress:
    enabled: true
    annotations:
      route.openshift.io/termination: "reencrypt"
      route.openshift.io/destination-ca-certificate-secret: capsule-proxy-root-secret
    hosts:
      - host: "capsule-proxy.example.com"
        paths: ["/"]

That is all the configuration needed for Capsule Proxy.

ProxyClusterScoped and OpenShift Projects

If the ProxyClusterScoped feature gate is enabled (not used in the example above, but is is a option to enable), it is recommended to create a GlobalProxySetting for Projects. OpenShift Projects are the equivalent of Kubernetes Namespaces, and when ProxyClusterScoped is active they can no longer be listed by default.

apiVersion: capsule.clastix.io/v1beta1
kind: GlobalProxySettings
metadata:
  name: openshift-projects
spec:
  rules:
  - subjects:
    - kind: Group
      name: oidc:org:devops:a
    clusterResources:
    - apiGroups:
      - "project.openshift.io"
      resources:
      - "projects"
      operations:
      - List
      selector:
        matchLabels:
            capsule.clastix.io/tenant: solar

Hosted control planes (HyperShift)

On hosted control planes each hosted cluster runs its own OAuth server, exposed on its own route (for example, oauth-clusters-<cluster-name>.<base-domain>) with a certificate signed by that cluster’s root CA.

The oc login --web flow retrieves the OAuth server metadata from /.well-known/oauth-authorization-server on the API endpoint before any credentials exist. Since Capsule Proxy requires authentication on every request, this lookup is rejected by default and the login aborts with unsupported protocol scheme "".

Public paths solve this (available since capsule-proxy v0.14.1): requests to a public path are proxied to the upstream API server without authentication, while everything else keeps requiring one.

options:
  extraArgs:
    - '--public-paths=/.well-known/oauth-authorization-server'

With this in place oc login <capsule-proxy-host> --web completes: the browser login and the token exchange run against the hosted cluster’s OAuth server, and the API traffic goes through the proxy as usual.

Since the browser and the token exchange talk to the OAuth server directly, the client must trust the certificate of that OAuth server: import the hosted cluster’s root CA on the client first, the same procedure as when logging in against the API server directly.

Console Customization

The OpenShift console can be customized. For example, the capsule-proxy can be added as a shortcut on the top right application menu with the ConsoleLink CR:

apiVersion: console.openshift.io/v1
kind: ConsoleLink
metadata:
  name: capsule-proxy-consolelink
spec:
  applicationMenu:
    imageURL: 'https://github.com/projectcapsule/capsule/raw/main/assets/logo/capsule.svg'
    section: 'Capsule'
  href: 'https://capsule-proxy.example.com'
  location: ApplicationMenu
  text: 'Capsule Proxy Kubernetes API'

It’s also possible to add links specific for certain namespaces, which are shown on the Namespace/Project overview. These can also be tenant specific by adding a NamespaceSelector:

apiVersion: console.openshift.io/v1
kind: ConsoleLink
metadata:
  name: namespaced-consolelink-solar
spec:
  text: "solar Docs"
  href: "https://linktothesolardocs.com"
  location: "NamespaceDashboard"
  namespaceDashboard:
    namespaceSelector:
      matchExpressions:
        - key: capsule.clastix.io/tenant
          operator: In
          values:
            - solar

Also a custom logo can be provided, for example by adding the Capsule logo.

Add these config lines to the existing cluster CR Console.

kubectl create configmap console-capsule-logo --from-file capsule-logo.png -n openshift-config

apiVersion: operator.openshift.io/v1
kind: Console
metadata:
  name: cluster
spec:
  customization:
    customLogoFile:
      key: capsule-logo.png
      name: console-capsule-logo
    customProductName: Capsule OpenShift Cluster

Conclusion

You now have a fully configured Capsule and Capsule Proxy installation on OpenShift, including console customizations, and the environment is ready to hand off to development teams.

4.1.5 - Rancher

How to install Capsule and the Capsule Proxy on Rancher

The integration between Rancher and Capsule, aims to provide a multi-tenant Kubernetes service to users, enabling:

a self-service approach
access to cluster-wide resources

to end-users.

Tenant users will have the ability to access Kubernetes resources through:

Rancher UI
Rancher Shell
Kubernetes CLI

On the other side, administrators need to manage the Kubernetes clusters through Rancher.

Rancher provides a feature called Projects to segregate resources inside a common domain. At the same time Projects doesn’t provide way to segregate Kubernetes cluster-scope resources.

Capsule as a project born for creating a framework for multi-tenant platforms, integrates with Rancher Projects enhancing the experience with Tenants.

Capsule allows tenants isolation and resources control in a declarative way, while enabling a self-service experience to tenants. With Capsule Proxy users can also access cluster-wide resources, as configured by administrators at Tenant custom resource-level.

You can read in detail how the integration works and how to configure it, in the following guides.

How to integrate Rancher Projects with Capsule Tenants How to enable cluster-wide resources and Rancher shell access.

capsule rancher addon

Tenants and Projects

This guide explains how to setup the integration between Capsule and Rancher Projects.

It then explains how for the tenant user, the access to Kubernetes resources is transparent.

Pre-requisites

An authentication provider in Rancher, e.g. an OIDC identity provider
A Tenant Member Cluster Role in Rancher

Configure an identity provider for Kubernetes

You can follow this general guide to configure an OIDC authentication for Kubernetes.

For a Keycloak specific setup you can check this resources list.

Known issues

Keycloak new URLs without `/auth` makes Rancher crash

Create the Tenant Member Cluster Role

A custom Rancher Cluster Role is needed to allow Tenant users, to read cluster-scope resources and Rancher doesn’t provide e built-in Cluster Role with this tailored set of privileges.

When logged-in to the Rancher UI as administrator, from the Users & Authentication page, create a Cluster Role named Tenant Member with the following privileges:

get, list, watch operations over IngressClasses resources.
get, list, watch operations over StorageClasses resources.
get, list, watch operations over PriorityClasses resources.
get, list, watch operations over Nodes resources.
get, list, watch operations over RuntimeClasses resources.

Configuration (administration)

Tenant onboarding

When onboarding tenants, the administrator needs to create the following, in order to bind the Project with the Tenant:

In Rancher, create a Project.

In the target Kubernetes cluster, create a Tenant, with the following specification:

kind: Tenant
...
spec:
  namespaceOptions:
    additionalMetadata:
      annotations:
        field.cattle.io/projectId: ${CLUSTER_ID}:${PROJECT_ID}
      labels:
        field.cattle.io/projectId: ${PROJECT_ID}

where $CLUSTER_ID and $PROEJCT_ID can be retrieved, assuming a valid $CLUSTER_NAME, as:

CLUSTER_NAME=foo
CLUSTER_ID=$(kubectl get cluster -n fleet-default ${CLUSTER_NAME} -o jsonpath='{.status.clusterName}')
PROJECT_IDS=$(kubectl get projects -n $CLUSTER_ID -o jsonpath="{.items[*].metadata.name}")
for project_id in $PROJECT_IDS; do echo "${project_id}"; done

Create the Tenant Member Project Role

A custom Project Role is needed to allow Tenant users, with minimum set of privileges and create and delete Namespaces.

Create a Project Role named Tenant Member that inherits the privileges from the following Roles:

read-only
create-ns

Usage

When the configuration administrative tasks have been completed, the tenant users are ready to use the Kubernetes cluster transparently.

For example can create Namespaces in a self-service mode, that would be otherwise impossible with the sole use of Rancher Projects.

Namespace creation

From the tenant user perspective both CLI and the UI are valid interfaces to communicate with.

From CLI

Tenants kubectl-logs in to the OIDC provider
Tenant creates a Namespace, as a valid OIDC-discoverable user.

the Namespace is now part of both the Tenant and the Project.

As administrator, you can verify with:

kubectl get tenant ${TENANT_NAME} -o jsonpath='{.status}'
kubectl get namespace -l field.cattle.io/projectId=${PROJECT_ID}

From UI

Tenants logs in to Rancher, with a valid OIDC-discoverable user (in a valid Tenant group).
Tenant user create a valid Namespace

the Namespace is now part of both the Tenant and the Project.

As administrator, you can verify with:

kubectl get tenant ${TENANT_NAME} -o jsonpath='{.status}'
kubectl get namespace -l field.cattle.io/projectId=${PROJECT_ID}

Additional administration

Project monitoring

Before proceeding is recommended to read the official Rancher documentation about Project Monitors.

In summary, the setup is composed by a cluster-level Prometheus, Prometheus Federator via which single Project-level Prometheus federate to.

Network isolation

Before proceeding is recommended to read the official Capsule documentation about NetworkPolicy at Tenant-level`.

Network isolation and Project Monitor

As Rancher’s Project Monitor deploys the Prometheus stack in a Namespace that is not part of neither the Project nor the Tenant Namespaces, is important to apply the label selectors in the NetworkPolicy ingress rules to the Namespace created by Project Monitor.

That Project monitoring Namespace will be named as cattle-project-<PROJECT_ID>-monitoring.

For example, if the NetworkPolicy is configured to allow all ingress traffic from Namespace with label capsule.clastix.io/tenant=foo, this label is to be applied to the Project monitoring Namespace too.

Then, a NetworkPolicy can be applied at Tenant-level with Capsule GlobalTenantResources. For example it can be applied a minimal policy for the wind Tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: wind-networkpolicies
spec:
  tenantSelector:
    matchLabels:
      capsule.clastix.io/tenant: wind
  resyncPeriod: 360s
  pruningOnDelete: true
  resources:
    - namespaceSelector:
        matchLabels:
          capsule.clastix.io/tenant: wind
      rawItems:
      - apiVersion: networking.k8s.io/v1
        kind: NetworkPolicy
        metadata:
          name: wind-minimal
        spec:
          podSelector: {}
          policyTypes:
            - Ingress
            - Egress
          ingress:
            # Intra-Tenant
            - from:
              - namespaceSelector:
                  matchLabels:
                    capsule.clastix.io/tenant: wind
            # Rancher Project Monitor stack
            - from:
              - namespaceSelector:
                  matchLabels:
                    role: monitoring
            # Kubernetes nodes
            - from:
              - ipBlock:
                  cidr: 192.168.1.0/24
          egress:
            # Kubernetes DNS server
            - to:
              - namespaceSelector: {}
                podSelector:
                  matchLabels:
                    k8s-app: kube-dns
                ports:
                  - port: 53
                    protocol: UDP
            # Intra-Tenant
            - to:
              - namespaceSelector:
                  matchLabels:
                    capsule.clastix.io/tenant: wind
            # Kubernetes API server
            - to:
              - ipBlock:
                  cidr: 10.43.0.1/32
                ports:
                  - port: 443

Capsule Proxy and Rancher Projects

This guide explains how to setup the integration between Capsule Proxy and Rancher Projects.

It then explains how for the tenant user, the access to Kubernetes cluster-wide resources is transparent.

Rancher Shell and Capsule

In order to integrate the Rancher Shell with Capsule it’s needed to route the Kubernetes API requests made from the shell, via Capsule Proxy.

The capsule-rancher-addon allows the integration transparently.

Install the Capsule addon

Add the Clastix Helm repository https://clastix.github.io/charts.

By updating the cache with Clastix’s Helm repository a Helm chart named capsule-rancher-addon is available.

Install keeping attention to the following Helm values:

proxy.caSecretKey: the Secret key that contains the CA certificate used to sign the Capsule Proxy TLS certificate (it should be"ca.crt" when Capsule Proxy has been configured with certificates generated with Cert Manager).
proxy.servicePort: the port configured for the Capsule Proxy Kubernetes Service (443 in this setup).
proxy.serviceURL: the name of the Capsule Proxy Service (by default "capsule-proxy.capsule-system.svc" hen installed in the capsule-system Namespace).

Rancher Cluster Agent

In both CLI and dashboard use cases, the Cluster Agent is responsible for the two-way communication between Rancher and the downstream cluster.

In a standard setup, the Cluster Agents communicates to the API server. In this setup it will communicate with Capsule Proxy to ensure filtering of cluster-scope resources, for Tenants.

Cluster Agents accepts as arguments:

KUBERNETES_SERVICE_HOST environment variable
KUBERNETES_SERVICE_PORT environment variable

which will be set, at cluster import-time, to the values of the Capsule Proxy Service. For example:

KUBERNETES_SERVICE_HOST=capsule-proxy.capsule-system.svc
(optional) KUBERNETES_SERVICE_PORT=9001. You can skip it by installing Capsule Proxy with Helm value service.port=443.

The expected CA is the one for which the certificate is inside the kube-root-ca ConfigMap in the same Namespace of the Cluster Agent (cattle-system).

Capsule Proxy

Capsule Proxy needs to provide a x509 certificate for which the root CA is trusted by the Cluster Agent. The goal can be achieved by, either using the Kubernetes CA to sign its certificate, or by using a dedicated root CA.

With the Kubernetes root CA

Note: this can be achieved when the Kubernetes root CA keypair is accessible. For example is likely to be possibile with on-premise setup, but not with managed Kubernetes services.

With this approach Cert Manager will sign certificates with the Kubernetes root CA for which it’s needed to be provided a Secret.

kubectl create secret tls -n capsule-system kubernetes-ca-key-pair --cert=/path/to/ca.crt --key=/path/to/ca.key

When installing Capsule Proxy with Helm chart, it’s needed to specify to generate Capsule Proxy Certificates with Cert Manager with an external ClusterIssuer:

certManager.externalCA.enabled=true
certManager.externalCA.secretName=kubernetes-ca-key-pair
certManager.generateCertificates=true

and disable the job for generating the certificates without Cert Manager:

options.generateCertificates=false

Enable tenant users access cluster resources

In order to allow tenant users to list cluster-scope resources, like Nodes, Tenants need to be configured with proper proxySettings, for example:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: wind
spec:
  owners:
  - kind: User
    name: alice
    proxySettings:
    - kind: Nodes
      operations:
      - List
[...]

Also, in order to assign or filter nodes per Tenant, it’s needed labels on node in order to be selected:

kubectl label node worker-01 capsule.clastix.io/tenant=wind

and a node selector at Tenant level:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: wind
spec:
  nodeSelector:
    capsule.clastix.io/tenant: wind
[...]

The final manifest is:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: wind
spec:
  owners:
  - kind: User
    name: alice
    proxySettings:
    - kind: Node
      operations:
      - List
  nodeSelector:
    capsule.clastix.io/tenant: wind

The same appplies for:

Nodes
StorageClasses
IngressClasses
PriorityClasses

More on this in the official documentation.

Configure OIDC authentication with Keycloak

Pre-requisites

Keycloak realm for Rancher
Rancher OIDC authentication provider

Keycloak realm for Rancher

These instructions is specific to a setup made with Keycloak as an OIDC identity provider.

Mappers

Add to userinfo Group Membership type, claim name groups
Add to userinfo Audience type, claim name client audience
Add to userinfo, full group path, Group Membership type, claim name full_group_path

Rancher OIDC authentication provider

Configure an OIDC authentication provider, with Client with issuer, return URLs specific to the Keycloak setup.

Use old and Rancher-standard paths with /auth subpath (see issues below).
Add custom paths, remove /auth subpath in return and issuer URLs.

Configuration

Configure Tenant users

In Rancher, configure OIDC authentication with Keycloak to use with Rancher.
In Keycloak, Create a Group in the rancher Realm: capsule.clastix.io.
In Keycloak, Create a User in the rancher Realm, member of capsule.clastix.io Group.
In the Kubernetes target cluster, update the CapsuleConfiguration by adding the "keycloakoidc_group://capsule.clastix.io" Kubernetes Group.
Login to Rancher with Keycloak with the new user.
In Rancher as an administrator, set the user custom role with get of Cluster.
In Rancher as an administrator, add the Rancher user ID of the just-logged in user as Owner of a Tenant.
(optional) configure proxySettings for the Tenant to enable tenant users to access cluster-wide resources.

4.1.6 - Managed Kubernetes

Capsule on managed Kubernetes offerings

Capsule Operator can be easily installed on a Managed Kubernetes Service. Since you do not have access to the Kubernetes APIs Server, you should check with the provider of the service:

the default cluster-admin ClusterRole is accessible the following Admission Webhooks are enabled on the APIs Server:

PodNodeSelector
LimitRanger
ResourceQuota
MutatingAdmissionWebhook
ValidatingAdmissionWebhook

AWS EKS

This is an example of how to install AWS EKS cluster and one user manged by Capsule. It is based on Using IAM Groups to manage Kubernetes access

Create EKS cluster:

export AWS_DEFAULT_REGION="eu-west-1"
export AWS_ACCESS_KEY_ID="xxxxx"
export AWS_SECRET_ACCESS_KEY="xxxxx"

eksctl create cluster \
--name=test-k8s \
--managed \
--node-type=t3.small \
--node-volume-size=20 \
--kubeconfig=kubeconfig.conf

Create AWS User alice using CloudFormation, create AWS access files and kubeconfig for such user:

cat > cf.yml << EOF
Parameters:
  ClusterName:
    Type: String
Resources:
  UserAlice:
    Type: AWS::IAM::User
    Properties:
      UserName: !Sub "alice-${ClusterName}"
      Policies:
      - PolicyName: !Sub "alice-${ClusterName}-policy"
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
          - Sid: AllowAssumeOrganizationAccountRole
            Effect: Allow
            Action: sts:AssumeRole
            Resource: !GetAtt RoleAlice.Arn
  AccessKeyAlice:
    Type: AWS::IAM::AccessKey
    Properties:
      UserName: !Ref UserAlice
  RoleAlice:
    Type: AWS::IAM::Role
    Properties:
      Description: !Sub "IAM role for the alice-${ClusterName} user"
      RoleName: !Sub "alice-${ClusterName}"
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Principal:
            AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
          Action: sts:AssumeRole
Outputs:
  RoleAliceArn:
    Description: The ARN of the Alice IAM Role
    Value: !GetAtt RoleAlice.Arn
    Export:
      Name:
        Fn::Sub: "${AWS::StackName}-RoleAliceArn"
  AccessKeyAlice:
    Description: The AccessKey for Alice user
    Value: !Ref AccessKeyAlice
    Export:
      Name:
        Fn::Sub: "${AWS::StackName}-AccessKeyAlice"
  SecretAccessKeyAlice:
    Description: The SecretAccessKey for Alice user
    Value: !GetAtt AccessKeyAlice.SecretAccessKey
    Export:
      Name:
        Fn::Sub: "${AWS::StackName}-SecretAccessKeyAlice"
EOF

eval aws cloudformation deploy --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides "ClusterName=test-k8s" \
  --stack-name "test-k8s-users" --template-file cf.yml

AWS_CLOUDFORMATION_DETAILS=$(aws cloudformation describe-stacks --stack-name "test-k8s-users")
ALICE_ROLE_ARN=$(echo "${AWS_CLOUDFORMATION_DETAILS}" | jq -r ".Stacks[0].Outputs[] | select(.OutputKey==\"RoleAliceArn\") .OutputValue")
ALICE_USER_ACCESSKEY=$(echo "${AWS_CLOUDFORMATION_DETAILS}" | jq -r ".Stacks[0].Outputs[] | select(.OutputKey==\"AccessKeyAlice\") .OutputValue")
ALICE_USER_SECRETACCESSKEY=$(echo "${AWS_CLOUDFORMATION_DETAILS}" | jq -r ".Stacks[0].Outputs[] | select(.OutputKey==\"SecretAccessKeyAlice\") .OutputValue")

eksctl create iamidentitymapping --cluster="test-k8s" --arn="${ALICE_ROLE_ARN}" --username alice --group capsule.clastix.io

cat > aws_config << EOF
[profile alice]
role_arn=${ALICE_ROLE_ARN}
source_profile=alice
EOF

cat > aws_credentials << EOF
[alice]
aws_access_key_id=${ALICE_USER_ACCESSKEY}
aws_secret_access_key=${ALICE_USER_SECRETACCESSKEY}
EOF

eksctl utils write-kubeconfig --cluster=test-k8s --kubeconfig="kubeconfig-alice.conf"
cat >> kubeconfig-alice.conf << EOF
      - name: AWS_PROFILE
        value: alice
      - name: AWS_CONFIG_FILE
        value: aws_config
      - name: AWS_SHARED_CREDENTIALS_FILE
        value: aws_credentials
EOF

Export “admin” kubeconfig to be able to install Capsule:

export KUBECONFIG=kubeconfig.conf

Install Capsule and create a tenant where alice has ownership. Use the default Tenant example:

kubectl apply -f https://raw.githubusercontent.com/clastix/capsule/master/config/samples/capsule_v1beta1_tenant.yaml

Based on the tenant configuration above the user alice should be able to create namespace. Switch to a new terminal and try to create a namespace as user alice:

# Unset AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY if defined
unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
kubectl create namespace test --kubeconfig="kubeconfig-alice.conf"

Azure AKS

This reference implementation introduces the recommended starting (baseline) infrastructure architecture for implementing a multi-tenancy Azure AKS cluster using Capsule. See CoAKS.

Charmed Kubernetes

Canonical Charmed Kubernetes is a Kubernetes distribution coming with out-of-the-box tools that support deployments and operational management and make microservice development easier. Combined with Capsule, Charmed Kubernetes allows users to further reduce the operational overhead of Kubernetes setup and management.

The Charm package for Capsule is available to Charmed Kubernetes users via Charmhub.io.

4.2 - Best Practices

Best Practices when running Capsule in production

This is general advice you should consider before making Kubernetes Distribution consideration. They are partly relevant for Multi-Tenancy with Capsule.

For a complete configuration combining the workload and networking policies, see the combined Tenant baseline.

Authentication

User authentication for the platform should be handled via a central OIDC-compatible identity provider system (e.g., Keycloak, Azure AD, Okta, or any other OIDC-compliant provider). The rationale is that other central platform components, such as ArgoCD, Grafana, Headlamp, or Harbor, should also integrate with the same authentication mechanism. This enables a unified login experience and reduces administrative complexity in managing users and permissions.

Capsule relies on native Kubernetes RBAC, so it’s important to consider how the Kubernetes API handles user authentication.

OCI Pull-Cache

By default, Kubernetes clusters pull images directly from upstream registries like docker.io, quay.io, ghcr.io, or gcr.io. In production environments, this can lead to issues, especially because Docker Hub enforces rate limits that may cause image pull failures with just a few nodes or frequent deployments (e.g., when pods are rescheduled).

To ensure availability, performance, and control over container images, it’s essential to provide an on-premise OCI mirror. This mirror should be configured via the CRI (Container Runtime Interface) by defining it as a mirror endpoint in registries.conf for default registries (e.g., docker.io). This way, all nodes automatically benefit from caching without requiring developers to change image URLs.

Secrets Management

In more complex environments with multiple clusters and applications, managing secrets manually via YAML or Helm is no longer practical. Instead, a centralized secrets management system should be established, such as Vault, AWS Secrets Manager, Azure Key Vault, or the CNCF project OpenBao (formerly the Vault community fork).

To integrate these external secret stores with Kubernetes, the External Secrets Operator (ESO) is a recommended solution. It automatically syncs defined secrets from external sources as Kubernetes secrets, and supports dynamic rotation, access control, and auditing.

If no external secret store is available, there should at least be a secure way to store sensitive data in Git. In our ecosystem, we provide a solution based on SOPS (Secrets OPerationS) for this use case; called the sops-operator.

👉 Demonstration

4.2.1 - Tenants

A complete Tenant combining the workload and networking best-practice baseline

Use this complete Tenant as a starting point for application namespaces. It combines the compatible baseline policies from Workloads and Networking into one ordered spec.rules list. All namespaces in this Tenant receive the same baseline.

Included Policies

Area	Combined configuration
Resources	Default CPU, memory, and ephemeral-storage requests; remove CPU limits; match memory limits to requests; default ephemeral-storage limits.
Capacity	Share one illustrative quota across all Tenant namespaces.
QoS	Deny `BestEffort` after resource mutation.
Placement	Use the default scheduler, exclude control-plane nodes, deny control-plane and wildcard tolerations, and reject Pods created with `nodeName`.
Workload types	Deny DaemonSets.
Pod isolation	Set `hostUsers: false` and manage the namespace’s Pod Security labels at `restricted`.
Images	Allow the approved registry path and require `Always` pull policy.
Priority	Default to and allow only `tenant-standard`.
Services	Allow only `ClusterIP` and deny every external IP.
Namespace trust	Deny adding or changing `company.com/system` on Tenant namespaces.

Before Applying

Replace solar, solar-owner, and registry.example.com/solar with your Tenant, owner, and approved registry path. Publish immutable image references in that registry; the example’s registry matcher does not require a digest.
Size requests, storage defaults, and the shared quota from measured usage and available capacity. The numbers below illustrate a profile, not a universal production budget. There is deliberately no CPU-limit quota.
Create the non-preempting tenant-standard PriorityClass before admitting workloads that use this Tenant. The Tenant references the class; it does not create it.
Use Linux nodes with user namespace support. Review kernel, runtime, and volume support before enabling hostUsers: false. For incompatible workloads, define a separate, reviewed namespace profile.
Set all three Pod Security version labels to a version supported by your cluster. This example pins them to v1.36. Applications must supply security contexts that satisfy restricted; these rules do not generate those fields.

Reference

Explore the ordered rules below. Select a card to open its documentation, or view, copy, and download the complete Tenant YAML, including ownership and PriorityClass settings.

View complete Tenant YAML Owners, priority settings, and all rules

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  # Default and allow the tenant-standard PriorityClass.
  priorityClasses:
    default: tenant-standard
    allowed: [tenant-standard]
  rules:

    # One shared budget across all namespaces in this Tenant.
    - quota:
        - name: application-budget
          hard:
            requests.cpu: "20"
            requests.memory: 40Gi
            limits.memory: 40Gi
            requests.ephemeral-storage: 100Gi
            limits.ephemeral-storage: 200Gi
            pods: "100"
            services: "100"

    # Apply placement and user-namespace settings on Pod creation.
    - mutate:
        - action: merge
          workloads:
            security:
              hostUsers: false
            placement:
              affinity:
                nodeAffinity:
                  requiredDuringSchedulingIgnoredDuringExecution:
                    nodeSelectorTerms:
                      - matchExpressions:
                          - key: node-role.kubernetes.io/control-plane
                            operator: DoesNotExist
                          - key: node-role.kubernetes.io/master
                            operator: DoesNotExist
      enforce:
        action: allow
        workloads:
          targets: [pod]
          placement:
            schedulers:
              - exact: [default-scheduler]

    # Defaults for application containers, sidecars, and init containers.
    - enforce:
        action: deny
        workloads:
          targets: [pod/containers, pod/initcontainers]
          resources:
            requests:
              cpu:
                policy: Default
                value: 250m
              memory:
                policy: Default
                value: 512Mi
              ephemeral-storage:
                policy: Default
                value: 1Gi
            limits:
              cpu:
                policy: Remove
              memory:
                policy: MatchRequest
              ephemeral-storage:
                policy: Default
                value: 2Gi

    # Normalize limits when a workload supplies Pod-level resources.
    - enforce:
        action: deny
        workloads:
          targets: [pod]
          resources:
            limits:
              cpu:
                policy: Remove
              memory:
                policy: MatchRequest

    # Deny BestEffort Pods after resource mutation.
    - enforce:
        action: deny
        workloads:
          targets: [pod]
          qosClasses: [BestEffort]

    # Reject control-plane and wildcard tolerations.
    - enforce:
        action: deny
        workloads:
          targets: [pod]
          placement:
            tolerations:
              - key:
                  exact:
                    - node-role.kubernetes.io/control-plane
                    - node-role.kubernetes.io/master
              - key: {exp: '^$'}
                operators: [Exists]

    # Check direct assignment only at creation; allow scheduler-bound updates.
    - enforce:
        action: deny
        conditions:
          - name: direct-node-assignment
            expression: >-
              request.operation == 'CREATE' && has(object.spec.nodeName) && object.spec.nodeName != ''
        workloads:
          targets: [pod]

    # Keep kind-level denials separate from workload property checks.
    - enforce:
        action: deny
        workloads:
          targets: [daemonset]

    # Allow approved images with Always pull policy.
    - enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
            - pod/initcontainers
            - pod/ephemeralcontainers
            - pod/volumes
          registries:
            - exp: '^registry\.example\.com/solar/.+$'
              policy: [Always]

    # Allow only ClusterIP Services.
    - enforce:
        action: allow
        services:
          types: [ClusterIP]

    # Deny all Service external IPs.
    - enforce:
        action: deny
        services:
          externalIPs: {}

    # Apply and reconcile Pod Security labels on the Namespace itself.
    - enforce:
        action: allow
        metadata:
          - apiGroups: [v1]
            kinds: [Namespace]
            labels:
              pod-security.kubernetes.io/enforce:
                managed: restricted
                required: true
                values:
                  - exact: [restricted]
              pod-security.kubernetes.io/enforce-version:
                managed: v1.36
                required: true
                values:
                  - exact: [v1.36]
              pod-security.kubernetes.io/warn:
                managed: restricted
                required: true
                values:
                  - exact: [restricted]
              pod-security.kubernetes.io/warn-version:
                managed: v1.36
                required: true
                values:
                  - exact: [v1.36]
              pod-security.kubernetes.io/audit:
                managed: restricted
                required: true
                values:
                  - exact: [restricted]
              pod-security.kubernetes.io/audit-version:
                managed: v1.36
                required: true
                values:
                  - exact: [v1.36]

    # Reserve this trusted-platform label for administrator-managed namespaces.
    - enforce:
        action: deny
        metadata:
          - apiGroups: [v1]
            kinds: [Namespace]
            labels:
              company.com/system:
                values:
                  - exp: ".*"

Namespace rulesIn declaration order

One shared budget across all namespaces in this Tenant.

- quota:
    - name: application-budget
      hard:
        requests.cpu: "20"
        requests.memory: 40Gi
        limits.memory: 40Gi
        requests.ephemeral-storage: 100Gi
        limits.ephemeral-storage: 200Gi
        pods: "100"
        services: "100"

Apply placement and user-namespace settings on Pod creation.

- mutate:
    - action: merge
      workloads:
        security:
          hostUsers: false
        placement:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: node-role.kubernetes.io/control-plane
                        operator: DoesNotExist
                      - key: node-role.kubernetes.io/master
                        operator: DoesNotExist
  enforce:
    action: allow
    workloads:
      targets: [pod]
      placement:
        schedulers:
          - exact: [default-scheduler]

Defaults for application containers, sidecars, and init containers.

- enforce:
    action: deny
    workloads:
      targets: [pod/containers, pod/initcontainers]
      resources:
        requests:
          cpu:
            policy: Default
            value: 250m
          memory:
            policy: Default
            value: 512Mi
          ephemeral-storage:
            policy: Default
            value: 1Gi
        limits:
          cpu:
            policy: Remove
          memory:
            policy: MatchRequest
          ephemeral-storage:
            policy: Default
            value: 2Gi

Normalize limits when a workload supplies Pod-level resources.

- enforce:
    action: deny
    workloads:
      targets: [pod]
      resources:
        limits:
          cpu:
            policy: Remove
          memory:
            policy: MatchRequest

Deny BestEffort Pods after resource mutation.

- enforce:
    action: deny
    workloads:
      targets: [pod]
      qosClasses: [BestEffort]

Reject control-plane and wildcard tolerations.

- enforce:
    action: deny
    workloads:
      targets: [pod]
      placement:
        tolerations:
          - key:
              exact:
                - node-role.kubernetes.io/control-plane
                - node-role.kubernetes.io/master
          - key: {exp: '^$'}
            operators: [Exists]

Check direct assignment only at creation; allow scheduler-bound updates.

- enforce:
    action: deny
    conditions:
      - name: direct-node-assignment
        expression: >-
          request.operation == 'CREATE' && has(object.spec.nodeName) && object.spec.nodeName != ''
    workloads:
      targets: [pod]

Keep kind-level denials separate from workload property checks.

- enforce:
    action: deny
    workloads:
      targets: [daemonset]

Allow approved images with Always pull policy.

- enforce:
    action: allow
    workloads:
      targets:
        - pod/containers
        - pod/initcontainers
        - pod/ephemeralcontainers
        - pod/volumes
      registries:
        - exp: '^registry\.example\.com/solar/.+$'
          policy: [Always]

Allow only ClusterIP Services.

- enforce:
    action: allow
    services:
      types: [ClusterIP]

Deny all Service external IPs.

- enforce:
    action: deny
    services:
      externalIPs: {}

Apply and reconcile Pod Security labels on the Namespace itself.

- enforce:
    action: allow
    metadata:
      - apiGroups: [v1]
        kinds: [Namespace]
        labels:
          pod-security.kubernetes.io/enforce:
            managed: restricted
            required: true
            values:
              - exact: [restricted]
          pod-security.kubernetes.io/enforce-version:
            managed: v1.36
            required: true
            values:
              - exact: [v1.36]
          pod-security.kubernetes.io/warn:
            managed: restricted
            required: true
            values:
              - exact: [restricted]
          pod-security.kubernetes.io/warn-version:
            managed: v1.36
            required: true
            values:
              - exact: [v1.36]
          pod-security.kubernetes.io/audit:
            managed: restricted
            required: true
            values:
              - exact: [restricted]
          pod-security.kubernetes.io/audit-version:
            managed: v1.36
            required: true
            values:
              - exact: [v1.36]

Reserve this trusted-platform label for administrator-managed namespaces.

- enforce:
    action: deny
    metadata:
      - apiGroups: [v1]
        kinds: [Namespace]
        labels:
          company.com/system:
            values:
              - exp: ".*"

How the Policies Combine

Capsule applies mutations before validation. On Pod creation, the resource rules supply missing requests and storage limits, remove CPU limits, and align memory limits with requests. Explicit requests and storage limits survive the Default policies. Placement mutation adds the control-plane exclusions and sets hostUsers: false; enforcement then checks the resulting Pod.

The workload policies target Pods, including controller-created Pods. The DaemonSet rule separately rejects the controller itself. Add explicit controller targets if you also want resource or image checks on saved controller templates. Keep the DaemonSet denial and conditional direct-assignment denial separate: adding workload properties to a targets-only rule changes its meaning.

The quota rule generates solar-application-budget, a GlobalResourceQuota shared across this Tenant’s namespaces. It bounds aggregate consumption; the resource defaults do not impose maximum requests or reserve node capacity by themselves. Retain capacity for failover and system services as described in capacity planning.

The restricted Pod Security labels are managed values. The platform-label denial checks added or changed metadata; unchanged existing values are skipped on updates. Remove any existing tenant-controlled company.com/system labels before using them as a network trust boundary.

The memory-overcommit and ExternalName examples on the individual pages are deliberate exceptions. Add them only under reviewed namespace selectors, with the selecting labels under platform control. Review rule order and scope when composing further rules so a later decision does not unintentionally relax the baseline.

Complete the Platform Setup

Apply the native NetworkPolicy GlobalTenantResource alongside this Tenant. It distributes default-deny, same-Tenant, DNS, and platform-ingress policies; NetworkPolicies are not embedded in this Tenant manifest. Review its CNI requirements and the permissions needed to prevent tenants from adding policies that widen access.

Use the additional admission policies for requirements such as PDB validation, emptyDir.sizeLimit, debugging access, and certificate issuer restrictions. Those examples require their respective admission mechanisms and are not installed by this Tenant.

Configure authentication, registry mirrors, and secret management at platform level. Review existing workloads and Services before rollout; admission policies do not remove previously admitted resources.

4.2.2 - Workloads

Define workload profiles and plan capacity for resource pressure and eviction

Use namespace rules to establish workload defaults and validate the resulting Pods. Every Capsule policy example below is a complete Tenant manifest with an owner and spec.rules. Replace solar and solar-owner with your Tenant name and owner identity. Each example is an independent configuration; to adopt multiple policies, combine their spec.rules entries into one Tenant manifest. Namespace selectors can give different namespaces different profiles; keep the labels selecting mandatory profiles under platform control.

See the combined Tenant baseline for resource, placement, image, and networking policies together. Add the PDB policy below for namespace profiles that must support routine maintenance.

Resource Management

Resource settings are both a scheduling contract and a runtime safety boundary. Kubernetes schedules a Pod from its resource requests, while the kubelet and container runtime enforce its resource limits. Correct values help the scheduler place Pods safely, keep one workload from affecting its neighbours, and give autoscalers meaningful data.

CPU and memory need different treatment:

CPU is compressible. Under contention, a container receives CPU time in proportion to its request; without contention, it can use spare CPU. A CPU limit is a hard ceiling and can throttle an otherwise healthy application.
Memory is not compressible. A memory request informs scheduling, while a memory limit protects the node and neighbouring workloads. A container that reaches its memory limit can be terminated with an out-of-memory (OOM) kill.

See Resource Management for Pods and Containers for the complete Kubernetes behavior.

Common Approaches

Teams commonly use one of the following patterns:

No requests or limits results in BestEffort QoS. The scheduler reserves no CPU or memory, and any memory use exceeds its memory request. This is suitable only for disposable workloads.
Requests only results in Burstable QoS. The scheduler accounts for the workload and it can use spare capacity. Without a memory limit, however, a leak can consume all memory available on the node.
Requests lower than limits results in Burstable QoS and allows controlled bursts. CPU can be throttled at its limit, and memory above the request is more exposed during node pressure.
Requests equal limits for CPU and memory results in Guaranteed QoS. It reserves the full declared capacity and reduces memory-pressure exposure, but prevents CPU bursting and can lower cluster utilization.
A CPU request without a CPU limit, plus an equal memory request and limit, results in Burstable QoS. It preserves CPU bursting while reserving and bounding memory. This is the recommended baseline for general workloads.

A CPU limit can be useful when a workload needs a fixed CPU ceiling. Exclusive CPU allocation under the CPU Manager static policy additionally requires eligible Guaranteed containers with integer CPU requests. Guaranteed should not be the automatic target for every production Pod.

Recommended Baseline

Recommended resource policy

For every application container, sidecar, and init container:

set a realistic CPU request;
do not set a CPU limit; and
set a memory request and memory limit to the same value.

For example:

apiVersion: v1
kind: Pod
metadata:
  name: api
spec:
  containers:
    - name: api
      image: registry.example.com/solar/api:1.0.0
      imagePullPolicy: Always
      resources:
        requests:
          cpu: 250m
          memory: 512Mi
        limits:
          memory: 512Mi

The baseline can be visualized as two independent resource decisions that produce one Pod QoS class:

flowchart TB
  workload["Container resources"]
  cpu["CPU<br/>request: 250m<br/>limit: none"]
  memory["Memory<br/>request: 512Mi<br/>limit: 512Mi"]
  cpuResult["Scheduled CPU share<br/>Can use spare CPU"]
  memoryResult["512Mi scheduled<br/>Bounded at 512Mi"]
  qos["Pod QoS: Burstable"]

  workload --> cpu
  workload --> memory
  cpu --> cpuResult
  memory --> memoryResult
  cpuResult --> qos
  memoryResult --> qos

This gives the scheduler an honest view of the CPU needed under contention and the maximum memory the container can consume. It also lets the container use idle CPU above 250m without being throttled by an artificial ceiling.

Declare the memory request explicitly. If a limit is present without a request and no admission mechanism supplies a default, Kubernetes copies the limit to the request. Relying on that default makes the intended scheduling contract less obvious to readers and policy tools.

Treat a different memory request and limit as a deliberate overcommit policy, not as a default. It can improve density, but a container using more memory than its request is more vulnerable to eviction when its node is under memory pressure.

Apply this baseline with workload resource rules:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets: [pod/containers, pod/initcontainers]
          resources:
            requests:
              cpu:
                policy: Default
                value: 250m
              memory:
                policy: Default
                value: 512Mi
            limits:
              cpu:
                policy: Remove
              memory:
                policy: MatchRequest
    # Normalize Pod-level limits too, when a workload supplies Pod resources.
    - enforce:
        action: deny
        workloads:
          targets: [pod]
          resources:
            limits:
              cpu:
                policy: Remove
              memory:
                policy: MatchRequest

Replace solar-owner with your owner identity and size the defaults for your workloads. Default preserves existing requests, Remove deletes CPU limits, and MatchRequest sets memory limits to the corresponding requests. These resource policies mutate Pod creation before enforcement; ordinary Pod updates are validated without resource mutation. They also cover controller-created Pods. Add explicit controller targets if you want template validation and resource mutation when controllers are saved.

Size the Values Step by Step

Measure representative usage. Include normal traffic, startup, background work, and known peak periods. For a new workload, begin with a conservative estimate and revise it after collecting metrics.
Choose the CPU request. Set it to the CPU share the container needs to make reliable progress during contention. Avoid both a token request that overstates available cluster capacity and an oversized request that leaves nodes unnecessarily unschedulable. CPU-based Horizontal Pod Autoscaling also depends on an accurate CPU request.
Choose the memory boundary. Use the observed high-water mark plus enough headroom for normal variation. Set both the memory request and limit to this value. A repeatedly OOMKilled container needs investigation and usually a corrected value; increasing the limit blindly can hide a memory leak.
Cover the whole Pod. Size application containers, sidecars, and init containers. One unconfigured container changes the Pod’s effective resource behavior and can undermine the policy.
Observe and revise. Check utilization, OOM events, pending Pods, and application latency after deployment. Revisit values when the workload or traffic profile changes. A Vertical Pod Autoscaler in recommendation mode can provide useful starting data without changing Pods automatically.

QoS Classes

Kubernetes derives a Pod’s Quality of Service (QoS) class from CPU and memory settings. With container-level resources:

Guaranteed: every container has equal, non-zero CPU and memory requests and limits.
Burstable: some CPU or memory resources are declared, but the Pod does not meet the Guaranteed criteria.
BestEffort: no CPU or memory requests or limits are declared.

Where supported, Pod-level resources also participate in classification. Ephemeral-storage requests and limits do not determine QoS.

The recommended baseline intentionally produces Burstable Pods because CPU limits are absent. Equal memory requests and limits make memory reservations explicit, but a QoS class does not guarantee protection from eviction. See Eviction and Capacity Planning for the resource-specific decisions, including disk pressure.

Capsule can allow, deny, or audit QoS classes. A good tenant baseline is to reject BestEffort Pods and permit the intentional Burstable pattern described above.

For example, reject BestEffort without forcing all workloads into Guaranteed:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets: [pod]
          qosClasses: [BestEffort]

Resource mutations run before this check. When combined with the recommended resource baseline, its defaults supply requests before QoS is evaluated. This QoS-only example does not add requests or limits, ensure every container is sized, or provide a disk or inode reservation.

PriorityClasses

QoS describes how resources are declared. A PriorityClass describes how important one Pod is relative to another. These mechanisms are independent: a Burstable Pod can have a higher priority than a Guaranteed Pod.

A PriorityClass is cluster-scoped and maps a name to an integer. Higher-priority Pods are considered earlier in the scheduling queue. By default, a pending higher-priority Pod can also preempt lower-priority Pods when that would make it schedulable. The kubelet also considers priority during node-pressure eviction. Priority does not reserve capacity and does not give a running container more CPU time; requests and limits still define that behavior.

Define a small set of classes with clear purposes. For example, a standard tenant class can be non-preempting:

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: tenant-standard
value: 1000
preemptionPolicy: Never
globalDefault: false
description: "Default priority for tenant application workloads"

Workload owners select an approved class on the Pod template:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      priorityClassName: tenant-standard
      containers:
        - name: api
          image: registry.example.com/solar/api:1.0.0
          imagePullPolicy: Always
          resources:
            requests:
              cpu: 250m
              memory: 512Mi
            limits:
              memory: 512Mi

Use elevated, preempting classes only for workloads whose unavailability would prevent the platform or other applications from recovering. Do not use them to compensate for undersized requests or insufficient cluster capacity. In a multi-tenant cluster, restrict which PriorityClasses each tenant may select; otherwise a tenant could displace another tenant’s Pods. Capsule can allow classes and assign a tenant default, and Kubernetes ResourceQuota can limit consumption by PriorityClass.

Platform Guardrails

Application owners should size their workloads, while the platform supplies safe fallbacks and validation:

Use Capsule workload resource rules to remove CPU limits, default requests, and keep memory limits aligned with memory requests.
Use a LimitRange for simple namespace defaults and minimum or maximum values. Do not configure a default CPU limit, because that would reverse the recommended policy.
Use ResourceQuota, Global Resource Quotas, or Resource Pools to control aggregate tenant consumption. Quotas are capacity boundaries; they do not replace accurate per-container sizing.
Monitor for missing requests, CPU throttling, OOM kills, and sustained usage close to the configured boundaries. Defaults should be a temporary safety net, not permanent sizing by accident.

Before deploying a workload, verify that every container has a CPU request, no CPU limit, and equal memory request and limit; that the expected QoS class is Burstable; and that any selected PriorityClass is approved for the tenant.

Overprovisioning and Overcommit

The terms overprovisioning and overcommit are sometimes used interchangeably, although they describe different capacity strategies:

Workload overcommit lets declared or potential workload demand exceed immediately available capacity. It improves utilization when workloads do not peak together, but creates contention when they do.
Cluster overprovisioning keeps deliberate spare capacity available. It costs more, but lets new replicas schedule while an autoscaler adds nodes or while the platform recovers from a failure.

The right approach depends on the resource. CPU contention delays work; memory contention can terminate Pods. They should therefore have different defaults.

CPU Overcommit

Kubernetes places Pods according to CPU requests, so the total CPU requests on a node normally cannot exceed its allocatable CPU. With no CPU limits, however, the potential CPU demand of the running workloads can be much higher than the node’s capacity. This is intentional CPU overcommit: containers can use otherwise idle CPU, and their requests determine their relative share when several containers need CPU at the same time.

The recommended baseline supports CPU overcommit safely when requests remain honest. Do not lower requests merely to fit more Pods onto a node. An undersized request makes scheduling and capacity reports misleading, reduces the workload’s CPU share during contention, and distorts CPU-utilization-based Horizontal Pod Autoscaling.

There is no universal safe CPU overcommit ratio. Choose it from workload measurements and service objectives, and account for correlated events such as traffic peaks, rollouts, batch schedules, and node failures. Watch for:

sustained node CPU saturation and growing run queues;
application latency or processing backlogs;
Horizontal Pod Autoscalers remaining at their maximum replica count; and
pending Pods when node capacity or autoscaler limits are exhausted.

Memory Overcommit

Memory is overcommitted when the sum of memory requests fits on a node but the workloads can collectively use more memory than the node provides. A common way to create this condition is to set memory requests lower than memory limits. The scheduler considers the lower requests, while each container can grow toward its higher limit.

Our baseline avoids deliberate application memory overcommit by setting each memory request equal to its limit. Node services, runtime overhead, and failover still need their own capacity budget; this setting alone does not prevent node memory pressure.

Use memory overcommit only as an explicit exception for workloads that are well understood, restart-tolerant, and unlikely to peak together. Keep a hard memory limit, retain node headroom, and expect increased eviction risk whenever usage exceeds the request. Avoid memory overcommit for critical or stateful workloads, uncertain memory profiles, and applications with expensive recovery.

Monitor the exception using both scheduling and runtime signals:

memory requests compared with node allocatable memory;
memory limits compared with node capacity;
working-set usage compared with requests and limits; and
node memory pressure, evictions, and container OOM kills.

For a measured exception, this complete Tenant includes the recommended baseline followed by a rule selecting only batch namespaces. The last rule replaces the earlier MatchRequest limit policy for regular and init container memory in namespaces labeled example.com/capacity-profile: batch:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets: [pod/containers, pod/initcontainers]
          resources:
            requests:
              cpu:
                policy: Default
                value: 250m
              memory:
                policy: Default
                value: 512Mi
            limits:
              cpu:
                policy: Remove
              memory:
                policy: MatchRequest
    # Normalize Pod-level limits too, when a workload supplies Pod resources.
    - enforce:
        action: deny
        workloads:
          targets: [pod]
          resources:
            limits:
              cpu:
                policy: Remove
              memory:
                policy: MatchRequest
    # Override container memory limits only in batch namespaces.
    - namespaceSelector:
        matchLabels:
          example.com/capacity-profile: batch
      enforce:
        action: deny
        workloads:
          targets: [pod/containers, pod/initcontainers]
          resources:
            limits:
              memory:
                policy: Ratio
                value: "2"

A missing memory limit becomes twice its positive request; an explicit limit above that ratio is denied. Earlier request defaults and CPU policies remain in effect. If workloads also use Pod-level memory resources, configure that limit policy for the intended burst too. A later Preserve policy instead clears earlier management and constraints for that same target, field, and resource; it does not undo Kubernetes defaults. See resource policy ordering.

Cluster Headroom

Even well-sized workloads need room for failover, rolling updates, sudden scale-outs, and the delay before new nodes become ready. Define the required headroom from those recovery objectives rather than relying on capacity that happens to be idle.

Headroom can be provided by keeping additional nodes running or, when using Cluster Autoscaler, by scheduling low-priority placeholder Pods with resource requests. Real workloads preempt the placeholders and use the reserved space immediately. The displaced placeholder Pods become pending and can trigger a replacement node. The Cluster Autoscaler overprovisioning guide describes this pattern.

Placeholder Pods need a dedicated priority below application Pods, but not below the autoscaler’s expendable-Pod cutoff if they should trigger scale-up. The application class must permit preemption to displace them: tenant-standard with preemptionPolicy: Never above cannot do that. Test the replacement and scale-down behavior, and keep application state out of placeholders. A PriorityClass only decides which Pods give way; it does not create capacity by itself.

Recommended Strategy

Overprovisioning baseline

Overcommit CPU through accurate requests and no CPU limits.
Do not overcommit memory by default; keep memory request equal to limit.
Maintain explicit cluster headroom for failover and node scale-up time.
Use quotas to bound tenant reservations and autoscaling to add capacity.
Treat higher memory overcommit or priority-based reservations as measured, monitored exceptions.

Review reservation ratios separately for each node pool and tenant. Aggregate cluster averages can hide a saturated pool or a single noisy tenant. Revisit the ratios after workload growth, node-type changes, autoscaler changes, or a capacity-related incident.

Eviction and Capacity Planning

Plan for the resource that can run out. A CPU/memory QoS class is neither a capacity reservation for other resources nor a general eviction ranking.

Distinguish the failure mechanisms

Situation	What happens	What to plan
CPU contention	Work slows; CPU limits can throttle. CPU saturation is not a kubelet node-pressure eviction signal.	Realistic CPU requests, latency targets, and scale-out capacity.
Container memory limit or node OOM	The kernel can kill a container; its restart policy determines recovery. This differs from a kubelet evicting the Pod.	Measured memory bounds, system reservations, and investigation of `OOMKilled`.
Node memory pressure	Kubelet may evict Pods to reclaim memory.	Memory requests versus actual usage, priorities, and node headroom.
Disk space pressure	Kubelet first attempts reclamation and can then evict Pods. CPU/memory QoS is not protection.	Storage requests, logs, writable layers, image caches, and filesystem free space.
Inode or PID pressure	Pods can be evicted even with spare memory or disk bytes. Priority matters; these resources have no Pod requests.	File counts, process counts, and per-node limits.
Local ephemeral-storage limit exceeded	With storage accounting enabled, the Pod can be evicted even without node-wide disk pressure.	Per-container storage limits and bounded scratch volumes.

For threshold handling and reclamation, see node-pressure eviction. CPU and memory limit behavior is described under resource management; local storage accounting has its own requirements and limits.

For a node-wide OOM, Linux process selection also uses kubelet-provided OOM score adjustments influenced by QoS and memory requests. That is a different mechanism from pressure-eviction ranking. See the kubelet OOM policy.

Scheduler preemption is a separate decision that makes room for a pending higher-priority Pod; it can happen without node pressure. preemptionPolicy: Never prevents a Pod from preempting others, not from being a victim. See priority and preemption.

A drain normally uses the Eviction API, which checks PodDisruptionBudgets. Kubelet node-pressure eviction does not honor a PDB; hard thresholds can terminate immediately. A PDB limits some voluntary disruptions and creates no spare capacity. See disruption budgets and the PDB maintenance policy below.

Why the pressure signal matters more than QoS

For memory, eviction ranking considers usage above the request first, then lower Pod priority, then usage minus the request. Disk-byte ranking uses storage consumption and requests for the affected filesystem, not CPU/memory QoS. Filesystem layout determines which usage is counted. Inode/PID pressure cannot compare against a Pod request for those resources. See the Kubernetes eviction ordering.

Consider two Pods at the same priority on a node using one filesystem for local Pod storage:

Pod	CPU/memory QoS	Ephemeral-storage request	Local storage use
A	`Guaranteed`	None	`5Gi`
B	`Burstable`	`8Gi`	`4Gi`

Under disk-byte pressure, A exceeds its storage request while B does not. With valid usage statistics, A ranks before B despite being Guaranteed. Adding CPU limits to B would not fix the full disk. Monitor the actual filesystem signal; inode exhaustion is a different case from running out of bytes.

Likewise, under memory pressure, a high-priority Pod using more than its memory request can rank before a lower-priority Pod still within its request. Pods within requests can still be evicted if the node cannot recover, for example when system daemons exceed their reservations.

Include local storage in the namespace profile

Requests for ephemeral-storage inform scheduling. Limits cover accounted container logs, writable layers, and disk-backed emptyDir usage, with Pod-level accounting rules. They are not an instantaneous filesystem quota. Memory-backed emptyDir consumes memory instead. Persistent volumes need their own storage planning. See local ephemeral storage and memory-backed volumes.

For workloads with local scratch data, add measured storage defaults:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets: [pod/containers, pod/initcontainers]
          resources:
            requests:
              ephemeral-storage:
                policy: Default
                value: 1Gi
            limits:
              ephemeral-storage:
                policy: Default
                value: 2Gi

These are fallback values, not maximum allowed sizes: Default preserves explicit settings. Use quota to bound namespace consumption, configure log rotation, and size image-cache space and inodes separately. Ephemeral-storage policies cannot target Pod-level spec.resources.

Capacity planning checklist

Start with Node Allocatable. Budget system and Kubernetes daemons and eviction margins before application capacity. Reserve overhead for the runtime and platform Pods as well. See system resource reservations.
Track reservations and usage separately. Requests determine placement; actual peaks determine contention. Keep memory overcommit deliberate, and include storage bytes, free inodes, and PID availability in monitoring.
Plan per eligible node pool. Affinity, taints, zones, and volume topology can make free capacity elsewhere unusable. For example, three eligible nodes with 24Gi allocatable memory each have only 48Gi after one fails, before failover or rollout demand. 60Gi of requests would not fit that scenario.
Budget recovery time. Allow for replica failover, rollout surge, image pulls, and autoscaler delay. Priority and PDBs cannot compensate for missing capacity. Test that replacement Pods fit the surviving topology.
Diagnose the actual signal. Use Pod events/status and node conditions to distinguish Evicted, OOMKilled, and FailedScheduling. kubectl top covers CPU/memory usage; it does not establish disk, inode, or PID headroom.

Pod Disruption

A PodDisruptionBudget (PDB) protects application availability during voluntary evictions. An overly restrictive budget can also stop a shared node from being emptied, delaying node upgrades or replacement. One tenant’s Pod can therefore hold up maintenance affecting other tenants. PDBs also constrain Cluster Autoscaler scale-down, so a blocked eviction can leave otherwise removable nodes running.

Common causes and the corresponding Capsule policies are:

Cause	Operational effect	Capsule policy
`maxUnavailable: 0`, `minAvailable: "100%"`, or an integer minimum equal to the replica count	No healthy replica can be voluntarily evicted	Require `evictableReplicas.min: 1`
Scaling down makes a percentage budget round up to all replicas	A previously usable budget permits no disruption; for example, `minAvailable: "75%"` allows one at four replicas but none at three	Check replica changes on the controller and `/scale` against the same minimum
Multiple PDBs select a Pod	The Eviction API can reject the request with HTTP 500	Set `allowOverlap: false`
An unhealthy Running Pod is covered by `IfHealthyBudget` while its application is already below budget	An unhealthy Pod can hold up a drain	Allow only `AlwaysAllow` in `unhealthyPodEvictionPolicies`

The Eviction API checks the live budget. AlwaysAllow permits unhealthy Running Pods to leave even when that budget is unsatisfied; healthy Pods still obey it. An omitted PDB eviction policy means IfHealthyBudget. See the Kubernetes unhealthy eviction policies.

For a namespace profile that must support routine maintenance, use this complete Tenant example. It checks Pods and Deployment/StatefulSet templates, including PDB writes and supported replica changes:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          example.com/profile: production
      enforce:
        action: allow
        workloads:
          targets:
            - pod
            - deployment
            - statefulset
          disruptionBudgets:
            allowOverlap: false
            evictableReplicas:
              min: 1
            unhealthyPodEvictionPolicies:
              - AlwaysAllow

The rule applies only to namespaces labeled example.com/profile: production. Keep that label under platform control; omit namespaceSelector to apply the policy throughout the Tenant. PDB authors must explicitly set unhealthyPodEvictionPolicy: AlwaysAllow; Capsule validates it without adding the field. No PDB is required or created. For the complete field and target reference, see PDB enforcement.

Choose disruption allowances with the application owner. min: 1 permits the only replica of a singleton to be evicted when its PDB allows that, so it cannot promise uninterrupted service. Run enough replicas and place them across the failure domains you need to tolerate. Add evictableReplicas.max only when you also need an upper bound on the per-controller configured allowance. Scaling to zero is exempt from the replica-count check; this policy does not impose a minimum workload size.

Capacity still matters

Capsule checks configuration at the desired replica count, not the live status.disruptionsAllowed. After an eviction, a replacement may remain Pending because of capacity, affinity, taints, or volume placement. Further evictions can then be blocked despite a compliant policy. Reserve headroom and test recovery on the nodes that will remain available.

Before maintenance, inspect PDB currentHealthy, desiredHealthy, and disruptionsAllowed alongside Pod readiness and scheduling events. Review all matching selectors when an eviction fails. A policy change does not repair existing objects, and concurrent admissions across separate resources can race. Coordinate an application or budget repair with its owner when necessary; deleting a conflicting PDB remains possible.

PDBs cover voluntary API evictions. Kubelet node-pressure eviction does not honor them, including under disk pressure. Keep resource sizing and eviction planning alongside these policies; a PDB does not provide spare resources or protect a node from exhaustion.

Placement Constraints

Use workload mutations to establish scheduling restrictions and enforcement to reject incompatible requests. The effective namespace profile is applied to new Pods, including Pods created by workload controllers.

Disallow Control Planes

Keep control-plane node role labels and taints managed by the platform. For ordinary tenant applications, add required node affinity that excludes both the current and legacy control-plane labels:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: merge
          workloads:
            placement:
              affinity:
                nodeAffinity:
                  requiredDuringSchedulingIgnoredDuringExecution:
                    nodeSelectorTerms:
                      - matchExpressions:
                          - key: node-role.kubernetes.io/control-plane
                            operator: DoesNotExist
                          - key: node-role.kubernetes.io/master
                            operator: DoesNotExist
      enforce:
        action: allow
        workloads:
          targets: [pod]
          placement:
            schedulers:
              - exact: [default-scheduler]
    - enforce:
        action: deny
        workloads:
          targets: [pod]
          placement:
            tolerations:
              - key:
                  exact:
                    - node-role.kubernetes.io/control-plane
                    - node-role.kubernetes.io/master
              # An empty key with Exists tolerates every taint key.
              - key: {exp: '^$'}
                operators: [Exists]
    - enforce:
        action: deny
        conditions:
          - name: direct-node-assignment
            expression: >-
              request.operation == 'CREATE' && has(object.spec.nodeName) && object.spec.nodeName != ''              
        workloads:
          targets: [pod]

Affinity merge combines the exclusions with existing required constraints; it does not replace an application’s other node restrictions. The scheduler allow-list keeps this example on the native scheduler. The last rule rejects Pods created with nodeName, which would bypass scheduling, while allowing normal updates after the scheduler binds a Pod. Tolerations permit scheduling onto tainted nodes; they do not select nodes or prevent kubelet pressure evictions. See taints and tolerations.

These settings apply to newly admitted Pods, not existing ones. Keep node label/taint writes and pods/binding permissions restricted to platform components. If you use another approved scheduler or different control-plane labels, adapt the profile to that setup.

Workload Types

Reserve DaemonSets for platform-managed agents when application tenants should not run one Pod on every eligible node:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets: [daemonset]

With no workload properties, targets apply the action to the kind itself. This rejects DaemonSet creation and main-resource updates in the Tenant’s namespaces; existing objects are not removed. Other kinds are unaffected. Keep this rule separate from resource or image policies, because adding those properties changes targets into policy scopes. See workload targets.

Security

User Namespaces

For supported Linux node pools, mutate[].workloads.security.hostUsers: false maps users inside a Pod to different host IDs. Root inside the container therefore does not become host root. Enable this only after verifying kernel, runtime, and volume filesystem support; host namespaces and some volume configurations are incompatible. See Kubernetes user namespaces.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: merge
          workloads:
            security:
              hostUsers: false

This sets the value on Pod creation, even if the submitted value was true. It does not change the container’s runAsUser or migrate existing Pods. Apply it to namespaces whose workloads and nodes support user namespaces, and use Pod Security Admission for the remaining security-context requirements. Capsule does not configure node sysctls or runtime prerequisites.

Pod Security Standards

Use Pod Security Admission to apply the privileged, baseline, or restricted standards. Start new application profiles at restricted; use warning and audit modes to assess existing workloads before enforcing a stricter profile.

Distribute the namespace labels through managed metadata rules. Explicitly select Namespace; wildcard kinds do not opt into Namespace metadata management. This example pins the policy to v1.36; choose a policy version supported by your cluster and review it during upgrades.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        metadata:
          - apiGroups: [v1]
            kinds: [Namespace]
            labels:
              pod-security.kubernetes.io/enforce:
                managed: restricted
                required: true
                values:
                  - exact: [restricted]
              pod-security.kubernetes.io/enforce-version:
                managed: v1.36
                required: true
                values:
                  - exact: [v1.36]
              pod-security.kubernetes.io/warn:
                managed: restricted
                required: true
                values:
                  - exact: [restricted]
              pod-security.kubernetes.io/warn-version:
                managed: v1.36
                required: true
                values:
                  - exact: [v1.36]
              pod-security.kubernetes.io/audit:
                managed: restricted
                required: true
                values:
                  - exact: [restricted]
              pod-security.kubernetes.io/audit-version:
                managed: v1.36
                required: true
                values:
                  - exact: [v1.36]

Capsule sets managed values during admission and reconciles them on existing namespaces for this unconditional rule. A user’s conflicting label value is overwritten. Kubernetes then checks admitted Pods against the selected standard; changing namespace labels does not retroactively evict running Pods. For different profiles, use namespace selectors whose profile labels are controlled by the platform. Prefer narrow, reviewed exceptions over allowing tenants to select privileged themselves.

Container Images

Use an approved registry and immutable image references. When requiring imagePullPolicy: Always, remember that the runtime can still reuse cached layers; Always does not mean downloading every layer again. Private-image credential verification also depends on the kubelet’s KubeletEnsureSecretPulledImages support and verification policy. Review the image pull and credential verification behavior for your Kubernetes version.

For a namespace profile that requires registry checks on each container start:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
            - pod/initcontainers
            - pod/ephemeralcontainers
            - pod/volumes
          registries:
            - exp: '^registry\.example\.com/solar/.+$'
              policy: [Always]

This permits only references in that registry path with Always pull policy; it validates the submitted policy rather than changing it. Registry allow-listing is separate from registry authentication and image signing. Budget image-cache storage and registry availability as part of capacity planning.

4.2.3 - Networking

Control Service exposure and network isolation in tenant namespaces

Services

For ordinary application namespaces, start with ClusterIP Services and expose applications through a platform-managed Gateway or Ingress. Give namespaces that need other Service types an explicit profile through Service enforcement rules.

See the combined Tenant baseline for Service restrictions together with workload and namespace policies.

Recommended Baseline

Setting	Default policy for application namespaces	Reason
`ClusterIP`	Allow, including headless Services.	Supports internal service discovery. NetworkPolicy still needs to control who can connect.
`NodePort`	Deny.	Opens a port on node addresses, creating an exposure path outside the managed application entry point.
`LoadBalancer`	Deny unless approved.	Can provision externally reachable infrastructure and normally allocates node ports too.
`ExternalName`	Deny unless approved.	Lets a tenant alias a Service name to another DNS name; review destinations before enabling it.
`spec.externalIPs`	Deny on every Service type.	Lets a Service claim traffic for an IP independently of its type.

These are recommended rules to install, not Capsule’s automatic defaults. See the Kubernetes Service types for their networking behavior. The upstream external IP deprecation proposal explains the traffic-interception risk of user-selected external IPs.

This complete Tenant applies the baseline to all its namespaces:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        services:
          types: [ClusterIP]
    - enforce:
        action: deny
        services:
          externalIPs: {}

Replace solar and solar-owner with your Tenant name and owner identity. The type allow-list rejects NodePort, LoadBalancer, and ExternalName. Under action: deny, externalIPs: {} rejects every supplied external IP, including IPv4 and IPv6, while allowing Services without external IPs. This separate check matters because a ClusterIP Service can also set spec.externalIPs.

These checks run on Service creation and updates. They do not remove existing Services or close existing exposure when a rule is installed. Review existing Services during rollout; subsequent updates must satisfy the effective rules.

Scope Exceptions to a Namespace Profile

Keep the baseline for ordinary namespaces and add narrowly scoped exceptions for workloads that need them. Protect the namespace labels selecting an exception so tenant users cannot grant themselves a more permissive profile. See Namespace metadata rules.

For example, this complete Tenant retains the baseline and permits an ExternalName pointing to database.example.com only in namespaces labeled example.com/service-profile: external-database:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        services:
          types: [ClusterIP]
    - enforce:
        action: deny
        services:
          externalIPs: {}
    - namespaceSelector:
        matchLabels:
          example.com/service-profile: external-database
      enforce:
        action: allow
        services:
          types: [ExternalName]
          externalNames:
            hostnames:
              - exact: [database.example.com]

In selected namespaces, ClusterIP remains allowed and the approved ExternalName is added. Other hostnames, NodePort, LoadBalancer, and all external IPs remain denied. Non-selected namespaces retain the baseline. Hostname constraints alone do not allow the ExternalName type; the exception must include both the type and its hostname constraint.

The examples are alternative configurations of the same Tenant. Combine the desired spec.rules entries into one manifest when adopting other policies. Review rule order and scope: later matching rules can override earlier decisions for the same Service value.

An ExternalName allow-list checks the submitted DNS name, not its resolved IP addresses. It does not restrict connections made directly to other names or IPs. Keep egress controls in place and consider who controls the approved DNS name. See ExternalName Services.

Approve LoadBalancers Deliberately

Some workloads need a dedicated load balancer. Treat that as a separate namespace profile, with a reviewed provider configuration:

Control the implementation and exposure. Allow LoadBalancer only in selected namespaces. Require the appropriate class or provider annotations for an internal load balancer where needed. Use metadata rules for annotations and enforcement conditions to deny incompatible Service fields. An allowed address range alone does not make a load balancer private.
Account for node ports. Denying the NodePort Service type does not stop a permitted LoadBalancer from allocating node ports. If the implementation routes directly to Pods, require spec.allocateLoadBalancerNodePorts: false and reject explicit spec.ports[].nodePort values. Changing that flag on an existing Service does not release previously allocated ports; remove them explicitly. Other implementations need node ports, so constrain access with node firewalls and provider controls. See load balancer node-port allocation.
Require the intended client networks. Use explicit spec.loadBalancerSourceRanges and verify the provider enforces them; an unsupported provider can ignore this field. Capsule’s LoadBalancer CIDR checks validate both spec.loadBalancerIP and spec.loadBalancerSourceRanges. They do not require source ranges specifically when loadBalancerIP is present, so use a condition-based deny rule if missing source ranges must be rejected. See the Service API.
Control provider-specific options and consumption. Review annotations that select public exposure, address pools, or firewall behavior. Bound load balancer counts with quotas and review the resulting addresses and firewall rules after provisioning.

Keep Network and Access Controls in Place

Service admission controls configuration; it does not establish tenant network isolation. Apply NetworkPolicies with a supporting CNI and test the actual ingress and egress paths. Address translation can affect which source or destination a policy sees, particularly with load balancers. See NetworkPolicy behavior.

Also review these paths outside the Service type rules:

Restrict direct writes to EndpointSlice, legacy Endpoints, and services/status through RBAC. Endpoint writes can redirect Service traffic; keep them with trusted controllers unless a workload explicitly needs that capability. See RBAC guidance on traffic redirection.
Apply Pod Security Standards to restrict hostNetwork and hostPort. Blocking NodePort Services does not restrict these Pod settings.
Control who can publish Gateway routes or Ingresses and which hosts they may use. A permitted ClusterIP backend can still be deliberately published by those controllers; see Ingress enforcement.

Network-Policies

Use GlobalTenantResource to distribute a common network profile across Tenant namespaces. Start with default-deny ingress and egress, then allow the flows applications need. The CNI must support and enforce native Kubernetes NetworkPolicies.

Native NetworkPolicies with GlobalTenantResource

Native NetworkPolicies are additive: an additional allow policy can widen the effective access. Protecting the replicated baseline alone is therefore not a mandatory isolation boundary. Restrict tenant permissions to create additional NetworkPolicies when the platform must own that boundary, or use an administrator-enforced policy mechanism supported by your CNI. Both source egress and destination ingress must allow a connection. See the Kubernetes policy-combination rules.

This example creates four policies per selected namespace. Verify replication readiness and test same-Tenant access, cross-Tenant denial, DNS, and platform ingress before relying on the profile. Replication readiness confirms resource delivery; it does not prove the CNI has applied the rules. Tune resyncPeriod for the number of namespaces you manage.

This complete, cluster-scoped GlobalTenantResource creates four native networking.k8s.io/v1 NetworkPolicies in every namespace of every selected Tenant. It has no metadata.namespace. tenantSelector: {} selects all Tenants; replace it with a label selector to roll out the profile gradually. Use resources[].namespaceSelector when only some namespaces need this profile. The replication identity must have permission to manage NetworkPolicies in the destination namespaces.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-network-baseline
spec:
  scope: Namespace
  tenantSelector: {}
  resyncPeriod: 60s
  resources:
    - policy:
        creation: Owner
        protect: true
        deletion: Remove
      rawItems:
        - apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: capsule-default-deny
          spec:
            podSelector: {}
            policyTypes: [Ingress, Egress]

        - apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: capsule-allow-same-tenant
          spec:
            podSelector: {}
            policyTypes: [Ingress, Egress]
            ingress:
              - from:
                  - namespaceSelector:
                      matchLabels:
                        capsule.clastix.io/tenant: "{{tenant.name}}"
            egress:
              - to:
                  - namespaceSelector:
                      matchLabels:
                        capsule.clastix.io/tenant: "{{tenant.name}}"

        - apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: capsule-allow-dns
          spec:
            podSelector: {}
            policyTypes: [Egress]
            egress:
              - to:
                  # Both selectors apply to the same destination Pod.
                  - namespaceSelector:
                      matchLabels:
                        kubernetes.io/metadata.name: kube-system
                    podSelector:
                      matchLabels:
                        k8s-app: kube-dns
                ports:
                  - protocol: UDP
                    port: 53
                  - protocol: TCP
                    port: 53

        - apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: capsule-allow-platform-ingress
          spec:
            podSelector: {}
            policyTypes: [Ingress]
            ingress:
              - from:
                  - namespaceSelector:
                      matchLabels:
                        company.com/system: "true"

Capsule renders {{tenant.name}} separately for each destination Tenant using Fast Templates, and sets the destination namespace on each generated policy. The selector uses Capsule’s namespace membership label, capsule.clastix.io/tenant.

The resulting profile permits:

Traffic	Allowance in this profile
Between Pods in the same Tenant, including the same namespace	Ingress and egress on all ports.
From Tenant Pods to cluster DNS	TCP and UDP port 53 to the selected DNS Pods.
From Pods in trusted platform namespaces to Tenant Pods	Ingress on all ports; narrow the Pod selectors and ports for your monitoring or ingress controllers.
To another Tenant, the internet, or other platform destinations	No allowance from this profile; add a reviewed policy for required flows.

For namespace-level isolation within a Tenant, replace the same-tenant peers with podSelector: {} so they select Pods only in the policy’s namespace. Install the namespace metadata restriction below on every application Tenant before trusting company.com/system.

Check your DNS deployment’s namespace and labels; the example assumes kube-system and k8s-app: kube-dns. See DNS troubleshooting. NodeLocal DNSCache or host-networked DNS may need different CNI-specific allowances. Test DNS over both UDP and TCP instead of relying on a fixed Service IP. The platform workloads also need an egress policy permitting their connections if their own namespaces isolate egress.

protect: true protects the replicated objects from tenant modification or deletion through Capsule admission. It does not stop someone from creating another NetworkPolicy. With deletion: Remove, removing the replication or a destination from its selection removes the policies it created; plan those changes as changes to network access.

Deny Namespace Metadata

The platform-ingress policy trusts namespaces carrying company.com/system: "true". Reserve that label for administrator-managed platform namespaces outside application Tenants. Restrict who can label those namespaces through RBAC.

Use a metadata enforcement rule to deny the label on application Tenant namespaces. This complete Tenant replaces the legacy namespaceOptions.forbiddenLabels configuration:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        metadata:
          - apiGroups: [v1]
            kinds: [Namespace]
            labels:
              company.com/system:
                values:
                  - exp: ".*"

The explicit Namespace kind targets the Namespace object itself. The rule rejects the label on creation and when an update adds or changes its value, including "true", "false", and an empty value. Namespaces without the label and updates removing it remain allowed. Apply this rule to every application Tenant and merge it with the Service rules above when using both profiles. Keep it unconditional so changing a namespace label cannot opt out of the restriction. Review later rules that could override this denial.

Metadata validation skips unchanged values on updates. This rule therefore does not remove an existing label or reject an update that leaves its value unchanged. Remove any existing tenant-controlled company.com/system labels before enabling the platform-ingress allowance.

4.2.4 - Admission Policies

Recommended Admission Policies to enforce best practices in multi-tenant environments.

As Capsule we try to provide a secure multi-tenant environment out of the box, there are however some additional Admission Policies you should consider to enforce best practices in your cluster. Since Capsule only covers the core multi-tenancy features, such as Namespaces, Resource Quotas, Network Policies, and Container Registries, Classes, you should consider using an additional Admission Controller to enforce best practices on workloads and other resources.

Custom

Create custom Policies and reuse data provided via Tenant Status to enforce your own rules.

Owner Validation

Class Validation

Let’s say we have the following namespaced ObjectBucketClaim resource:

apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: admission-class
  namespace: solar-production
  finalizers:
    - objectbucket.io/finalizer
  labels:
    bucket-provisioner: openshift-storage.ceph.rook.io-bucket
spec:
  additionalConfig:
    maxSize: 2G
  bucketName: test-some-uid
  generateBucketName: test
  objectBucketName: obc-test-test
  storageClassName: ocs-storagecluster-ceph-rgw

However since we are allowing Tenant Users to create these ObjectBucketClaims we might want to consider validating the storageClassName field to ensure that only allowed StorageClasses are used.

---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-tenant-class
spec:
  validationFailureAction: Enforce
  rules:
  - name: restrict-storage-class
    context:
      - name: classes
        apiCall:
          urlPath: "/apis/capsule.clastix.io/v1beta2/tenants"
          jmesPath: "items[?contains(status.namespaces, '{{ request.namespace }}')].status.classes | [0]"

      - name: storageClass
        variable:
          jmesPath: "request.object.spec.storageClassName || 'NONE'"
    match:
      resources:
        kinds:
        - ObjectBucketClaim
        namespaceSelector:
          matchExpressions:
          - key: capsule.clastix.io/tenant
            operator: Exists
    validate:
      message: "storageclass {{ storageClass }} is not allowed in tenant ({{classes.storage}})"
      deny:
        conditions:
          - key:   "{{classes.storage}}"
            operator: AnyNotIn
            value:  "{{ storageClass }}"

Workloads

Policies to harden workloads running in a multi-tenant environment.

Pod Disruption Budgets

Pod Disruption Budgets (PDBs) are a way to limit the number of concurrent disruptions to your Pods. In multi-tenant environments, it is recommended to enforce the usage of PDBs to ensure that tenants do not accidentally or maliciously block cluster operations.

MaxUnavailable

A PodDisruptionBudget which sets its maxUnavailable value to zero prevents all voluntary evictions including Node drains which may impact maintenance tasks. This policy enforces that if a PodDisruptionBudget specifies the maxUnavailable field it must be greater than zero.

---
# Source: https://kyverno.io/policies/other/pdb-maxunavailable/pdb-maxunavailable/
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: pdb-maxunavailable
  annotations:
    policies.kyverno.io/title: PodDisruptionBudget maxUnavailable Non-Zero
    policies.kyverno.io/category: Other
    kyverno.io/kyverno-version: 1.9.0
    kyverno.io/kubernetes-version: "1.24"
    policies.kyverno.io/subject: PodDisruptionBudget
    policies.kyverno.io/description: >-
      A PodDisruptionBudget which sets its maxUnavailable value to zero prevents
      all voluntary evictions including Node drains which may impact maintenance tasks.
      This policy enforces that if a PodDisruptionBudget specifies the maxUnavailable field
      it must be greater than zero.      
spec:
  validationFailureAction: Enforce
  background: false
  rules:
    - name: pdb-maxunavailable
      match:
        any:
          - resources:
              kinds:
                - PodDisruptionBudget
              namespaceSelector:
                matchExpressions:
                - key: capsule.clastix.io/tenant
                  operator: Exists
      validate:
        message: "The value of maxUnavailable must be greater than zero."
        pattern:
          spec:
            =(maxUnavailable): ">0"

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: pdb-maxunavailable
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
    - apiGroups: ["policy"]
      apiVersions: ["v1"]
      operations: ["CREATE", "UPDATE"]
      resources: ["poddisruptionbudgets"]
    namespaceSelector:
      matchExpressions:
      - key: capsule.clastix.io/tenant
        operator: Exists
  validations:
  - expression: |
      !has(object.spec.maxUnavailable) ||
      string(object.spec.maxUnavailable).contains('%') ||
      object.spec.maxUnavailable > 0      
    message: "The value of maxUnavailable must be greater than zero or a percentage."
    reason: Invalid
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: pdb-maxunavailable-binding
spec:
  policyName: pdb-maxunavailable
  validationActions: ["Deny"]

MinAvailable

When a Pod controller which can run multiple replicas is subject to an active PodDisruptionBudget, if the replicas field has a value equal to the minAvailable value of the PodDisruptionBudget it may prevent voluntary disruptions including Node drains which may impact routine maintenance tasks and disrupt operations. This policy checks incoming Deployments and StatefulSets which have a matching PodDisruptionBudget to ensure these two values do not match.

---
# Source: https://kyverno.io/policies/other/pdb-minavailable/pdb-minavailable/
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: pdb-minavailable-check
  annotations:
    policies.kyverno.io/title: Check PodDisruptionBudget minAvailable
    policies.kyverno.io/category: Other
    kyverno.io/kyverno-version: 1.9.0
    kyverno.io/kubernetes-version: "1.24"
    policies.kyverno.io/subject: PodDisruptionBudget, Deployment, StatefulSet
    policies.kyverno.io/description: >-
      When a Pod controller which can run multiple replicas is subject to an active PodDisruptionBudget,
      if the replicas field has a value equal to the minAvailable value of the PodDisruptionBudget
      it may prevent voluntary disruptions including Node drains which may impact routine maintenance
      tasks and disrupt operations. This policy checks incoming Deployments and StatefulSets which have
      a matching PodDisruptionBudget to ensure these two values do not match.      
spec:
  validationFailureAction: Enforce
  background: false
  rules:
    - name: pdb-minavailable
      match:
        any:
          - resources:
              kinds:
                - Deployment
                - StatefulSet
              namespaceSelector:
                matchExpressions:
                - key: capsule.clastix.io/tenant
                  operator: Exists
      preconditions:
        all:
        - key: "{{`{{ request.operation | 'BACKGROUND' }}`}}"
          operator: AnyIn
          value:
          - CREATE
          - UPDATE
        - key: "{{`{{ request.object.spec.replicas | '1' }}`}}"
          operator: GreaterThan
          value: 0
      context:
        - name: minavailable
          apiCall:
            urlPath: "/apis/policy/v1/namespaces/{{`{{ request.namespace }}`}}/poddisruptionbudgets"
            jmesPath: "items[?label_match(spec.selector.matchLabels, `{{`{{ request.object.spec.template.metadata.labels }}`}}`)] | [0].spec.minAvailable | default(`0`)"
      validate:
        message: >-
          The matching PodDisruptionBudget for this resource has its minAvailable value equal to the replica count
          which is not permitted.          
        deny:
          conditions:
            any:
              - key: "{{`{{ request.object.spec.replicas }}`}}"
                operator: Equals
                value: "{{`{{ minavailable }}`}}"

Deployment Replicas higher than PDB

PodDisruptionBudget resources are useful to ensuring minimum availability is maintained at all times.Introducing a PDB where there are already matching Pod controllers may pose a problem if the author is unaware of the existing replica count. This policy ensures that the minAvailable value is not greater or equal to the replica count of any matching existing Deployment. If other Pod controllers should also be included in this check, additional rules may be added to the policy which match those controllers.

---
# Source: https://kyverno.io/policies/other/deployment-replicas-higher-than-pdb/deployment-replicas-higher-than-pdb/
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: deployment-replicas-higher-than-pdb
  annotations:
    policies.kyverno.io/title: Ensure Deployment Replicas Higher Than PodDisruptionBudget
    policies.kyverno.io/category: Other
    policies.kyverno.io/subject: PodDisruptionBudget, Deployment
    kyverno.io/kyverno-version: 1.11.4
    kyverno.io/kubernetes-version: "1.27"
    policies.kyverno.io/description: >-
      PodDisruptionBudget resources are useful to ensuring minimum availability is maintained at all times.
      Introducing a PDB where there are already matching Pod controllers may pose a problem if the author
      is unaware of the existing replica count. This policy ensures that the minAvailable value is not
      greater or equal to the replica count of any matching existing Deployment. If other Pod controllers
      should also be included in this check, additional rules may be added to the policy which match those
      controllers.      
spec:
  validationFailureAction: Enforce
  background: true
  rules:
  - name: deployment-replicas-greater-minAvailable
    match:
      any:
      - resources:
          kinds:
          - PodDisruptionBudget
          operations:
          - CREATE
          - UPDATE
          namespaceSelector:
            matchExpressions:
            - key: capsule.clastix.io/tenant
              operator: Exists
    context:
    - name: deploymentreplicas
      apiCall:
        jmesPath: items[?label_match(`{{`{{ request.object.spec.selector.matchLabels }}`}}`, spec.template.metadata.labels)] || `[]`
        urlPath: /apis/apps/v1/namespaces/{{`{{request.namespace}}`}}/deployments
    preconditions:
      all:
      - key: '{{`{{ length(deploymentreplicas) }}`}}'
        operator: GreaterThan
        value: 0
      - key: '{{`{{ request.object.spec.minAvailable || "" }}`}}'
        operator: NotEquals
        value: ''
    validate:
      message: >-
        PodDisruption budget minAvailable ({{`{{ request.object.spec.minAvailable }}`}}) cannot be
        greater than or equal to the replica count of any matching existing Deployment.
        There are {{`{{ length(deploymentreplicas) }}`}} Deployments which match this labelSelector
        having {{`{{ deploymentreplicas[*].spec.replicas }}`}} replicas.        
      foreach:
        - list: deploymentreplicas
          deny:
            conditions:
              all:
              - key: "{{`{{ request.object.spec.minAvailable }}`}}"
                operator: GreaterThanOrEquals
                value: "{{`{{ element.spec.replicas }}`}}"

CNPG Cluster

---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: pdb-cnpg-cluster-validation
  annotations:
    policies.kyverno.io/title: Check PodDisruptionBudget minAvailable for cnpgCluster
    policies.kyverno.io/category: Other
    kyverno.io/kyverno-version: 1.9.0
    kyverno.io/kubernetes-version: "1.24"
    policies.kyverno.io/subject: PodDisruptionBudget, Cluster
    policies.kyverno.io/description: >-
      When a Pod controller which can run multiple replicas is subject to an active PodDisruptionBudget,
      if the replicas field has a value equal to the minAvailable value of the PodDisruptionBudget
      it may prevent voluntary disruptions including Node drains which may impact routine maintenance
      tasks and disrupt operations. This policy checks incoming CNPG Clusters and their .spec.enablePDB setting.      
spec:
  validationFailureAction: Enforce
  background: false
  rules:
    - name: pdb-cnpg-cluster-validation
      match:
        any:
          - resources:
              kinds:
                - postgresql.cnpg.io/v1/Cluster
              namespaceSelector:
                matchExpressions:
                - key: capsule.clastix.io/tenant
                  operator: Exists
      preconditions:
        any:
        - key: "{{request.operation || 'BACKGROUND'}}"
          operator: AnyIn
          value:
          - CREATE
          - UPDATE
      validate:
        message: >-
          Set `.spec.enablePDB` to `false` for CNPG Clusters when the number of instances is lower than 2.          
        deny:
          conditions:
            all:
              - key: "{{request.object.spec.enablePDB }}"
                operator: Equals
                value: true
              - key: "{{request.object.spec.instances }}"
                operator: LessThan
                value: 2

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: pdb-cnpg-cluster-validation
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
    - apiGroups: ["postgresql.cnpg.io"]
      apiVersions: ["v1"]
      operations: ["CREATE", "UPDATE"]
      resources: ["clusters"]
    namespaceSelector:
      matchExpressions:
      - key: capsule.clastix.io/tenant
        operator: Exists
  validations:
  - expression: |
      !has(object.spec.enablePDB) ||
      object.spec.enablePDB == false ||
      (has(object.spec.instances) && object.spec.instances >= 2)      
    message: "Set `.spec.enablePDB` to `false` for CNPG Clusters when the number of instances is lower than 2."
    messageExpression: |
      'Set `.spec.enablePDB` to `false` for CNPG Clusters when the number of instances is lower than 2. Current instances: ' +
      string(has(object.spec.instances) ? object.spec.instances : 1)      
    reason: Invalid
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: pdb-cnpg-cluster-validation-binding
spec:
  policyName: pdb-cnpg-cluster-validation
  validationActions: ["Deny"]

Enforce EmptDir Requests/Limits

By Defaults emptyDir Volumes do not have any limits. This could lead to a situation, where a tenant fills up the node disk. To avoid this, you can enforce limits on emptyDir volumes. You may also consider restricting the usage of emptyDir with the medium: Memory option, as this could lead to memory exhaustion on the node.

---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: tenant-workload-restrictions
spec:
  rules:
    - name: default-emptydir-sizelimit
      match:
        any:
        - resources:
            kinds:
            - Pod
            namespaceSelector:
              matchExpressions:
              - key: capsule.clastix.io/tenant
                operator: Exists
      mutate:
        foreach:
        - list: "request.object.spec.volumes[]"
          preconditions:
            all:
            - key: "{{element.keys(@)}}"
              operator: AnyIn
              value: emptyDir
            - key: "{{element.emptyDir.sizeLimit || ''}}"
              operator: Equals
              value: ''
          patchesJson6902: |-
            - path: "/spec/volumes/{{elementIndex}}/emptyDir/sizeLimit"
              op: add
              value: 250Mi

Block Ephemeral Containers

Ephemeral containers, enabled by default in Kubernetes 1.23, allow users to use the kubectl debug functionality and attach a temporary container to an existing Pod. This may potentially be used to gain access to unauthorized information executing inside one or more containers in that Pod. This policy blocks the use of ephemeral containers.

---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: block-ephemeral-containers
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
      # 1) Regular Pods (ensure spec doesn't carry ephemeralContainers)
      - apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]
        operations: ["CREATE","UPDATE"]
        scope: "Namespaced"
      # 2) Subresource used by `kubectl debug` to inject ephemeral containers
      - apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods/ephemeralcontainers"]
        operations: ["UPDATE","CREATE"]  # UPDATE is typical, CREATE included for future-proofing
        scope: "Namespaced"
  validations:
    # Deny any request that targets the pods/ephemeralcontainers subresource
    - expression: request.subResource != "ephemeralcontainers"
      message: "Ephemeral (debug) containers are not permitted (subresource)."
    # For direct Pod create/update, allow only if the field is absent or empty
    - expression: >
        !has(object.spec.ephemeralContainers) ||
        size(object.spec.ephemeralContainers) == 0        
      message: "Ephemeral (debug) containers are not permitted in Pod specs."
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: block-ephemeral-containers-binding
spec:
  policyName: block-ephemeral-containers
  validationActions: ["Deny"]
  matchResources:
    namespaceSelector:
      matchExpressions:
        - key: capsule.clastix.io/tenant
          operator: Exists

# Source: https://kyverno.io/policies/other/block-ephemeral-containers/block-ephemeral-containers/
---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: block-ephemeral-containers
  annotations:
    policies.kyverno.io/title: Block Ephemeral Containers
    policies.kyverno.io/category: Other
    policies.kyverno.io/severity: medium
    kyverno.io/kyverno-version: 1.6.0
    policies.kyverno.io/minversion: 1.6.0
    kyverno.io/kubernetes-version: "1.23"
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/description: >-
      Ephemeral containers, enabled by default in Kubernetes 1.23, allow users to use the
      `kubectl debug` functionality and attach a temporary container to an existing Pod.
      This may potentially be used to gain access to unauthorized information executing inside
      one or more containers in that Pod. This policy blocks the use of ephemeral containers.      
spec:
  validationFailureAction: Enforce
  background: true
  rules:
  - name: block-ephemeral-containers
    match:
      any:
      - resources:
          kinds:
            - Pod
          namespaceSelector:
            matchExpressions:
            - key: capsule.clastix.io/tenant
              operator: Exists
    validate:
      message: "Ephemeral (debug) containers are not permitted."
      pattern:
        spec:
          X(ephemeralContainers): "null"

Certificates

Deny ClusterIssuer in Certificates

Often when working in multi-tenant environments, you want to ensure that tenants are not using ClusterIssuers to issue certificates, but rather use namespaced Issuers within their own namespace. This policy enforces that cert-manager.io/v1/Certificate resources do not reference ClusterIssuers and that the Issuer referenced is in the same namespace as the Certificate.

---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: certificates-only-local-issuer
spec:
  validationFailureAction: Enforce
  background: true
  rules:
    - name: deny-clusterissuer-in-certificates
      match:
        any:
          - resources:
              kinds:
                - cert-manager.io/v1/Certificate
              namespaceSelector:
                matchExpressions:
                - key: capsule.clastix.io/tenant
                  operator: Exists
      validate:
        message: "Certificates must not reference ClusterIssuers; use a namespaced Issuer in the same namespace."
        deny:
          conditions:
            any:
              - key: "{{ request.object.spec.issuerRef.kind || 'Issuer' }}"
                operator: Equals
                value: "ClusterIssuer"

    - name: deny-cross-namespace-issuerref-in-certificates
      match:
        any:
          - resources:
              kinds:
                - cert-manager.io/v1/Certificate
      validate:
        message: "Certificates must reference an Issuer in the same namespace (spec.issuerRef.namespace must be empty or equal to the Certificate namespace)."
        deny:
          conditions:
            any:
              # If issuerRef.namespace is set and differs from the Certificate namespace -> deny
              - key: "{{request.object.spec.issuerRef.namespace || '' }}"
                operator: NotEquals
                value: ""
                # AND also not equal to request namespace
              - key: "{{ request.object.spec.issuerRef.namespace || request.namespace  }}"
                operator: NotEquals
                value: "{{ request.namespace }}"

Deny ClusterIssuer in Gateways

Deny to usage of ClusterIssuers in Gateways by checking for the cert-manager.io/cluster-issuer annotation. This ensures that tenants use namespaced issuer mechanisms instead.

This requires extra permissions to allow Kyverno to read Gateway resources:

admissionController:
  rbac:
    clusterRole:
      extraResources:
       - apiGroups: ["gateway.networking.k8s.io"]
         resources: ["*"]
         verbs: ["get", "list", "watch"]

---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: gateways-deny-cluster-issuer-annotation
spec:
  validationFailureAction: Enforce
  background: false
  rules:
    - name: deny-cert-manager-cluster-issuer-annotation
      match:
        any:
          - resources:
              kinds:
                - gateway.networking.k8s.io/v1/Gateway
              namespaceSelector:
                matchExpressions:
                - key: capsule.clastix.io/tenant
                  operator: Exists
      validate:
        message: "Gateways must not use cert-manager.io/cluster-issuer; use namespaced issuer mechanisms instead."
        deny:
          conditions:
            any:
              - key: "{{ request.object.metadata.annotations.\"cert-manager.io/cluster-issuer\" || '' }}"
                operator: NotEquals
                value: ""

Selective ClusterIssuers

Allow certain ClusterIssuers within Tenants:

---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: certificates-restrict-clusterissuer-by-tenant-label
spec:
  validationFailureAction: Enforce
  background: true
  rules:
    - name: allow-clusterissuer-only-when-managed-by-matches
      match:
        any:
          - resources:
              kinds:
                - cert-manager.io/v1/Certificate
              namespaceSelector:
                matchExpressions:
                  - key: capsule.clastix.io/tenant
                    operator: Exists
      context:
        - name: certManagedBy
          variable:
            jmesPath: request.object.metadata.labels."capsule.clastix.io/managed-by" || ''
        - name: clusterIssuerManagedBy
          apiCall:
            urlPath: /apis/cert-manager.io/v1/clusterissuers/{{ request.object.spec.issuerRef.name }}
            jmesPath: metadata.labels."company.com/tenant" || ''
            default: ""
      preconditions:
        all:
          - key: "{{ request.object.spec.issuerRef.kind || 'Issuer' }}"
            operator: Equals
            value: ClusterIssuer
          - key: "{{request.operation || 'BACKGROUND'}}"
            operator: AnyIn
            value:
            - CREATE
            - UPDATE
      validate:
        message: >-
          ClusterIssuer is only allowed when the Certificate label
          capsule.clastix.io/managed-by ({{certManagedBy}}) matches the referenced ClusterIssuer label
          company.com/tenant ({{clusterIssuerManagedBy}}).          
        deny:
          conditions:
            any:
              - key: "{{certManagedBy}}"
                operator: Equals
                value: ""
              - key: "{{clusterIssuerManagedBy}}"
                operator: Equals
                value: ""
              - key: "{{certManagedBy}}"
                operator: NotEquals
                value: "{{clusterIssuerManagedBy}}"

4.3 - Concepts

Key design decisions, personas, templating, and metadata reference for Capsule

Concept references explain reusable behavior shared across Capsule APIs:

Parameter Schemas and Dynamic Forms: validate parameterized APIs with JSON Schema and Kubernetes CEL, and populate client forms from arbitrary Kubernetes resources.
Managed Resources and Server-Side Apply: understand field ownership, adoption, protection, cleanup policies, and processed-resource status.

4.3.1 - Architecture

Architecture references and considerations

Key Decisions

Introducing a new separation of duties can lead to a significant paradigm shift. This has technical implications and may also impact your organizational structure. Therefore, when designing a multi-tenant platform pattern, carefully consider the following aspects. As Cluster Administrator, ask yourself:

🔑 How much ownership can be delegated to Tenant Owners (Platform Users)?

The answer to this question may be influenced by the following aspects:

Are the Cluster Administrators willing to grant permissions to Tenant Owners?
- You might have a problem with know-how and probably your organisation is not yet pushing Kubernetes itself enough as a key strategic platform. The key here is enabling Platform Users through good UX and know-how transfers
Who is responsible for the deployed workloads within the Tenants?
- If Platform Administrators are still handling this, a true “shift left” has not yet been achieved.
Who gets paged during a production outage within a Tenant’s application?
- You’ll need robust monitoring that enables Tenant Owners to clearly understand and manage what’s happening inside their own tenant.
Are your customers technically capable of working directly with the Kubernetes API?
- If not, you may need to build a more user-friendly platform with better UX, for example a multi-tenant ArgoCD setup, or UI layers like Headlamp.

Personas

In Capsule, we introduce a new persona called the Tenant Owner. The goal is to enable Cluster Administrators to delegate tenant management responsibilities to Tenant Owners. Here’s how it works:

Capsule Administrators

Configure Capsule Administrators. The ClusterRoles assigned to Administrators can be configured in the CapsuleConfiguration as well.

They are promoted to Tenant-Owners for all available tenants. Effectively granting them the ability to manage all namespaces within the cluster, across all tenants.

Note: Granting Capsule Administrator rights should be done with caution, as it provides extensive control over the cluster’s multi-tenant environment. When granting Capsule Administrator rights, the entity gets the privileges to create any namespace (also not part of capsule tenants) and the privileges to delete any tenant namespaces.

Capsule Administrators can:

Create and manage namespaces via labels in any tenant.
Create namespaces outside of tenants.
Delete namespaces in any tenant.

Administrators come in handy in bootstrap scenarios or GitOps scenarios where certain users/serviceaccounts need to be able to manage namespaces for all tenants.

Capsule Users

Configure Capsule Users

Any entity which needs to interact with tenants and their namespaces must be defined as a Capsule User. This is where the flexibility of Capsule comes into play. You can define users or groups as Capsule Users, allowing them to create and manage namespaces within any tenant they have access to. If they are not defined as Capsule Users, any interactions will be ignored by Capsule. Often a best practice is to define a single group which identifies all your tenant users. This way you can have one generic group for all your users and then use additional groups to separate responsibilities (e.g. administrators vs normal users).

Only one entry is needed to identify a Capsule User. This is only important for Namespace Admission..

Capsule Users Admission

You can always verify the effective Capsule Users by checking the Configuration Status. As this is variable with Tenant Owners, the status will always show the effective users:

kubectl get capsuleconfiguration default -o jsonpath='{.status.users}' | jq

[
  {
    "kind": "Group",
    "name": "oidc:kubernetes:admin"
  },
  {
    "kind": "Group",
    "name": "projectcapsule.dev"
  },
  {
    "kind": "User",
    "name": "test-user"
  }
]

Tenant Owners

Configure Tenant Owners

Every Tenant Owner must be a Capsule User. By using the TenantOwner CRD this is automatically handeled.

They manage the namespaces within their tenants and perform administrative tasks confined to their tenant boundaries. This delegation allows teams to operate more autonomously while still adhering to organizational policies. Tenant Owners can be used to shift reposnsability of one tenant towards this user group. promoting them to the SPOC of all namespaces within the tenant.

Tenant Owners can:

Create and manage namespaces within their tenant.
Delete namespaces within their tenant.

Capsule provides robust tools to strictly enforce tenant boundaries, ensuring that each tenant operates within its defined limits. This separation of duties promotes both security and efficient resource management.

Layouts

Let’s discuss different Tenant Layouts which could be used . These are just approaches we have seen, however you might also find a combination of these which fits your use-case.

Tenant As A Service

With this approach you essentially just provide your Customers with the Tenant on your cluster. The rest is their responsibility. This concludes to a shared responsibility model. This can be achieved when also the Tenant Owners are responsible for everything they are provisiong within their Tenant’s namespaces.

Resourcepool Dashboard

Scheduling

Workload distribution across your compute infrastructure can be approached in various ways, depending on your specific priorities. Regardless of the use case, it’s essential to preserve maximum flexibility for your platform administrators. This means ensuring that:

Nodes can be drained or deleted at any time.
Cluster updates can be performed at any time.
The number of worker nodes can be scaled up or down as needed.

If your cluster architecture prevents any of these capabilities, or if certain applications block the enforcement of these policies, you should reconsider your approach.

Dedicated

Strong tenant isolation, ensuring that any noisy neighbor effects remain confined within individual tenants (tenant responsibility). This approach may involve higher administrative overhead and costs compared to shared compute. It also provides enhanced security by dedicating nodes to a single customer/application. It is recommended, at a minimum, to separate the cluster’s operator workload from customer workloads.

Dedicated Nodepool

Shared

With this approach you share the nodes amongst all Tenants, therefore giving you more potential for optimizing resources on a node level. It’s a common pattern to separate the controllers needed to power your Distribution (operators) from the actual workload. This ensures smooth operations for the cluster

Overview:

✅ Designed for cost efficiency .
✅ Suitable for applications that typically experience low resource fluctuations and run with multiple replicas.
❌ Not ideal for applications that are not cloud-native ready, as they may adversely affect the operation of other applications or the maintenance of node pools.
❌ Not ideal if strong isolation is required

Shared Nodepool

We provide the concept of ResourcePools or CustomQuotas to manage resources cross namespaces. There’s some further aspects you must think about with shared approaches:

4.3.2 - Parameter Schemas and Dynamic Forms

Validate parameterized APIs with JSON Schema and Kubernetes CEL, and enrich clients with live Kubernetes resource selectors.

Capsule parameter schemas define the contract between a template and the values submitted by a consuming API object. Resource Permits currently use spec.paramSchema to validate ResourcePermit.spec.params before context loading and rendering.

spec.paramSchema on the referenced template is the complete contract for ResourcePermit.spec.params. It accepts JSON Schema 2020-12 keywords, Kubernetes-compatible x-kubernetes-validations CEL rules, and Capsule’s x-capsule-form presentation extension.

Capsule validates parameters during ResourcePermit admission, before loading context or rendering resources. Invalid parameters are rejected and no ResourcePermit is created. Failures that require runtime context or rendering remain visible on an accepted request as Created with Ready=False; Capsule does not apply any resources.

JSON Schema 2020-12

Use ordinary JSON Schema for types, required fields, enums, ranges, lengths, patterns, and conditional structure:

paramSchema:
  $schema: https://json-schema.org/draft/2020-12/schema
  type: object
  additionalProperties: false
  required:
    - subjectKind
    - subjectName
  properties:
    subjectKind:
      type: string
      enum: [User, Group, ServiceAccount]
    subjectName:
      type: string
      minLength: 1
  allOf:
    - if:
        properties:
          subjectKind:
            const: ServiceAccount
        required: [subjectKind]
      then:
        properties:
          subjectName:
            pattern: '^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?$'

With this schema:

ServiceAccount with operations/resource-permit-runner is valid;
ServiceAccount with resource-permit-runner is invalid; and
User with alice@example.com remains valid.

The example represents a ServiceAccount as namespace/name. When rendering an RBAC Subject, split that value into the subject’s namespace and name fields.

Other useful JSON Schema keywords include dependentRequired, dependentSchemas, contains, prefixItems, allOf, anyOf, oneOf, and unevaluatedProperties.

Kubernetes CEL rules

Use x-kubernetes-validations for readable cross-field and collection invariants. It has the same shape as validation rules generated from Kubebuilder’s +kubebuilder:validation:XValidation marker. Capsule converts the parameter schema to a Kubernetes structural schema and delegates evaluation to Kubernetes’ CEL validator.

paramSchema:
  type: object
  additionalProperties: false
  required:
    - subjectKind
    - subjectName
  properties:
    subjectKind:
      type: string
      enum: [User, Group, ServiceAccount]
    subjectName:
      type: string
      minLength: 1
  x-kubernetes-validations:
    - rule: >-
        self.subjectKind != 'ServiceAccount' ||
        self.subjectName.matches('^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?$')        
      message: ServiceAccount subjects must use namespace/name
      reason: FieldValueInvalid
      fieldPath: .subjectName

self is the value at the schema node containing the extension. At the root it is the whole parameters object. Use has(self.field) for optional properties.

The accepted validation-rule fields are:

Field	Meaning
`rule`	Required CEL expression returning a boolean
`message`	Static failure message; required when a rule contains line breaks
`messageExpression`	CEL expression returning a dynamic failure string
`reason`	`FieldValueInvalid`, `FieldValueForbidden`, `FieldValueRequired`, or `FieldValueDuplicate`
`fieldPath`	Relative field path associated with the error
`optionalOldSelf`	Kubernetes transition-rule behavior for an optional previous value

Rule and message-expression syntax is compiled when the template is admitted. Evaluation uses Kubernetes CEL cost limits.

Parameter validation has no previous parameter object. A transition rule using oldSelf therefore behaves like Kubernetes validation on create: an ordinary transition rule is skipped, while optionalOldSelf: true can explicitly handle an absent previous value. Use self for request-time constraints.

Dynamic failure messages are supported:

x-kubernetes-validations:
  - rule: "self.subjectKind != 'ServiceAccount' || self.subjectName.contains('/')"
    message: invalid ServiceAccount path
    messageExpression: >-
      'subjectName ' + self.subjectName + ' must use namespace/name for a ServiceAccount'      
    reason: FieldValueInvalid
    fieldPath: .subjectName

If messageExpression fails or produces an invalid message, evaluation falls back to message, then to the rule text.

Rules may also be attached to structural properties, array items, and additional-property schemas. Their self value is scoped accordingly:

paramSchema:
  type: object
  properties:
    subjects:
      type: array
      items:
        type: object
        required: [name]
        properties:
          name:
            type: string
        x-kubernetes-validations:
          - rule: "!self.name.startsWith('system:')"
            messageExpression: "'subject ' + self.name + ' uses a reserved prefix'"
            fieldPath: .name

CEL rules must be attached to Kubernetes structural schema nodes. Do not place them inside $defs, if/then, or another JSON-Schema-only branch. Put the rule on a root, property, item, or additional-property node that can see the required values. Standard JSON Schema may still use those constructs alongside CEL.

Choose the smallest validation mechanism that expresses the invariant:

Requirement	Mechanism
Type, enum, range, length, or regex	JSON Schema property keyword
Field required for a discriminator value	JSON Schema `if`/`then`
Relationship between fields	`x-kubernetes-validations` CEL
Collection-item relationship	Nested CEL rule or CEL collection macro
Populate choices from live objects	`x-capsule-form`
Authoritatively load a selected object	Template `context.resources`

Dynamic forms and live cluster data

x-capsule-form is an optional JSON Schema vendor extension for compatible clients such as a Headlamp plugin. It populates a string field from live Kubernetes objects without changing server-side JSON Schema semantics.

This complete example combines JSON Schema, conditional validation, CEL, and a live Secret selector:

paramSchema:
  $schema: https://json-schema.org/draft/2020-12/schema
  type: object
  additionalProperties: false
  required:
    - subjectKind
    - subjectName
    - credentialsSecret
  properties:
    subjectKind:
      type: string
      enum: [User, Group, ServiceAccount]
    subjectName:
      type: string
      minLength: 1
    credentialsSecret:
      type: string
      pattern: '^[a-z0-9.-]+/[a-z0-9.-]+$'
      x-capsule-form:
        widget: kubernetes-resource
        source:
          apiVersion: v1
          kind: Secret
          namespace: request
          labelSelector: access.example.com/selectable=true
        option:
          labelTemplate: '{{ .metadata.name }}'
          valueTemplate: '{{ .metadata.namespace }}/{{ .metadata.name }}'
  allOf:
    - if:
        properties:
          subjectKind:
            const: ServiceAccount
        required: [subjectKind]
      then:
        properties:
          subjectName:
            pattern: '^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?$'
  x-kubernetes-validations:
    - rule: >-
        self.subjectKind != 'Group' ||
        !self.subjectName.startsWith('system:')        
      message: Groups using the system prefix cannot be requested
      reason: FieldValueForbidden
      fieldPath: .subjectName

The accepted form-extension shape is:

x-capsule-form:
  widget: kubernetes-resource       # required
  source:                           # required
    apiVersion: group.example.io/v1 # required; v1 is also valid
    kind: Example                   # required; one exact kind
    namespace: request              # optional: request, "*", or a namespace
    labelSelector: key=value        # optional
    fieldSelector: metadata.name=x  # optional
  option:                           # optional
    labelTemplate: '{{ .metadata.name }}'
    valueTemplate: '{{ .metadata.namespace }}/{{ .metadata.name }}'

Unknown widgets and extension fields are rejected. option may be omitted; both templates then default to {{ .metadata.name }}.

Resource source

Field	Required	Meaning
`apiVersion`	Yes	Exact version such as `v1`, `apps/v1`, or an arbitrary CRD version
`kind`	Yes	Exact Kubernetes kind; wildcards are unsupported
`namespace`	No	List scope for a namespaced GVK
`labelSelector`	No	Kubernetes label selector passed to the list request
`fieldSelector`	No	Kubernetes field selector passed to the list request

namespace supports four modes:

Value	Behavior
omitted	Request namespace for namespaced GVKs; cluster endpoint for cluster-scoped GVKs
`request`	Explicitly use the ResourcePermit namespace
`*`	List a namespaced GVK across every namespace
namespace name	List only that literal namespace

Any exact discoverable GVK is supported, including CRDs. Capsule validates API-version syntax, selectors, namespaces, extension fields, and option-template syntax when the template is admitted. It does not require the GVK to exist at that time, allowing the template to be installed before its CRD.

The form client uses Kubernetes discovery to determine whether a GVK is namespaced or cluster-scoped. For example:

properties:
  gatewayClass:
    type: string
    x-capsule-form:
      widget: kubernetes-resource
      source:
        apiVersion: gateway.networking.k8s.io/v1
        kind: GatewayClass
      option:
        labelTemplate: '{{ .metadata.name }} — {{ .spec.controllerName }}'
        valueTemplate: '{{ .metadata.name }}'

To list a namespaced type across all namespaces:

properties:
  sourceConfigMap:
    type: string
    x-capsule-form:
      widget: kubernetes-resource
      source:
        apiVersion: v1
        kind: ConfigMap
        namespace: '*'
        labelSelector: access.example.com/exportable=true
      option:
        valueTemplate: '{{ .metadata.namespace }}/{{ .metadata.name }}'

The user’s Kubernetes RBAC must permit the corresponding list operation.

Option templates

The listed object is an unstructured map at the Go-template root:

option:
  labelTemplate: '{{ .metadata.name }} ({{ .metadata.namespace }})'
  valueTemplate: '{{ .metadata.namespace }}/{{ .metadata.name }}'

Both fields default to {{ .metadata.name }}. They use Capsule’s safe Go-template function map; environment-reading functions are unavailable. Keep values deterministic because the rendered value is stored in spec.params.

Nested form schemas

Capsule finds and validates x-capsule-form recursively in properties, array items, tuple and prefix items, $defs, definitions, allOf, anyOf, oneOf, and conditional schemas:

paramSchema:
  type: object
  properties:
    targets:
      type: array
      items:
        type: object
        properties:
          secret:
            type: string
            x-capsule-form:
              widget: kubernetes-resource
              source:
                apiVersion: v1
                kind: Secret
                namespace: request

Arrays and multiple selections

For one or more selections, define an array and place x-capsule-form on its string items schema. A compatible client may display this as a multi-select control or as repeatable resource selectors:

properties:
  clusterRoles:
    type: array
    minItems: 1
    uniqueItems: true
    items:
      type: string
      x-capsule-form:
        widget: kubernetes-resource
        source:
          apiVersion: rbac.authorization.k8s.io/v1
          kind: ClusterRole
  namespaces:
    type: array
    minItems: 1
    uniqueItems: true
    items:
      type: string
      x-capsule-form:
        widget: kubernetes-resource
        source:
          apiVersion: v1
          kind: Namespace

minItems enforces at least one selection and uniqueItems prevents duplicate generator output. The submitted values remain ordinary arrays in spec.params and can be iterated with nested Go-template range actions.

Client and security contract

A compatible client:

traverses JSON Schema and locates x-capsule-form;
resolves the exact apiVersion and kind through Kubernetes discovery;
selects the namespaced, all-namespaces, or cluster list endpoint;
passes the configured label and field selectors;
renders a label and value for every unstructured object; and
submits the chosen value through ordinary spec.params.

Discovery and listing must use the logged-in user’s credentials, not the template’s execution ServiceAccount. Otherwise a dropdown could disclose objects the user cannot list.

x-capsule-form is presentation metadata, not an authorization boundary:

it grants no list permission;
it does not prove that a value came from the dropdown;
it does not load the selected object into rendering context; and
it does not validate that the GVK is installed.

Validate submitted values with JSON Schema or CEL. For authoritative lookup, load the selected object through spec.context.resources, which uses the recorded template ServiceAccount:

paramSchema:
  type: object
  required: [crd]
  properties:
    crd:
      type: string
      x-capsule-form:
        widget: kubernetes-resource
        source:
          apiVersion: apiextensions.k8s.io/v1
          kind: CustomResourceDefinition
context:
  resources:
    - apiVersion: apiextensions.k8s.io/v1
      kind: CustomResourceDefinition
      name: '{{ .crd }}'
      index: crdInfo
      optional: false

If the context object is missing or the execution identity cannot read it, the request becomes Ready=False.

4.3.3 - Managed Resources and Server-Side Apply

Resource ownership, server-side apply, protection, cleanup policies, and status.

Capsule uses a shared resource manager to apply Kubernetes objects, track their ownership, and prune them when they leave scope. TenantResource, GlobalTenantResource, ResourcePermitTemplate, and GlobalResourcePermitTemplate share spec.resources[].policy. Each block controls its generated resources, allowing namespaces within a tenant to receive different configuration and lifecycle behavior.

Apply and ownership model

Capsule uses Kubernetes server-side apply (SSA) with a stable field manager for each managing parent. A ResourcePermit manager is derived from the request UID:

projectcapsule.dev/resource/resourcepermit/<request-uid>

SSA allows Capsule and another manager to own different fields on the same object. When both managers claim the same field, Kubernetes reports an apply conflict unless Capsule is configured to force ownership.

For Resource Permits, Capsule renders every target before approval and stores the concrete manifests and their policies in .status.request.resources. That snapshot—not the mutable source template—is the source of truth for review, apply, and cleanup.

Resource policy

One policy applies to every target in a resource group:

resources:
  - policy:
      creation: Owner
      protect: true
      force: false
      deletion: Remove
    targets:
      - apiVersion: v1
        kind: ConfigMap
        metadata:
          name: temporary-settings

Field	Values	Default	Behavior
`creation`	`Owner`, `Merge`	`Owner`	Whether an existing object may be adopted
`protect`	`true`, `false`	`true`	Whether admission blocks direct changes while managed
`force`	`true`, `false`	`false`	Whether SSA may take conflicting field ownership
`deletion`	`Remove`, `Orphan`	`Remove`	What happens when the parent stops managing the object
`condition`	CEL boolean expression	Omitted	Whether to apply rendered content to each destination

Owner requires Capsule to have created the target. If an object with the same identity already exists and was not created for the managing API, reconciliation fails. Merge adopts an existing object when possible and creates it otherwise. Use Merge only when the workflow intentionally manages fields on pre-existing objects.

force: true resolves SSA conflicts by taking ownership of the rendered fields. It does not bypass Kubernetes RBAC, admission, or schema validation, and can displace another controller’s ownership.

With protect: true, Capsule marks the object as protected. The generic Capsule admission webhook rejects direct updates and deletion while the parent manages it, except for the controller and the resolved execution ServiceAccount. Disable protection only when another actor must modify the object during its managed lifetime.

With deletion: Remove, when multiple Capsule resource managers share an adopted target, removing one manager preserves the shared tracking and protection metadata while another Capsule resource manager remains. The final departing manager removes that metadata without deleting fields owned by external managers.

Cleanup depends on both the deletion policy and whether Capsule created or adopted the object:

Policy	Object created by Capsule	Existing object adopted with `Merge`
`Remove`	Delete the object	Relinquish managed fields and tracking
`Orphan`	Keep the object and remove tracking/protection	Keep the object and applied fields, then remove tracking/protection

Orphan can leave privileges or configuration behind after the parent expires, so it should be an explicit design choice.

Apply conditions

Set spec.resources[].policy.condition on TenantResource, GlobalTenantResource, ResourcePermitTemplate, or GlobalResourcePermitTemplate. One expression applies independently to every destination produced by that resource block; separate blocks can use different conditions. The expression must return a boolean and may contain up to 4096 characters. Omit the field to keep unconditional apply; an empty or whitespace-only expression is not a substitute for omitting it.

Input	Meaning
`object`	The existing destination resource, read using the execution identity; `null` if it does not exist. It is not the rendered candidate.
`now`	The UTC timestamp at evaluation time. Use `timestamp(...)` and `duration(...)` for time comparisons.

Apply conditions use CEL directly. There is no self, .self, Go-template expansion, or access to template parameters/context in this expression. Load context and render the desired resource through the existing template API. The condition then decides whether that rendered resource may be applied.

For example, create a target only when it is absent:

policy:
  condition: "object == null"

To initialize a target and then allow a refresh after five minutes:

policy:
  condition: |
    object == null ||
    !has(object.metadata.annotations) ||
    !('keys.example.org/rotated-at' in object.metadata.annotations) ||
    now >= timestamp(object.metadata.annotations['keys.example.org/rotated-at']) + duration('5m')

The generator must write the keys.example.org/rotated-at annotation on each successful refresh. Guard missing objects and fields before accessing them; use bracket notation for annotation keys containing dots or slashes. A missing timestamp initializes this example, while a malformed timestamp causes an evaluation error. See the complete rotation example for the template, retention behavior, and verification steps.

Outcome	Apply behavior
Condition omitted	Existing unconditional apply behavior.
`true`	Apply using the block’s creation, force, and protection policies; normal authorization and ownership checks still apply.
`false`	Skip rendered content and preserve its last-apply timestamp. Reconcile protection and retain the current policy for already managed targets. A never-applied target is not adopted, protected, or modified.
Invalid syntax or non-boolean result type	Admission rejects the expression at `spec.resources[i].policy.condition`.
Evaluation or target-read error	The target is not written; the failure is reported through existing status. A forbidden read is not treated as an absent object.

There is no configurable action for a false result. Conditions do not provide an all-or-nothing transaction across targets: each target is evaluated separately. They also do not change SSA retention. To preserve old fields owned by Capsule, include them in subsequent rendered resources, as the age-key example does.

For example, changing protect: true to protect: false removes protection on an already managed target even while its condition is false, provided no other Capsule resource manager still shares the target. Protection is also reconciled when the condition becomes true or is removed; protection established during a skipped apply does not persist after a sole manager disables it. Changing deletion: Remove to deletion: Orphan takes effect for subsequent cleanup. creation and force govern the next permitted content apply; changing them does not override a false condition. Rendered labels, annotations, and data remain unchanged, including key material and rotation timestamps. Capsule may patch its own protection metadata, so the target’s resource version can change. When that metadata already matches, a skipped reconciliation performs no target write. Policy changes that fail to reconcile are reported through existing status.

Evaluation lifecycle

Resource	Evaluation lifecycle	Suitable use
`TenantResource` / `GlobalTenantResource`	Render and evaluate during reconciliation, including periodic resync.	Recurring namespace configuration and key rotation.
`ResourcePermitTemplate` / `GlobalResourcePermitTemplate`	Render a request snapshot, then evaluate against live targets during preflight, activation, and explicit failure retries. Active permits do not periodically reevaluate.	Conditional provisioning during an approved permit’s activation.

Permit approval conditions (spec.approvals.conditions) control approval eligibility. Apply conditions control target writes after those approval rules are satisfied. In particular, an Active permit may contain skipped targets; use approval conditions when the entire permit must be gated. The permit guide provides namespaced/global examples, required permissions, and status diagnostics.

Scope, tracking, and cleanup

For namespaced targets, Capsule normally defaults an omitted namespace to the parent workflow’s namespace. An explicitly rendered namespace is preserved when the workflow is trusted to operate across namespaces. Cluster-scoped GVKs are applied without a namespace.

A namespaced parent cannot own a cluster-scoped object through a Kubernetes owner reference. APIs such as ResourcePermit therefore record every processed target and use a finalizer for explicit cascading cleanup. Successful targets remain recorded even if a later target fails, allowing retries and finalization to handle partial application.

Managed objects may carry labels and annotations such as:

metadata:
  labels:
    projectcapsule.dev/created-by: resource-permit
    projectcapsule.dev/managed-by: resource-permit
    projectcapsule.dev/protected-by: resource-permit
  annotations:
    projectcapsule.dev/active-until: "2026-08-31T11:00:00Z"
    projectcapsule.dev/resourcepermit-service-account: system:serviceaccount:capsule-system:resource-permit-runner

The exact metadata depends on the parent API, policy, and lifecycle stage. Capsule sets tracking metadata independently of the rendered manifest, so a template cannot forge whether an object was created or adopted.

Status and failures

Managed APIs use a shared processed-resource status shape. A ResourcePermit example is:

status:
  size: 2
  processedItems:
    - group: rbac.authorization.k8s.io
      version: v1
      kind: Role
      namespace: solar-uat
      name: incident-1042-editor
      status:
        clusterScoped: false
        created: true
        lastApply: "2026-08-31T10:02:13Z"
        type: Ready
        status: "True"
    - group: rbac.authorization.k8s.io
      version: v1
      kind: ClusterRole
      name: incident-1042-cluster-reader
      status:
        clusterScoped: true
        created: true
        lastApply: "2026-08-31T10:02:14Z"
        type: Ready
        status: "True"

Each item identifies its GVK, name, namespace and scope, whether it was created or adopted, its last apply time, and its latest condition or error. Skipped targets report ConditionNotMet: apply skipped. A previously applied target keeps its last successful apply time; a never-applied target has none. The retained policy records the last successful policy reconciliation and determines subsequent cleanup. Ownership violations, SSA conflicts, RBAC denials, unavailable GVKs, invalid manifests, and admission failures are reported through this status and the parent conditions.

kubectl get resourcepermit REQUEST -n NAMESPACE -o json \
  | jq '.status | {size, processedItems, conditions}'

Do not edit controller-owned rendered manifests or processed-item status to recover from a failure. Correct the underlying object, permissions, template, or policy and let reconciliation retry.

4.3.4 - Known Metadata

Labels and Annotations in Capsule Items

Labels

Labels commonly used in Capsule items are listed below. These labels are applied to the corresponding resources by the Capsule controller and can be used for filtering and selection purposes.

`capsule.clastix.io/tenant`

Description	Target Objects	Audience
Established the connection between Object and Tenant. It’s value indicates the owning `Tenant`	* `Namespaces` having a relationship to `Tenants`	`Controller`

`projectcapsule.dev/tenant`

Description	Target Objects	Audience
Established the connection between Object and Tenant. It’s value indicates the owning `Tenant`. Long term replacement for `capsule.clastix.io/tenant` and `capsule.clastix.io/managed-by` labels.	* All namespaced items within a `Tenant` `Namespace` become the corresponding Tenant Label via Mutating Admission.	`Controller`

`capsule.clastix.io/managed-by`

Description	Target Objects	Audience
Established the connection between Object and Tenant. It’s value indicates the owning `Tenant`. Long term replacement for `capsule.clastix.io/tenant`	* All namespaced items within a `Tenant` `Namespace` become the corresponding Tenant Label via Mutating Admission. This label is still added to keep compatibility wiht the Capsule Proxy.	`User`

`projectcapsule.dev/managed-by`

Description	Target Objects
Indicator which controller of capsule or `Custom Resource` is responsible for managing the corresponding Object. Mainly used in Replications to establish that objects are at least managed by one Replications.	* `Any Object being influenced by Replications`	`User`

`projectcapsule.dev/created-by`

`projectcapsule.dev/name`

Description	Target Objects	Audience
Label for tracking internal name or allowing for faster selects. Mainly used to identify relevant rulestatus	* `Tenant Namespaces`	`Controller`

`projectcapsule.dev/cordoned`

Description	Target Objects	Audience
Indicator that a namespace is cordoned (when value equals `true`)	* `Tenant Namespaces`	`User`

`projectcapsule.dev/pool`

Description	Target Objects	Audience
Allocation of Resourcepool via ResourcePoolClaims	* `ResourcePoolClaims`	`User`

4.3.5 - Templating

Templating in Capsule Items

Fast Templates

For simple template cases we provide a fast templating engine. With this engine, you can use Go templates syntax to reference Tenant and Namespace fields. There are no operators or anything else supported.

Available fields are:

{{tenant.name}}: The Name of the Tenant
{{namespace}}: The Name of the namespace within the tenant (current context)

Sprout Templating

Our template library is mainly based on the upstream implementation from Sprout. You can find the all available functions here:

https://docs.atom.codes/sprout/registries/list-of-all-registries

We have removed certain functions which could exploit runtime information. Therefor the following functions are not available:

env
expandEnv

Data

You can provide structured data for each Tenant which can be used in templating:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  data:
    bool: true
    foo: bar
    list:
    - a
    - b
    number: 123
    obj:
      nested: value

Replication Context

Templates rendered by a TenantResource or GlobalTenantResource always have the metadata of the replication object available under $.replications.metadata. An explicit resources[].context is not required.

Commonly used fields include:

$.replications.metadata.name
$.replications.metadata.namespace for a namespaced TenantResource
$.replications.metadata.labels
$.replications.metadata.annotations
$.replications.metadata.uid
$.replications.metadata.resourceVersion
$.replications.metadata.generation
$.replications.metadata.ownerReferences
$.replications.metadata.finalizers

For example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ $.replications.metadata.name }}
  labels:
    replication-name: {{ $.replications.metadata.name | quote }}
    team: {{ index $.replications.metadata.labels "company.example/team" | quote }}
data:
  sourceNamespace: {{ $.replications.metadata.namespace | quote }}

metadata.namespace is absent for a cluster-scoped GlobalTenantResource. The context excludes managedFields and the kubectl.kubernetes.io/last-applied-configuration annotation.

Function Library

Custom Functions we provide in our template package.

getResourceByName

getResourceByName finds a Kubernetes object by metadata.name in a resource collection loaded through resources[].context.resources[].index.

getResourceByName(name string, resources any) map[string]any

The function is pipe-friendly. The indexed resource collection is passed last:

{{- $management := $.mgmt | getResourceByName "tenant-management" -}}

If no object matches, the function returns an empty map. This makes it suitable for optional resources:

{{- with ($.mgmt | getResourceByName "optional-settings") }}
data:
  source: {{ .metadata.name | quote }}
{{- end }}

If multiple objects have the requested name, rendering fails because the match is ambiguous. Use getResourceByNamespacedName when an index can contain objects from multiple namespaces.

mustGetResourceByName

mustGetResourceByName is the required variant of getResourceByName. Rendering fails if no object matches or if multiple objects have the requested name.

mustGetResourceByName(name string, resources any) map[string]any

Example using a ConfigMap value containing YAML:

{{- $management := $.mgmt | mustGetResourceByName "tenant-management" -}}
{{- $team := $management.data | get "team-a" | fromYAML -}}
subjects:
{{ $team.subjects | toYAML | indent 2 }}

getResourceByNamespacedName

getResourceByNamespacedName finds an object by both metadata.namespace and metadata.name. Use it when a context index contains resources from more than one namespace.

getResourceByNamespacedName(namespace string, name string, resources any) map[string]any

The namespace and name are supplied before the piped resource collection:

{{- $management := $.mgmt | getResourceByNamespacedName "solar-system" "tenant-management" -}}

If no object matches, the function returns an empty map. Rendering fails if the namespace and name still identify multiple objects.

deterministicUUID

deterministicUUID generates a deterministic, RFC-4122–compliant UUID (version 5 + RFC4122 variant) from a set of input strings. It is designed for use in templates where you need stable, repeatable IDs derived from meaningful inputs (e.g. cluster name, tenant, role name), instead of random UUIDs.

This is especially useful for:

Crossplane / IaC resources that must not change IDs across reconciles

The function takes any number of strings and turns them into a UUID in a fully deterministic way.

What that means in practice:

If you call it twice with the same values, you get the same UUID
If any input changes, the UUID changes too
There is no randomness involved
The output is always a valid UUID

So from the outside, it behaves just like a normal UUID, just deterministic.

deterministicUUID(parts ...string) string

Example usage:

{{ deterministicUUID "cluster-a" "app-123" "tenant-x" "some-role" }}

generateAgeKey

generateAgeKey generates a new age X25519 key pair for use with age. It returns both the private identity and the public recipient key.

This is useful in templates where a resource needs to create an age-compatible encryption identity, for example when generating Kubernetes Secrets that are later used for encrypting or decrypting data.

This is especially useful for:

Bootstrap secrets that need an age identity
Generating encryption keys during initial provisioning
Creating age recipient keys for systems that need to encrypt data for a generated identity

The function does not return a plain string. It returns an object with two fields:

Identity: the private age identity, e.g. AGE-SECRET-KEY-1...
Recipient: the public age recipient, e.g. age1...

What that means in practice:

Each call generates a new key pair
The identity is the private key and must be treated as secret
The recipient is the public key and can be shared with systems that need to encrypt data
The output is not deterministic
Calling this function during every reconcile may rotate the generated key unless the result is persisted

generateAgeKey() any

Example usage:

{{ $key := generateAgeKey }}
apiVersion: v1
kind: Secret
metadata:
  name: age-key
type: Opaque
stringData:
  identity: {{ $key.Identity | quote }}
  recipient: {{ $key.Recipient | quote }}

Output:

apiVersion: v1
kind: Secret
metadata:
  name: age-key
type: Opaque
stringData:
  identity: "AGE-SECRET-KEY-1..."
  recipient: "age1..."

Because this function creates a new random key pair on every call, it should usually only be used when the generated Secret is created once and then reused. For continuously reconciled resources, prefer generating the key in controller logic and persisting it before using it in templates.

generateAgePQKey

generateAgePQKey generates a new age post-quantum hybrid key pair for use with age. It returns both the private identity and the public recipient key.

This is useful in templates where a resource needs to create an age-compatible encryption identity using the newer hybrid recipient format.

This is especially useful for:

Bootstrap secrets that should use age hybrid keys
Generating encryption keys during initial provisioning
Creating public recipient keys for systems that need to encrypt data for a generated hybrid identity
Future-facing age encryption setups where hybrid keys are preferred

The function does not return a plain string. It returns an object with two fields:

Identity: the private age hybrid identity, e.g. AGE-SECRET-KEY-PQ-1...
Recipient: the public age recipient, e.g. age1...

What that means in practice:

Each call generates a new key pair
The identity is the private key and must be treated as secret
The recipient is the public key and can be shared with systems that need to encrypt data
The output is not deterministic
Calling this function during every reconcile may rotate the generated key unless the result is persisted

generateAgePQKey() any

Example Usage:

{{ $key := generateAgePQKey }}
apiVersion: v1
kind: Secret
metadata:
  name: age-pq-key
type: Opaque
stringData:
  identity: {{ $key.Identity | quote }}
  recipient: {{ $key.Recipient | quote }}

Output:

apiVersion: v1
kind: Secret
metadata:
  name: age-pq-key
type: Opaque
stringData:
  identity: "AGE-SECRET-KEY-PQ-1..."
  recipient: "age1..."

4.4 - Authentication

Integrate Capsule with Authentication of your Kubernetes cluster

OIDC

In the following guide, we’ll use Keycloak an Open Source Identity and Access Management server capable to authenticate users via OIDC and release JWT tokens as proof of authentication.

Configuring OIDC Server

Configure Keycloak as OIDC server:

Add a realm called caas, or use any existing realm instead
Add a group projectcapsule.dev
Add a user alice assigned to group projectcapsule.dev
Add an OIDC client called kubernetes (Public)
Add an OIDC client called kubernetes-auth (Confidential (Client Secret))

$ KEYCLOAK=sso.clastix.io
$ REALM=kubernetes-auth
$ OIDC_ISSUER=${KEYCLOAK}/realms/${REALM}

$ curl -k -s https://${OIDC_ISSUER}/protocol/openid-connect/token \
     -d grant_type=password \
     -d response_type=id_token \
     -d scope=openid \
     -d client_id=${OIDC_CLIENT_ID} \
     -d client_secret=${OIDC_CLIENT_SECRET} \
     -d username=${USERNAME} \
     -d password=${PASSWORD} | jq

{
   "access_token":"ACCESS_TOKEN",
   "refresh_token":"REFRESH_TOKEN",
   "id_token": "ID_TOKEN",
   "token_type":"bearer",
   "scope": "openid groups profile email"
}

To introspect the ID_TOKEN token run:

$ curl -k -s https://${OIDC_ISSUER}/protocol/openid-connect/introspect \
     -d token=${ID_TOKEN} \
     --user kubernetes-auth:${OIDC_CLIENT_SECRET} | jq

The result will be like the following:

{
  "exp": 1601323086,
  "iat": 1601322186,
  "aud": "kubernetes",
  "typ": "ID",
  "azp": "kubernetes",
  "preferred_username": "alice",
  "email_verified": false,
  "acr": "1",
  "groups": [
    "capsule.clastix.io"
  ],
  "client_id": "kubernetes",
  "username": "alice",
  "active": true
}

Configuring Kubernetes API Server

Authentication Configuration (Recommended)

apiVersion: apiserver.config.k8s.io/v1beta1
kind: AuthenticationConfiguration
jwt:
- issuer:
    url: https://${OIDC_ISSUER}
    audiences:
    - kubernetes
    - kubernetes-auth
    audienceMatchPolicy: MatchAny
  claimMappings:
    username:
      claim: 'email'
      prefix: ""
    groups:
      claim: 'groups'
      prefix: ""
  certificateAuthority: <PEM encoded CA certificates>

This file must be present and consistent across all kube-apiserver instances in the cluster. Add the following flag to the kube-apiserver manifest:

spec:
  containers:
  - command:
    - kube-apiserver
    ...
    - --authentication-configuration-file=/etc/kubernetes/authentication/authentication.yaml

spec:
  containers:
  - command:
    - kube-apiserver
    ...
    - --oidc-issuer-url=https://${OIDC_ISSUER}
    - --oidc-ca-file=/etc/kubernetes/oidc/ca.crt
    - --oidc-client-id=kubernetes
    - --oidc-username-claim=preferred_username
    - --oidc-groups-claim=groups
    - --oidc-username-prefix=-

KinD

As reference, here is an example of a KinD configuration for OIDC Authentication, which can be useful for local testing:

apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
nodes:
  - role: control-plane
    kubeadmConfigPatches:
     - |
       kind: ClusterConfiguration
       apiServer:
           extraArgs:
             oidc-issuer-url: https://${OIDC_ISSUER}
             oidc-username-claim: preferred_username
             oidc-client-id: kubernetes
             oidc-username-prefix: "keycloak:"
             oidc-groups-claim: groups
             oidc-groups-prefix: "keycloak:"
             enable-admission-plugins: PodNodeSelector

Configuring kubectl

There are two options to use kubectl with OIDC:

OIDC Authenticator
Use the –token option

Plugin

kubectl oidc-login setup \
	--oidc-issuer-url=https://${OIDC_ISSUER} \
	--oidc-client-id=kubernetes-auth \
	--oidc-client-secret=${OIDC_CLIENT_SECRET}

Manual

To use the OIDC Authenticator, add an oidc user entry to your kubeconfig file:

$ kubectl config set-credentials oidc \
    --auth-provider=oidc \
    --auth-provider-arg=idp-issuer-url=https://${OIDC_ISSUER} \
    --auth-provider-arg=idp-certificate-authority=/path/to/ca.crt \
    --auth-provider-arg=client-id=kubernetes-auth \
    --auth-provider-arg=client-secret=${OIDC_CLIENT_SECRET} \
    --auth-provider-arg=refresh-token=${REFRESH_TOKEN} \
    --auth-provider-arg=id-token=${ID_TOKEN} \
    --auth-provider-arg=extra-scopes=groups

To use the --token option:

$ kubectl config set-credentials oidc --token=${ID_TOKEN}

Point the kubectl to the URL where the Kubernetes APIs Server is reachable:

$ kubectl config set-cluster mycluster \
    --server=https://kube.projectcapsule.io:6443 \
    --certificate-authority=~/.kube/ca.crt

If your APIs Server is reachable through the capsule-proxy, make sure to use the URL of the capsule-proxy.

Create a new context for the OIDC authenticated users:

$ kubectl config set-context alice-oidc@mycluster \
    --cluster=mycluster \
    --user=oidc

As user alice, you should be able to use kubectl to create some namespaces:

$ kubectl --context alice-oidc@mycluster create namespace wind-production
$ kubectl --context alice-oidc@mycluster create namespace wind-development
$ kubectl --context alice-oidc@mycluster create namespace water-marketing

4.5 - Backup & Restore

Run Backups and Restores of Tenants

Velero is a backup and restore solution that performs data protection, disaster recovery and migrates Kubernetes cluster from on-premises to the Cloud or between different Clouds.

When coming to backup and restore in Kubernetes, we have two main requirements:

Configurations backup
Data backup

The first requirement aims to backup all the resources stored into etcd database, for example: namespaces, pods, services, deployments, etc. The second is about how to backup stateful application data as volumes.

The main limitation of Velero is the multi tenancy. Currently, Velero does not support multi tenancy meaning it can be only used from admin users and so it cannot provided “as a service” to the users. This means that the cluster admin needs to take care of users’ backup.

Assuming you have multiple tenants managed by Capsule, for example wind and water, as cluster admin, you can to take care of scheduling backups for:

Tenant cluster resources
Namespaces belonging to each tenant

Create backup of a tenant

Create a backup of the tenant solar. It consists in two different backups:

backup of the tenant resource
backup of all the resources belonging to the tenant

To back up the solar tenant selectively, label the tenant as:

kubectl label tenant wind capsule.clastix.io/tenant=solar

and create the backup

velero create backup solar-tenant \
    --include-cluster-resources=true \
    --include-resources=tenants.capsule.clastix.io \
    --selector capsule.clastix.io/tenant=solar

resulting in the following Velero object:

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: solar-tenant
spec:
  defaultVolumesToRestic: false
  hooks: {}
  includeClusterResources: true
  includedNamespaces:
  - '*'
  includedResources:
  - tenants.capsule.clastix.io
  labelSelector:
    matchLabels:
      capsule.clastix.io/tenant: solar
  metadata: {}
  storageLocation: default
  ttl: 720h0m0s

Create a backup of all the resources belonging to the solar tenant namespaces:

velero create backup solar-namespaces \
    --include-cluster-resources=false \
    --include-namespaces solar-production,solar-development,solar-marketing

resulting in the following Velero object:

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: solar-namespaces
spec:
  defaultVolumesToRestic: false
  hooks: {}
  includeClusterResources: false
  includedNamespaces:
  - solar-production
  - solar-development
  - solar-marketing
  metadata: {}
  storageLocation: default
  ttl: 720h0m0s

Velero requires an Object Storage backend where to store backups, you should take care of this requirement before using Velero.

Restore a tenant from the backup

To recover the tenant after a disaster, or to migrate it to another cluster, create a restore from the previous backups:

velero create restore --from-backup solar-tenant
velero create restore --from-backup solar-namespaces

Using Velero to restore a Capsule tenant can lead to an incomplete recovery of tenant because the namespaces restored with Velero do not have the OwnerReference field used to bind the namespaces to the tenant. For this reason, all restored namespaces are not bound to the tenant:

kubectl get tnt
NAME   STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR     AGE
water    active   9                 5                 {"pool":"water"}    34m
wind  active   9                 8                 {"pool":"wind"}  33m
solar    active   9                 0 # <<<           {"pool":"solar"}    54m

To avoid this problem you can use the script velero-restore.sh located under the hack/ folder:

./velero-restore.sh --kubeconfig /path/to/your/kubeconfig --tenant "wind" restore

Running this command, we are going to patch the tenant’s namespaces manifests that are actually ownerReferences-less. Once the command has finished its run, you got the tenant back.

kubectl get tnt
NAME   STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR     AGE
water    active   9                 5                 {"pool":"water"}    44m
solar  active   9                 8                 {"pool":"wind"}  43m
wind    active   9                 3 # <<<           {"pool":"solar"}    12s

4.6 - Day-2 Operations

Monitoring, backup and restore, and troubleshooting for production Capsule clusters

4.6.1 - Monitoring

Monitoring Capsule Items and Tenants

The Capsule dashboard allows you to track the health and performance of Capsule manager and tenants, with particular attention to resources saturation, server responses, and latencies. Prometheus and Grafana are requirements for monitoring Capsule.

OpenTelemetry Tracing

Capsule can export OpenTelemetry traces for admission webhook requests through the OTLP gRPC exporter. Enable tracing with manager.options.tracing.enabled=true and set manager.options.tracing.endpoint to your collector or tracing backend, for example tempo.monitoring-system.svc:4317. The exporter supports insecure transport for local clusters, TLS server name overrides, gzip compression, export timeouts, custom OTLP headers, and basic authentication. For production, prefer manager.options.tracing.basicAuth.existingSecret so credentials are loaded from a Kubernetes Secret instead of being rendered into controller arguments. Use manager.options.tracing.sampleRatio to control trace volume.

Tenants

Instrumentation for Tenants.

Metrics

The following Metrics are exposed and can be used for monitoring:

# HELP capsule_tenant_condition Provides per tenant condition status for each condition
# TYPE capsule_tenant_condition gauge
capsule_tenant_condition{condition="Cordoned",tenant="solar"} 0
capsule_tenant_condition{condition="Ready",tenant="solar"} 1


# HELP capsule_tenant_namespace_condition Provides per namespace within a tenant condition status for each condition
# TYPE capsule_tenant_namespace_condition gauge
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="earth",tenant="solar"} 0
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="fire",tenant="solar"} 0
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="foild",tenant="solar"} 0
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="green",tenant="solar"} 0
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="solar",tenant="solar"} 0
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="wind",tenant="solar"} 0
capsule_tenant_namespace_condition{condition="Ready",target_namespace="earth",tenant="solar"} 1
capsule_tenant_namespace_condition{condition="Ready",target_namespace="fire",tenant="solar"} 1
capsule_tenant_namespace_condition{condition="Ready",target_namespace="foild",tenant="solar"} 1
capsule_tenant_namespace_condition{condition="Ready",target_namespace="green",tenant="solar"} 1
capsule_tenant_namespace_condition{condition="Ready",target_namespace="solar",tenant="solar"} 1
capsule_tenant_namespace_condition{condition="Ready",target_namespace="wind",tenant="solar"} 1

# HELP capsule_tenant_namespace_count Total number of namespaces currently owned by the tenant
# TYPE capsule_tenant_namespace_count gauge
capsule_tenant_namespace_count{tenant="solar"} 6

# HELP capsule_tenant_namespace_relationship Mapping metric showing namespace to tenant relationships
# TYPE capsule_tenant_namespace_relationship gauge
capsule_tenant_namespace_relationship{target_namespace="earth",tenant="solar"} 1
capsule_tenant_namespace_relationship{target_namespace="fire",tenant="solar"} 1
capsule_tenant_namespace_relationship{target_namespace="soil",tenant="solar"} 1
capsule_tenant_namespace_relationship{target_namespace="green",tenant="solar"} 1
capsule_tenant_namespace_relationship{target_namespace="solar",tenant="solar"} 1
capsule_tenant_namespace_relationship{target_namespace="wind",tenant="solar"} 1

# HELP capsule_tenant_resource_limit Current resource limit for a given resource in a tenant
# TYPE capsule_tenant_resource_limit gauge
capsule_tenant_resource_limit{resource="limits.cpu",resourcequotaindex="0",tenant="solar"} 2
capsule_tenant_resource_limit{resource="limits.memory",resourcequotaindex="0",tenant="solar"} 2.147483648e+09
capsule_tenant_resource_limit{resource="pods",resourcequotaindex="1",tenant="solar"} 7
capsule_tenant_resource_limit{resource="requests.cpu",resourcequotaindex="0",tenant="solar"} 2
capsule_tenant_resource_limit{resource="requests.memory",resourcequotaindex="0",tenant="solar"} 2.147483648e+09

# HELP capsule_tenant_resource_usage Current resource usage for a given resource in a tenant
# TYPE capsule_tenant_resource_usage gauge
capsule_tenant_resource_usage{resource="limits.cpu",resourcequotaindex="0",tenant="solar"} 0
capsule_tenant_resource_usage{resource="limits.memory",resourcequotaindex="0",tenant="solar"} 0
capsule_tenant_resource_usage{resource="namespaces",resourcequotaindex="",tenant="solar"} 2
capsule_tenant_resource_usage{resource="pods",resourcequotaindex="1",tenant="solar"} 0
capsule_tenant_resource_usage{resource="requests.cpu",resourcequotaindex="0",tenant="solar"} 0
capsule_tenant_resource_usage{resource="requests.memory",resourcequotaindex="0",tenant="solar"} 0

Custom Metrics

You can gather more information based on the status of the tenants. These can be scrapped via Kube-State-Metrics CustomResourcesState Metrics. With these you have the possibility to create custom metrics based on the status of the tenants.

Here as an example with the kube-prometheus-stack chart, set the following values:

kube-state-metrics:
  rbac:
    extraRules:
      - apiGroups: [ "capsule.clastix.io" ]
        resources: ["tenants"]
        verbs: [ "list", "watch" ]
  customResourceState:
    enabled: true
    config:
      spec:
        resources:
          - groupVersionKind:
              group: capsule.clastix.io
              kind: "Tenant"
              version: "v1beta2"
            labelsFromPath:
              name: [metadata, name]
            metrics:
              - name: "tenant_size"
                help: "Count of namespaces in the tenant"
                each:
                  type: Gauge
                  gauge:
                    path: [status, size]
                commonLabels:
                  custom_metric: "yes"
                labelsFromPath:
                  capsule_tenant: [metadata, name]
                  kind: [ kind ]
              - name: "tenant_state"
                help: "The operational state of the Tenant"
                each:
                  type: StateSet
                  stateSet:
                    labelName: state
                    path: [status, state]
                    list: [Active, Cordoned]
                commonLabels:
                  custom_metric: "yes"
                labelsFromPath:
                  capsule_tenant: [metadata, name]
                  kind: [ kind ]
              - name: "tenant_namespaces_info"
                help: "Namespaces of a Tenant"
                each:
                  type: Info
                  info:
                    path: [status, namespaces]
                    labelsFromPath:
                      tenant_namespace: []
                commonLabels:
                  custom_metric: "yes"
                labelsFromPath:
                  capsule_tenant: [metadata, name]
                  kind: [ kind ]

This example creates three custom metrics:

tenant_size is a gauge that counts the number of namespaces in the tenant.
tenant_state is a state set that shows the operational state of the tenant.
tenant_namespaces_info is an info metric that shows the namespaces of the tenant.

4.6.2 - Backup & Restore

Run Backups and Restores of Tenants

Velero is a backup and restore solution that performs data protection, disaster recovery and migrates Kubernetes cluster from on-premises to the Cloud or between different Clouds.

When coming to backup and restore in Kubernetes, we have two main requirements:

Configurations backup
Data backup

Assuming you have multiple tenants managed by Capsule, for example wind and water, as cluster admin, you can to take care of scheduling backups for:

Tenant cluster resources
Namespaces belonging to each tenant

Create backup of a tenant

Create a backup of the tenant solar. It consists in two different backups:

backup of the tenant resource
backup of all the resources belonging to the tenant

To back up the solar tenant selectively, label the tenant as:

kubectl label tenant wind capsule.clastix.io/tenant=solar

and create the backup

velero create backup solar-tenant \
    --include-cluster-resources=true \
    --include-resources=tenants.capsule.clastix.io \
    --selector capsule.clastix.io/tenant=solar

resulting in the following Velero object:

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: solar-tenant
spec:
  defaultVolumesToRestic: false
  hooks: {}
  includeClusterResources: true
  includedNamespaces:
  - '*'
  includedResources:
  - tenants.capsule.clastix.io
  labelSelector:
    matchLabels:
      capsule.clastix.io/tenant: solar
  metadata: {}
  storageLocation: default
  ttl: 720h0m0s

Create a backup of all the resources belonging to the solar tenant namespaces:

velero create backup solar-namespaces \
    --include-cluster-resources=false \
    --include-namespaces solar-production,solar-development,solar-marketing

resulting in the following Velero object:

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: solar-namespaces
spec:
  defaultVolumesToRestic: false
  hooks: {}
  includeClusterResources: false
  includedNamespaces:
  - solar-production
  - solar-development
  - solar-marketing
  metadata: {}
  storageLocation: default
  ttl: 720h0m0s

Velero requires an Object Storage backend where to store backups, you should take care of this requirement before using Velero.

Restore a tenant from the backup

To recover the tenant after a disaster, or to migrate it to another cluster, create a restore from the previous backups:

velero create restore --from-backup solar-tenant
velero create restore --from-backup solar-namespaces

kubectl get tnt
NAME   STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR     AGE
water    active   9                 5                 {"pool":"water"}    34m
wind  active   9                 8                 {"pool":"wind"}  33m
solar    active   9                 0 # <<<           {"pool":"solar"}    54m

To avoid this problem you can use the script velero-restore.sh located under the hack/ folder:

./velero-restore.sh --kubeconfig /path/to/your/kubeconfig --tenant "wind" restore

Running this command, we are going to patch the tenant’s namespaces manifests that are actually ownerReferences-less. Once the command has finished its run, you got the tenant back.

kubectl get tnt
NAME   STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR     AGE
water    active   9                 5                 {"pool":"water"}    44m
solar  active   9                 8                 {"pool":"wind"}  43m
wind    active   9                 3 # <<<           {"pool":"solar"}    12s

5 - Tenants

Understand principles and concepts of Capsule Tenants

Capsule is a framework to implement multi-tenant and policy-driven scenarios in Kubernetes. In this tutorial, we’ll focus on a hypothetical case covering the main features of the Capsule Operator. This documentation is styled in a tutorial format, and it’s designed to be read in sequence. We’ll start with the basics and then move to more advanced topics.

Acme Corp, our sample organization, is building a Container as a Service platform (CaaS) to serve multiple lines of business, or departments, e.g. Solar, Wind, Water. Each department has its team of engineers that are responsible for the development, deployment, and operating of their digital products. We’ll work with the following actors:

Bill: the cluster administrator from the operations department of Acme Corp.
Alice: the project leader in the Solar & Green departments. She is responsible for a team made of different job responsibilities: e.g. developers, administrators, SRE engineers, etc.
Joe: works as a lead developer of a distributed team in Alice’s organization.
Bob: is the head of engineering for the Water department, the main and historical line of business at Acme Corp.

This scenario will guide you through the following topics.

5.1 - Namespaces

Namespace to Tenant relation

Assignment

Alice, once logged with her credentials, can create a new Namespace in her Tenant, as simply issuing:

kubectl create ns solar-production

Alice started the name of the Namespace prepended by the name of the Tenant: this is not a strict requirement but it is highly suggested because it is likely that many different Tenants would like to call their Namespaces production, test, or demo, etc. The enforcement of this naming convention is optional and can be controlled by the cluster administrator with forceTenantPrefix option.

Alice can deploy any resource in any of the Namespaces. That is because she is the owner of the tenant solar and therefore she has full control over all Namespaces assigned to that Tenant.

kubectl -n solar-development run nginx --image=docker.io/nginx
kubectl -n solar-development get pods

Every Namespaces assigned to a Tenant has an owner reference pointing to the Tenant object itself. In Addition each Namespaces has a label capsule.clastix.io/tenant=<tenant_name> identifying the Tenant it belongs to (Read More).

The Namespaces are tracked as part of the Tenant status:

$ kubectl get tnt solar -o yaml
...
status:
  ...

  # Simplie list of namespaces
  namespaces:
  - solar-dev
  - solar-prod
  - solar-test

  # Size (Amount of namespaces)
  size: 3

  # Detailed information about each namespace
  spaces:
  - conditions:
    - lastTransitionTime: "2025-12-04T10:23:17Z"
      message: reconciled
      reason: Succeeded
      status: "True"
      type: Ready
    - lastTransitionTime: "2025-12-04T10:23:17Z"
      message: not cordoned
      reason: Active
      status: "False"
      type: Cordoned
    metadata: {}
    name: solar-prod
    uid: ad8ea663-9457-4b00-ac67-0778c4160171
  - conditions:
    - lastTransitionTime: "2025-12-04T10:23:25Z"
      message: reconciled
      reason: Succeeded
      status: "True"
      type: Ready
    - lastTransitionTime: "2025-12-04T10:23:25Z"
      message: not cordoned
      reason: Active
      status: "False"
      type: Cordoned
    metadata: {}
    name: solar-test
    uid: 706e3d30-af2b-4acc-9929-acae7b887ab9
  - conditions:
    - lastTransitionTime: "2025-12-04T10:23:33Z"
      message: reconciled
      reason: Succeeded
      status: "True"
      type: Ready
    - lastTransitionTime: "2025-12-04T10:23:33Z"
      message: not cordoned
      reason: Active
      status: "False"
      type: Cordoned
    metadata: {}
    name: solar-dev
    uid: e4af5283-aad8-43ef-b8b8-abe7092e25d0

By default the following rules apply for namespaces:

A Namespace can not be moved from a Tenant to another one (or anywhere else).
Namespaces are deleted when the Tenant is deleted.

If you feel like these rules are too restrictive, you must implement your own custom logic to handle these cases, for example, with Finalizers for Namespaces.

If namespaces are not correctly assigned to tenants, make sure to evaluate your Capsule Users Configuration.

Multiple Tenants

A single team is likely responsible for multiple lines of business. For example, in our sample organization Acme Corp., Alice is responsible for both the Solar and Green lines of business. It’s more likely that Alice requires two different Tenants, for example, solar and green to keep things isolated.

By design, the Capsule operator does not permit a hierarchy of Tenants, since all Tenants are at the same levels. However, we can assign the ownership of multiple Tenants to the same user or group of users.

Bill, the cluster admin, creates multiple Tenants having alice as owner:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User

and

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: green
spec:
  owners:
  - name: alice
    kind: User

Alternatively, the ownership can be assigned to a group called solar-and-green for both Tenants:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: solar-and-green
    kind: Group

See Ownership for more details on how to assign ownership to a group of users.

The two tenants remain isolated from each other in terms of resources assignments, e.g. ResourceQuotas, Nodes, StorageClasses and IngressClasses, and in terms of governance, e.g. NetworkPolicies, PodSecurityPolicies, Trusted Registries, etc.

When Alice logs in, she has access to all namespaces belonging to both the solar and green Tenants.

Tenant Prefix

We recommend to use the forceTenantPrefix for production environments.

If the forceTenantPrefix option is enabled, which is not the case by default, the Namespaces are automatically assigned to the right tenant by Capsule because the operator does a lookup on the tenant names.

For example, Alice creates a Namespace called solar-production and green-production:

kubectl create ns solar-production
kubectl create ns green-production

And they are assigned to the Tenant based on their prefix:

$ kubectl get tnt
NAME    STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR   AGE
green   Active                     1                                 3m26s
solar   Active                     1                                 3m26s

However alice can create any Namespace, which does not have a prefix of any of the Tenants she owns, for example production:

$ kubectl create ns production
Error from server (Forbidden): admission webhook "owner.namespace.capsule.clastix.io" denied the request: The Namespace prefix used doesn't match any available Tenant

Label

The default behavior, if the forceTenantPrefix option is not enabled, Alice needs to specify the Tenant name as a label capsule.clastix.io/tenant=<desired_tenant> in the Namespace manifest:

kind: Namespace
apiVersion: v1
metadata:
  name: solar-production
  labels:
    capsule.clastix.io/tenant: solar

If not specified, Capsule will deny with the following message: Unable to assign Namespace to Tenant:

$ kubectl create ns solar-production
Error from server (Forbidden): admission webhook "owner.namespace.capsule.clastix.io" denied the request: Please use capsule.clastix.io/tenant label when creating a namespace

Administration

Administration of Namespaces is done by the Administrators, who can assign Namespaces to Tenants, Migrate Namespaces between Tenants and remove Namespaces from Tenants.

Adding Namespace to a Tenant

When you want to join existing Namespace to a Tenant, you can use the following command to set the OwnerReference of the Namespace to point to the Tenant definition. This is mainly intended to join existing Namespaces to a Tenant that were created before the Tenant itself.

export TARGET_TENANT_NAME=green
export TARGET_NAMESPACE=green
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: $TARGET_NAMESPACE
  labels:
    capsule.clastix.io/tenant: $TARGET_TENANT_NAME
  ownerReferences:
  - apiVersion: capsule.clastix.io/v1beta2
    kind: Tenant
    name: $TARGET_TENANT_NAME
    uid: $(kubectl get tnt/$TARGET_TENANT_NAME -o jsonpath='{.metadata.uid}')
EOF

Migrate Namespace between Tenants

When you want to migrate existing Namespace from a Tenant to another Tenant, you can use the following command to set the OwnerReference of the Namespace to point to the new Tenant definition.

export TARGET_TENANT_NAME=wind
export TARGET_NAMESPACE=green
kubectl patch namespace "$TARGET_NAMESPACE" \
  --type=merge \
  --patch "$(cat <<EOF
{
  "metadata": {
    "labels": {
      "capsule.clastix.io/tenant": "$TARGET_TENANT_NAME"
    },
    "ownerReferences": [
      {
        "apiVersion": "capsule.clastix.io/v1beta2",
        "kind": "Tenant",
        "name": "$TARGET_TENANT_NAME",
        "uid": "$(kubectl get tenant "$TARGET_TENANT_NAME" -o jsonpath='{.metadata.uid}')",
        "controller": true,
        "blockOwnerDeletion": true
      }
    ]
  }
}
EOF
)"

Remove Namespace from a Tenant

When you want to remove existing Namespace from a Tenant, you can use the following command to remove the OwnerReference of the Namespace to point to the Tenant definition. Removing a Namespace from a Tenant will not delete the Namespace itself, but it will remove the Tenant’s control over it and other artifacts like LimitRange, ResourceQuota, NetworkPolicy, RuleStatus etc. will not be applied anymore.

export TARGET_NAMESPACE=green
kubectl patch namespace "$TARGET_NAMESPACE" \
  --type=merge \
  --patch '{"metadata":{"labels":{"capsule.clastix.io/tenant":null},"ownerReferences":[]}}'

Namespace Scope

By setting enforcement at the Namespace level, i.e. spec.resourceQuotas.scope=Namespace, Capsule does not aggregate the resources usage and all enforcement is done at the Namespace level.

Namespace Quotas

The cluster admin, can control how many Namespaces Alice, creates by setting a quota in the Tenant manifest spec.namespaceOptions.quota:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  namespaceOptions:
    quota: 3

Alice can create additional Namespaces according to the quota:

kubectl create ns solar-development
kubectl create ns solar-test

While Alice creates Namespaces, the Capsule controller updates the status of the Tenant so Bill, the cluster admin, can check the status:

$ kubectl describe tenant solar
...
status:
  Namespaces:
    solar-development
    solar-production
    solar-test
  Size:   3 # current namespace count
  State:  Active
...

Once the Namespace quota assigned to the tenant has been reached, Alice cannot create further Namespaces:

$ kubectl create ns solar-training
Error from server (Cannot exceed Namespace quota: please, reach out to the system administrators):
admission webhook "namespace.capsule.clastix.io" denied the request.

The enforcement on the maximum number of Namespaces per Tenant is the responsibility of the Capsule controller via its Dynamic Admission Webhook capability.

Termination

Capsule keeps it’s managed resources as long as possible. Meaning even if a Namespace is terminated we verify the following things, before any capsule managed resources are finally removed:

There are no pods in the namespace left (including pods with finalizers). Users must take own action to resolve finalizing pods.
All other namespaced resources have removed finalizers and are deleted (capsule will initiate that step)
Finally capsule managed resources are removed.

This ensures proper cleanup of namespaces without having namespaces being stuck in Terminating phase and requiring administrator attention.

If you are running a Kubernetes Version below 1.33 you should make sure, that the FeatureGate OrderedNamespaceDeletion is enabled. This already enforces this order by default:

All Pods must be absent
Only after all pods are absent other namespaced items are removed.

As stated in the following KEP.

Cordon

It is possible to cordon a Namespace from a Tenant, preventing anything from being changed within this Namespace. This is useful for production Namespaces where you want to avoid any accidental changes or if you have some sort of change freeze period.

This action can be performed by the TenantOwner by adding the label projectcapsule.dev/cordoned=true to the Namespace:

kubectl patch namespace solar-production --patch '{"metadata": {"labels": {"projectcapsule.dev/cordoned": "true"}}}' --as alice --as-group projectcapsule.dev

To uncordon the Namespace, simply remove the label or set it to false:

kubectl patch namespace solar-production --patch '{"metadata": {"labels": {"projectcapsule.dev/cordoned": "false"}}}' --as alice --as-group projectcapsule.dev

Note: If the entire Tenant is cordoned all Namespaces within the Tenant will be cordoned as well. Meaning a single Namespace can not be uncordoned if the Tenant is cordoned.

5.2 - Permissions

Grant permissions for tenants

Administrators

Administrators are users that have full control over all Tenants and their namespaces. They are typically cluster administrators or operators who need to manage the entire cluster and all its Tenants. However as administrator you are automatically Owner of all Tenants.Tenants This means that administrators can create, delete, and manage namespaces and other resources within any Tenant, given you are using label assignments for tenants.

Ownership

Capsule introduces the principal, that tenants must have owners (Tenant Owners). The owner of a tenant is a user or a group of users that have the right to create, delete, and manage the tenant’s namespaces and other tenant resources. However an owner does not have the permissions to manage the tenants they are owner of. This is still done by cluster-administrators.

At any time you are able to verify which users or groups are owners of a tenant by checking the owners field of the Tenant status subresource:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
...
status:
  owners:
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: Group
    name: oidc:org:devops:a
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    - mega-admin
    - controller
    kind: ServiceAccount
    name: system:serviceaccount:capsule:controller
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: User
    name: alice

To explain these entries, let’s inspect one of them:

kind: It can be User, Group or ServiceAccount
name: Is the reference name of the user, group or serviceaccount we want to bind
clusterRoles: ClusterRoles which are bound for each namespace of the tenant to the owner. By default, Capsule assigns admin and capsule-namespace-deleter roles to each owner, but you can customize them as explained in Owner Roles section.

With this information available you

Tenant Owners

Tenant Owners can be declared as dedicated cluster scoped Resources called TenantOwner. This allows the cluster admin to manage the ownership of tenants in a more flexible way, for example by adding labels and annotations to the TenantOwner resources.

apiVersion: capsule.clastix.io/v1beta2
kind: TenantOwner
metadata:
  labels:
    team: devops
  name: devops
spec:
  kind: Group
  name: "oidc:org:devops:a"

This TenantOwner can now be matched by any tenant. Essentially we define on a per tenant basis which TenantOwners should be owners of the tenant (Each item under spec.permissions.matchOwners is understood as OR selection.):

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  labels:
    kubernetes.io/metadata.name: solar
  name: solar
spec:
  permissions:
    matchOwners:
      - matchLabels:
          team: devops
      - matchLabels:
          customer: x

Since the ownership is now loosely coupled, all TenantOwners matching the given labels will be owners of the tenant. We can verify this via the .status.owners field of the Tenant resource:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
...
status:
  owners:
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: Group
    name: oidc:org:devops:a
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: User
    name: alice

This can also be combined with direct owner declarations. In the example, both alice user and all TenantOwners with label team: devops and TenantOwners with label customer: x will be owners of the solar tenant.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: User
    name: alice
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: ServiceAccount
    name: system:serviceaccount:capsule:controller
  permissions:
    matchOwners:
    - matchLabels:
        team: devops
    - matchLabels:
        customer: x

If we create a TenantOwner where the .spec.name and .spec.kind matches one of the owners declared in the tenant, the entries wille be merged. That’s mainly relevant for the clusterRoles:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantOwner
metadata:
  labels:
    customer: x
  name: controller
spec:
  kind: ServiceAccount
  name: "system:serviceaccount:capsule:controller"
  clusterRoles:
    - "mega-admin"
    - "controller"

Again we can verify the resulting owners via the .status.owners field of the Tenant resource:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
...
status:
  owners:
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: Group
    name: oidc:org:devops:a
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    - mega-admin
    - controller
    kind: ServiceAccount
    name: system:serviceaccount:capsule:controller
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: User
    name: alice

We can see that the system:serviceaccount:capsule:controller ServiceAccount now has additional mega-admin and controller roles assigned.

Implicit Tenant Assignment

If a TenantOwner is created all Tenants are always matching the label projectcapsule.dev/tenant on TenantOwner with the name of the Tenant. This means that if you create a TenantOwner with the name solar, it will automatically become owner of the solar Tenant. This can only be done for one tenant at a time (because the label is unique) and is intended that way.

apiVersion: capsule.clastix.io/v1beta2
kind: TenantOwner
metadata:
  labels:
    projectcapsule.dev/tenant: "solar"
  name: solar-test-gitops-reconciler
spec:
  kind: ServiceAccount
  name: "system:serviceaccount:solar-test:gitops-reconciler"

With this Tenant specification:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec: {}

We can observe that the owner is automatically assigned for the Tenant solar:

kubectl get tnt solar -o yaml

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  cordoned: false
  preventDeletion: false
status:
  owners:
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: ServiceAccount
    name: system:serviceaccount:solar-test:gitops-reconciler

Aggregation

All subjects defined in TenantOwner resources are automatically considered Capsule Users and don’t need to mentioned further in the CapsuleConfiguration User Scope. If you don’t want this behavior, you can disable it by setting aggregate: false in the TenantOwner spec:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantOwner
metadata:
  labels:
    customer: x
  name: controller
spec:
  kind: ServiceAccount
  name: "system:serviceaccount:capsule:controller"
  aggregate: false

Users

Bill, the cluster admin, receives a new request from Acme Corp’s CTO asking for a new Tenant to be onboarded and Alice user will be the TenantOwner. Bill then assigns Alice’s identity of alice in the Acme Corp. identity management system. Since Alice is a TenantOwner, Bill needs to assign alice the Capsule group defined by –capsule-user-group option, which defaults to projectcapsule.dev.

To keep things simple, we assume that Bill just creates a client certificate for authentication using X.509 Certificate Signing Request, so Alice’s certificate has "/CN=alice/O=projectcapsule.dev".

Bill creates a new Tenant solar in the CaaS management portal according to the Tenant’s profile:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User

Bill checks if the new Tenant is created and operational:

kubectl get tenant solar
NAME   STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR   AGE
solar    Active                     0                                 33m

Note that namespaces are not yet assigned to the new Tenant. The Tenant owners are free to create their namespaces in a self-service fashion and without any intervention from Bill.

Once the new Tenant solar is in place, Bill sends the login credentials to Alice. Alice can log in using her credentials and check if she can create a namespace

kubectl auth can-i create namespaces
yes

or even delete the namespace

kubectl auth can-i delete ns -n solar-production
yes

However, cluster resources are not accessible to Alice

kubectl auth can-i get namespaces
no

kubectl auth can-i get nodes
no

kubectl auth can-i get persistentvolumes
no

including the Tenant resources

kubectl auth can-i get tenants
no

Groups

In the example above, Bill assigned the ownership of solar Tenant to alice user. If another user, e.g. Bob needs to administer the solar Tenant, Bill can assign the ownership of solar Tenant to such user too:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  - name: bob
    kind: User

However, it’s more likely that Bill assigns the ownership of the solar Tenant to a group of users instead of a single one, especially if you use OIDC Authentication. Bill creates a new group account solar-users in the Acme Corp. identity management system and then he assigns Alice and Bob identities to the solar-users group.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: solar-users
    kind: Group

With the configuration above, any user belonging to the solar-users group will be the owner of the solar Tenant with the same permissions of Alice. For example, Bob can log in with his credentials and issue

kubectl auth can-i create namespaces
yes

All the groups you want to promote to TenantOwners must be part of the Group Scope. You have to add solar-users to the CapsuleConfiguration Group Scope to make it work.

ServiceAccounts

You can use the Group subject to grant ServiceAccounts the ownership of a Tenant. For example, you can create a group of ServiceAccounts and assign it to the Tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: system:serviceaccount:tenant-system:robot
    kind: ServiceAccount

Bill can create a ServiceAccount called robot, for example, in the tenant-system namespace and leave it to act as TenantOwner of the solar Tenant

kubectl --as system:serviceaccount:tenant-system:robot --as-group projectcapsule.dev auth can-i create namespaces
yes

since each service account in a namespace is a member of following group:

system:serviceaccounts:{service-account-namespace}

You have to add system:serviceaccounts:{service-account-namespace} to the CapsuleConfiguration Group Scope or system:serviceaccounts:{service-account-namespace}:{service-account-name} to the CapsuleConfiguration User Scope to make it work.

Owner Roles

By default, all TenantOwners will be granted with two ClusterRole resources using the RoleBinding API:

admin: the Kubernetes default one, admin, that grants most of the namespace scoped resources
capsule-namespace-deleter: a custom clusterrole, created by Capsule, allowing to delete the created namespaces

You can observe this behavior when you get the Tenant solar:

$ kubectl get tnt solar -o yaml
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  labels:
    kubernetes.io/metadata.name: solar
  name: solar
spec:
  ingressOptions:
    hostnameCollisionScope: Disabled
  limitRanges: {}
  networkPolicies: {}
  owners:
  # -- HERE -- #
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: User
    name: alice
    labels:
      projectcapsule.dev/sample: "true"
    annotations:
      projectcapsule.dev/sample: "true"
  resourceQuotas:
    scope: Tenant
status:
  namespaces:
  - solar-production
  - solar-system
  size: 2
  state: Active

In the example below, assuming the TenantOwner creates a namespace solar-production in Tenant solar, you’ll see the Role Bindings giving the TenantOwner full permissions on the Tenant namespaces:

$ kubectl get rolebinding -n solar-production
NAME                                        ROLE                                    AGE
capsule-solar-0-admin                       ClusterRole/admin                       111m
capsule-solar-1-capsule-namespace-deleter   ClusterRole/capsule-namespace-deleter   111m

When Alice creates the namespaces, the Capsule controller assigns to Alice the following permissions, so that Alice can act as the admin of all the Tenant namespaces:

$ kubectl get rolebinding -n solar-production -o yaml
apiVersion: v1
items:
- apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    creationTimestamp: "2024-02-25T14:02:36Z"
    labels:
      capsule.clastix.io/role-binding: 8fb969aaa7a67b71
      capsule.clastix.io/tenant: solar
      projectcapsule.dev/sample: "true"
    annotations:
      projectcapsule.dev/sample: "true"
    name: capsule-solar-0-admin
    namespace: solar-production
    ownerReferences:
    - apiVersion: capsule.clastix.io/v1beta2
      blockOwnerDeletion: true
      controller: true
      kind: Tenant
      name: solar
      uid: 1e6f11b9-960b-4fdd-82ee-7cd91a2db052
    resourceVersion: "2980"
    uid: 939da5ae-7fec-4300-8db2-223d3049b43f
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: admin
  subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: alice
- apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    creationTimestamp: "2024-02-25T14:02:36Z"
    labels:
      capsule.clastix.io/role-binding: b8822dde20953fb1
      capsule.clastix.io/tenant: solar
      projectcapsule.dev/sample: "true"
    annotations:
      projectcapsule.dev/sample: "true"
    name: capsule-solar-1-capsule-namespace-deleter
    namespace: solar-production
    ownerReferences:
    - apiVersion: capsule.clastix.io/v1beta2
      blockOwnerDeletion: true
      controller: true
      kind: Tenant
      name: solar
      uid: 1e6f11b9-960b-4fdd-82ee-7cd91a2db052
    resourceVersion: "2982"
    uid: bbb4cd79-ce0d-41b0-a52d-dbed71a9b48a
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: capsule-namespace-deleter
  subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: alice
kind: List
metadata:
  resourceVersion: ""

In some cases, the cluster admin needs to narrow the range of permissions assigned to TenantOwners by assigning a Cluster Role with less permissions than above. Capsule supports the dynamic assignment of any ClusterRole resources for each TenantOwner.

For example, assign user Joe the Tenant ownership with only view permissions on Tenant namespaces:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  - name: joe
    kind: User
    clusterRoles:
      - view

you’ll see the new Role Bindings assigned to Joe:

$ kubectl get rolebinding -n solar-production
NAME                                        ROLE                                    AGE
capsule-solar-0-admin                       ClusterRole/admin                       114m
capsule-solar-1-capsule-namespace-deleter   ClusterRole/capsule-namespace-deleter   114m
capsule-solar-2-view                        ClusterRole/view                        1s

so that Joe can only view resources in the Tenant namespaces:

kubectl --as joe --as-group projectcapsule.dev auth can-i delete pods -n solar-production
no

Please, note that, despite created with more restricted permissions, a TenantOwner can still create namespaces in the Tenant because he belongs to the projectcapsule.dev group. If you want a user not acting as TenantOwner, but still operating in the Tenant, you can assign additional RoleBindings without assigning him the Tenant ownership.

Custom ClusterRoles are also supported. Assuming the cluster admin creates:

kubectl apply -f - << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant-resources
rules:
- apiGroups: ["capsule.clastix.io"]
  resources: ["tenantresources"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
EOF

These permissions can be granted to Joe

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  - name: joe
    kind: User
    clusterRoles:
      - view
      - tenant-resources

For the given configuration, the resulting RoleBinding resources are the following ones:

$ kubectl -n solar-production get rolebindings
NAME                                              ROLE                                            AGE
capsule-solar-0-admin                               ClusterRole/admin                               90s
capsule-solar-1-capsule-namespace-deleter           ClusterRole/capsule-namespace-deleter           90s
capsule-solar-2-view                                ClusterRole/view                                90s
capsule-solar-3-tenant-resources                    ClusterRole/prometheus-servicemonitors-viewer   25s

Role Aggregation

Sometimes the admin role is missing certain permissions. You can aggregate the admin role with a custom role, for example, gateway-resources:

kubectl apply -f - << EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: gateway-resources
  labels:
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
- apiGroups: ["gateway.networking.k8s.io"]
  resources: ["gateways"]
  verbs: ["*"]
EOF

Proxy Owner Authorization

This feature will be deprecated in a future release of Capsule. Instead use ProxySettings

When you are using the Capsule Proxy, the tenant owner can list the cluster-scoped resources. You can control the permissions to cluster scoped resources by defining proxySettings for a tenant owner.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: joe
    kind: User
    clusterRoles:
      - view
      - tenant-resources

Promotion

As Tenant Owner you can perform ServiceAccount Promotion.

Owner Promotion

Within a Tenant, a ServiceAccount can be promoted to a TenantOwner. For example, Alice can create a ServiceAccount called robot in the solar Tenant and promote it to be a TenantOwner (This requires Alice to be an owner of the Tenant as well):

kubectl label sa gitops-reconcile -n green-test owner.projectcapsule.dev/promote=true --as alice --as-group projectcapsule.dev

Note: Promotion is only triggered on the label owner.projectcapsule.dev/promote with the value true

We can now verify if the promotion was successful by checking the Tenant status:

kubectl get tnt green  -o jsonpath='{.status.owners}' | jq

[
  {
    "clusterRoles": [
      "capsule-namespace-provisioner",
      "capsule-namespace-deleter"
    ],
    "kind": "ServiceAccount",
    "name": "system:serviceaccount:green-test:gitops-reconcile"
  },
 {
    "clusterRoles": [
      "view",
      "tenant-resources"
    ],
    "kind": "User",
    "name": "joe"
  }
]

Now the ServiceAccount robot can create namespaces in the solar Tenant:

kubectl create ns green-valkey--as system:serviceaccount:green-test:gitops-reconcile

To revoke the promotion, Alice can just remove the label:

kubectl label sa gitops-reconcile -n green-test owner.projectcapsule.dev/promote-  --as alice --as-group projectcapsule.dev

This feature must be enabled in the CapsuleConfiguration. The ClusterRoles assigned to promoted ServiceAccounts can be configured in the CapsuleConfiguration as well.

You can also dis/enable Owner Promotion per Tenant. By default it’s enabled, however since it’s disabled in the CapsuleConfiguration it can’t be used, unless that’s enabled as well.

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  permissions:
    promotions:
      allowOwnerPromotion: false

Rule based Promotion

Additional Rolebindings

With Tenant rolebindings you can distribute namespaced rolebindings to all namespaces which are assigned to a namespace. Essentially it is then ensured the defined rolebindings are present and reconciled in all namespaces of the Tenant. This is useful if users should have more insights on Tenant basis. Let’s look at an example.

Assuming a cluster-administrator creates the following clusterRole:

kubectl apply -f - << EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: prometheus-servicemonitors-viewer
rules:
- apiGroups: ["monitoring.coreos.com"]
  resources: ["servicemonitors"]
  verbs: ["get", "list", "watch"]
EOF

Now the cluster-administrator creates wants to bind this clusterRole in each namespace of the solar Tenant. He creates a tenantRoleBinding:

kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  additionalRoleBindings:
  - clusterRoleName: 'prometheus-servicemonitors-viewer'
    subjects:
    - apiGroup: rbac.authorization.k8s.io
      kind: User
      name: joe
    labels:
      projectcapsule.dev/sample: "true"
    annotations:
      projectcapsule.dev/sample: "true"
EOF

As you can see the subjects is a classic rolebinding subject. This way you grant permissions to the subject user Joe, who only can list and watch servicemonitors in the solar tenant namespaces, but has no other permissions.

Strict

If you have strict RBAC enabled for the controller, you need to ensure that the controller ServiceAccount is allowed to create RoleBindings for the specified ClusterRole. Because of RBAC escalation prevention, the controller must either hold the bind verb on that ClusterRole or hold all of its permissions. The default owner ClusterRoles (admin, capsule-namespace-provisioner and capsule-namespace-deleter) are covered out of the box; for any other ClusterRole you have two options.

The narrowest option is adding the ClusterRole to manager.rbac.bindableClusterRoles, which grants the controller the bind verb on it — note that the list replaces the default, so keep admin in it. For the above example:

manager:
  rbac:
    bindableClusterRoles:
      - admin
      - prometheus-servicemonitors-viewer

Alternatively, you can aggregate the ClusterRole’s permissions into the controller. The Controller Aggregates ClusterRoles with the labels (OR):

projectcapsule.dev/aggregate-to-controller: "true"
projectcapsule.dev/aggregate-to-controller-instance: {{ .Release.Name }}

So for the above example, you can label the prometheus-servicemonitors-viewer ClusterRole like this:

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: prometheus-servicemonitors-viewer
  labels:
    projectcapsule.dev/aggregate-to-controller: "true"
rules:
- apiGroups: ["monitoring.coreos.com"]
  resources: ["servicemonitors"]
  verbs: ["get", "list", "watch"]

Built-in ClusterRoles

We strongly recommend you use custom ClusterRoles for your Tenant rolebindings, but you can also use built-in ClusterRoles (admin (default for Tenant Owners), view and edit). For example, if you want to give the view permissions to Joe in all namespaces of the solar Tenant, you can use the built-in view ClusterRole.

In that case it also makes sense to use ClusterRole Aggregation. In the following example we are creating custom aggregated ClusterRoles for these three built-in clusterroles, to allow interactions with the GatewayAPI resources:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant:admins:extension
  labels:
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways
      - httproutes
      - grpcroutes
      - tlsroutes
      - tcproutes
      - udproutes
      - referencegrants
      - backendtlspolicies
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways/status
      - httproutes/status
      - grpcroutes/status
      - tlsroutes/status
      - tcproutes/status
      - udproutes/status
      - referencegrants/status
      - backendtlspolicies/status
    verbs: ["get"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies
      - backendtrafficpolicies
      - securitypolicies
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies/status
      - backendtrafficpolicies/status
      - securitypolicies/status
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant:members:extension
  labels:
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
rules:
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways
      - httproutes
      - grpcroutes
      - tlsroutes
      - tcproutes
      - udproutes
      - referencegrants
      - backendtlspolicies
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways/status
      - httproutes/status
      - grpcroutes/status
      - tlsroutes/status
      - tcproutes/status
      - udproutes/status
      - referencegrants/status
      - backendtlspolicies/status
    verbs: ["get"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies
      - backendtrafficpolicies
      - securitypolicies
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies/status
      - backendtrafficpolicies/status
      - securitypolicies/status
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant:viewers:extension
  labels:
    rbac.authorization.k8s.io/aggregate-to-view: "true"
rules:
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways
      - httproutes
      - grpcroutes
      - tlsroutes
      - tcproutes
      - udproutes
      - referencegrants
      - backendtlspolicies
    verbs: ["get", "list", "watch"]
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways/status
      - httproutes/status
      - grpcroutes/status
      - tlsroutes/status
      - tcproutes/status
      - udproutes/status
      - referencegrants/status
      - backendtlspolicies/status
    verbs: ["get"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies
      - backendtrafficpolicies
      - securitypolicies
    verbs: ["get", "list", "watch", "create"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies/status
      - backendtrafficpolicies/status
      - securitypolicies/status
    verbs: ["get"]

Selective Distribution

You may have the use-case where you want to distribute different ClusterRoles to different namespaces of the same Tenant. For example, you want to give view permissions to a operational group in all namespaces of the solar Tenant with environment=production label, but you want to give edit permissions to the operations group inall other namespaces. You can achieve this by leveraging GlobalTenantResources:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: operators-rolebindings
spec:
  resyncPeriod: 60s
  resources:
    - namespaceSelector:
        matchExpressions:
        - key: environment
          operator: NotIn
          values:
          - prod
      rawItems:
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: RoleBinding
          metadata:
            name: operators-rw
          subjects:
          - kind: Group
            name: tenant:{{tenant.name}}:operators
            namespace: "{{namespace}}"
          roleRef:
            kind: ClusterRole
            name: view
            apiGroup: rbac.authorization.k8s.io

    - namespaceSelector:
        matchLabels:
          environment: prod
      rawItems:
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: RoleBinding
          metadata:
            name: operators-view-only
          subjects:
          - kind: Group
            name: tenant:{{tenant.name}}:operators
            namespace: "{{namespace}}"
          roleRef:
            kind: ClusterRole
            name: edit
            apiGroup: rbac.authorization.k8s.io

Custom Resources

Capsule grants admin permissions to the TenantOwners but is only limited to their namespaces. To achieve that, it assigns the ClusterRole admin to the TenantOwner. This ClusterRole does not permit the installation of custom resources in the namespaces.

In order to leave the TenantOwner to create Custom Resources in their namespaces, the cluster admin defines a proper Cluster Role. For example:

kubectl apply -f - << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: argoproj-provisioner
rules:
- apiGroups:
  - argoproj.io
  resources:
  - applications
  - appprojects
  verbs:
  - create
  - get
  - list
  - watch
  - update
  - patch
  - delete
EOF

Bill can assign this role to any namespace in the Alice’s Tenant by setting it in the Tenant manifest:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  - name: joe
    kind: User
  additionalRoleBindings:
    - clusterRoleName: 'argoproj-provisioner'
      subjects:
        - apiGroup: rbac.authorization.k8s.io
          kind: User
          name: alice
        - apiGroup: rbac.authorization.k8s.io
          kind: User
          name: joe

With the given specification, Capsule will ensure that all Alice’s namespaces will contain a RoleBinding for the specified Cluster Role. For example, in the solar-production namespace, Alice will see:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: capsule-solar-argoproj-provisioner
  namespace: solar-production
subjects:
  - kind: User
    apiGroup: rbac.authorization.k8s.io
    name: alice
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: argoproj-provisioner

With the above example, Capsule is leaving the TenantOwner to create namespaced custom resources.

Take Note: a TenantOwner having the admin scope on its namespaces only, does not have the permission to create Custom Resources Definitions (CRDs) because this requires a cluster admin permission level. Only Bill, the cluster admin, can create CRDs. This is a known limitation of any multi-tenancy environment based on a single shared control plane.

5.3 - Administration

Administrative controls on tenants

Cordoning

Bill needs to cordon a Tenant and its Namespaces for several reasons:

Avoid accidental resource modification(s) including deletion during a Production Freeze Window
During the Kubernetes upgrade, to prevent any workload updates
During incidents or outages
During planned maintenance of a dedicated nodes pool in a BYOD scenario

With the default installation of Capsule all CREATE, UPDATE and DELETE operations performed by Capsule Users are dropped. Any Updates to Subresources (i.e. status updates) and events are allowed to proceed as usual. If you wish to allow specific Operations, you can change the values for the Cordoning Admission via Values (eg. allow Pod/DELETE operations):

webhooks:
  hooks:
    cordoning:
      matchConditions:

        - name: skip-pod-create-delete
          expression: '!(request.resource.resource == "pods" && request.operation in ["DELETE"])'

        # Default conditions to ignore subresources and events
        - name: ignore-subresources
          expression: '!has(request.subResource) || request.subResource == ""'
        - name: ignore-events
          expression: 'request.resource.resource != "events"'

This is possible by just toggling the specific Tenant specification:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  cordoned: true
  owners:
  - kind: User
    name: alice

Any operation performed by Alice, the TenantOwner, will be rejected by the Admission controller:

kubectl delete pod --all -n solar-test --as alice --as-group projectcapsule.dev

Error from server (Forbidden): admission webhook "cordoning.misc.projectcapsule.dev" denied the request: The current namespace 'solar-test' is cordoned. The attempted operation DELETE for /v1/Pod/nginx-deployment-56f567c7cb-pj86t is not permitted during cordoning status.

Uncordoning can be done by removing the said specification key:

$ cat <<EOF | kubectl apply -f -
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  cordoned: false
  owners:
  - kind: User
    name: alice
EOF

$ kubectl --as alice --as-group projectcapsule.dev -n solar-dev create deployment nginx --image nginx
deployment.apps/nginx created

Status of cordoning is also reported in the state of the Tenant:

kubectl get tenants
NAME     STATE    NAMESPACE QUOTA   NAMESPACE COUNT   NODE SELECTOR    AGE
bronze   Active                     2                                  3d13h
gold     Active                     2                                  3d13h
solar    Cordoned                   4                                  2d11h
silver   Active                     2                                  3d13h

Force Tenant-Prefix

Use this if you want to disable/enable the Tenant name prefix to specific Tenants, overriding global forceTenantPrefix in CapsuleConfiguration. When set to ’true’, it enforces Namespaces created for this Tenant to be named with the Tenant name prefix, separated by a dash (i.e. for Tenant ‘foo’, Namespace names must be prefixed with ‘foo-’), this is useful to avoid Namespace name collision. When set to ‘false’, it allows Namespaces created for this Tenant to be named anything. Overrides CapsuleConfiguration global forceTenantPrefix for the Tenant only. If unset, Tenant uses CapsuleConfiguration’s forceTenantPrefix

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  forceTenantPrefix: true

Deletion Protection

Sometimes it is important to protect business critical Tenants from accidental deletion. This can be achieved by toggling preventDeletion specification key on the Tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  preventDeletion: true

5.4 - Metadata

Inherit additional metadata on Tenant resources.

Managed

By default all namespaced resources within a Namespace which are part of a Tenant labeled at admission with the following labels:

capsule.clastix.io/managed-by: <tenant-name> (Legacy label)
projectcapsule.dev/tenant: <tenant-name>

The labels are used by Capsule to identify resources belonging to a specific tenant. This is currently important for the Capsule Proxy to filter resources accordingly.

Namespaces

ManagedMetadataOnly

Disallows any further metadata to be added to the Namespaces created by the tenant owners.

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  namespaceOptions:
    managedMetadataOnly: true

RequiredMetadata

Deprecated

This feature will be deprecated in a future release of Capsule. Instead use Metadata Rules. Migration Guide Here

The cluster admin can enforce tenant owners to add specific metadata as Labels and Annotations to the Namespaces they create. This is a useful feature to enforce a set of Rules based on Labels.

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  namespaceOptions:
    requiredMetadata:
      labels:
        env: "^(prod|test|dev)$"
      annotations:
        example.corp/cost-center: "^INV-[0-9]{4}$"

If you add these properties to a Tenant, and there’s already a Namespace in that Tenant that does not comply with the required metadata, the Namespace will have admission errors until the required metadata is added to it.

Example with Rules:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  namespaceOptions:
    requiredMetadata:
      labels:
        env: "^(prod|test|dev)$"
      annotations:
        example.corp/cost-center: "^INV-[0-9]{4}$"

  rules:
    # Select a subset of namespaces (enviornment=prod) to allow further registries
    - namespaceSelector:
        matchExpressions:
          - key: env
            operator: In
            values: ["prod"]
      enforce:
        registries:
         -  url: "harbor/v2/prod-registry/.*"
            policy: [ "ifNotPresent" ]

AdditionalMetadataList

Deprecated

This feature will be deprecated in a future release of Capsule. Instead use Metadata Rules. Migration Guide Here

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - name: alice
      kind: User
  namespaceOptions:
    additionalMetadataList:
      - annotations:
          templated-annotation: {{ tenant.name }}
        labels:
          templated-label: {{ namespace }}

The cluster admin can “taint” the namespaces created by tenant owners with additional metadata as labels and annotations. There is no specific semantic assigned to these labels and annotations: they will be assigned to the namespaces in the tenant as they are created. However you have the option to be more specific by selecting to which namespaces you want to assign what kind of metadata:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  namespaceOptions:
    additionalMetadataList:
    # An item without any further selectors is applied to all namspaces
    - annotations:
        storagelocationtype: s3
      labels:
        projectcapsule.dev/backup: "true"

    # Select a subset of namespaces to apply metadata on
    - namespaceSelector:
        matchExpressions:
          - key: projectcapsule.dev/low_security_profile
            operator: NotIn
            values: ["true"]
      labels:
        pod-security.kubernetes.io/enforce: baseline

    - namespaceSelector:
        matchExpressions:
          - key: projectcapsule.dev/low_security_profile
            operator: In
            values: ["true"]
      labels:
        pod-security.kubernetes.io/enforce: privileged

AdditionalMetadata

Deprecated

This feature will be deprecated in a future release of Capsule. Instead use Metadata Rules. Migration Guide Here

The cluster admin can “taint” the namespaces created by tenant owners with additional metadata as labels and annotations. There is no specific semantic assigned to these labels and annotations: they will be assigned to the namespaces in the tenant as they are created. This can help the cluster admin to implement specific use cases as, for example, leave only a given tenant to be backed up by a backup service.

Assigns additional labels and annotations to all namespaces created in the solar tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  namespaceOptions:
    additionalMetadata:
      annotations:
        storagelocationtype: s3
      labels:
        projectcapsule.dev/backup: "true"

When the tenant owner creates a namespace, it inherits the given label and/or annotation:

apiVersion: v1
kind: Namespace
metadata:
  annotations:
    storagelocationtype: s3
  labels:
    capsule.clastix.io/tenant: solar
    kubernetes.io/metadata.name: solar-production
    name: solar-production
    projectcapsule.dev/backup: "true"
  name: solar-production
  ownerReferences:
  - apiVersion: capsule.clastix.io/v1beta2
    blockOwnerDeletion: true
    controller: true
    kind: Tenant
    name: solar
spec:
  finalizers:
  - kubernetes
status:
  phase: Active

Deny labels and annotations on Namespaces

Deprecated

This feature will be deprecated in a future release of Capsule. Instead use Metadata Rules. Migration Guide Here

By default, capsule allows tenant owners to add and modify any label or annotation on their namespaces.

But there are some scenarios, when tenant owners should not have an ability to add or modify specific labels or annotations (for example, this can be labels used in Kubernetes network policies which are added by cluster administrator).

Bill, the cluster admin, can deny Alice to add specific labels and annotations on namespaces:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  namespaceOptions:
    forbiddenAnnotations:
      denied:
          - foo.acme.net
          - bar.acme.net
      deniedRegex: .*.acme.net
    forbiddenLabels:
      denied:
          - foo.acme.net
          - bar.acme.net
      deniedRegex: .*.acme.net
  owners:
  - name: alice
    kind: User

Nodes

Warning

Due to CVE-2021-25735 this feature is only supported for Kubernetes version older than: v1.18.18, v1.19.10, v1.20.6, v1.21.0

When using capsule together with capsule-proxy, Bill can allow Tenant Owners to modify Nodes.

By default, it will allow tenant owners to add and modify any label or annotation on their nodes.

But there are some scenarios, when tenant owners should not have an ability to add or modify specific labels or annotations (there are some types of labels or annotations, which must be protected from modifications - for example, which are set by cloud-providers or autoscalers).

Bill, the cluster admin, can deny Tenant Owners to add or modify specific labels and annotations on Nodes:

apiVersion: capsule.clastix.io/v1beta2
kind: CapsuleConfiguration
metadata:
  name: default
spec:
  nodeMetadata:
    forbiddenAnnotations:
      denied:
        - foo.acme.net
        - bar.acme.net
      deniedRegex: .*.acme.net
    forbiddenLabels:
      denied:
        - foo.acme.net
        - bar.acme.net
      deniedRegex: .*.acme.net
  userGroups:
    - projectcapsule.dev
    - system:serviceaccounts:default

Services

Deprecated

This feature will be deprecated in a future release of Capsule. Instead use Metadata Rules. Migration Guide Here

The cluster admin can “taint” the services created by the tenant owners with additional metadata as labels and annotations.

Assigns additional labels and annotations to all services created in the solar tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  serviceOptions:
    additionalMetadata:
      annotations:
        storagelocationtype: s3
      labels:
        projectcapsule.dev/backup: "true"

When the tenant owner creates a service in a tenant namespace, it inherits the given label and/or annotation:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: solar-production
  labels:
    projectcapsule.dev/backup: "true"
  annotations:
    storagelocationtype: s3
spec:
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  selector:
    run: nginx
  type: ClusterIP

Pods

Deprecated

This feature will be deprecated in a future release of Capsule. Instead use Metadata Rules. Migration Guide Here

The cluster admin can “taint” the pods created by the tenant owners with additional metadata as labels and annotations.

Assigns additional labels and annotations to all services created in the solar tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  podOptions:
    additionalMetadata:
      annotations:
        storagelocationtype: s3
      labels:
        projectcapsule.dev/backup: "true"

When the tenant owner creates a service in a tenant namespace, it inherits the given label and/or annotation:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  namespace: solar-production
  labels:
    projectcapsule.dev/backup: "true"
  annotations:
    storagelocationtype: s3
...

5.5 - Rules

Configure policies and restrictions on a per-Namespace basis with Rules

Rules documentation

The Rules documentation is the main reference for the rules API. Find all mutation and enforcement properties there, together with conditions, audiences, and rule evaluation order.

This page covers Tenant-specific configuration through Tenant.spec.rules: namespace selection, templating, and quota generation.

Enforcement rules allow Bill, the cluster administrator, to set policies and restrictions on a per-Tenant basis. These rules are enforced by Capsule admission webhooks when Alice, the TenantOwner, creates or modifies resources in her Namespaces. With the rule construct, namespaces within the same tenant can be profiled differently depending on their metadata.

Namespace Selector

By default, a rule applies to all namespaces within a Tenant. To apply a rule only to a subset of namespaces, use namespaceSelector. The selector follows the standard Kubernetes label selector semantics.

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  ...
  rules:
    # Matches all Namespaces and enforces the rule for all of them.
    - enforce:
        action: allow
        workloads:
          registries:
            - exact:
                - harbor/v2/customer-registry/debian:latest
              policy: ["IfNotPresent"]

    # Selects a subset of namespaces (environment=prod) to allow additional registries.
    - namespaceSelector:
        matchExpressions:
          - key: environment
            operator: In
            values: ["prod"]
      enforce:
        action: allow
        workloads:
          registries:
            - exp: "harbor/v2/prod-registry/.*"
              policy: ["IfNotPresent"]

Rules are combined together. In this example, all namespaces within the solar tenant can use the exact harbor/v2/customer-registry/debian:latest image, while namespaces labeled with environment=prod can also use images from harbor/v2/prod-registry/*.

Templating

Namespace rule bodies are rendered as Go templates before they are written into the per-namespace RuleStatus. This allows administrators to define generic Tenant rules that are rendered differently for each Namespace. Templates can use the tenant and namespace objects as context, including metadata such as names, labels, and annotations.

For example, the following rule allows an image reference based on the current Tenant and Namespace name:

rules:
  - enforce:
      action: allow
      workloads:
        registries:
          - exact:
              - "{{ .tenant.metadata.name }}/{{ .namespace.metadata.name }}/app:1"

For a Tenant named solar and a Namespace named solar-prod, this rule is rendered into an exact registry reference of solar/solar-prod/app:1.

Labels and annotations can also be used. Because many Kubernetes label and annotation keys contain dashes, dots, or slashes, use the index function when accessing them:

rules:
  - enforce:
      action: allow
      workloads:
        registries:
          - exact:
              - "{{ index .namespace.metadata.labels \"registry-prefix\" }}/app:1"

If the Namespace has the label registry-prefix=harbor/team-a, the rendered registry rule becomes harbor/team-a/app:1.

Templates are rendered after namespaceSelector matching and before rule evaluation. This means a selected rule can use the concrete Namespace context while preserving the original rule order. Rule order remains important because the last matching allow or deny rule wins.

If a template references a missing key, Capsule marks the Tenant as not ready and reports the rendering error in the Tenant status. This prevents partially rendered or ambiguous rules from being applied silently.

Quotas

Tenant rules can generate GlobalResourceQuota objects:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - name: alice
      kind: User
  rules:
    - namespaceSelector:
        matchLabels:
          company.example/tier: application
      quota:
        - name: shared-compute
          hard:
            requests.cpu: "8"
            requests.memory: 16Gi
            limits.cpu: "8"
            limits.memory: 16Gi
        - name: object-counts
          hard:
            services: "20"
            count/horizontalpodautoscalers.autoscaling: "10"

Capsule creates one GlobalResourceQuota per entry in the rule’s quota list. The generated selector combines:

The rule’s namespace selector.
The Tenant membership label.

The Tenant membership requirement prevents a rule from selecting another Tenant’s namespaces. Generated quotas are reconciled and pruned with the Tenant rule lifecycle.

Every quota entry requires a DNS label-compatible name which must be unique across all rules in the Tenant. The name is the durable identity of the generated GlobalResourceQuota: changing limits, selectors, or rule ordering updates the existing object. Renaming or removing the quota entry replaces or deletes it.

Generated resource names use <tenant-name>-<quota-name>.

Quota accounting is independent of a rule’s request audience. An audience can limit other rule behavior but does not partition the shared resource budget. Quota definitions are not copied into namespace RuleStatus objects. They are reconciled directly from the Tenant into cluster-scoped GlobalResourceQuota objects, which remain the authoritative source for admission and accounting.

Existing Tenant.spec.resourceQuotas behavior remains available for compatibility. Use rule-generated or directly managed GlobalResourceQuota objects when an atomic shared limit across namespaces is required.

Migration

This guide demonstrates how to migrate from Tenant.spec.resourceQuotas to rule-generated GlobalResourceQuota objects. The example Tenant has two resource quotas defined:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  namespaceOptions:
    quota: 3
  resourceQuotas:
    scope: Tenant
    items:
    - hard:
        limits.cpu: "8"
        limits.memory: 16Gi
        requests.cpu: "8"
        requests.memory: 16Gi
    - hard:
        pods: "10"

Verify what the current usage is for the Tenant’s namespaces (for all namespace of the Tenant solar):

NAMESPACE    NAME              REQUEST                                                    LIMIT                                                   AGE
solar-dev    capsule-solar-0   requests.cpu: 0/6800m, requests.memory: 0/15616Mi          limits.cpu: 0/5, limits.memory: 0/14848Mi               87m
solar-dev    capsule-solar-1   pods: 0/4                                                                                                          87m
solar-prod   capsule-solar-0   requests.cpu: 600m/7400m, requests.memory: 384Mi/16000Mi   limits.cpu: 1500m/6500m, limits.memory: 768Mi/15616Mi   87m
solar-prod   capsule-solar-1   pods: 3/7                                                                                                          87m
solar-test   capsule-solar-0   requests.cpu: 600m/7400m, requests.memory: 384Mi/16000Mi   limits.cpu: 1500m/6500m, limits.memory: 768Mi/15616Mi   87m
solar-test   capsule-solar-1   pods: 3/7                                                                                                          87m

Now we update the Tenant to propagate the same quotas through rules, we are not removing the old resourceQuotas yet, to avoid any race conditions with the quota usage. The new Tenant spec looks like this:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  namespaceOptions:
    quota: 3
  resourceQuotas:
    scope: Tenant
    items:
    - hard:
        limits.cpu: "8"
        limits.memory: 16Gi
        requests.cpu: "8"
        requests.memory: 16Gi
    - hard:
        pods: "10"
  rules:
    - quota:
        - name: shared-compute
          hard:
            requests.cpu: "8"
            requests.memory: 16Gi
            limits.cpu: "8"
            limits.memory: 16Gi
        - name: object-counts
          hard:
            pods: "10"

After applying we can verify that ResourceQuotas are created for each rule quota. The important value is the actual allocation:

NAMESPACE    NAME                                        REQUEST                                                    LIMIT                                                   AGE

# NEW /OLD
solar-dev    capsule-global-quota-27d5c64b6c978b1b0f9c   pods: 0/4                                                                                                          102m
solar-dev    capsule-solar-1                             pods: 0/4                                                                                                          3h13m

# NEW /OLD
solar-dev    capsule-global-quota-cf84f5e52f0542edd93c   requests.cpu: 0/6800m, requests.memory: 0/15616Mi          limits.cpu: 0/5, limits.memory: 0/14848Mi               102m
solar-dev    capsule-solar-0                             requests.cpu: 0/6800m, requests.memory: 0/15616Mi          limits.cpu: 0/5, limits.memory: 0/14848Mi               3h13m

# NEW /OLD
solar-prod   capsule-global-quota-27d5c64b6c978b1b0f9c   pods: 3/7                                                                                                          102m
solar-prod   capsule-solar-1                             pods: 3/7                                                                                                          3h13m

# NEW /OLD
solar-prod   capsule-global-quota-cf84f5e52f0542edd93c   requests.cpu: 600m/7400m, requests.memory: 384Mi/16000Mi   limits.cpu: 1500m/6500m, limits.memory: 768Mi/15616Mi   102m
solar-prod   capsule-solar-0                             requests.cpu: 600m/7400m, requests.memory: 384Mi/16000Mi   limits.cpu: 1500m/6500m, limits.memory: 768Mi/15616Mi   3h13m

# NEW /OLD
solar-test   capsule-global-quota-27d5c64b6c978b1b0f9c   pods: 3/7                                                                                                          102m
solar-test   capsule-solar-1                             pods: 3/7                                                                                                          3h13m

# NEW /OLD
solar-test   capsule-global-quota-cf84f5e52f0542edd93c   requests.cpu: 600m/7400m, requests.memory: 384Mi/16000Mi   limits.cpu: 1500m/6500m, limits.memory: 768Mi/15616Mi   102m
solar-test   capsule-solar-0                             requests.cpu: 600m/7400m, requests.memory: 384Mi/16000Mi   limits.cpu: 1500m/6500m, limits.memory: 768Mi/15616Mi   3h13m

As you can see, the GlobalResourceQuota objects are created and the actual usage is preserved. Now we can safely remove the old resourceQuotas from the Tenant spec, and the new quotas will be enforced by the rules.

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  namespaceOptions:
    quota: 3
  rules:
    - quota:
        - name: shared-compute
          hard:
            requests.cpu: "8"
            requests.memory: 16Gi
            limits.cpu: "8"
            limits.memory: 16Gi
        - name: object-counts
          hard:
            pods: "10"

After applying we only see the new GlobalResourceQuota objects, and the actual usage is preserved:

NAMESPACE    NAME                                        REQUEST                                                    LIMIT                                                   AGE
solar-dev    capsule-global-quota-27d5c64b6c978b1b0f9c   pods: 0/4                                                                                                          121m
solar-dev    capsule-global-quota-cf84f5e52f0542edd93c   requests.cpu: 0/6800m, requests.memory: 0/15616Mi          limits.cpu: 0/5, limits.memory: 0/14848Mi               121m

solar-prod   capsule-global-quota-27d5c64b6c978b1b0f9c   pods: 3/7                                                                                                          121m
solar-prod   capsule-global-quota-cf84f5e52f0542edd93c   requests.cpu: 600m/7400m, requests.memory: 384Mi/16000Mi   limits.cpu: 1500m/6500m, limits.memory: 768Mi/15616Mi   121m

solar-test   capsule-global-quota-27d5c64b6c978b1b0f9c   pods: 3/7                                                                                                          121m
solar-test   capsule-global-quota-cf84f5e52f0542edd93c   requests.cpu: 600m/7400m, requests.memory: 384Mi/16000Mi   limits.cpu: 1500m/6500m, limits.memory: 768Mi/15616Mi   121m

GlobalResourceQuotas with Proxy

When you are using the Capsule-Proxy you can allow users to list and get the GlobalResourceQuota objects in their Tenant. Add a GlobalTenantResource to provide the necessary RBAC permissions to the Tenant users:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: capsule-proxy-settings
spec:
  scope: Tenant
  resyncPeriod: 30s
  resources:
    - generators:
        - missingKey: zero
          template: |
            ---
            apiVersion: capsule.clastix.io/v1beta1
            kind: GlobalProxySettings
            metadata:
              name: {{ $.tenant.metadata.name }}-proxy-settings
            spec:
              rules:
              - subjects:
                {{- range $.tenant.status.owners }}
                - kind: {{ .kind }}
                  name: {{ .name }}
                {{- end }}
                clusterResources:
                - apiGroups:
                  - "capsule.clastix.io"
                  resources:
                  - "globalresourcequotas"
                  operations:
                  - List
                  selector:
                    matchLabels:
                      projectcapsule.dev/tenant: {{ $.tenant.metadata.name }}

Permissions

Declare permission distribution rules for the selected namespaces.

Bindings

With Tenant RoleBindings you can distribute namespaced RoleBindings to all namespaces which are assigned to a Tenant. This ensures the defined RoleBindings are present and reconciled in all namespaces of the Tenant. This is useful if users should have more insights on a Tenant basis. Let’s look at an example.

Assuming a cluster-administrator creates the following clusterRole:

kubectl apply -f - << EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: prometheus-servicemonitors-viewer
rules:
- apiGroups: ["monitoring.coreos.com"]
  resources: ["servicemonitors"]
  verbs: ["get", "list", "watch"]
EOF

Now the cluster administrator wants to bind this ClusterRole in each namespace of the solar Tenant. They can configure this with a Tenant manifest:

kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  rules:
    - permissions:
        bindings:
          - clusterRoleName: 'prometheus-servicemonitors-viewer'
            subjects:
              - kind: User
                name: alice
            labels:
              projectcapsule.dev/sample: "true"
            annotations:
              projectcapsule.dev/sample: "true"
EOF

As you can see, subjects uses the standard RoleBinding subject format. This grants permissions to the subject user alice, who can get, list, and watch ServiceMonitors in the solar Tenant namespaces, but has no other permissions.

Strict

If you have strict RBAC enabled for the controller, you need to ensure that the controller ServiceAccount has the permission to create RoleBindings for the specified ClusterRole. The Controller Aggregates ClusterRoles with the labels (OR):

projectcapsule.dev/aggregate-to-controller: "true"
projectcapsule.dev/aggregate-to-controller-instance: {{ .Release.Name }}

So for the above example, you need to label the prometheus-servicemonitors-viewer ClusterRole like this:

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: prometheus-servicemonitors-viewer
  labels:
    projectcapsule.dev/aggregate-to-controller: "true"
rules:
- apiGroups: ["monitoring.coreos.com"]
  resources: ["servicemonitors"]
  verbs: ["get", "list", "watch"]

Distribution

You may have the use-case where you want to distribute different ClusterRoles to different namespaces of the same Tenant. For example, you want to give view permissions to an operational group in all namespaces of the solar Tenant with environment=prod label, but you want to give edit permissions to the operations group in all other namespaces. You can achieve this by leveraging GlobalTenantResources:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - name: alice
      kind: User
    - name: joe
      kind: User
  rules:
    - namespaceSelector:
        matchExpressions:
        - key: environment
          operator: NotIn
          values:
          - prod
      permissions:
        bindings:
          - clusterRoleName: 'edit'
            subjects:
              - kind: Group
                name: tenant:{{ .tenant.metadata.name }}:operators
    - namespaceSelector:
        matchLabels:
          environment: prod
      permissions:
        bindings:
          - clusterRoleName: 'view'
            subjects:
              - kind: Group
                name: tenant:{{ .tenant.metadata.name }}:operators

Built-in ClusterRoles

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant:admins:extension
  labels:
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways
      - httproutes
      - grpcroutes
      - tlsroutes
      - tcproutes
      - udproutes
      - referencegrants
      - backendtlspolicies
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways/status
      - httproutes/status
      - grpcroutes/status
      - tlsroutes/status
      - tcproutes/status
      - udproutes/status
      - referencegrants/status
      - backendtlspolicies/status
    verbs: ["get"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies
      - backendtrafficpolicies
      - securitypolicies
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies/status
      - backendtrafficpolicies/status
      - securitypolicies/status
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant:members:extension
  labels:
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
rules:
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways
      - httproutes
      - grpcroutes
      - tlsroutes
      - tcproutes
      - udproutes
      - referencegrants
      - backendtlspolicies
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways/status
      - httproutes/status
      - grpcroutes/status
      - tlsroutes/status
      - tcproutes/status
      - udproutes/status
      - referencegrants/status
      - backendtlspolicies/status
    verbs: ["get"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies
      - backendtrafficpolicies
      - securitypolicies
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies/status
      - backendtrafficpolicies/status
      - securitypolicies/status
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant:viewers:extension
  labels:
    rbac.authorization.k8s.io/aggregate-to-view: "true"
rules:
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways
      - httproutes
      - grpcroutes
      - tlsroutes
      - tcproutes
      - udproutes
      - referencegrants
      - backendtlspolicies
    verbs: ["get", "list", "watch"]
  - apiGroups: ["gateway.networking.k8s.io"]
    resources:
      - gateways/status
      - httproutes/status
      - grpcroutes/status
      - tlsroutes/status
      - tcproutes/status
      - udproutes/status
      - referencegrants/status
      - backendtlspolicies/status
    verbs: ["get"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies
      - backendtrafficpolicies
      - securitypolicies
    verbs: ["get", "list", "watch", "create"]
  - apiGroups: ["gateway.envoyproxy.io"]
    resources:
      - clienttrafficpolicies/status
      - backendtrafficpolicies/status
      - securitypolicies/status
    verbs: ["get"]

Custom Resources

In order to leave the TenantOwner to create Custom Resources in their namespaces, the cluster admin defines a proper Cluster Role. For example:

kubectl apply -f - << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: argoproj-provisioner
rules:
- apiGroups:
  - argoproj.io
  resources:
  - applications
  - appprojects
  verbs:
  - create
  - get
  - list
  - watch
  - update
  - patch
  - delete
EOF

Bill can assign this role to any namespace in the Alice’s Tenant by setting it in the Tenant manifest:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - name: alice
      kind: User
    - name: joe
      kind: User
  rules:
    - permissions:
        bindings:
          - clusterRoleName: 'argoproj-provisioner'
            subjects:
              - apiGroup: rbac.authorization.k8s.io
                kind: User
                name: alice
              - apiGroup: rbac.authorization.k8s.io
                kind: User
                name: joe

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: capsule-solar-argoproj-provisioner
  namespace: solar-production
subjects:
  - kind: User
    apiGroup: rbac.authorization.k8s.io
    name: alice
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: argoproj-provisioner

With the above example, Capsule is leaving the TenantOwner to create namespaced custom resources.

Take Note: a TenantOwner having the admin scope on its namespaces only, does not have the permission to create Custom Resources Definitions (CRDs) because this requires a cluster admin permission level. Only Bill, the cluster admin, can create CRDs. This is a known limitation of any multi-tenancy environment based on a single shared control plane.

Promotions

As an administrator, you can define promotion rules. A promotion rule selects ServiceAccounts within a Tenant based on specified conditions and assigns them predefined ClusterRoles.

The selected ClusterRoles are then applied across all namespaces belonging to the Tenant, or a selected subset of namespaces, with the corresponding ServiceAccounts configured as subjects. This allows a ServiceAccount in one namespace to automatically receive equivalent permissions in other namespaces of the same Tenant.

This feature is particularly useful in scenarios involving Tenant Replications, where consistent permissions across namespaces are required.

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  ...
  rules:
    - permissions:
        promotions:
          # Every promoted ServiceAccount receives this ClusterRole in all Namespaces of Tenant solar.
          - clusterRoles:
              - "configmap-replicator"

          # Every promoted ServiceAccount with the matching labels receives this ClusterRole.
          - clusterRoles:
              - "secret-replicator"
            selector:
              matchLabels:
                super: "account"

    - namespaceSelector:
        matchExpressions:
          - key: env
            operator: In
            values: ["prod"]
      permissions:
        promotions:
          # Promoted ServiceAccounts receive this ClusterRole only in namespaces matching env=prod.
          - clusterRoles:
              - "secret-replicator:prod"

Make sure the ClusterRoles exist. Otherwise, the corresponding Tenant reports a reconciliation error:

conditions:
- lastTransitionTime: "2026-02-16T23:08:59Z"
  message: 'cannot sync rolebindings items: rolebindings.rbac.authorization.k8s.io
    "tenant-replicator" not found'

If you run Capsule in Strict Mode, the controller must be allowed to grant the corresponding permissions to the ServiceAccount in all selected Namespaces. You can aggregate the same ClusterRoles to the controller:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: configmap-replicator
  labels:
    projectcapsule.dev/aggregate-to-controller: "true"
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "create", "patch", "watch", "list", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secret-replicator
  labels:
    projectcapsule.dev/aggregate-to-controller: "true"
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "create", "patch", "watch", "list", "delete"]

As a Tenant Owner, Alice can promote ServiceAccounts by labeling them with projectcapsule.dev/promote=true. This feature must be enabled in the CapsuleConfiguration. If the feature is disabled, admission fails:

Error from server (Forbidden): admission webhook "serviceaccounts.projectcapsule.dev" denied the request: service account promotion is disabled. Contact cluster administrators

When the feature is enabled, the following command succeeds, assuming alice is a Tenant Owner of the solar Tenant:

kubectl label sa gitops-reconcile -n solar-test projectcapsule.dev/promote=true --as alice --as-group projectcapsule.dev

Verify the promotion in the Tenant status:

kubectl get tnt solar -o jsonpath='{.status.promotions}' | jq

Example status:

[
  {
    "clusterRoles": [
      "tenant-replicator"
    ],
    "kind": "ServiceAccount",
    "name": "system:serviceaccount:solar-test:gitops-reconcile",
    "targets": [
      "solar-test",
      "solar-prod"
    ]
  }
]

You can verify that the RoleBinding was distributed to other namespaces of the solar Tenant:

kubectl get rolebinding -n solar-prod

NAME                               ROLE                                    AGE
..
capsule:managed:7ad688b586eada40   ClusterRole/configmap-replicator        21s
..

To revoke the promotion, Alice can remove the label:

kubectl label sa gitops-reconcile -n solar-test projectcapsule.dev/promote- --as alice --as-group projectcapsule.dev

5.6 - Enforcement

Configure policies and restrictions on tenant-basis

Deprecated

This page covers legacy enforcement configuration. Use Rules for all current policy and restriction management.

Scheduling

LimitRanges

This feature will be deprecated in a future release of Capsule. Instead use TenantReplications

Bill, the cluster admin, can also set Limit Ranges for each Namespace in Alice’s Tenant by defining limits for pods and containers in the Tenant spec:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
...
  limitRanges:
    items:
      - limits:
          - type: Pod
            min:
              cpu: "50m"
              memory: "5Mi"
            max:
              cpu: "1"
              memory: "1Gi"
      - limits:
          - type: Container
            defaultRequest:
              cpu: "100m"
              memory: "10Mi"
            default:
              cpu: "200m"
              memory: "100Mi"
            min:
              cpu: "50m"
              memory: "5Mi"
            max:
              cpu: "1"
              memory: "1Gi"
      - limits:
          - type: PersistentVolumeClaim
            min:
              storage: "1Gi"
            max:
              storage: "10Gi"

Limits will be inherited by all the Namespaces created by Alice. In our case, when Alice creates the Namespace solar-production, Capsule creates the following:

apiVersion: v1
kind: LimitRange
metadata:
  name: capsule-solar-0
  namespace: solar-production
spec:
  limits:
    - max:
        cpu: "1"
        memory: 1Gi
      min:
        cpu: 50m
        memory: 5Mi
      type: Pod
---
apiVersion: v1
kind: LimitRange
metadata:
  name: capsule-solar-1
  namespace: solar-production
spec:
  limits:
    - default:
        cpu: 200m
        memory: 100Mi
      defaultRequest:
        cpu: 100m
        memory: 10Mi
      max:
        cpu: "1"
        memory: 1Gi
      min:
        cpu: 50m
        memory: 5Mi
      type: Container
---
apiVersion: v1
kind: LimitRange
metadata:
  name: capsule-solar-2
  namespace: solar-production
spec:
  limits:
    - max:
        storage: 10Gi
      min:
        storage: 1Gi
      type: PersistentVolumeClaim

Note: being the limit range specific of single resources, there is no aggregate to count.

Alice doesn’t have permission to change or delete the resources according to the assigned RBAC profile.

kubectl -n solar-production auth can-i patch resourcequota
no
kubectl -n solar-production auth can-i delete resourcequota
no
kubectl -n solar-production auth can-i patch limitranges
no
kubectl -n solar-production auth can-i delete limitranges
no

LimitRange Distribution with TenantReplications

In the future Cluster-Administrators must distribute LimitRanges via TenantReplications. This is a more flexible and powerful way to distribute LimitRanges, as it allows to distribute any kind of resource, not only LimitRanges. Here’s an example of how to distribute a LimitRange to all the Namespaces of a tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: solar-limitranges
  namespace: solar-system
spec:
  resyncPeriod: 60s
  resources:
    - namespaceSelector:
        matchLabels:
          capsule.clastix.io/tenant: solar
      rawItems:
        - apiVersion: v1
          kind: LimitRange
          metadata:
            name: cpu-resource-constraint
          spec:
            limits:
            - default: # this section defines default limits
                cpu: 500m
              defaultRequest: # this section defines default requests
                cpu: 500m
              max: # max and min define the limit range
                cpu: "1"
              min:
                cpu: 100m
              type: Container

PriorityClasses

Pods can have priority. Priority indicates the importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the scheduler tries to preempt (evict) lower priority Pods to make scheduling of the pending Pod possible. See Kubernetes documentation.

In a multi-tenant cluster, not all users can be trusted, as a tenant owner could create Pods at the highest possible priorities, causing other Pods to be evicted/not get scheduled.

To prevent misuses of Pod PriorityClass, Bill, the cluster admin, can enforce the allowed Pod PriorityClass at tenant level:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  priorityClasses:
    matchLabels:
      env: "production"

With the said Tenant specification, Alice can create a Pod resource if spec.priorityClassName equals to:

Any PriorityClass which has the label env with the value production

If a Pod is going to use a non-allowed PriorityClass, it will be rejected by the Validation Webhook enforcing it.

Assign Pod PriorityClass as tenant default

Note: This feature supports type PriorityClass only on API version scheduling.k8s.io/v1

This feature allows specifying a custom default value on a Tenant basis, bypassing the global cluster default (globalDefault=true) that acts only at the cluster level.

It’s possible to assign each Tenant a PriorityClass which will be used, if no PriorityClass is set on pod basis:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  priorityClasses:
    default: "tenant-default"
    matchLabels:
      env: "production"

Let’s create a PriorityClass which is used as the default:

kubectl apply -f - << EOF
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: tenant-default
  labels:
    env: "production"
value: 1313
preemptionPolicy: Never
globalDefault: false
description: "This is the default PriorityClass for the solar-tenant"
EOF

Note the globalDefault: false which is important to avoid the PriorityClass to be used as the default for all the Tenants. If a Pod has no value for spec.priorityClassName, the default value for PriorityClass (tenant-default) will be used.

RuntimeClasses

Pods can be assigned different RuntimeClasses. With the assigned runtime you can control Container Runtime Interface (CRI) is used for each pod. See Kubernetes documentation for more information.

To prevent misuses of Pod RuntimeClasses, Bill, the cluster admin, can enforce the allowed PodRuntimeClasses at Tenant level:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  runtimeClasses:
    matchLabels:
      env: "production"

With the said Tenant specification, Alice can create a Pod resource if spec.runtimeClassName equals to:

Any RuntimeClass which has the label env with the value production

If a Pod is going to use a non-allowed RuntimeClass, it will be rejected by the Validation Webhook enforcing it.

Assign Runtime Class as tenant default

This feature allows specifying a custom default value on a Tenant basis- It’s possible to assign each tenant a Runtime which will be used, if no Runtime is set on pod basis:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  runtimeClasses:
    default: "tenant-default"
    matchLabels:
      env: "production"

Let’s create a RuntimeClass which is used as the default:

kubectl apply -f - << EOF
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: tenant-default
  labels:
    env: "production"
handler: myconfiguration
EOF

If a Pod has no value for spec.runtimeclass, the default value for RuntimeClass (tenant-default) will be used.

NodeSelector

Bill, the cluster admin, can dedicate a pool of worker nodes to the solar Tenant, to isolate the tenant applications from other noisy neighbors.

These nodes are labeled by Bill as pool=renewable

kubectl get nodes --show-labels

NAME                      STATUS   ROLES             AGE   VERSION   LABELS
...
worker06.acme.com         Ready    worker            8d    v1.25.2 pool=renewable
worker07.acme.com         Ready    worker            8d    v1.25.2   pool=renewable
worker08.acme.com         Ready    worker            8d    v1.25.2   pool=renewable

PodNodeSelector

This approach requires PodNodeSelector Admission Controller plugin to be active. If the plugin is not active, the pods will be scheduled to any node. If your distribution does not support this feature, you can use Expression Node Selectors.

The label pool=renewable is defined as .spec.nodeSelector in the Tenant manifest:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  nodeSelector:
    pool: renewable
    kubernetes.io/os: linux

The Capsule controller makes sure that any Namespace created in the Tenant has the annotation: scheduler.alpha.kubernetes.io/node-selector: pool=renewable. This annotation tells the scheduler of Kubernetes to assign the node selector pool=renewable to all the Pods deployed in the Tenant. The effect is that all the Pods deployed by Alice are placed only on the designated pool of nodes.

Multiple node selector labels can be defined as in the following snippet:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  nodeSelector:
    pool: renewable
    kubernetes.io/os: linux
    kubernetes.io/arch: amd64
    hardware: gpu

Any attempt of Alice to change the selector on the Pods will result in an error from the PodNodeSelector Admission Controller plugin.

kubectl auth can-i edit ns -n solar-production
no

Dynamic resource allocation (DRA)

Dynamic Resource Allocation (DRA) is a Kubernetes capability that allows Pods to request and use shared resources, typically external devices such as hardware accelerators. See Kubernetes documentation for more information.

Bill can assign a set of dedicated DeviceClasses to tell the solar Tenant what devices they can request.

apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: gpu.example.com
  labels:
    env: "production"
spec:
  selectors:
    - cel:
        expression: device.driver == 'gpu.example.com' && device.attributes['gpu.example.com'].type
          == 'gpu'
  extendedResourceName: example.com/gpu

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - name: alice
      kind: User
  deviceClasses:
    matchLabels:
      env: "production"

With the said Tenant specification, Alice can create a ResourceClaim or ResourceClaimTemplate resource if spec.devices.requests[].deviceClassName ( ResourceClaim) or spec.spec.devices.requests[].deviceClassName ( ResourceClaimTemplate) equals to:

Any DeviceClass, which has the label env with the value production

If any of the devices in the ResourceClaim or ResourceClaimTemplate spec is going to use a non-allowed DeviceClass, the entire request will be rejected by the Validation Webhook enforcing it.

Alice now can create a ResourceClaim using only an allowed DeviceClass:

apiVersion: resource.k8s.io/v1
kind: ResourceClaim
metadata:
  name: example-resource-claim
  namespace: solar-production
spec:
  devices:
    requests:
      - name: gpu-request
        exactly:
          deviceClassName: 'gpu.example.com'

Connectivity

Services

ExternalIPs

Specifies the external IPs that can be used in Services with type ClusterIP. An empty list means no IPs are allowed, which is recommended in multi-tenant environments (can be misused for traffic hijacking):

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  serviceOptions:
    externalIPs:
      allowed: []

Deny labels and annotations

By default, capsule allows Tenant owners to add and modify any label or annotation on their Services.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  serviceOptions:
    forbiddenAnnotations:
      denied:
          - loadbalancer.class.acme.net
      deniedRegex: .*.acme.net
    forbiddenLabels:
      denied:
          - loadbalancer.class.acme.net
      deniedRegex: .*.acme.net

Deny Service Types

Bill, the cluster admin, can prevent the creation of Services with specific Service types.

NodePort

When dealing with a shared multi-tenant scenario, multiple NodePort services can start becoming cumbersome to manage. The reason behind this could be related to the overlapping needs by the Tenant owners, since a NodePort is going to be open on all nodes and, when using hostNetwork=true, accessible to any Pod although any specific NetworkPolicy.

Bill, the cluster admin, can block the creation of Services with NodePort service type for a given Tenant

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  serviceOptions:
    allowedServices:
      nodePort: false

With the above configuration, any attempt of Alice to create a Service of type NodePort is denied by the Validation Webhook enforcing it. Default value is true.

ExternalName

Service with the type of ExternalName has been found subject to many security issues. To prevent TenantOwners to create services with the type of ExternalName, the cluster admin can prevent a tenant to create them:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  serviceOptions:
    allowedServices:
      externalName: false

With the above configuration, any attempt of Alice to create a Service of type externalName is denied by the Validation Webhook enforcing it. Default value is true.

LoadBalancer

Same as previously, the Service of type of LoadBalancer could be blocked for various reasons. To prevent TenantOwners to create these kinds of Services, the cluster admin can Tenant a tenant to create them:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  serviceOptions:
    allowedServices:
      loadBalancer: false

With the above configuration, any attempt of Alice to create a Service of type LoadBalancer is denied by the Validation Webhook enforcing it. Default value is true.

GatewayClasses

Note: This feature is offered only by API type GatewayClass in group gateway.networking.k8s.io version v1.

GatewayClass is cluster-scoped resource defined by the infrastructure provider. This resource represents a class of Gateways that can be instantiated. Read More

Bill can assign a set of dedicated GatewayClasses to the solar Tenant to force the applications in the solar Tenant to be published only by the assigned Gateway Controller:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  gatewayOptions:
    allowedClasses:
      matchLabels:
        env: "production"

With the said Tenant specification, Alice can create a Gateway resource if spec.gatewayClassName equals to:

Any GatewayClass which has the label env with the value production

If an Gateway is going to use a non-allowed GatewayClass, it will be rejected by the Validation Webhook enforcing it.

Alice can create an Gateway using only an allowed GatewayClass:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example-gateway
  namespace: solar-production
spec:
  gatewayClassName: customer-class
  listeners:
  - name: http
    protocol: HTTP
    port: 80

Any attempt of Alice to use a non-valid GatewayClass, or missing it, is denied by the Validation Webhook enforcing it.

Assign GatewayClass as tenant default

Note: The Default GatewayClass must have a label which is allowed within the tenant. This behavior is only implemented this way for the GatewayClass default.

This feature allows specifying a custom default value on a Tenant basis. Currently there is no global default feature for a GatewayClass. Each Gateway must have a spec.gatewayClassName set.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  gatewayOptions:
    allowedClasses:
      default: "tenant-default"
      matchLabels:
        env: "production"

Here’s how the Tenant default GatewayClass could look like:

kubectl apply -f - << EOF
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: tenant-default
  labels:
    env: "production"
spec:
  controllerName: example.com/gateway-controller
EOF

If a Gateway has no value for spec.gatewayClassName, the tenant-default GatewayClass is automatically applied to the Gateway resource.

Ingresses

Assign Ingress Hostnames

Bill can control ingress hostnames in the solar Tenant to force the applications to be published only using the given hostname or set of hostnames:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  ingressOptions:
    allowedHostnames:
      allowed:
        - solar.acmecorp.com
      allowedRegex: ^.*acmecorp.com$

The Capsule controller assures that all Ingresses created in the Tenant can use only one of the valid hostnames. Alice can create an Ingress using any allowed hostname:

kubectl apply -f - << EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  namespace: solar-production
spec:
  ingressClassName: solar
  rules:
  - host: web.solar.acmecorp.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx
            port:
              number: 80
EOF

Any attempt of Alice to use a non-valid hostname is denied by the Validation Webhook enforcing it.

Control Hostname collision in Ingresses

In a multi-tenant environment, as more and more ingresses are defined, there is a chance of collision on the hostname leading to unpredictable behavior of the Ingress Controller. Bill, the cluster admin, can enforce hostname collision detection at different scope levels:

Cluster
Tenant
Namespace
Disabled (default)

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  - name: joe
    kind: User
  ingressOptions:
    hostnameCollisionScope: Tenant

When a TenantOwner creates an Ingress resource, Capsule will check the collision of hostname in the current ingress with all the hostnames already used, depending on the defined scope.

For example, Alice, one of the TenantOwners, creates an Ingress:

kubectl apply -f - << EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  namespace: solar-production
spec:
  rules:
  - host: web.solar.acmecorp.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx
            port:
              number: 80
EOF

Another user, Joe creates an Ingress having the same hostname:

kubectl apply -f - << EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  namespace: solar-development
spec:
  rules:
  - host: web.solar.acmecorp.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx
            port:
              number: 80
EOF

When a collision is detected at scope defined by spec.ingressOptions.hostnameCollisionScope, the creation of the Ingress resource will be rejected by the Validation Webhook enforcing it. When spec.ingressOptions.hostnameCollisionScope=Disabled (default), no collision detection is made at all.

Deny Wildcard Hostname in Ingresses

Bill, the cluster admin, can deny the use of wildcard hostname in Ingresses. Let’s assume that Acme Corp. uses the domain acme.com.

As a TenantOwner of solar, Alice creates an Ingress with the host like - host: "*.acme.com". That can lead problems for the water tenant because Alice can deliberately create ingress with host: water.acme.com.

To avoid this kind of problems, Bill can deny the use of wildcard hostnames in the following way:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - name: alice
      kind: User
  ingressOptions:
    allowWildcardHostnames: false

Doing this, Alice will not be able to use *.water.acme.com, being the tenant owner of solar and green only.

IngressClasses

An Ingress Controller is used in Kubernetes to publish services and applications outside of the cluster. An Ingress Controller can be provisioned to accept only Ingresses with a given IngressClass.

Bill can assign a set of dedicated IngressClass to the solar Tenant to force the applications in the solar tenant to be published only by the assigned Ingress Controller:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  ingressOptions:
    allowedClasses:
      matchLabels:
        env: "production"

With the said Tenant specification, Alice can create a Ingress resource if spec.ingressClassName or metadata.annotations."kubernetes.io/ingress.class" equals to:

Any IngressClass which has the label env with the value production

If an Ingress is going to use a non-allowed IngressClass, it will be rejected by the Validation Webhook enforcing it.

Alice can create an Ingress using only an allowed IngressClass:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  namespace: solar-production
spec:
  ingressClassName: legacy
  rules:
  - host: solar.acmecorp.com
    http:
      paths:
      - backend:
          service:
            name: nginx
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific

Any attempt of Alice to use a non-valid Ingress Class, or missing it, is denied by the Validation Webhook enforcing it.

Assign Ingress Class as tenant default

Note: This feature is offered only by API type IngressClass in group networking.k8s.io version v1. However, resource Ingress is supported in networking.k8s.io/v1 and networking.k8s.io/v1beta1

This feature allows specifying a custom default value on a Tenant basis, bypassing the global cluster default (with the annotation metadata.annotations.ingressclass.kubernetes.io/is-default-class=true) that acts only at the cluster level. More information: Default IngressClass

It’s possible to assign each Tenant an IngressClass which will be used, if a class is not set on Ingress basis:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  ingressOptions:
    allowedClasses:
      default: "tenant-default"
      matchLabels:
        env: "production"

Here’s how the Tenant default IngressClass could look like:

kubectl apply -f - << EOF
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    env: "production"
    app.kubernetes.io/component: controller
  name: tenant-default
  annotations:
    ingressclass.kubernetes.io/is-default-class: "false"
spec:
  controller: k8s.io/customer-nginx
EOF

If an Ingress has no value for spec.ingressClassName or metadata.annotations."kubernetes.io/ingress.class", the tenant-default IngressClass is automatically applied to the Ingress resource.

NetworkPolicies

Deprecated

This feature will be deprecated in a future release of Capsule. Instead use TenantReplications. This is also true if you would like other NetworkPolicy implementation like Cilium.

Kubernetes network policies control network traffic between Namespaces and between pods in the same Namespace. Bill, the cluster admin, can enforce network traffic isolation between different Tenants while leaving to Alice, the TenantOwner, the freedom to set isolation between Namespaces in the same Tenant or even between pods in the same Namespace.

To meet this requirement, Bill needs to define network policies that deny pods belonging to Alice’s Namespaces to access pods in Namespaces belonging to other Tenants, e.g. Bob’s Tenant water, or in system Namespaces, e.g. kube-system.

Keep in mind, that because of how the NetworkPolicies API works, the users can still add a policy which contradicts what the Tenant has set, resulting in users being able to circumvent the initial limitation set by the Tenant admin. Two options can be put in place to mitigate this potential privilege escalation: 1. providing a restricted role rather than the default admin one 2. using Calico’s GlobalNetworkPolicy, or Cilium’s CiliumClusterwideNetworkPolicy which are defined at the cluster-level, thus creating an order of packet filtering.

Also, Bill can make sure pods belonging to a Tenant Namespace cannot access other network infrastructures like cluster nodes, load balancers, and virtual machines running other services.

Bill can set network policies in the Tenant manifest, according to the requirements:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  networkPolicies:
    items:
    - policyTypes:
      - Ingress
      - Egress
      egress:
      - to:
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
              - 192.168.0.0/16
      ingress:
      - from:
        - namespaceSelector:
            matchLabels:
              capsule.clastix.io/tenant: water
        - podSelector: {}
        - ipBlock:
            cidr: 192.168.0.0/16
      podSelector: {}

The Capsule controller, watching for Namespace creation, creates the Network Policies for each Namespace in the Tenant.

Alice has access to network policies:

kubectl -n solar-production get networkpolicies
NAME              POD-SELECTOR   AGE
capsule-solar-0   <none>         42h

Alice can create, patch, and delete additional network policies within her Namespaces:

kubectl -n solar-production auth can-i get networkpolicies
yes

kubectl -n solar-production auth can-i delete networkpolicies
yes

kubectl -n solar-production auth can-i patch networkpolicies
yes

For example, she can create:

kubectl apply -f - << EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  labels:
  name: production-network-policy
  namespace: solar-production
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
EOF

Check all the network policies

kubectl -n solar-production get networkpolicies
NAME                          POD-SELECTOR   AGE
capsule-solar-0               <none>         42h
production-network-policy     <none>         3m

And delete the Namespace network policies:

kubectl -n solar-production delete networkpolicy production-network-policy

Any attempt of Alice to delete the Tenant network policy defined in the tenant manifest is denied by the Validation Webhook enforcing it. Any deletion by a cluster-administrator will cause the network policy to be recreated by the Capsule controller.

NetworkPolicy Distribution with TenantReplications

In the future Cluster-Administrators must distribute NetworkPolicies via TenantReplications. This is a more flexible and powerful way to distribute NetworkPolicies, as it allows to distribute any kind of resource. Here’s an example of how to distribute a CiliumNetworkPolicy to all the Namespaces of a Tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: solar-limitranges
  namespace: solar-system
spec:
  resyncPeriod: 60s
  resources:
    - namespaceSelector:
        matchLabels:
          capsule.clastix.io/tenant: solar
      rawItems:
        - apiVersion: "cilium.io/v2"
          kind: CiliumNetworkPolicy
          metadata:
            name: "l3-rule"
          spec:
            endpointSelector:
              matchLabels:
                role: backend
            ingress:
            - fromEndpoints:
              - matchLabels:
                  role: frontend

Storage

PersistentVolumes

For selected PVs that have no Tenant ownership label, administrators can grant additional volume access through namespace rules. These rules support restore handoffs with label selectors, caller audiences, and CEL conditions while preserving the Tenant ownership checks below.

Any Tenant owner is able to create a PersistentVolumeClaim that, backed by a given StorageClass, will provide volumes for their applications.

In most cases, once a PersistentVolumeClaim is deleted, the bounded PersistentVolume will be recycled due.

However, in some scenarios, the StorageClass or the provisioned PersistentVolume itself could change the retention policy of the volume, keeping it available for recycling and being consumable for another Pod.

In such a scenario, Capsule enforces the Volume mount only to the Namespaces belonging to the Tenant on which it’s been consumed, by adding a label to the Volume as follows.

apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    pv.kubernetes.io/provisioned-by: rancher.io/local-path
  creationTimestamp: "2022-12-22T09:54:46Z"
  finalizers:
  - kubernetes.io/pv-protection
  labels:
    capsule.clastix.io/tenant: solar
  name: pvc-1b3aa814-3b0c-4912-9bd9-112820da38fe
  resourceVersion: "2743059"
  uid: 9836ae3e-4adb-41d2-a416-0c45c2da41ff
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 10Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: melange
    namespace: caladan
    resourceVersion: "2743014"
    uid: 1b3aa814-3b0c-4912-9bd9-112820da38fe

Once the PeristentVolume become available again, it can be referenced by any PersistentVolumeClaim in the solar Tenant Namespace resources.

If another Tenant, like green, tries to use it, it will get an error:

$ kubectl describe pv pvc-9788f5e4-1114-419b-a830-74e7f9a33f5d
Name:              pvc-9788f5e4-1114-419b-a830-74e7f9a33f5d
Labels:            capsule.clastix.io/tenant=solar
Annotations:       pv.kubernetes.io/provisioned-by: rancher.io/local-path
Finalizers:        [kubernetes.io/pv-protection]
StorageClass:      standard
Status:            Available
...

$ cat /tmp/pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: melange
  namespace:  green-energy
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi
  volumeName: pvc-9788f5e4-1114-419b-a830-74e7f9a33f5d

$ kubectl apply -f /tmp/pvc.yaml
Error from server: error when creating "/tmp/pvc.yaml": admission webhook "pvc.capsule.clastix.io" denied the request: PeristentVolume pvc-9788f5e4-1114-419b-a830-74e7f9a33f5d cannot be used by the following Tenant, preventing a cross-tenant mount

StorageClasses

Persistent storage infrastructure is provided to Tenants. Different types of storage requirements, with different levels of QoS, eg. SSD versus HDD, are available for different tenants according to the Tenant’s profile. To meet these different requirements, Bill, the cluster admin can provision different StorageClasses and assign them to the tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  storageClasses:
    matchLabels:
      env: "production"

With the said Tenant specification, Alice can create a Persistent Volume Claims if spec.storageClassName equals to:

Any StorageClass which has the label env with the value production

Capsule assures that all PersistentVolumeClaims created by Alice will use only one of the valid storage classes. Assume the StorageClass ceph-rbd has the label env: production:

kubectl apply -f - << EOF
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc
  namespace: solar-production
spec:
  storageClassName: ceph-rbd
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 12Gi
EOF

If a PersistentVolumeClaim is going to use a non-allowed Storage Class, it will be rejected by the Validation Webhook enforcing it.

Assign Storage Class as tenant default

Note: This feature supports type StorageClass only on API version storage.k8s.io/v1

This feature allows specifying a custom default value on a Tenant basis, bypassing the global cluster default (.metadata.annotations.storageclass.kubernetes.io/is-default-class=true) that acts only at the cluster level. See the Default Storage Class section on Kubernetes documentation.

It’s possible to assign each tenant a StorageClass which will be used, if no value is set on PersistentVolumeClaim basis:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  storageClasses:
    default: "tenant-default"
    matchLabels:
      env: "production"

Here’s how the new StorageClass could look like:

kubectl apply -f - << EOF
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: tenant-default
  labels:
    env: production
  annotations:
    storageclass.kubernetes.io/is-default-class: "false"
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
EOF

If a PersistentVolumeClaim has no value for spec.storageClassName the tenant-default value will be used on new PersistentVolumeClaim resources.

Images

PullPolicy

Bill is a cluster admin providing a Container as a Service platform using shared nodes.

Alice, a TenantOwner, can start container images using private images: according to the Kubernetes architecture, the kubelet will download the layers on its cache.

Bob, an attacker, could try to schedule a Pod on the same node where Alice is running her Pods backed by private images: they could start new Pods using ImagePullPolicy=IfNotPresent and be able to start them, even without required authentication since the image is cached on the node.

To avoid this kind of attack, Bill, the cluster admin, can force Alice, the TenantOwner, to start her Pods using only the allowed values for ImagePullPolicy, enforcing the Kubelet to check the authorization first.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  imagePullPolicies:
  - Always

Allowed values are: Always, IfNotPresent, Never. As defined by the Kubernetes API

Any attempt of Alice to use a disallowed imagePullPolicies value is denied by the Validation Webhook enforcing it.

Images Registries

Bill, the cluster admin, can set a strict policy on the applications running into Alice’s Tenant: he’d like to allow running just images hosted on a list of specific container registries.

The spec.containerRegistries addresses this task and can provide a combination with hard enforcement using a list of allowed values.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  containerRegistries:
    allowed:
    - docker.io
    - quay.io
    allowedRegex: 'internal.registry.\\w.tld'

In case of Pod running non-FQCI (non fully qualified container image) containers, the container registry enforcement will disallow the execution. If you would like to run a bbusybox:latest container that is commonly hosted on Docker Hub, the TenantOwner has to specify its name explicitly, like docker.io/library/busybox:latest.

A Pod running internal.registry.foo.tld/capsule:latest as registry will be allowed, as well internal.registry.bar.tld since these are matching the regular expression.

A catch-all regex entry as .* allows every kind of registry, which would be the same result of unsetting .spec.containerRegistries at all.

Any attempt of Alice to use a not allowed .spec.containerRegistries value is denied by the Validation Webhook enforcing it.

5.7 - Quotas

Strategies on granting quotas on tenant-basis

With help of Capsule, Bill, the cluster admin, can set and enforce resources quota and limits for Alice’s Tenant.

Resource Quota

Deprecated

This feature will be deprecated in a future release of Capsule. Instead use Rules Quota. Migration Guide Here

With help of Capsule, Bill, the cluster admin, can set and enforce resources quota and limits for Alice’s Tenant. Set resources quota for each Namespace in the Alice’s Tenant by defining them in the Tenant spec:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
  - name: alice
    kind: User
  namespaceOptions:
    quota: 3
  resourceQuotas:
    scope: Tenant
    items:
    - hard:
        limits.cpu: "8"
        limits.memory: 16Gi
        requests.cpu: "8"
        requests.memory: 16Gi
    - hard:
        pods: "10"

The resource quotas above will be inherited by all the Namespaces created by Alice. In our case, when Alice creates the Namespace solar-production, Capsule creates the following resource quotas:

kind: ResourceQuota
apiVersion: v1
metadata:
  name: capsule-solar-0
  namespace: solar-production
  labels:
    tenant: solar
spec:
  hard:
    limits.cpu: "8"
    limits.memory: 16Gi
    requests.cpu: "8"
    requests.memory: 16Gi
---
kind: ResourceQuota
apiVersion: v1
metadata:
  name: capsule-wind-1
  namespace: solar-production
  labels:
    tenant: solar
spec:
  hard:
    pods : "10"

Alice can create any resource according to the assigned quotas:

kubectl -n solar-production create deployment nginx --image nginx:latest --replicas 4

At Namespaces solar-production level, Alice can see the used resources by inspecting the status in ResourceQuota:

kubectl -n solar-production get resourcequota capsule-solar-1 -o yaml
...
status:
  hard:
    pods: "10"
    services: "50"
  used:
    pods: "4"

When defining ResourceQuotas you might want to consider distributing LimitRanges via Tenant Replications:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: solar-limitranges
  namespace: solar-system
spec:
  resyncPeriod: 60s
  resources:
    - namespaceSelector:
        matchLabels:
          capsule.clastix.io/tenant: solar
      rawItems:
        - apiVersion: v1
          kind: LimitRange
          metadata:
            name: cpu-resource-constraint
          spec:
            limits:
            - default: # this section defines default limits
                cpu: 500m
              defaultRequest: # this section defines default requests
                cpu: 500m
              max: # max and min define the limit range
                cpu: "1"
              min:
                cpu: 100m
              type: Container

Tenant Scope

By setting enforcement at Tenant level, i.e. spec.resourceQuotas.scope=Tenant, Capsule aggregates resources usage for all Namespaces in the Tenant and adjusts all the ResourceQuota usage as aggregate. In such case, Alice can check the used resources at the Tenant level by inspecting the annotations in ResourceQuota object of any Namespace in the Tenant:

kubectl -n solar-production get resourcequotas capsule-solar-1 -o yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  annotations:
    quota.capsule.clastix.io/used-pods: "4"
    quota.capsule.clastix.io/hard-pods: "10"
...

kubectl -n solar-development get resourcequotas capsule-solar-1 -o yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  annotations:
    quota.capsule.clastix.io/used-pods: "4"
    quota.capsule.clastix.io/hard-pods: "10"
...

When the aggregate usage for all Namespaces crosses the hard quota, then the native ResourceQuota Admission Controller in Kubernetes denies Alice’s request to create resources exceeding the quota:

kubectl -n solar-development create deployment nginx --image nginx:latest --replicas 10

Alice cannot schedule more pods than the admitted at Tenant aggregate level.

kubectl -n solar-development get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-55649fd747-6fzcx   1/1     Running   0          12s
nginx-55649fd747-7q6x6   1/1     Running   0          12s
nginx-55649fd747-86wr5   1/1     Running   0          12s
nginx-55649fd747-h6kbs   1/1     Running   0          12s
nginx-55649fd747-mlhlq   1/1     Running   0          12s
nginx-55649fd747-t48s5   1/1     Running   0          7s

and

kubectl -n solar-production get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-55649fd747-52fsq   1/1     Running   0          22m
nginx-55649fd747-9q8n5   1/1     Running   0          22m
nginx-55649fd747-r8vzr   1/1     Running   0          22m
nginx-55649fd747-tkv7m   1/1     Running   0          22m

Custom Resources

Deprecated

This feature will be deprecated in a future release of Capsule. Instead use Custom Quotas.

Kubernetes offers by default ResourceQuota resources, aimed to limit the number of basic primitives in a Namespace.

Capsule already provides the sharing of these constraints across the Tenant Namespaces, however, limiting the amount of namespaced Custom Resources instances is not upstream-supported.

Starting from Capsule v0.1.1, this can be done using a special annotation in the Tenant manifest.

Imagine the case where a Custom Resource named mysqls in the API group databases.acme.corp/v1 usage must be limited in the Tenant solar: this can be done as follows.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
  annotations:
    quota.resources.capsule.clastix.io/mysqls.databases.acme.corp_v1: "3"
spec:
  additionalRoleBindings:
  - clusterRoleName: mysql-namespace-admin
    subjects:
      - kind: User
        name: alice
  owners:
  - name: alice
    kind: User

The Additional Role Binding referring to the Cluster Role mysql-namespace-admin is required to let Alice manage their Custom Resource instances.

The pattern for the quota.resources.capsule.clastix.io annotation is the following:

quota.resources.capsule.clastix.io/${PLURAL_NAME}.${API_GROUP}_${API_VERSION}

You can figure out the required fields using kubectl api-resources.

When alice will create a MySQL instance in one of their Tenant Namespace, the Cluster Administrator can easily retrieve the overall usage.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
  annotations:
    quota.resources.capsule.clastix.io/mysqls.databases.acme.corp_v1: "3"
    used.resources.capsule.clastix.io/mysqls.databases.acme.corp_v1: "1"
spec:
  owners:
  - name: alice
    kind: User

6 - Rules

Configure policies and restrictions on a per-Namespace basis with Rules

Namespace rules let administrators configure behavior for resources in Tenant namespaces. Use namespace selectors to give namespaces within the same Tenant different profiles. Rules can set workload properties during admission and validate the resulting resources.

Rules cover two areas:

Mutate: apply typed workload settings to new Pods in selected namespaces.
Enforcement: control allowed workloads, ingress hostnames, service types, NetworkPolicy egress CIDRs, and namespace metadata.

See Conditions for conditional mutation and enforcement.

Order and scope

Capsule first mutates the admission object, then enforces policies against the result. Both phases use the rules selected for the namespace and requesting subject.

A Tenant rule’s namespaceSelector selects its namespaces; omitting it selects all namespaces in that Tenant. Matching Tenant rules retain their declaration order in the namespace’s effective RuleStatus, including rules containing only mutate. Tenant/Namespace templating and audience filtering apply to both mutation and enforcement.

Admission order

Mutate. Apply applicable metadata mutations, then resource request/limit mutations, then mutate entries in list order within the effective rule order. Each entry and its conditions see the object after preceding mutations. When entries change the same property, its merge or replacement behavior determines the result.
Enforce. Evaluate applicable policies against the resulting admission object, including mutated values and values injected by Kubernetes. A mutation must satisfy the applicable enforcement policies; an incompatible combination rejects the request.

Mutation runs across the applicable rules before enforcement. Capsule does not alternate mutation and enforcement for each rule. mutate and enforce are independent sibling blocks; enforce.action does not select or disable mutations.

Some settings under enforce, such as managed metadata and resource defaults, produce mutations in the first phase. Their enforce.conditions are checked before those changes and evaluated again during enforcement. See Enforcement conditions for which object each phase uses.

Resource scope

mutate[].workloads applies when a Pod is created, including Pods created by Deployments, StatefulSets, and other controllers. security.readOnlyRootFilesystem and registries.imagePullPolicy also apply to newly added ephemeral containers on UPDATE pods/ephemeralcontainers; all other workload mutation properties are skipped on that subresource. Controller templates and existing containers are not rewritten. Other Pod updates, deletes, and subresources do not run workload mutations. Rule or namespace-label changes affect new Pods and new ephemeral containers within the property’s supported scope. Each rule supports up to 64 mutation entries. Placement mutations do not accumulate duplicate entries when admission is reinvoked.

Enforcement runs for the resource types and operations supported by each policy. enforce.workloads.targets selects native workload kinds and policy locations. Targets-only entries apply the action to the kind; entries with workload policies scope those checks. Controller targets explicitly enable template validation and resource request/limit mutation on controller creation and updates. Omitted targets preserve Pod-only defaults. Enforcement targets do not control mutate[].workloads.targets, whose separate selection rules include all container groups when pod is selected. Conditions can narrow a policy’s scope but cannot expand its supported operations.

Audience

Use audience to restrict a rule to requests made by specific users, groups, service accounts, or Capsule-defined subject categories. The property belongs to the root of a rule, alongside mutate and enforce:

spec:
  rules:
    - audience:
        - kind: Group
          name: system:authenticated
      enforce:
        action: allow
        metadata:
          - kinds:
              - ConfigMap
            annotations:
              example.corp/cost-center:
                required: true
                values:
                  - exp: "^INV-[0-9]{4}$"

When audience is omitted or empty, the rule applies to every request selected by the rule.

When an audience is configured, the rule applies if the requesting subject matches at least one entry. In other words, entries are combined with logical OR semantics. Audience matching is performed consistently for both validation and mutation, so a subject excluded from a rule is neither validated nor mutated by that rule.

Audience filtering uses the actual identity on the Pod admission request. Pods created by a workload controller are normally submitted by that controller’s service account, not by the user who originally created the Deployment or Job. Account for that distinction when combining audience with mutation or enforcement.

The supported standard audience kinds are User, Group, and ServiceAccount:

spec:
  rules:
    - audience:
        # Match one exact Kubernetes username.
        - kind: User
          name: alice@example.com

        # Match any request carrying this group.
        - kind: Group
          name: oidc:engineering

        # Match one Kubernetes service account.
        - kind: ServiceAccount
          name: system:serviceaccount:delivery:deployer
      enforce:
        action: deny
        metadata:
          - apiGroups:
              - v1
            kinds:
              - Namespace
            labels:
              pod-security.kubernetes.io/enforce:
                managed: restricted

For User, Group, and ServiceAccount, name is compared with the identity information provided in the Kubernetes admission request. A service account is represented by its canonical Kubernetes username:

system:serviceaccount:<namespace>:<service-account-name>

For example, a request from the deployer service account in the delivery namespace has the username system:serviceaccount:delivery:deployer.

Custom

The Custom kind exposes audiences based on Capsule’s internal identity and tenant resolution. Its name must be one of the supported values below.

name	description
`CapsuleUser`	Matches subjects listed by `configuration.Users()`.
`Administrator`	Matches subjects listed by `configuration.Administrators()`.
`TenantOwner`	Matches an owner of the tenant resolved for the current request. A request cannot match when no tenant can be resolved.
`Controller`	Matches the service account used by the Capsule controller.

Custom audiences can be combined with standard audiences. The following rule applies to Capsule users, Capsule administrators, tenant owners, the Capsule controller, or members of the Kubernetes system:masters group:

spec:
  rules:
    - audience:
        - kind: Custom
          name: CapsuleUser
        - kind: Custom
          name: Administrator
        - kind: Custom
          name: TenantOwner
        - kind: Custom
          name: Controller
        - kind: Group
          name: system:masters
      enforce:
        action: allow
        metadata:
          - apiGroups:
              - v1
            kinds:
              - Namespace
            annotations:
              example.corp/cost-center:
                default: II-1

TenantOwner is request-scoped. Capsule first resolves the tenant associated with the admission request and then checks the requesting subject against that tenant’s owners. This makes it suitable for rules that should affect tenant owners but not unrelated Capsule users:

spec:
  rules:
    - audience:
        - kind: Custom
          name: TenantOwner
      enforce:
        action: allow
        metadata:
          - kinds:
              - ConfigMap
            labels:
              owner-managed:
                default: "true"

Controller specifically identifies the Capsule controller service account. It is useful when internal reconciliation requests need different policy behavior from requests made by ordinary users:

spec:
  rules:
    - audience:
        - kind: Custom
          name: Controller
      enforce:
        action: allow
        metadata:
          - kinds:
              - Secret
            labels:
              capsule.clastix.io/reconciled:
                managed: "true"

Unknown audience kinds and unsupported Custom names are rejected when the rule is admitted. This catches spelling mistakes and prevents a rule from being silently configured with an audience that can never match.

Match expressions

Rule fields that accept match expressions use a common structure. A matcher must define at least one of exact or exp. Both fields may be set together; in that case, the matcher succeeds when either the exact list or the regular expression matches.

exact:
  - value-a
  - value-b
exp: "value-[0-9]+"

Field	Description
`exact`	Up to 64 exact values per matcher. The matcher succeeds when the evaluated value equals one of the listed values.
`exp`	A Go regular expression of up to 4,096 Unicode characters, matched against the evaluated value.
`negate`	Negates the final match result. This applies to both `exact` and `exp`.

These limits apply wherever match expressions are used, including placement and localhost profile matchers. Policies exceeding either limit are rejected when saved. Both limits still apply when exact and exp are used together or negate is enabled. Existing policies exceeding these limits must be reduced before they can be updated.

Anchor regular expressions with ^ and $ to match the whole string.

For example, this matcher matches registry.local/team-a/app:1.0.0, registry.local/team-b/app:1.0.0, or any reference under registry.local/shared/*:

exact:
  - registry.local/team-a/app:1.0.0
  - registry.local/team-b/app:1.0.0
exp: "registry.local/shared/.*"

With negate: true, the final match result is inverted. This means negation applies to exact values as well as regular expressions:

exact:
  - registry.local/blocked/app:1.0.0
exp: "registry.local/deprecated/.*"
negate: true

This matcher succeeds for every value except registry.local/blocked/app:1.0.0 and values matching registry.local/deprecated/.*.

Conditions

Conditions determine whether a mutation entry or enforcement rule applies to the current admission request. They are optional and belong alongside action at mutate[].conditions or enforce.conditions.

Location	Scope
`mutate[].conditions`	The entire mutation entry, within the property’s supported targets and operations.
`enforce.conditions`	The entire `enforce` block: workloads, Services, metadata, ingress, and storage, including metadata and resource request/limit mutations.

A false mutation condition skips only its mutation entry. A false enforcement condition skips that enforce block, while sibling mutate entries and other rules continue. Namespace selection and audience filtering apply first.

Conditions are evaluated only when the entry or enforcement policy applies to the resource and operation. A workload-only rule does not evaluate its conditions for a Service request. If the same enforce block also contains Service policies, those policies use the same conditions. Use separate rules when resource types need different conditions.

Expressions

Each entry has an expression containing one Boolean CEL expression and an optional name used in error messages. Names must be unique within a mutation entry or enforce block and use a DNS label of up to 63 characters. Each entry or block supports up to 64 conditions; each expression can contain up to 4096 characters and has a bounded evaluation cost. Expressions are compiled when rules are validated and cached for reuse.

Variable	Meaning
`object`	The current resource, using its Kubernetes field structure. Metadata conditions can inspect the full object, including fields such as `spec` or ConfigMap `data`.
`request`	Admission metadata, such as `operation`, `namespace`, `name`, `userInfo`, `kind`, `resource`, `subResource`, and `dryRun`. Raw objects and admission options are not exposed here.
`volume`	The referenced PersistentVolume during additional volume access evaluation; `null` for other condition consumers. The snapshot is read-only.

Check absent fields with has(...) and map membership with in. The missing-value example demonstrates both. The Pod node selector is at object.spec.nodeSelector; it is not a container property. Use request.userInfo for request identity, while keeping existing audience settings for straightforward subject selection.

Evaluation

No conditions means the mutation entry or enforce block applies within its supported scope.
All conditions must return true for it to apply. Use || within an expression for alternatives.
Any false condition skips the entry or block, including when another condition produces an error.
If none is false, an evaluation error rejects admission and identifies the condition.
Invalid syntax and non-Boolean expressions are rejected when saving the rule. Templated expressions are checked after rendering into the namespace’s RuleStatus.

Mutation conditions

Without conditions, a workload mutation applies to every Pod selected by the rule. Add conditions to the mutation entry, alongside action and workloads, to apply its changes only when all Boolean CEL expressions return true. A false condition skips that entry; later entries and other rules continue. Conditions on the sibling enforce block do not control these mutations.

Conditions see the current Pod immediately before the mutation entry runs, including changes from preceding mutations. They do not see changes from their own entry. Conditions run on Pod creation and, for applicable security.readOnlyRootFilesystem and registries.imagePullPolicy mutations, when adding ephemeral containers. They cannot expand a property’s supported targets or operations. They select mutations; mutation values are supplied by the workload properties.

Set a value only when it is missing

merge can overwrite an existing map value. To use a setting as a default, check whether that specific key is absent:

mutate:
  - action: merge
    conditions:
      - name: missing-os-selector
        expression: >-
          !has(object.spec.nodeSelector) || !('kubernetes.io/os' in object.spec.nodeSelector)          
    workloads:
      placement:
        nodeSelector:
          kubernetes.io/os: linux

A Pod with kubernetes.io/os: windows keeps its value. A Pod with only another selector key receives the Linux selector and retains the other key. If an earlier mutation already set the OS selector, this entry is skipped.

A condition gates every property in its mutation entry. For example, adding placement.tolerations beside placement.nodeSelector above would make those tolerations depend on the OS selector being absent too. Put unconditional changes or changes with different conditions in separate entries.

For a field Kubernetes fills before admission, see default a scheduler with a condition. That example uses conditional replace to override default-scheduler while preserving custom scheduler names.

Enforcement conditions

enforce.conditions applies to every applicable policy in the enforcement block. For ingress policies, the conditions are evaluated only when the incoming resource kind is selected by ingress.types. Metadata policies can inspect the full resource, including its spec or data.

For storage volume rules, object is the incoming PVC and volume is the referenced PV. These conditions govern additional access to PVs without a Tenant label; they do not revoke existing Tenant-owned volume access. Keep volume-specific conditions in their own storage block because other condition consumers see volume as null.

Conditions under enforce also gate metadata defaults, managed metadata, and resource request/limit mutations. On Pod creation, applicable enforcement conditions are evaluated against the same object before metadata and resource mutations run. Capsule then applies metadata mutations, resource policies, and the ordered mutate entries. During enforcement, conditions are evaluated again against the resulting admission object. A condition based on a mutated field can therefore have a different result in the two phases.

On Pod updates, conditional placement enforcement is reevaluated even when only labels or another condition input changes. Typed workload mutations apply on creation, with the additional security.readOnlyRootFilesystem and registries.imagePullPolicy scope for new ephemeral containers. Conditions do not expand a policy’s supported operations or subresources.

Managed metadata

Managed metadata sets labels or annotations during admission, overwriting existing values. For rules without conditions, the RuleStatus controller also reconciles these values onto existing objects.

When enforce.conditions is nonempty, managed metadata is applied only on admission requests that satisfy those conditions. The RuleStatus controller skips background reconciliation for that rule, even if its expression is true. Keep managed metadata in an unconditional rule when existing objects must be kept in sync.

Independent resource conditions

Split resource policies into separate rules when they need independent conditions. Repeat any namespace selector or audience settings that should apply to both rules:

rules:
  - enforce:
      action: deny
      conditions:
        - name: restricted-pods
          expression: |
            has(object.metadata.labels) &&
            'example.com/restricted' in object.metadata.labels &&
            object.metadata.labels['example.com/restricted'] == 'true'            
      workloads:
        placement:
          nodeSelector:
            - key: {exact: [infrastructure.example.com/pool]}
              values: {exact: [dedicated]}
  - enforce:
      action: deny
      conditions:
        - name: external-traffic
          expression: object.spec.type == 'NodePort'
      services:
        types: [NodePort]

The first rule denies the selected node-selector entry on matching Pods. The second denies NodePort Services independently. The Service-specific expression is evaluated only for Service requests because its rule has only Service policies. When sharing an enforce block across resource types, write conditions that handle every selected type, using has(...) or request.kind as needed.

Skipping an enforce block removes it from the ordered allow/deny/audit evaluation, including any allow-list requirement it would otherwise introduce.

Find the failing condition

A mutation condition error includes its location and optional condition name:

rules[0].mutate[1]: conditions[0] ("shared-pool"): ...

Indices are zero-based within the effective rules and mutation entries evaluated for this request. Inspect the namespace’s effective RuleStatus and its audience selection when tracing a rendered rule back to its source. A mutation condition gates every property in that entry.

Mutation errors identify the affected property where applicable. For example, an affinity merge that exceeds the term limit identifies affinity: nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution together with the rule and mutation indices. Enforcement validation condition errors identify the enforce block, rule index, and condition:

enforce: enforcement rule[0]: conditions[0] ("restricted-pods"): ...

Condition errors during metadata or resource mutation instead identify rules[0].enforce before the condition index and name.

6.1 - Mutate

Set resource properties during admission with typed mutations

Use spec.rules[].mutate to set resource properties during admission. It is an ordered list of entries, each with an optional action and a typed resource block. Currently, workloads applies to Pods on creation; security.readOnlyRootFilesystem and registries.imagePullPolicy also apply to newly added ephemeral containers. Mutation targets select compatible Pod locations, with pod including all container groups.

mutate and enforce are sibling keys. Mutation changes the resource; enforcement validates the result. Either can be used alone, and mutated values remain subject to applicable enforcement policies.

See Order and scope for the admission sequence and Conditions for conditional mutations.

Action

Each mutate entry supports an action field:

Action	Behavior
`merge` (default)	Apply supplied values while retaining unrelated settings. Map keys are set, list entries are added or updated, and affinity restrictions are combined according to the property’s rules.
`replace`	Replace each supplied property in full. Other properties remain unchanged.

The action applies to every property in that entry. Use separate entries when properties need different actions:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: merge
          workloads:
            placement:
              nodeSelector:
                kubernetes.io/os: linux
        - action: replace
          workloads:
            placement:
              tolerations:
                - key: infrastructure.example.com/dedicated
                  operator: Equal
                  value: shared
                  effect: NoSchedule

This retains other node-selector keys, but replaces the entire tolerations list. Other admission controllers can subsequently add tolerations.

Replacement happens at the supplied property’s boundary. For example, placement.nodeSelector is replaced as one map and placement.affinity as one object, including all its branches. The placement reference explains which list entries match during a merge and how required affinity is combined. Scalar settings such as security.hostUsers, security.readOnlyRootFilesystem, and registries.imagePullPolicy overwrite selected values with either action.

Omitted and empty values

Value in the mutation	Effect
Property omitted or `null`	Leave the existing property unchanged with either action.
Empty `placement.nodeSelector`, `placement.tolerations`, `placement.topologySpreadConstraints`, `placement.affinity`, or `registries.imagePullSecrets` with `replace`	Clear that property.
Empty `placement: {}`, `security: {}`, or `registries: {}`	Supply no changes; another mutation property must be supplied.
`security.hostUsers: false` or `security.readOnlyRootFilesystem: false`	Set an explicit Boolean value; `false` is not an omission.

For example, this clears node selectors and affinity while retaining tolerations and topology-spread constraints:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: replace
          workloads:
            placement:
              nodeSelector: {}
              affinity: {}

6.1.1 - Workloads

Configure Pod placement, security, and image pull policies with ordered mutations

Workload mutations set Pod properties under spec.rules[].mutate[].workloads using native Kubernetes Pod syntax. Use placement settings to select schedulers, assign node pools, add tolerations, spread workloads across topology domains, and configure affinity. Use security settings to configure Pod user namespaces, seccomp/AppArmor profiles, and read-only container root filesystems. Use registry settings to configure container image pull policies and Pod image pull secret references. Placement enforcement validates the resulting placement settings using matchers.

This page covers configuration and mutation behavior. The Reference provides a complete Tenant combining all workload mutation properties with placement, security-profile, and image pull policy enforcement.

Property	Field under `workloads`	Pod field	Enforcement
Scheduler	`placement.scheduler`	`spec.schedulerName`	Scheduler matchers
Node selectors	`placement.nodeSelector`	`spec.nodeSelector`	Node-selector matchers
Tolerations	`placement.tolerations`	`spec.tolerations`	Toleration matchers
Topology spread constraints	`placement.topologySpreadConstraints`	`spec.topologySpreadConstraints`	Spread matchers
Affinity	`placement.affinity`	`spec.affinity`	Affinity matchers
Seccomp	`security.seccompProfile`	`spec.securityContext.seccompProfile`	Seccomp matchers
AppArmor	`security.appArmorProfile`	`spec.securityContext.appArmorProfile`	AppArmor matchers
Read-only root filesystem	`security.readOnlyRootFilesystem`	Selected containers: `securityContext.readOnlyRootFilesystem`	Mutation only.
Image pull secrets	`registries.imagePullSecrets`	`spec.imagePullSecrets`	Mutation only.
Image pull policy	`registries.imagePullPolicy`	Selected containers: `imagePullPolicy`	Registry pull policies
Host user namespace	`security.hostUsers`	`spec.hostUsers`	Mutation only.

Configure workload mutations

Each mutation entry has an optional action and a workloads block containing at least one property to set. Scheduling fields live under workloads.placement; security fields live under workloads.security. targets stays directly under workloads and applies to all configured groups. The action applies to each supplied field inside a group: replacing placement.scheduler preserves an omitted placement.nodeSelector and all omitted security fields. Empty placement: {} or security: {} groups supply no mutations; use an explicit empty map or list on a supported field to clear it. For conditional mutations, see Conditions.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          environment: production
      mutate:
        - action: merge
          workloads:
            placement:
              nodeSelector:
                kubernetes.io/os: linux
      enforce:
        action: allow
        workloads:
          placement:
            nodeSelector:
              - key:
                  exact: [kubernetes.io/os]
                values:
                  exact: [linux]

In selected namespaces, this rule sets the Linux node selector. The allow rule then rejects any additional node-selector entries. mutate and enforce are sibling keys and can be used independently. enforce.action does not select or disable mutations.

Workload mutations apply on Pod creation. security.readOnlyRootFilesystem and registries.imagePullPolicy also apply to newly added ephemeral containers. See Order and scope for the mutate-then-enforce sequence, rule selection, and supported operations.

Targets

Set mutate[].workloads.targets to select where the entry applies. Omitting targets, using targets: [], or selecting pod includes all compatible locations. For mutation, pod includes every regular, init, and ephemeral container. Native sidecars belong to the init-container group.

The table lists supported mutation targets. ✅ indicates mutation support; ❌ means that property cannot be mutated through that target. Controller and volume targets are supported by enforcement but are rejected by the mutation API.

Pods created by controllers are still mutated

Every new Pod is evaluated against the applicable Pod mutation rules, whether created directly or by a Deployment, StatefulSet, DaemonSet, Job, CronJob, or another controller. Stored controller templates remain unchanged. The resulting Pods still receive applicable mutations when they are created.

Namespace selection, mutation targets, conditions, and audience still determine which mutations apply. Audience matching uses the identity creating the Pod, usually the controller’s service account.

Target	Placement / Image pull secrets	`security.hostUsers`	Seccomp / AppArmor	`security.readOnlyRootFilesystem` / `registries.imagePullPolicy`
Omitted, `[]`, or `pod`	✅ Pod-level fields	✅ Pod user namespace	✅ Pod-level profiles only	✅ All container groups
`pod/containers`	❌	❌	❌	✅ Regular containers
`pod/initcontainers`	❌	❌	❌	✅ Init containers and native sidecars
`pod/ephemeralcontainers`	❌	❌	❌	✅ Newly added ephemeral containers

Placement includes placement.scheduler, placement.nodeSelector, placement.tolerations, placement.topologySpreadConstraints, and placement.affinity. These properties, security.hostUsers, security.seccompProfile, security.appArmorProfile, and registries.imagePullSecrets require pod or omitted/empty targets. Seccomp and AppArmor mutation preserves explicit container profiles with both merge and replace; container-specific profile mutation is not supported.

security.readOnlyRootFilesystem and registries.imagePullPolicy support container-specific mutation targets. Use separate mutation entries to combine Pod-level settings with a narrower container selection. Controller and volume targets are not supported for typed workload mutations.

Targets combine by inclusion: listing pod alongside a narrower target still selects all groups. A targets-only mutation is invalid because it has no property to set. Enforcement targets are configured independently and have property-specific matching semantics.

Regular and init containers are mutated on Pod creation. Ephemeral containers are mutated only when newly added through UPDATE pods/ephemeralcontainers. An existing ephemeral container is left unchanged, even if the rule changed since it was added. Other Pod updates, deletes, and subresources do not apply these mutations. On an ephemeral update, the entry’s conditions see the full current Pod, including earlier mutations, but only the new ephemeral containers' securityContext.readOnlyRootFilesystem and imagePullPolicy values can change.

Placement

Scheduler

placement.scheduler sets the Pod’s spec.schedulerName. Use the name of a scheduler configured in your cluster; Capsule selects the scheduler but does not deploy it. See Kubernetes multiple schedulers.

To select a scheduler for every new Pod covered by the rule, use replace:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: replace
          workloads:
            placement:
              scheduler: tenant-scheduler

Action	Behavior
`merge`	Fill an empty `spec.schedulerName`. Preserve every non-empty value, including `default-scheduler`.
`replace`	Set `spec.schedulerName` to the configured value when the entry’s conditions match.

Omitting placement.scheduler or setting it to null leaves the current value unchanged. An empty or blank configured name is invalid. To select the built-in scheduler, set scheduler: default-scheduler explicitly.

Default a scheduler with a condition

Kubernetes fills an omitted or empty spec.schedulerName with default-scheduler before Capsule receives the admission request. To use a tenant scheduler as the default while preserving custom scheduler names, combine replace with a mutation condition:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: replace
          conditions:
            - name: default-scheduler
              expression: >-
                !has(object.spec.schedulerName) || object.spec.schedulerName in ['', 'default-scheduler']                
          workloads:
            placement:
              scheduler: tenant-scheduler

Scheduler in the submitted Pod	Result with this condition
Omitted or empty	`tenant-scheduler`
`default-scheduler`	`tenant-scheduler`
`batch-scheduler`	`batch-scheduler`

The condition cannot distinguish an omitted value from an explicitly selected default-scheduler: both reach admission with the built-in name. This rule deliberately replaces both. A custom scheduler supplied by the user or an earlier mutation makes the condition false and is retained.

Keep this scheduler default in its own mutation entry. Conditions gate every property in an entry, so adding node selectors or tolerations to it would make those changes depend on the scheduler condition too. Namespace selection and audience still determine which rules apply. Only new Pods are changed; existing Pods and controller templates retain their settings.

The final scheduler name remains subject to scheduler enforcement. Allow the configured default in any applicable scheduler allow-list, or admission will reject the mutated Pod.

Node selectors

placement.nodeSelector is a map of node-label keys and values. Every entry must match a node for the Pod to be scheduled there.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: merge
          workloads:
            placement:
              nodeSelector:
                infrastructure.example.com/pool: shared

Action	Behavior
`merge`	Set the supplied keys, overwriting their existing values. Retain other keys.
`replace`	Replace the entire map. Use `nodeSelector: {}` to clear it.

For a Pod whose selector is {infrastructure.example.com/pool: private, disk: ssd}, the example produces:

Action	Resulting node selector
`merge`	`{infrastructure.example.com/pool: shared, disk: ssd}`
`replace`	`{infrastructure.example.com/pool: shared}`

Later mutations can overwrite the same key. For defaults, see set a value only when it is missing. Use node-selector enforcement to restrict which keys and values remain on the admitted Pod.

Tolerations

placement.tolerations is a list of native Pod tolerations. A toleration permits scheduling onto a node with a matching taint; it does not require that node. Use a node selector or required node affinity to restrict placement to a node pool.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: merge
          workloads:
            placement:
              tolerations:
                - key: infrastructure.example.com/dedicated
                  operator: Equal
                  value: shared
                  effect: NoExecute
                  tolerationSeconds: 60

Action	Behavior
`merge`	Add or update the toleration identified by key + operator + value + effect. An omitted operator means `Equal`.
`replace`	Replace the entire list, including tolerations with other identities. Use `tolerations: []` to clear it.

The duration is not part of the merge identity. A matching toleration receives the supplied tolerationSeconds. Omitting the duration removes an existing limit and makes the toleration unlimited.

Existing toleration	Supplied toleration	Result with `merge`
`pool=private`, `Equal`, `NoSchedule`	`pool=shared`, `Equal`, `NoSchedule`	Both remain because their values differ.
`pool=shared`, `Equal`, `NoExecute`, 30 seconds	Same identity, 60 seconds	One toleration with a 60-second duration.
`pool=shared`, `Equal`, `NoExecute`, 30 seconds	Same identity, no duration	One toleration with unlimited duration.

Changing a key, operator, value, or effect adds another toleration under merge. Use replace when only the supplied list should remain. Other admission controllers may subsequently inject additional tolerations. All final entries are subject to toleration enforcement, including Kubernetes-injected entries.

Topology spread constraints

placement.topologySpreadConstraints is a list of native Pod spread constraints. Each constraint specifies a topology domain and the Pods to count when spreading.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: merge
          workloads:
            placement:
              topologySpreadConstraints:
                - topologyKey: kubernetes.io/hostname
                  whenUnsatisfiable: ScheduleAnyway
                  maxSkew: 1
                  labelSelector:
                    matchLabels:
                      app: checkout

This expresses a preference to spread Pods labeled app: checkout across hosts. Use DoNotSchedule when the spread constraint must prevent scheduling that would violate it.

Action	Behavior
`merge`	Add or replace the entire constraint identified by `topologyKey` + `whenUnsatisfiable`. Retain constraints with other identities.
`replace`	Replace the entire list. Use `topologySpreadConstraints: []` to clear it.

If the Pod already has a kubernetes.io/hostname / ScheduleAnyway constraint with maxSkew: 3, the example replaces it with maxSkew: 1 and the supplied selector. Old selector fields and optional fields are not carried forward. An existing topology.kubernetes.io/zone / DoNotSchedule constraint remains with merge and is removed with replace unless it is also supplied.

Changing whenUnsatisfiable creates a different merge identity, even if the topology key is unchanged. For allowed topology keys, skew ranges, and selector operators, see spread enforcement.

Affinity

placement.affinity uses the native Pod structure with three branches: nodeAffinity, podAffinity, and podAntiAffinity. Each can contain required and preferred terms.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: merge
          workloads:
            placement:
              affinity:
                nodeAffinity:
                  requiredDuringSchedulingIgnoredDuringExecution:
                    nodeSelectorTerms:
                      - matchExpressions:
                          - key: topology.kubernetes.io/zone
                            operator: In
                            values: [zone-a, zone-b]
                podAffinity:
                  preferredDuringSchedulingIgnoredDuringExecution:
                    - weight: 50
                      podAffinityTerm:
                        topologyKey: kubernetes.io/hostname
                        labelSelector:
                          matchLabels:
                            app: cache
                podAntiAffinity:
                  preferredDuringSchedulingIgnoredDuringExecution:
                    - weight: 100
                      podAffinityTerm:
                        topologyKey: kubernetes.io/hostname
                        labelSelector:
                          matchLabels:
                            app: checkout

This requires a node in zone-a or zone-b, prefers a host with cache Pods, and prefers to separate checkout Pods across hosts. The referenced node labels must exist on the intended nodes. Omitting namespace selection in these Pod-affinity terms limits their Pod matching to the incoming Pod’s namespace.

Merge

Affinity term	Behavior with `merge`
Required node affinity	Require both the existing restrictions and the supplied restrictions. Combine existing and supplied alternatives to preserve that AND relationship.
Required Pod affinity or anti-affinity	Add distinct complete terms. All required terms remain required; identical terms are retained once.
Preferred affinity of any type	Add distinct complete terms or replace the weight of a matching term.

For example, existing node alternatives (disk=ssd OR arch=arm64) combined with supplied zone=a become (disk=ssd AND zone=a) OR (arch=arm64 AND zone=a). Required restrictions accumulate across applicable mutations. Capsule rejects combinations that would produce more than 256 node-selector alternatives. An empty node-selector term matches no nodes.

Preferred-term identity includes the complete node preference or Pod-affinity term, excluding its weight. Requirement and selector-value order are normalized for matching. Supplying the same term with weight 80 updates an existing weight 50; changing the selector adds a distinct preferred term.

Replace

replace replaces all of placement.affinity, including branches omitted from the mutation. Supplying only nodeAffinity also removes existing podAffinity and podAntiAffinity. Use affinity: {} to clear all three branches.

For allowed types, required/preferred modes, selector requirements, and namespace scope, see affinity enforcement. Its matcher syntax uses one flat list across all three affinity types.

Security

Seccomp and AppArmor mutations set only the Pod-level profile on creation. Use targets: [pod] or omit mutation targets for these Pod-level properties. enforce.workloads.targets selects enforcement locations independently. Both merge and replace preserve explicit regular, init, and ephemeral container profiles. Containers without an override inherit the Pod default. See the Pod targets table and profile target example for how enforcement checks those defaults and overrides.

Read-only root filesystem

security.readOnlyRootFilesystem sets the Boolean on every selected container’s securityContext. Like security.hostUsers, both merge and replace set the supplied value, including an explicit false. Omitted or null leaves the value unchanged. Later matching entries can overwrite earlier values; all other container security-context fields are preserved.

This full Tenant makes all container root filesystems read-only in namespaces labeled filesystem-profile: readonly. Namespaces labeled filesystem-profile: init-only apply the setting only to init containers, including native sidecars. Other namespaces keep their submitted values.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          filesystem-profile: readonly
      mutate:
        - action: merge
          workloads:
            targets: [pod]
            security:
              readOnlyRootFilesystem: true
    - namespaceSelector:
        matchLabels:
          filesystem-profile: init-only
      mutate:
        - action: replace
          workloads:
            targets: [pod/initcontainers]
            security:
              readOnlyRootFilesystem: true

In the first profile, pod also selects new ephemeral containers. Choose pod/containers for regular containers only or pod/ephemeralcontainers for new debug containers only. Neither the rule nor a namespace-label change rewrites existing containers. Pods declaring spec.os.name: windows are skipped.

A read-only root filesystem does not make mounted volumes read-only. Applications that write to paths such as /tmp need suitable writable volume mounts. See the Kubernetes security-context documentation. This field is a mutation setting; there is no corresponding enforce.workloads.security.readOnlyRootFilesystem matcher.

Host user namespace

security.hostUsers selects whether the Pod uses the host user namespace.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - mutate:
        - action: merge
          workloads:
            security:
              hostUsers: false

Supplied value	Behavior with `merge` or `replace`
`false`	Request a separate user namespace for the Pod.
`true`	Use the host user namespace.

Both actions overwrite an existing Boolean. A later entry can change false to true or the reverse. This property is independent of a container’s runAsUser and has no corresponding enforce.workloads.security.hostUsers matcher.

The Kubernetes version, operating system, and container runtime must support the requested setting. Kubernetes validates incompatible Pod settings; Capsule does not adjust other security or host-namespace fields. See Kubernetes user namespaces.

Seccomp

security.seccompProfile sets spec.securityContext.seccompProfile on new Linux Pods. Containers inherit this Pod default unless their own security context supplies a seccomp profile. This includes regular, init, and ephemeral containers.

Property	Meaning
`type: RuntimeDefault`	Use the container runtime’s default seccomp profile. Omit `localhostProfile`.
`type: Localhost`	Use a seccomp profile file on the node. Set `localhostProfile` to its relative path.
`type: Unconfined`	Disable seccomp filtering. Omit `localhostProfile`.

Seccomp mutation behavior

Action	Behavior
`merge`	Set the configured profile only when the Pod has no `spec.securityContext.seccompProfile`. Preserve any supplied profile, including its localhost path.
`replace`	Replace the complete Pod profile with the configured type and localhost path.
Property omitted	Preserve the Pod’s seccomp profile.

Merge treats the profile as one property: it does not fill a missing localhostProfile inside an already supplied profile. A Localhost profile must include its path. Replacing Localhost with RuntimeDefault removes the old path.

Other security-context fields and explicit container profiles are preserved. Use seccomp enforcement to prevent container overrides from bypassing the intended profile. Mutations use the existing conditions and order, run only on Pod creation, and skip Pods declaring spec.os.name: windows. They do not change running Pods or controller templates.

Default to RuntimeDefault

This complete Tenant defaults seccomp in namespaces labeled security-profile: confined. The allow rule requires every container’s effective profile to be RuntimeDefault, including containers added later through pods/ephemeralcontainers.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          security-profile: confined
      mutate:
        - action: merge
          workloads:
            security:
              seccompProfile:
                type: RuntimeDefault
      enforce:
        action: allow
        workloads:
          security:
            seccompProfiles:
              - types: [RuntimeDefault]

An explicit container-level Unconfined profile remains unchanged by mutation and is rejected by enforcement. Namespaces without the selector label do not receive either rule effect.

Set a local seccomp path

Set type: Localhost together with localhostProfile. The path is relative to the kubelet’s seccomp directory and must not be absolute or contain .. path segments. For example, profiles/solar.json refers to /var/lib/kubelet/seccomp/profiles/solar.json when the kubelet uses its default root directory. It is a file on the node, independent of the container’s filesystem. See the Kubernetes seccomp tutorial.

This Tenant replaces the Pod default with that local profile and allows only that profile for the effective container checks:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          security-profile: local-seccomp
      mutate:
        - action: replace
          workloads:
            security:
              seccompProfile:
                type: Localhost
                localhostProfile: profiles/solar.json
      enforce:
        action: allow
        workloads:
          security:
            seccompProfiles:
              - types: [Localhost]
                localhostProfiles:
                  - exact: [profiles/solar.json]

Use action: merge instead to supply this profile only when the Pod profile is absent. With the allow-list above, an existing different profile is then preserved by merge and rejected by enforcement.

Install the file on every eligible node before using the rule, or combine the rule with placement that selects prepared nodes. Capsule writes the profile reference; it does not distribute or inspect the file. A Pod can pass admission and still fail to start if the profile is unavailable on its node.

AppArmor

security.appArmorProfile sets spec.securityContext.appArmorProfile on new Linux Pods. The Pod value supplies the default for containers that have no AppArmor override. Use this mutation only for workloads placed on nodes with AppArmor support. Explicit RuntimeDefault can prevent a Pod from starting on a node without that support. See the AppArmor prerequisites.

Property	Meaning
`type: RuntimeDefault`	Use the container runtime’s default AppArmor profile. Omit `localhostProfile`.
`type: Localhost`	Use an AppArmor profile already loaded on the node. Set `localhostProfile` to its loaded name.
`type: Unconfined`	Disable AppArmor confinement. Omit `localhostProfile`.

AppArmor mutation behavior

Action	Behavior
`merge`	Set the configured profile only when the Pod has no `spec.securityContext.appArmorProfile`. Preserve any supplied profile, including its localhost name.
`replace`	Replace the complete Pod profile with the configured type and localhost name.
Property omitted	Preserve the Pod’s AppArmor profile.

Merge does not complete a partially supplied Localhost profile. Replacing a Localhost profile with RuntimeDefault removes the old localhostProfile. Other security-context fields and container overrides, including legacy AppArmor annotations, remain unchanged. Pair mutation with AppArmor enforcement to validate these effective container choices.

The existing conditions and order apply. Mutations run only on Pod creation and skip Pods declaring spec.os.name: windows; they do not change running Pods or controller templates.

Default the AppArmor profile

This Tenant supplies RuntimeDefault and requires it for every effective container profile in the selected namespaces:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          security-profile: apparmor
      mutate:
        - action: merge
          workloads:
            security:
              appArmorProfile:
                type: RuntimeDefault
      enforce:
        action: allow
        workloads:
          security:
            appArmorProfiles:
              - types: [RuntimeDefault]

Set a local AppArmor profile

For AppArmor, localhostProfile is the loaded profile name, such as solar-confined. It is not a filesystem path to the profile definition. The name must match the profile loaded on the node. See Specifying AppArmor confinement.

This Tenant replaces the Pod default with that name and restricts effective container profiles to the same choice:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          security-profile: local-apparmor
      mutate:
        - action: replace
          workloads:
            security:
              appArmorProfile:
                type: Localhost
                localhostProfile: solar-confined
      enforce:
        action: allow
        workloads:
          security:
            appArmorProfiles:
              - types: [Localhost]
                localhostProfiles:
                  - exact: [solar-confined]

Use action: merge to preserve an existing Pod profile instead of replacing it; the enforcement rule still rejects any different effective profile. Load solar-confined on every eligible node before applying the rule, or restrict placement to prepared nodes. Capsule does not load AppArmor profiles or verify that their definitions are consistent across nodes.

Registries

Configure registry-related mutations under mutate[].workloads.registries. Set imagePullPolicy for selected containers and imagePullSecrets for the Pod. The targets table shows which locations support each property. Both settings apply to Linux and Windows Pods.

Image pull policy

Set registries.imagePullPolicy to one of the native Kubernetes values:

Value	Container image behavior
`Always`	Resolve the image through the registry at each container start; cached image layers can still be reused.
`IfNotPresent`	Pull the image when it is not already available on the node.
`Never`	Use an image already available on the node; the container cannot start if it is missing.

See Kubernetes image pull policies for runtime behavior. Capsule changes the container’s policy; it does not pull images or create registry credentials.

Mutation configuration	Behavior
`action: merge` or omitted action	Overwrite the selected containers’ pull policies.
`action: replace`	Overwrite the selected containers’ pull policies.
`registries` or `imagePullPolicy` omitted or `null`	Preserve existing pull policies.
`registries: {}`	Leave pull policies unchanged. The mutation entry still needs a property to set.
Empty string or a value other than `Always`, `IfNotPresent`, or `Never`	Reject the rule configuration.

Kubernetes fills an omitted pull policy before mutation. Capsule overwrites both that default and an explicit user value with either action: merge does not mean “only when unset” for this property. Later applicable mutation entries can overwrite it again. Conditions gate the whole entry and see changes made by preceding entries.

registries is a settings object. The mutation applies the same policy to every selected container, regardless of its image registry. To restrict image sources or validate the resulting pull policy, use registry enforcement, where registries is a list of matchers with an optional policy allowlist.

Apply different namespace profiles

This Tenant uses Always for all container groups in namespaces labeled image-profile: always-pull and allows images only from registry.k8s.io. Namespaces labeled image-profile: init-only receive the mutation only for init containers and native sidecars, preserving regular-container policies.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          image-profile: always-pull
      mutate:
        - action: replace
          workloads:
            targets: [pod]
            registries:
              imagePullPolicy: Always
      enforce:
        action: allow
        workloads:
          targets: [pod/containers, pod/initcontainers, pod/ephemeralcontainers]
          registries:
            - exact: [registry.k8s.io]
              policy: [Always]
    - namespaceSelector:
        matchLabels:
          image-profile: init-only
      mutate:
        - action: merge
          workloads:
            targets: [pod/initcontainers]
            registries:
              imagePullPolicy: Always

Request	Result from these rules
New Pod in `always-pull`, with omitted, `Never`, or `IfNotPresent` policies	Regular and init containers receive `Always`; images must pass the registry allowlist.
New ephemeral container in `always-pull`	The new container receives `Always`; registry enforcement still applies.
New Pod in `init-only`	Init containers and native sidecars receive `Always`; regular containers keep their policies.
Namespace with neither profile label	Neither rule applies.

Existing containers and stored controller templates are unchanged. Pods created by controllers receive the mutation when admitted. Ordinary Pod updates do not rewrite pull policies, and ephemeral-container updates change only newly added containers. Image-volume pull policies are outside the mutation targets.

Image pull secrets

registries.imagePullSecrets sets the Pod’s spec.imagePullSecrets using a list of references with a name field. This is a Pod-level setting: use targets: [pod], targets: [], or omit targets. Container-only targets are rejected, including when clearing the list.

Configuration	Behavior
`merge` or omitted action	Keep each existing name once, in first-occurrence order, then append missing configured names.
`replace`	Set the complete list to the configured references.
`imagePullSecrets: []` with `replace`	Clear the Pod’s image pull secret references.
`imagePullSecrets: []` with `merge`	Remove duplicate references; otherwise preserve the existing list.
Property omitted or `null`	Preserve the list with either action.

Merge retains each name already supplied on the Pod once, including ServiceAccount defaults. Replace can remove those references. Later mutation entries see and can change the resulting list. Repeated names on the incoming Pod are removed, and overlapping rules or repeated admission do not introduce duplicates. Omitting the property leaves the list untouched. Each entry accepts up to 64 references with unique, valid Secret names; duplicate configured names are rejected for both actions.

Referenced Secrets must exist in the Pod’s namespace and contain suitable registry credentials. Kubernetes accepts kubernetes.io/dockerconfigjson or kubernetes.io/dockercfg Secrets for this purpose; see image pull secrets. Capsule writes only the references: it does not create, copy, read, or verify the Secrets. Provision the credentials in each selected namespace separately, for example with TenantResource or GlobalTenantResource.

This Tenant adds a shared reference in one namespace profile, replaces the list in another, and clears it in a third. Create platform-registry or team-registry in the corresponding namespaces before workloads need to pull private images.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          pull-credentials: shared
      mutate:
        - action: merge
          workloads:
            targets: [pod]
            registries:
              imagePullSecrets:
                - name: platform-registry
    - namespaceSelector:
        matchLabels:
          pull-credentials: dedicated
      mutate:
        - action: replace
          workloads:
            registries:
              imagePullSecrets:
                - name: team-registry
    - namespaceSelector:
        matchLabels:
          pull-credentials: none
      mutate:
        - action: replace
          workloads:
            registries:
              imagePullSecrets: []

These rules apply only when a Pod is created, including Pods created by controllers. They do not rewrite controller templates, ServiceAccounts, existing Pods, or the secret list when adding ephemeral containers. Configure both imagePullPolicy and imagePullSecrets in one registries block when using the Pod target. For a container-specific pull policy, use a separate mutation entry.

Reference

This complete Tenant combines every supported workload mutation property with placement, security-profile, and image pull policy enforcement. It selects namespaces labeled example.com/application: checkout and configures workloads for Linux nodes.

The tenant-scheduler scheduler, node pool, and zone labels must exist in the cluster. Eligible nodes must support user namespaces and AppArmor. Provision the checkout-registry image pull Secret in each selected namespace. Application Pods should carry app.kubernetes.io/part-of: checkout so the spread and Pod-affinity selectors describe the intended workload.

The first mutation entry replaces the built-in scheduler default while preserving custom scheduler names. The second entry independently applies the remaining placement settings, supplies missing Pod-level security profiles, and sets all selected container root filesystems read-only and image pull policies to Always. It also adds checkout-registry to the Pod’s image pull secret references. Provide writable volume mounts for application paths that need them.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: checkout
spec:
  owners:
    - kind: User
      name: checkout-owner
  rules:
    - namespaceSelector:
        matchLabels:
          example.com/application: checkout
      mutate:
        - action: replace
          conditions:
            - name: default-scheduler
              expression: >-
                !has(object.spec.schedulerName) || object.spec.schedulerName in ['', 'default-scheduler']                
          workloads:
            placement:
              scheduler: tenant-scheduler
        - action: merge
          workloads:
            targets: [pod]
            security:
              readOnlyRootFilesystem: true
              hostUsers: false
              seccompProfile:
                type: RuntimeDefault
              appArmorProfile:
                type: RuntimeDefault
            registries:
              imagePullPolicy: Always
              imagePullSecrets:
                - name: checkout-registry
            placement:
              nodeSelector:
                kubernetes.io/os: linux
                infrastructure.example.com/pool: shared
              tolerations:
                - key: infrastructure.example.com/dedicated
                  operator: Equal
                  value: shared
                  effect: NoSchedule
              # Application Pods must carry app.kubernetes.io/part-of: checkout.
              topologySpreadConstraints:
                - topologyKey: topology.kubernetes.io/zone
                  whenUnsatisfiable: DoNotSchedule
                  maxSkew: 1
                  labelSelector:
                    matchLabels:
                      app.kubernetes.io/part-of: checkout
              affinity:
                nodeAffinity:
                  requiredDuringSchedulingIgnoredDuringExecution:
                    nodeSelectorTerms:
                      - matchExpressions:
                          - key: topology.kubernetes.io/zone
                            operator: In
                            values: [zone-a, zone-b]
                podAffinity:
                  preferredDuringSchedulingIgnoredDuringExecution:
                    - weight: 50
                      podAffinityTerm:
                        topologyKey: topology.kubernetes.io/zone
                        labelSelector:
                          matchLabels:
                            app.kubernetes.io/part-of: checkout
                            app.kubernetes.io/component: cache
                podAntiAffinity:
                  preferredDuringSchedulingIgnoredDuringExecution:
                    - weight: 100
                      podAffinityTerm:
                        topologyKey: kubernetes.io/hostname
                        labelSelector:
                          matchLabels:
                            app.kubernetes.io/part-of: checkout
      enforce:
        action: allow
        workloads:
          targets:
            - pod
            - pod/containers
            - pod/initcontainers
            - pod/ephemeralcontainers
          # Allow any image source, but require the mutated pull policy.
          registries:
            - exp: '.*'
              policy: [Always]
          security:
            seccompProfiles:
              - types: [RuntimeDefault]
            appArmorProfiles:
              - types: [RuntimeDefault]
          placement:
            nodeSelector:
              - key: {exact: [kubernetes.io/os]}
                values: {exact: [linux]}
              - key: {exact: [infrastructure.example.com/pool]}
                values: {exact: [shared]}
              - key: {exp: '^placement\.example\.com/[a-z0-9-]+$'}
                values: {exp: '^[a-z0-9-]+$'}
            tolerations:
              - key: {exact: [infrastructure.example.com/dedicated]}
                operators: [Equal]
                values: {exact: [shared]}
                effects: [NoSchedule]
              - key:
                  exact:
                    - node.kubernetes.io/not-ready
                    - node.kubernetes.io/unreachable
                operators: [Exists]
                effects: [NoExecute]
                tolerationSeconds:
                  max: 600
                  allowUnlimited: false
              - key: {exact: [node.kubernetes.io/memory-pressure]}
                operators: [Exists]
                effects: [NoSchedule]
            topologySpreadConstraints:
              - topologyKey:
                  exact: [topology.kubernetes.io/zone, kubernetes.io/hostname]
                whenUnsatisfiable: [DoNotSchedule, ScheduleAnyway]
                maxSkew: {min: 1, max: 3}
                labelSelector:
                  required: true
                  requirements:
                    - key: {exact: [app.kubernetes.io/part-of]}
                      operators: [In]
                      values: {exact: [checkout]}
            affinity:
              - types: [nodeAffinity]
                modes: [required]
                requirements:
                  - key: {exact: [topology.kubernetes.io/zone]}
                    operators: [In]
                    values: {exact: [zone-a, zone-b]}
              - types: [nodeAffinity]
                modes: [preferred]
                weight: {min: 1, max: 100}
                requirements:
                  - key: {exact: [kubernetes.io/arch]}
                    operators: [In]
                    values: {exact: [amd64, arm64]}
              - types: [podAffinity, podAntiAffinity]
                modes: [preferred]
                topologyKey:
                  exact: [topology.kubernetes.io/zone, kubernetes.io/hostname]
                namespaceScope: SameNamespace
                weight: {min: 1, max: 100}
                labelSelector:
                  required: true
                  requirements:
                    - key: {exact: [app.kubernetes.io/part-of]}
                      operators: [In]
                      values: {exact: [checkout]}
                    - key: {exact: [app.kubernetes.io/component]}
                      operators: [In, NotIn]
                      values: {exact: [api, worker, cache]}

Submitted value or scope	Result
Scheduler omitted or `default-scheduler`	Replaced with `tenant-scheduler`.
Custom scheduler name	Preserved; the placement and security mutations still apply.
Existing image pull secret references	Each name is kept once in first-occurrence order; `checkout-registry` is appended if absent.
Image pull policy omitted, `IfNotPresent`, or `Never`	Set to `Always` on regular/init containers at creation and newly added ephemeral containers.
Root filesystem flag omitted or `false`	Set to `true` on regular/init containers at creation and newly added ephemeral containers.
Pod seccomp or AppArmor profile omitted	Defaulted to RuntimeDefault.
Explicit Pod or container profile	Preserved by mutation, then rejected unless its effective type is RuntimeDefault.
Existing node selector or required affinity	Combined with the configured restrictions, then checked by the placement allowlists.
Namespace without `example.com/application: checkout`	These rules do not apply.

Under enforce, the pod target validates placement and Pod-level profile defaults. The three container targets validate effective profiles and image pull policies for regular, init, and ephemeral containers, including container profile overrides. The registry matcher accepts all image sources while requiring Always. Under mutate, targets: [pod] selects all container groups for the root filesystem flag and image pull policy, and Pod-level fields, including image pull secrets, for the other properties; profile mutation still writes only Pod-level profiles. An Unconfined override or privileged container is rejected after mutation.

The toleration allowlist includes common Kubernetes-injected tolerations. Controllers such as DaemonSets may add others; include the entries needed by the workloads selected by your rule. The affinity validation uses one flat list for node affinity, Pod affinity, and Pod anti-affinity.

See the placement matcher reference for regular expressions, empty matchers, durations, selector operators, and namespace scope.

These mutations run on Pod creation. The root filesystem flag and image pull policy also apply to new ephemeral containers; the other properties are skipped on that subresource. Controller templates and existing containers are not rewritten. Profile enforcement also applies on subsequent Pod updates and pods/ephemeralcontainers updates. See Order and scope for rule composition and admission order.

6.2 - Enforce

Enforcement policies and restrictions on a per-Namespace basis with Rules

Namespace rules can enforce admission behavior for selected resources in Tenant namespaces. Each enforce block can define an action and one or more matchers.

Each policy chapter identifies its API configuration path, then opens with Rule Composition: Actions and order, followed by Admission scope and limits. Policy sections explain configuration and examples. The final Reference section provides complete Tenant examples. Use the policy reference to choose a chapter.

See Order and scope for how mutation precedes enforcement, and Conditions for conditional enforcement. Audience and match expressions are documented under Rules.

Rule Composition

Actions and order

Rules are evaluated in declaration order. If multiple allow or deny rules match the same evaluated value, the last matching allow or deny rule wins. If at least one allow rule is configured for a matcher and no allow or deny rule matches the evaluated value, Capsule denies the request. In other words, allow rules create an allow-list for that matcher. audit rules are purely observational: they never influence the allow/deny decision, and report matches through Kubernetes events. Some policies also add admission warnings; see the reporting behavior documented for each policy.

Each enforce block supports an action field:

Action	Behavior
`allow`	Allows the matching request and enables allow-list behavior for the matcher. If at least one allow rule exists and no allow or deny rule matches a value, Capsule denies that value. Additional constraints, such as image pull policy, must also be satisfied.
`deny`	Denies the matching request. A later matching `allow` rule can override it.
`audit`	Reports matching values without allowing or denying the request. NetworkPolicy CIDR rules emit Kubernetes events; other policies may also return admission warnings.

If action is omitted, Capsule treats the rule as deny.

Storage volume rules use these actions only for additional access to PVs without a Tenant ownership label. They preserve existing Tenant-owned volume access, and cannot override another Tenant’s PV label. A storage audit match never grants access.

Allow-list behavior is evaluated per matcher and per evaluated value. For example, if a registry allow rule exists for harbor/.*, a Pod image from docker.io/library/nginx:latest is denied unless another later or earlier allow rule also matches that image. Audit rules do not satisfy this allow-list requirement.

This precedence model allows both broad defaults and specific exceptions. For example, you can allow all Harbor images but deny a customer path afterwards:

rules:
  - enforce:
      action: allow
      workloads:
        registries:
          - exp: "harbor/.*"

  - enforce:
      action: deny
      workloads:
        registries:
          - exp: "harbor/customer/.*"

In this example, harbor/nginx:1.14.2 is allowed, while harbor/customer/app:1.0.0 is denied because the later, more specific deny rule also matches.

You can also deny broadly and allow a more specific exception afterwards:

rules:
  - enforce:
      action: deny
      workloads:
        registries:
          - exp: "harbor/customer/.*"

  - enforce:
      action: allow
      workloads:
        registries:
          - exp: "harbor/customer/prod-image/.*"

In this example, harbor/customer/test-image/app:1.0.0 is denied, while harbor/customer/prod-image/app:1.0.0 is allowed.

Audit

Use action: audit to observe workload usage without directly blocking the request. Audit rules emit Kubernetes events and add warnings to the admission response, but they do not allow or deny the request. If an allow-list is active for the same matcher and no allow rule matches the evaluated value, the request is still denied even when an audit rule matches.

For registry enforcement:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  ...
  rules:
    - enforce:
        action: audit
        workloads:
          targets:
            - pod/containers
          registries:
            - exp: "docker.io/.*"

Applying a Pod with docker.io/library/nginx:latest succeeds in this audit-only example because no registry allow-list is configured. The API server response contains an admission warning and Capsule emits a related event for the Pod.

For QoS enforcement:

rules:
  - enforce:
      action: audit
      workloads:
        qosClasses:
          - Burstable

Applying a Burstable Pod succeeds in this audit-only example because no QoS allow-list is configured. Capsule emits an event and returns an admission warning.

For scheduler enforcement:

rules:
  - enforce:
      action: audit
      workloads:
        placement:
          schedulers:
            - exact:
                - custom-scheduler

Applying a Pod with spec.schedulerName: custom-scheduler succeeds in this audit-only example because no scheduler allow-list is configured. Capsule emits an audit event and returns an admission warning.

When audit rules are used together with allow rules, the matching value must still be allowed explicitly. For example, an audited registry reference that does not match any registry allow rule is denied by the allow-list, but Capsule still emits the audit event before denying the request.

Admission scope and limits

Enforcement runs during admission for resources in the namespaces selected by the rule. Audience filters the caller, and conditions decide whether an enforcement block participates. Each policy chapter lists its supported resources, operations, subresources, and handling of missing values.

Mutation runs before validation. An allow decision applies to its evaluated policy and value; other applicable checks, RBAC, and Kubernetes API validation must still pass. In particular, an enclosing action: audit does not disable resource defaulting or mutation.

Enforcement evaluates incoming writes. Changing a rule or namespace profile does not by itself repair existing resources or revoke established bindings. Storage ownership, resource mutation, and checks involving related resources have specific limits documented in their chapters.

Reference

The following fields belong to spec.rules[].enforce. Each policy page includes a complete Tenant example and explains its supported resources, matching behavior, and admission scope.

Policy	Field under `enforce`	Complete Tenant example
Workloads	`workloads`	Workload reference
Metadata	`metadata`	Metadata reference
Services	`services`	Service reference
Ingress	`ingress`	Ingress reference
Network Policies	`network.policies.ingress.cidrs`, `network.policies.egress.cidrs`	NetworkPolicy reference
Storage	`storage.volumes`	Storage reference

To disallow CIDR-based ingress and egress grants in every namespace of a Tenant:

rules:
  - enforce:
      action: deny
      network:
        policies:
          ingress:
            cidrs:
              - 0.0.0.0/0
              - ::/0
          egress:
            cidrs:
              - 0.0.0.0/0
              - ::/0

Place this rules block under Tenant.spec. The rule rejects grants from or to any IPv4 or IPv6 CIDR, as well as unrestricted ingress or egress rules with omitted peers. Selector-based peers and empty grant lists pass their CIDR check. Each direction is independent; omit ingress or egress to leave that direction unconstrained. The NetworkPolicy reference shows how later rules can permit approved ranges in selected namespaces.

6.2.1 - Workloads

Workload enforcement

Workload enforcement is configured under spec.rules[].enforce.workloads. Use targets to select native workload kinds or locations inside their Pod specs. A targets-only rule matches the selected kinds themselves. When workload policies are present, targets scope those policies: resource management, OCI registries, scheduler names, QoS, placement, security profiles, and PodDisruptionBudgets.

Rules use the shared namespace selection, audiences, conditions, and actions and order. See Reference for a complete Tenant example.

Rule Composition

Actions and order

Rules are evaluated in declaration order after namespace selection, audience, and condition filtering. The last matching allow or deny decision wins for each evaluated value. An omitted action defaults to deny.

Action	Behavior
`allow`	Creates an allow-list for the evaluated matcher. A value must match an applicable allow or deny rule; otherwise an active allow-list rejects it.
`deny`	Rejects matching values, unless a later matching allow rule overrides the decision.
`audit`	Reports matches without allowing or denying them. Audit matches never satisfy an allow-list.

Each policy and value is evaluated independently. Allowing a registry does not bypass a scheduler, security, placement, or other applicable constraint.

For constraints on requests and limits and PodDisruptionBudgets, deny rejects violations and allow requires compliance. Their policy sections explain which values constitute a match and how audit events and admission warnings are reported.

Resource policies also have a mutation phase. The last applicable policy is effective for each target, resource, and request or limit field. The enclosing action: audit does not disable resource mutation. See Rule order and policy overrides for how Preserve and Default reset earlier constraints.

Admission scope and limits

Workload rules govern admission of the selected resources and Pod-spec locations. Workload Targets maps kinds to their Pod specs; the controller and Pod tables show where each policy takes effect.

Scope	Behavior
Targets omitted	Policies use their supported Pod locations. Controller templates require explicit controller targets.
Creation and updates	Validation checks the incoming resource on supported write paths. Policy-specific subresource handling is described below.
Resource mutation	Request and limit policies mutate Pods on creation only. With explicit controller targets, they also mutate controller templates on creation and update.
Pod subresources	Resource validation skips Pod subresources, including `ephemeralcontainers`. Registry and security policies use their supported container targets.
PDB relationships	Checks can run on workload writes, PDB writes, and supported controller replica changes, including `/scale`. See Evaluation for triggers and limits.
Existing resources	Rule or namespace-profile changes affect subsequent relevant admissions; they do not rewrite existing workloads.

A controller write and the Pods it later creates are separate admissions, often with different callers. Include the required targets and audiences for both paths. Conditions inspect the actual incoming resource, which can also be a PDB or a Scale object.

Kubernetes validation, LimitRanges, ResourceQuotas, and other admission policies still apply after Capsule’s checks. See Admission lifecycle for the resource policy stages and Interaction with Kubernetes resource controls for their combined behavior.

Workload Targets

Configure targets under spec.rules[].enforce.workloads.targets. The same list selects workloads for resource, registry, scheduler, QoS, placement, security, and PodDisruptionBudget policies; individual policies do not have separate target lists.

With no workload policies, targets apply the rule’s action to the selected workload kinds. For example, targets: [daemonset] with action: deny rejects DaemonSets.
With workload policies, targets scope those policies to the selected workloads and Pod-spec locations. Adding a policy means the rule checks that property instead of restricting the kind itself. Use separate rule entries when you need both kind restrictions and property checks.

Omitted or empty targets preserve the default Pod scopes described below and do not enable controller checks. An empty workload block has no effect. Namespace selectors, audiences, and enforce.conditions apply to both forms.

Each target selects the object being admitted. With workload policies configured, Capsule reads the Pod spec at the following location:

Target	Native API group / kind	Pod spec evaluated on that object
`pod`	core / Pod	`spec`
`deployment`	apps / Deployment	`spec.template.spec`
`statefulset`	apps / StatefulSet	`spec.template.spec`
`daemonset`	apps / DaemonSet	`spec.template.spec`
`replicaset`	apps / ReplicaSet	`spec.template.spec`
`replicationcontroller`	core / ReplicationController	`spec.template.spec`
`job`	batch / Job	`spec.template.spec`
`cronjob`	batch / CronJob	`spec.jobTemplate.spec.template.spec`

For example, cronjob/containers checks regular containers at spec.jobTemplate.spec.template.spec.containers[] on the CronJob itself. It does not select a Job or Pod by following its CronJob owner.

Requests match their own API group and kind, without following owner references. A custom resource named DaemonSet in another API group is not a native target. Targets use the exact lowercase values shown here; wildcards such as pod/*, deployment/*, and * are not supported.

Migration: targets-only rules previously had no effect. They now apply the action to the selected kinds, and an omitted action defaults to deny. Remove unused targets-only entries before upgrading if they were placeholders.

Controller targets

Controller targets check the stored Pod template during controller creation and main-resource updates. They do not wait for a Pod to be created. A denial rejects the controller write; an accepted template supplies the values Kubernetes later uses to create child workloads. Existing children are not rewritten by admission.

The table uses cronjob as an example. The same suffixes and policy scopes apply to every controller kind listed above, using that kind’s Pod-spec path. Paths in this table are relative to spec.jobTemplate.spec.template.spec. ✅ indicates support; ❌ means the policy does not apply to that target. Only policies configured in the rule are evaluated.

Policy	`cronjob`	`cronjob/containers`	`cronjob/initcontainers`	`cronjob/volumes`
Resource policies	✅ Pod-level resources and regular/init-container resources	✅ `containers[].resources`	✅ `initContainers[].resources`	❌
Registry policies	✅ Regular/init-container images and image volumes	✅ Regular container images	✅ Init container images	✅ Image volumes under `volumes[].image`
Placement policies	✅ Pod placement fields	❌	❌	❌
Seccomp / AppArmor	✅ Effective regular/init-container profiles	✅ Effective regular-container profiles	✅ Effective init-container profiles	❌
PDB policies	✅ Template labels: overlap and unhealthy eviction policy	❌	❌	❌

For example, deployment/initcontainers has the same scope as cronjob/initcontainers, but reads spec.template.spec.initContainers[]. Controller templates do not support /ephemeralcontainers. Resource policies cannot be combined with volume targets.

The placement row covers node selectors, tolerations, topology spread, and affinity. Scheduler and QoS policies always evaluate the whole selected Pod spec, even with a container or volume suffix. For example, cronjob/initcontainers with a scheduler policy checks spec.jobTemplate.spec.template.spec.schedulerName.

A whole-controller target includes container resources and image references; the explicit pod target has the narrower scope in the Pod table. For security profiles, the effective value is the container override or its inherited Pod default. A whole-controller target does not independently require a Pod-level default when every container supplies its own allowed profile.

Whole-controller PDB checks use the Pod template’s labels to find matching budgets. CronJobs, Jobs, and DaemonSets support overlap and unhealthy eviction policy checks, but not evictableReplicas. See the PDB target table for the complete mapping.

Resource request/limit policies also mutate the selected template locations on controller creation and updates. Targets do not extend mutate[].workloads to controllers; mutations configured there remain limited to Pod creation. See Admission lifecycle.

Conditions see the whole admitted controller: use object.spec.template.spec for a Deployment and object.spec.jobTemplate.spec.template.spec for a CronJob. Events concern that controller object. Namespace selection, audiences, and conditions must match each request independently.

Example: CronJob admission

This rule denies a forbidden scheduler or image in Deployment and CronJob templates while permitting compliant ones:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets: [deployment, cronjob]
          placement:
            schedulers:
              - exact: [forbidden-scheduler]
          registries:
            - exact: ['example.com/blocked/app:v1']

For the CronJob path, assuming the namespace, audience, and conditions match:

Admission request	Does this rule run?	Effect
Create or update the CronJob	✅ `cronjob` is selected	Checks the scheduler and image references under `spec.jobTemplate.spec.template.spec`. A match rejects the CronJob write.
The CronJob controller creates a Job	❌ `job` is not selected	The Job inherits the template, but this rule does not check its admission. Add `job` to check it.
The Job controller creates a Pod	❌ No Pod target is selected	The Pod inherits the template, but this rule does not check its admission. Add the relevant Pod targets to check it.

To check all three stages, include cronjob, job, and the Pod targets needed for the policy. For registry checks, use pod/containers, pod/initcontainers, pod/ephemeralcontainers, and/or pod/volumes for the image locations you want to cover; pod alone does not select images. Controller-created Jobs and Pods can have different callers, so include those identities in any audience filter.

With no workload policies, targets: [cronjob] instead acts on the kind: deny rejects CronJobs, while allow establishes a workload-kind allow-list. A targets-only allow-list must also allow job and pod for scheduled workloads to run. See Select Workloads.

Pod targets

When workload policies are present, Pod targets select these locations. ✅ indicates support; ❌ means the policy does not apply to that target.

Target	Resource policies	Registry policies	Placement policies	Seccomp / AppArmor	PDB policies
`pod`	✅ Pod-level `spec.resources`	❌	✅ Pod placement fields	✅ Pod default only	✅ Overlap and unhealthy eviction policy
`pod/containers`	✅ `spec.containers[].resources`	✅ Regular container images	❌	✅ Effective regular-container profiles	❌
`pod/initcontainers`	✅ `spec.initContainers[].resources`	✅ Init container images	❌	✅ Effective init-container profiles	❌
`pod/ephemeralcontainers`	❌	✅ Ephemeral container images	❌	✅ Effective ephemeral-container profiles	❌
`pod/volumes`	❌	✅ Image volumes under `spec.volumes[].image`	❌	❌	❌

For seccomp and AppArmor, an effective profile resolves a container override before the Pod default. Init containers include restartable sidecars. Selecting pod alone checks the default and leaves container overrides unchecked. See Target security profiles for a complete Tenant and a comparison of these scopes.

This table describes enforcement. Profile mutations under mutate[].workloads set Pod-level defaults on Pod creation, using pod or omitted mutation targets. Neither merge nor replace modifies container profile overrides. For security.readOnlyRootFilesystem and registries.imagePullPolicy mutations, pod selects all container groups and container targets can narrow the selection. See the separate mutation targets table. Enforcement targets do not control mutation.

Scheduler and QoS policies always evaluate the whole Pod selected by any Pod target. For example, pod/containers does not limit QoS calculation to regular containers. Resource policies cannot be combined with targets for ephemeral containers or volumes.

Omitting targets, or setting it to [], selects all compatible Pod locations: Pod-level resources, regular and init container resources, regular/init/ephemeral container images, image volumes, and Pod placement fields. Scheduler and QoS checks also apply. Profile policies check effective regular, init, and ephemeral container profiles. PDB overlap and unhealthy eviction policies check the Pod; eviction-count bounds require an explicit supported controller target. Resource-name compatibility is described under Targeting resource locations.

An explicit targets: [pod] is narrower than omitted targets: it selects Pod-level resources, placement, and security-profile defaults, together with scheduler and QoS checks. It does not select container resources or any image references. List the desired Pod parts explicitly when narrowing container, image, or profile policies.

For example, deny matching images only in init containers:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets: [pod/initcontainers]
          registries:
            - exp: "harbor/init-only/.*"

The same image reference in a regular container, ephemeral container, or image volume is unaffected by this rule. Combine targets to check multiple locations:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets: [pod/containers, pod/ephemeralcontainers]
          registries:
            - exp: "debug/.*"

This checks regular and ephemeral container images. It does not check init containers or image volumes.

Targets without policies match the whole kind. A targets-only rule with targets: [pod/containers] and action: deny rejects Pods, just like targets: [pod]. A part suffix narrows property checks only when workload policies are configured.

Target security profiles

This complete Tenant gives namespaces two different profile policies. Namespaces labeled profile-scope: pod-only require RuntimeDefault in the Pod’s security context. Namespaces labeled profile-scope: containers require RuntimeDefault for every effective regular, init, and ephemeral container profile. Namespaces with neither label are unaffected by these rules.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          profile-scope: pod-only
      enforce:
        action: allow
        workloads:
          targets: [pod]
          security:
            seccompProfiles:
              - types: [RuntimeDefault]
            appArmorProfiles:
              - types: [RuntimeDefault]
    - namespaceSelector:
        matchLabels:
          profile-scope: containers
      enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
            - pod/initcontainers
            - pod/ephemeralcontainers
          security:
            seccompProfiles:
              - types: [RuntimeDefault]
            appArmorProfiles:
              - types: [RuntimeDefault]

These rules supply no defaults. For Linux Pods, each mechanism is checked independently:

Submitted profile configuration	`profile-scope: pod-only`	`profile-scope: containers`
Pod RuntimeDefault; all containers inherit it	✅ Allowed	✅ Allowed
Pod RuntimeDefault; regular container overrides it with Unconfined	✅ Allowed	❌ Rejected
Pod RuntimeDefault; init container overrides it with Unconfined	✅ Allowed	❌ Rejected
Pod RuntimeDefault; ephemeral container overrides it with Unconfined	✅ Allowed	❌ Rejected
No Pod profile; every container explicitly uses RuntimeDefault	❌ Rejected	✅ Allowed
No profile at either level	❌ Rejected	❌ Rejected

List only pod/initcontainers to restrict the profile checks to init containers. Combine pod with the three container targets to require both an allowed Pod default and allowed effective container profiles. Omitting targets checks all three container groups without independently requiring a Pod default.

The security reference combines Pod-level defaulting with container enforcement. A container override survives mutation and must then pass the selected enforcement checks, including on pods/ephemeralcontainers updates.

Select Workloads

A targets-only allow rule establishes an allow-list across the eight supported native workload kinds. Other resource kinds remain outside this allow-list. Allow controller-created children explicitly:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          targets: [deployment, replicaset, pod]

This allows Deployments, ReplicaSets, and Pods while rejecting the other supported native workload kinds. Allowing a Deployment alone does not allow its ReplicaSets or Pods: each request is evaluated separately. Kind restrictions apply to creation and main-resource updates, not deletion or subresources.

Deny DaemonSets

Add a targets-only rule under Tenant.spec.rules to deny DaemonSets in namespaces labeled env: test:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          env: test
      enforce:
        action: deny
        workloads:
          targets: [daemonset]

This rejects DaemonSet creation and main-resource updates in the selected namespaces. It does not remove existing DaemonSets or block deletion or subresources. Other workload kinds and non-selected namespaces are unaffected by this rule. To cover all namespaces in the Tenant, omit namespaceSelector.

Use action: audit to record matching requests without blocking them. For kind restrictions, the last matching allow or deny rule wins; audit does not change that decision.

Resource Management

Configure CPU, memory, and other resource requests and limits, together with Pod QoS policies.

Requests and Limits

Resource policies let a Tenant administrator normalize and enforce the requests and limits of Pods created in Tenant namespaces. They cover common requirements that a Kubernetes LimitRange cannot express directly, including:

always removing a resource limit;
making a limit equal to its request;
defaulting a missing request or limit without replacing an explicit value;
deriving a missing limit from a request using a maximum ratio; and
rejecting or auditing explicit limits that exceed that ratio.

Resource policies are configured under spec.rules[].enforce.workloads.resources. The requests and limits maps are keyed by Kubernetes resource name. Each map entry contains a case-sensitive policy and, for policies that need one, a value.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  rules:
    - enforce:
        action: deny
        workloads:
          resources:
            limits:
              cpu:
                policy: Remove
              memory:
                policy: MatchRequest

At every compatible resource location, this example produces the following behavior:

CPU limits are removed, even when the submitted Pod defines them;
memory limits are set to the corresponding memory requests; and
because targets is omitted, the policy applies to Pod-level resources, regular containers, and init containers.

Before and after admission

Consider this submitted Pod:

apiVersion: v1
kind: Pod
metadata:
  name: api
spec:
  containers:
    - name: api
      image: registry.example.com/api:1.0.0
      resources:
        requests:
          cpu: 250m
          memory: 512Mi
        limits:
          cpu: "1"
          memory: 1Gi

Capsule admits it with the equivalent resource configuration:

resources:
  requests:
    cpu: 250m
    memory: 512Mi
  limits:
    memory: 512Mi

The CPU request is untouched, the CPU limit is removed, and the memory limit is replaced with the memory request. Because targets is omitted, Capsule applies the same policy independently to spec.resources, every regular container, and every init container in the Pod. The submitted example has no Pod-level or init container resources, so those locations remain empty.

If MatchRequest finds neither a request nor a limit, the resource is compliant and remains undefined. If it finds a limit without a corresponding request, it cannot derive a replacement and the final state violates the policy. With action: deny the Pod is rejected; with action: audit Capsule admits it and emits an audit event and admission warning.

Policy reference

After starting with the common example above, use this reference to choose the precise behavior for each request and limit.

Requests

The following policies are supported under resources.requests:

Policy	`value`	Mutation behavior	Validation behavior
`Preserve`	Not allowed	Leaves an existing or missing request unchanged.	Adds no constraint and clears earlier constraints for the same target and resource.
`Default`	Required	Sets the configured quantity only when the request is absent. An explicit request is preserved.	Adds no constraint and clears earlier constraints for the same target and resource.
`Remove`	Not allowed	Removes the request when it is present.	Requires the request to be absent in the final Pod.

Default a request without overriding tenant workloads that already specify it:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          resources:
            requests:
              cpu:
                policy: Default
                value: 100m
              memory:
                policy: Default
                value: 256Mi

If a container requests 500m CPU, Capsule preserves 500m. If the CPU request is missing, Capsule adds 100m.

Remove an extended resource request:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
          resources:
            requests:
              example.com/temporary-device:
                policy: Remove

Resource names must be valid Kubernetes qualified names. Custom and extended resource names can be used for container and init-container policies, subject to the normal Kubernetes rules for that resource.

Limits

The following policies are supported under resources.limits:

Policy	`value`	Mutation behavior	Validation behavior
`Preserve`	Not allowed	Leaves an existing or missing limit unchanged.	Adds no constraint and clears earlier constraints for the same target and resource.
`Default`	Required	Sets the configured quantity only when the limit is absent. An explicit limit is preserved.	Adds no constraint and clears earlier constraints for the same target and resource.
`Remove`	Not allowed	Removes the limit when it is present.	Requires the limit to be absent in the final Pod.
`MatchRequest`	Not allowed	If a request exists, sets the limit to exactly the request, replacing a different explicit limit. If no request exists, it does not add a limit.	Requires request and limit to be equal. When the request is absent, the limit must also be absent.
`Ratio`	Required	If a positive request exists and the limit is absent, sets the limit to `request * value`. An explicit limit is preserved for validation.	Requires a positive request and a limit no greater than `request * value`.

Default is a fill-only policy. It does not mean that every limit must equal the configured value. Use MatchRequest when Capsule should own the limit and keep it equal to the request, or Ratio when explicit tenant values are allowed within a bounded range.

Ratio limits

Ratio defines the maximum permitted limit as a multiple of the request. It is available only for limits and supports these resource names:

Resource	Calculation precision
`cpu`	Exact decimal arithmetic, rounded down to the nearest millicore.
`memory`	Exact decimal arithmetic, rounded down to a whole byte.
`ephemeral-storage`	Exact decimal arithmetic, rounded down to a whole byte.

The ratio must be greater than or equal to 1. Quote fractional ratios in YAML for clarity, for example "1.5".

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
          resources:
            limits:
              memory:
                policy: Ratio
                value: "1.5"

For a memory request of 1Gi, the maximum limit is 1536Mi:

Submitted limit	Mutation	Result with `action: deny`
Missing	Capsule adds `1536Mi`.	Admitted.
`1Gi`	Explicit value is preserved.	Admitted because it is below the maximum.
`1536Mi`	Explicit value is preserved.	Admitted because it equals the maximum.
`2Gi`	Explicit value is preserved.	Denied because it exceeds the maximum.

An explicit limit is never reduced by Ratio. This is deliberate: preserving the submitted value allows allow, deny, and audit to decide how a ratio violation should be handled. Use MatchRequest when the limit should always be rewritten instead.

Ratio requires a positive request. When the request is missing or zero, Capsule cannot calculate a default limit. The missing or non-positive request is therefore a policy violation. A deny action blocks it, an audit action reports it, and an allow action treats it as an allow-list miss and blocks it.

Capsule calculates ratios without floating-point arithmetic and rounds down so that the generated limit never exceeds the configured factor. For example, a CPU request of 101m with a ratio of 1.5 produces a limit of 151m, not 152m.

Targeting resource locations

Resource policies reuse enforce.workloads.targets to select resource locations. See Pod targets for the location table and Workload targets for controller targets. Ephemeral containers and volumes cannot be selected for resource policies.

When targets is omitted or empty, each resource policy applies to every compatible location: Pod-level spec.resources, regular containers, and init containers. An explicit targets list narrows that default.

Compatibility is evaluated per resource name. Kubernetes Pod-level resources support only cpu, memory, and huge-page resources. With omitted targets, those names apply at all three locations, while other valid names such as ephemeral-storage and extended resources apply only to regular and init containers. Capsule skips those incompatible names at spec.resources; it does not reject the rule or discard their container policies.

Only regular containers:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
          resources:
            limits:
              cpu:
                policy: Remove

Only init containers:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/initcontainers
          resources:
            requests:
              cpu:
                policy: Default
                value: 25m

Both regular and init containers, explicitly:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
            - pod/initcontainers
          resources:
            limits:
              memory:
                policy: MatchRequest

Only Pod-level resources:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod
          resources:
            requests:
              cpu:
                policy: Default
                value: 500m
              memory:
                policy: Default
                value: 1Gi
            limits:
              cpu:
                policy: Ratio
                value: "2"
              memory:
                policy: Ratio
                value: "1.5"

The Kubernetes API server must support spec.resources for Pod-level resource policies to be useful. Pod-level policies support cpu, memory, and huge-page resources. Because Ratio itself supports only CPU, memory, and ephemeral storage, a pod-level ratio can be configured for CPU or memory, but not for huge pages. Use an explicit pod target when the policy must not affect containers.

Target compatibility

When a workload block contains resources, every explicitly configured target in that block must support resource policies. A block that combines resource policies with pod/ephemeralcontainers or pod/volumes is invalid. Put registry or image-volume enforcement that needs those targets in a separate rule.

If pod is explicitly combined with a container target, every configured resource name must also be valid at Pod level. Omit targets when a policy should use the broad default and automatically skip only its incompatible Pod-level location.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  # Valid: resource and registry policies use separate target scopes.
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
          resources:
            limits:
              cpu:
                policy: Remove

    - enforce:
        action: deny
        workloads:
          targets:
            - pod/ephemeralcontainers
            - pod/volumes
          registries:
            - exp: "untrusted.example.com/.*"

Advanced behavior

The following concepts are mainly relevant when combining multiple rules, selectors, admission actions, or Kubernetes resource-management components.

Admission lifecycle

Resource policy admission has a mutation phase and a validation phase. Knowing which phase a policy uses is important when choosing a policy and an action.

Admission phase	Operations	Behavior
Mutation	Pod `CREATE`	Capsule applies the effective `Default`, `Remove`, `MatchRequest`, or `Ratio` mutation to the incoming Pod. `Preserve` does not change the field.
Validation	Pod `CREATE` and normal `UPDATE`	Capsule checks the final resource values for `Remove`, `MatchRequest`, and `Ratio`. `Preserve` and `Default` do not add a validation constraint.
Pod subresources	Any	Resource validation is skipped for Pod subresources, including `ephemeralcontainers`. Resource policies do not mutate ephemeral containers.

For Pods, resource mutation is limited to creation. Capsule does not try to rewrite resource fields on existing Pods, where Kubernetes immutability rules would normally reject the change. Normal Pod updates are still validated so that the policy describes the accepted final state.

With omitted targets or Pod targets, Capsule mutates and validates the resulting Pods without rewriting controller templates. With explicit controller targets, resource request/limit policies also mutate and validate the stored Pod template on controller creation and updates. Defaults therefore appear directly in the controller. Other workload mutations under mutate[].workloads remain Pod-only.

Important

The enclosing action does not disable mutation. A rule with action: audit still applies its resource mutation within the supported target and operation. The action controls the result of a remaining validation violation. For example, Ratio leaves an explicit limit unchanged, then deny rejects an excessive value while audit reports it without blocking the Pod.

Actions and compliance

The action belongs to the enclosing enforce block and applies to validation constraints created by Remove, MatchRequest, and Ratio:

Action	When the final value complies	When the final value violates the policy
`deny`	No deny decision is produced.	The Pod is denied.
`allow`	The policy produces an allow decision.	The value does not satisfy the resource allow-list and the Pod is denied unless a later matching rule allows it.
`audit`	No audit is emitted.	The Pod is admitted by this rule, and Capsule emits a Kubernetes event and admission warning. Other rules can still deny it.

As with other enforcement matchers, the last matching allow or deny decision wins. Audit decisions never override allow or deny decisions.

The following example denies memory limits above 1.5 times the request by default but permits up to 2 times the request in namespaces labeled burst-memory=true:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
          resources:
            limits:
              memory:
                policy: Ratio
                value: "1.5"

    - namespaceSelector:
        matchLabels:
          burst-memory: "true"
      enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
          resources:
            limits:
              memory:
                policy: Ratio
                value: "2"

A container with a 1Gi request and a 1792Mi limit violates the first rule but complies with the later allow rule. It is admitted only in a namespace that matches the selector on the second rule.

Audit explicit ratio violations without rejecting them:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: audit
        workloads:
          targets:
            - pod/containers
          resources:
            limits:
              ephemeral-storage:
                policy: Ratio
                value: "2"

Remember that a missing limit is still defaulted during create. The audit is emitted only when the final Pod remains noncompliant, such as when it contains an explicit excessive limit or a limit without a positive request.

Rule order and policy overrides

Rules are processed in declaration order after namespaceSelector and audience filtering. Resource policies are resolved independently for every combination of target, resource name, and field (request or limit). This allows CPU and memory, or requests and limits, to be managed independently.

For mutation, the last applicable policy for a target, resource name, and field is effective. A later Preserve therefore prevents an earlier mutation:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          resources:
            limits:
              cpu:
                policy: Remove

    - namespaceSelector:
        matchLabels:
          preserve-cpu-limit: "true"
      enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
            - pod/initcontainers
          resources:
            limits:
              cpu:
                policy: Preserve

In matching namespaces, the later Preserve policy leaves CPU limits intact. In other namespaces, the earlier Remove policy remains effective.

For validation, Preserve and Default also clear earlier constraints for the same target, field, and resource. Constraints declared after that reset point are still evaluated. This makes Preserve useful as a namespace-specific escape hatch and Default useful when switching from enforcement back to fill-only behavior.

For the request identity of controller-created Pods, see Audience.

Configuration validation

Capsule validates resource policy configuration before using it. Invalid rules are reported on the RuleStatus and are not silently accepted.

The following requirements apply:

At least one non-empty requests or limits map is required.
Policy names are case-sensitive.
value is required only by request Default, limit Default, and limit Ratio.
value must not be supplied to Preserve, Remove, or MatchRequest.
Default quantities must not be negative. A zero default is accepted, although Kubernetes or another admission policy can impose a stricter requirement.
A ratio must be greater than or equal to 1.
Ratio supports only cpu, memory, and ephemeral-storage.
pod/ephemeralcontainers, pod/volumes, and the deprecated pod/images target cannot be used in a workload block containing resource policies.
A request cannot use Remove while the same rule uses MatchRequest or Ratio for that resource’s limit, because those limit policies require the request.
When the pod target is explicitly present, resource names in that block are restricted to cpu, memory, and huge-page resources supported by Kubernetes pod-level resources.
When targets is omitted, Pod-incompatible resource names remain valid and are applied only to regular and init containers.

For example, this configuration is invalid:

resources:
  requests:
    memory:
      policy: Remove
  limits:
    memory:
      policy: Ratio
      value: "1.5"

The following is also invalid because value is not accepted by Remove:

resources:
  limits:
    cpu:
      policy: Remove
      value: "1"

Interaction with Kubernetes resource controls

Resource policies complement Kubernetes resource controls; they do not replace or disable them.

LimitRange can still apply its own defaults and min/max validation. Configure its constraints consistently with Capsule resource policies to avoid admission behavior that depends on webhook and admission-plugin order.
ResourceQuota, Global Resource Quota, and Resource Pools evaluate the resulting Pod resources according to their own semantics.
Other mutating webhooks can also change resources. Capsule’s generic mutating webhook requests reinvocation when another mutator changes the Pod, and the validating webhook checks the final object it receives.
Kubernetes performs its own resource validation after mutation. A resource configuration accepted by a Capsule policy can still be rejected by the API server or another admission policy.
A Vertical Pod Autoscaler or another controller that creates replacement Pods is subject to the policy when those Pods are admitted. Normal updates are validated but not mutated.

To inspect the effective resources after admission, query the Pod rather than only its workload-controller template:

kubectl get pod api -n solar-apps \
  -o jsonpath='{.spec.containers[*].resources}'

If admission is denied, Capsule’s error identifies the target path, resource field, and failed requirement. Audit violations appear as admission warnings and Kubernetes events associated with the Pod and Tenant.

QoS Classes

For example, reject BestEffort Pods in the Tenant namespaces:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  rules:
    - enforce:
        action: deny
        workloads:
          qosClasses:
            - BestEffort

QoS class enforcement allows administrators to allow, deny, or audit Pods based on their computed Kubernetes QoS class.

QoS rules are configured under enforce.workloads.qosClasses.

Supported QoS classes are:

QoS class	Description
`Guaranteed`	The Pod has CPU and memory requests and limits set so that requests equal limits.
`Burstable`	The Pod has at least one CPU or memory request or limit, but does not qualify as `Guaranteed`.
`BestEffort`	The Pod has no CPU or memory requests or limits.

Capsule evaluates the QoS class of the incoming Pod during create and update admission. If Kubernetes has already populated status.qosClass, Capsule can use that value; otherwise it computes the QoS class from the Pod specification.

Deny BestEffort Pods:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  ...
  rules:
    - enforce:
        action: deny
        workloads:
          qosClasses:
            - BestEffort

With this rule, a Pod without CPU or memory requests and limits is denied:

apiVersion: v1
kind: Pod
metadata:
  name: best-effort
spec:
  containers:
    - name: shell
      image: harbor/platform/debian:latest
      command: ["sleep", "infinity"]

Example rejection:

Error from server (Forbidden): error when creating "pod.yaml": admission webhook "pods.projectcapsule.dev" denied the request: QoS class "BestEffort" at status.qosClass is denied by namespace rule

Audit Burstable Pods:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: audit
        workloads:
          qosClasses:
            - Burstable

A matching Pod is admitted in this audit-only example, but Capsule emits an event and the API server response contains an admission warning. If a QoS allow-list is also configured and the Pod’s QoS class is not allowed, the Pod is denied while the audit event is still emitted.

Allow BestEffort only for selected namespaces:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          qosClasses:
            - BestEffort

    - namespaceSelector:
        matchLabels:
          allow-best-effort: "true"
      enforce:
        action: allow
        workloads:
          qosClasses:
            - BestEffort

Because later matching allow or deny rules take precedence, namespaces labeled allow-best-effort=true can run BestEffort Pods, while other namespaces cannot.

Placement

Configure scheduling matchers under enforce.workloads.placement, including schedulers, nodeSelector, tolerations, topologySpreadConstraints, and affinity. targets remains at enforce.workloads.targets and scopes all configured workload policies.

Configure placement policies under spec.rules[].enforce.workloads. Each property has its own matcher list:

Property	Evaluated value	Mutation
Scheduler names	`spec.schedulerName`	Set the scheduler
Node selectors	Each key/value pair in `spec.nodeSelector`	Set node selectors
Tolerations	Each complete entry in `spec.tolerations`	Set tolerations
Topology spread constraints	Each complete entry in `spec.topologySpreadConstraints`	Set spread constraints
Affinity	Each complete required or preferred affinity term	Set affinity

Read Order and scope for how mutation precedes enforcement, and Conditions to apply checks conditionally. The Reference combines all placement policies with resource and image rules. To configure mutations alongside enforcement, see the complete mutation example.

Scheduler policies evaluate one scheduler name. Node selectors, tolerations, topology spread constraints, and affinity are evaluated entry by entry. For these structured entries, fields within a matcher are combined with AND, and entries in the matcher list are alternatives. A match requires one complete matcher to match the entry; different matchers cannot authorize separate parts of the same affinity term or spread constraint.

The action rules apply to each evaluated value: allow enables an allow-list, deny rejects matches, and audit only records them. The last matching allow or deny wins. Every supplied entry must pass its policy; a single rejected entry denies the request. Mutated values and Kubernetes-injected values are evaluated in the same way.

Allow rules do not require a placement property to be present. Omitted properties and empty lists have no entries to validate. Use workload mutation to establish settings before enforcement. An omitted policy list or [] adds no matchers. For node selectors, tolerations, spread constraints, and affinity, [{}] matches every present entry. Scheduler matchers require a match expression instead.

Those four structured properties use the pod target or a whole-controller target such as deployment. Omitting targets includes Pod placement checks; selecting only container or volume targets skips them. Scheduler checks apply to the whole selected Pod spec even when a part is selected. See Workload targets for all target scopes.

For those four properties, main-resource Pod updates check changed values and recheck affinity and spread selectors when labels change. Subresources skip these checks.

Matcher fields

Scheduler matchers and the key, values, topologyKey, and namespaces fields use the common match expression structure: exact, exp, and optional negate.

Omit a matcher field to leave it unrestricted. A supplied expression must contain exact or exp; key: {} is invalid. Use exp: '^$' to match an empty string. The property references below list the additional fields supported by each matcher, such as operators, effects, and numeric bounds.

Selector policies

Topology-spread and Pod-affinity matchers use the following policy structure for labelSelector and, where supported, namespaceSelector:

Field	Behavior
`required`	When `true`, require at least one effective selector requirement.
`requirements`	Allowlist for individual selector requirements. Omission or an empty list leaves requirements unrestricted.

Each requirement matcher supports key, values, and operators. Label selectors support In, NotIn, Exists, and DoesNotExist. Every actual requirement must match one complete entry, and every supplied value must match its values expression. matchLabels entries are evaluated as singleton In requirements.

Exists and DoesNotExist have no values to test. Permitting those operators permits their valueless form; use operators: [In] when allowed values must restrict the selection. A requirement allowlist constrains supplied requirements; it does not require every listed key to appear.

Dynamic matchLabelKeys and mismatchLabelKeys are checked as In and NotIn requirements using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes behavior. required: true uses this effective selector, including any dynamic requirements.

Scheduler names

Configure scheduler matchers under enforce.workloads.placement.schedulers. Capsule checks spec.schedulerName during create and update admission.

Kubernetes defaults an omitted or empty name to default-scheduler before Pod admission. Enforcement checks that name unless a mutation changes it; the matcher itself does not fill an empty value. A conditional scheduler default can replace the built-in scheduler while preserving custom names.

Allow only these schedulers:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          placement:
            schedulers:
              - exact: [tenant-scheduler, batch-scheduler]

Pod scheduler name at enforcement	Result
`tenant-scheduler` or `batch-scheduler`	Allowed
`other-scheduler`	Denied
`default-scheduler`, including a name defaulted by Kubernetes	Denied

To allow a scheduler family as well as fixed names, combine exact and exp. Either one can satisfy this matcher:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          placement:
            schedulers:
              - exact: [default-scheduler, batch-scheduler]
                exp: '^tenant-[a-z0-9-]+$'

Deny one scheduler while leaving other names unrestricted by this rule:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          placement:
            schedulers:
              - exact: [unsafe-scheduler]

Alternatively, negate a trusted set to deny all other names:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          placement:
            schedulers:
              - exact: [default-scheduler, tenant-scheduler]
                negate: true

To record usage without changing the admission decision, use audit:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: audit
        workloads:
          placement:
            schedulers:
              - exact: [custom-scheduler]

This emits an event and an admission warning for custom-scheduler. Other policies still apply; an audit match cannot satisfy a scheduler allow-list.

Node selectors

Each matcher evaluates one key/value pair in spec.nodeSelector.

Field	Matches
`key`	The node-label key.
`values`	The node-label value, including an explicitly empty value.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          placement:
            nodeSelector:
              - key:
                  exact: [kubernetes.io/os]
                values:
                  exact: [linux]
              - key:
                  exp: '^placement\.example\.com/[a-z0-9-]+$'
                values:
                  exact: [shared, batch]

This allows kubernetes.io/os: linux and keys such as placement.example.com/pool: shared. It rejects kubernetes.io/os: windows, placement.example.com/pool: dedicated, and unlisted keys. A Pod with no node selector is allowed by this rule.

Tolerations

Each matcher evaluates one complete toleration in spec.tolerations.

Field	Matches
`key`	Taint key expression. An empty Pod key is a literal empty string.
`values`	Toleration value expression.
`operators`	`Equal` or `Exists`. An omitted Pod operator is `Equal`.
`effects`	`NoSchedule`, `PreferNoSchedule`, `NoExecute`, or the literal empty string `""`.
`tolerationSeconds`	Inclusive `min` and `max` bounds, with optional `allowUnlimited`.

An empty toleration key or effect has broad Kubernetes semantics. It does not match an allowlist of specific keys or effects. To allow an empty effect, include "" explicitly in effects.

An absent tolerationSeconds means unlimited. Bounds apply to finite durations; allowUnlimited defaults to true. Set it to false to require a finite duration:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          placement:
            tolerations:
              - key:
                  exact: [node.kubernetes.io/not-ready, node.kubernetes.io/unreachable]
                operators: [Exists]
                effects: [NoExecute]
                tolerationSeconds:
                  max: 600
                  allowUnlimited: false

This permits only the two listed NoExecute tolerations, each with a finite duration of at most 600 seconds. Add other matchers for any additional tolerations needed by your workloads or injected by Kubernetes.

Disallow every toleration

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          placement:
            tolerations:
              - {}

The empty matcher rejects every toleration, including injected ones. Kubernetes normally adds not-ready and unreachable tolerations, so this rule rejects ordinary Pods with those entries too. Use an allowlist when system tolerations must remain permitted. A Pod with no tolerations has no entry for this deny rule to match.

Topology spread constraints

Each matcher evaluates one complete entry in spec.topologySpreadConstraints.

Field	Matches
`topologyKey`	Topology-key expression.
`whenUnsatisfiable`	List containing `DoNotSchedule` or `ScheduleAnyway`.
`maxSkew`	Inclusive `min` and `max` bounds.
`minDomains`	Inclusive bounds; an omitted Pod value is evaluated as `1`.
`nodeAffinityPolicy`	`Honor` or `Ignore`; an omitted Pod value is `Honor`.
`nodeTaintsPolicy`	`Honor` or `Ignore`; an omitted Pod value is `Ignore`.
`labelSelector`	Selector policy for the Pods counted by the constraint.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          placement:
            topologySpreadConstraints:
              - topologyKey:
                  exact: [topology.kubernetes.io/zone, kubernetes.io/hostname]
                whenUnsatisfiable: [DoNotSchedule, ScheduleAnyway]
                maxSkew:
                  min: 1
                  max: 3
                labelSelector:
                  required: true
                  requirements:
                    - key:
                        exact: [app]
                      operators: [In]
                      values:
                        exact: [checkout]

This allows zone or host spreading with a skew of 1–3 and a selector for app: checkout. A constraint for another topology key, a skew of 4, or a selector using another key is rejected. The rule constrains supplied entries; it does not add a spread constraint or require one to be present.

Affinity

All three affinity types use one flat list under enforce.workloads.placement.affinity. Each matcher evaluates a complete required or preferred term from spec.affinity.

Field	Applies to	Behavior
`types`	All	Select `nodeAffinity`, `podAffinity`, or `podAntiAffinity`. Omission selects all compatible types.
`modes`	All	Select `required`, `preferred`, or both. Omission selects both.
`weight`	Preferred terms	Inclusive `min` and `max` bounds within 1–100. Requires `modes: [preferred]`.
`requirements`	Node affinity	Match `matchExpressions` using key, values, and operators.
`fieldRequirements`	Node affinity	Match `matchFields` using the same requirement structure.
`topologyKey`	Pod affinity and anti-affinity	Topology-key expression.
`labelSelector`	Pod affinity and anti-affinity	Selector policy for matching Pods.
`namespaceScope`	Pod affinity and anti-affinity	`SameNamespace` or `Any`. Omission imposes no scope restriction.
`namespaces`	Pod affinity and anti-affinity	Expression matched against every explicitly supplied namespace.
`namespaceSelector`	Pod affinity and anti-affinity	Selector policy for matching namespaces.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          placement:
            affinity:
              - types: [nodeAffinity]
                modes: [required, preferred]
                requirements:
                  - key:
                      exact: [topology.kubernetes.io/zone]
                    operators: [In]
                    values:
                      exact: [zone-a, zone-b]
              - types: [podAffinity, podAntiAffinity]
                modes: [preferred]
                topologyKey:
                  exact: [kubernetes.io/hostname]
                namespaceScope: SameNamespace
                weight:
                  min: 1
                  max: 100

This allows node affinity using the listed zone requirements and preferred Pod affinity or anti-affinity within the Pod’s namespace, grouped by host. Required Pod affinity and anti-affinity are rejected. The second matcher leaves Pod-label selectors unrestricted; add labelSelector to constrain them.

Node requirements also support Gt and Lt. The value matcher checks the literal numeric operand; it does not query node labels. When either node requirement list constrains a term, requirements from the other source must be explicitly allowed if present. For example, fieldRequirements: [{}] permits any native-valid field requirement alongside constrained requirements.

SameNamespace permits an omitted namespaces list or explicit references to the Pod’s own namespace, and requires the Pod term’s namespaceSelector to be absent. Even namespaceSelector: {} selects more than the current namespace and does not match this scope. Any leaves namespace scope unrestricted. A namespaces expression alone does not restrict namespaces selected through a namespaceSelector.

Type-specific fields only match types on which they are meaningful. Explicitly incompatible combinations are rejected when the policy is saved. Matching examines the submitted terms without listing nodes, Pods, or namespaces.

Pod Disruption Budgets

Use enforce.workloads.disruptionBudgets to constrain PodDisruptionBudgets (PDBs) covering selected workloads in a namespace. These policies can prevent overlapping budgets, require room for voluntary disruption at the desired replica count, and allow eviction of unhealthy Pods during maintenance.

The policies check existing PDBs; they do not create a PDB or require one to exist.

For example, require matching PDBs to leave room for at least one voluntary eviction per Deployment or StatefulSet with a nonzero desired replica count. This rule applies to every namespace in the Tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          targets: [deployment, statefulset]
          disruptionBudgets:
            evictableReplicas:
              min: 1

For three desired replicas, a PDB with minAvailable: 2 passes, while minAvailable: 3 is rejected. This checks the configured allowance; actual eviction still depends on workload health.

The Reference includes all three properties in the combined Tenant. For maintenance scenarios, see workload best practices.

The shared workloads.targets list selects the Pods and controller templates whose PDBs are checked. Capsule also checks PDB writes that affect those workloads in the same namespace. A controller need not have running Pods for its template to be checked.

Target	`allowOverlap`	`unhealthyPodEvictionPolicies`	`evictableReplicas`
Omitted or `pod`	✅ Pod labels	✅ PDBs selecting the Pod	❌
`deployment`, `statefulset`, `replicaset`, `replicationcontroller`	✅ Template labels	✅ PDBs selecting the template	✅ Desired `spec.replicas`, including `/scale`
`daemonset`, `job`, `cronjob`	✅ Template labels	✅ PDBs selecting the template	❌ No supported desired replica count
Container or volume part targets	❌	❌	❌

Controller targets are explicit opt-ins. Include pod as well to check labels on actual Pods, including those created by controllers. CronJobs use the Pod template inside their Job template. Part targets such as pod/containers and deployment/containers do not participate in PDB checks.

An evictableReplicas policy must explicitly select at least one of the four supported controller targets. Other targets in the same rule still participate in overlap and unhealthy-policy checks.

Evictable replicas

evictableReplicas.min and .max define inclusive bounds on the configured number of replicas that a PDB would allow to be evicted from a selected controller at its desired size, assuming those replicas are healthy. Both bounds are nonnegative integers. An omitted bound is unrestricted; for example, min: 1 requires room for at least one disruption without setting an upper limit.

Capsule calculates this allowance from the PDB and the controller’s desired spec.replicas:

PDB size field	Configured allowance
Integer `minAvailable`	Desired replicas minus `minAvailable`, with a minimum of zero
Percentage `minAvailable`	Desired replicas minus the rounded-up percentage of desired replicas, with a minimum of zero
Integer `maxUnavailable`	`maxUnavailable`, capped at the desired replica count
Percentage `maxUnavailable`	Rounded-up percentage of desired replicas, capped at the desired replica count
Both size fields omitted	Zero

Both percentage forms use Kubernetes’ upward rounding. For an allow rule with evictableReplicas.min: 1:

PDB configuration	Desired replicas	Allowance	Admission
`minAvailable: "75%"`	3	0	❌ Rejected
`minAvailable: "75%"`	4	1	✅ Allowed
`minAvailable: 2`	3	1	✅ Allowed
`maxUnavailable: "25%"`	1	1	✅ Allowed; the only replica can be evicted
`maxUnavailable: 0`	4	0	❌ Rejected

Replica changes are checked through both the controller resource and its /scale subresource, including requests from kubectl scale and autoscalers. With the first two rows above, scaling from four replicas down to three is rejected.

Zero replicas are exempt from the replica-count check. Scaling a controller to zero remains possible. Scaling back above zero checks the resulting allowance. Overlap and unhealthy-policy checks still apply to zero-replica templates.

Each matching PDB is checked against each selected controller independently. Capsule does not add replicas from multiple controllers or calculate a combined live PDB allowance. This can be stricter than Kubernetes for a PDB that covers multiple workloads. The max bound applies to this per-controller calculation; it does not cap aggregate live disruptions across multiple controllers. Use a PDB selector that covers the complete intended controller rather than an arbitrary subset of its Pods.

Configured allowance and live eviction

These checks use desired replicas, not status.disruptionsAllowed. Pending, unhealthy, missing or terminating Pods and rollout state can still prevent an eviction. A compliant configuration does not guarantee that a drain will finish. PDBs also do not limit direct Pod deletion or controller scale-down.

Unhealthy Pods

unhealthyPodEvictionPolicies matches the effective value of the PDB’s spec.unhealthyPodEvictionPolicy:

Value	Behavior for unhealthy Running Pods
`IfHealthyBudget`	Eviction requires the application’s healthy Pod count to satisfy its budget. This is also the effective value when the PDB omits the field.
`AlwaysAllow`	Unhealthy Running Pods may be evicted even when the healthy budget is not satisfied. Healthy Pods still obey the PDB.

An allow rule listing only AlwaysAllow rejects a matching PDB that omits this field or sets IfHealthyBudget. Alternatively, a deny rule listing IfHealthyBudget rejects that value, including omission. An omitted or empty policy list adds no restriction. These rules validate the field; they do not populate it. See Kubernetes’ unhealthy Pod eviction policies for their effect during a drain.

Overlap

allowOverlap: false requires at most one PDB to select each Pod or selected controller template. Kubernetes accepts overlapping PDBs, but an eviction can fail when more than one PDB selects the Pod. See API-initiated eviction.

Matching uses Pod or Pod-template labels and each PDB’s matchLabels and matchExpressions in the same namespace. With policy/v1, a null selector matches no Pods, while an empty selector {} matches every Pod in the namespace. An omitted allowOverlap field adds no constraint. A later allow rule with allowOverlap: true can permit overlap, but it does not change Kubernetes' eviction behavior.

For example, two PDBs selecting app: frontend can be created before any matching Pod or selected controller template exists. Creating a matching workload is then rejected. If the workload already exists, creating the second PDB is rejected instead. Deleting a conflicting PDB or changing selectors to remove the overlap allows subsequent writes.

Evaluation

Checks run on both sides of the relationship:

Request	PDB checks
Selected Pod creation or changed labels	Matching PDB overlap and unhealthy eviction policies; label changes through Pod `/status` are also checked
Selected controller creation or changed template labels	Matching PDB policies for that controller target
Supported controller replica change with an applicable `evictableReplicas` policy, including `/scale`	Resulting configured eviction allowance and the other applicable PDB policies
PDB creation or relevant spec change	Policies for existing selected Pods and controller templates that its selector matches
Unchanged relevant fields	No new PDB check
Deletion	Allowed by these policies, so a conflicting PDB can be removed

A PDB with no matching selected workload can be created. The workload is checked when it is later admitted, allowing either creation order without making PDB existence a requirement.

Actions differ between numeric/boolean constraints and the list of policy values:

Property	`allow`	`deny`	`audit`
`allowOverlap` and `evictableReplicas`	Requires compliance	Rejects violations	Reports violations
`unhealthyPodEvictionPolicies`	Accepts listed values	Rejects listed values	Reports listed values

An omitted action defaults to deny. Normal rule ordering applies independently to each property: the last matching allow/deny decision wins. Allowing one property does not bypass a failure of another. Audit rules emit events without changing the admission decision. Omitted properties never override an earlier rule.

Namespace selection, audiences, and enforcement conditions use the usual rules scope. Conditions inspect the actual incoming object: a PDB during a PDB write, a controller during a controller write, and an autoscaling/v1 Scale during a /scale update. Include all relevant actors and write paths when using these gates; a condition restricted to Pod creation does not protect controller, PDB, or scale updates.

Changing a rule or namespace profile affects subsequent relevant admissions; it does not repair existing PDBs or immediately revalidate stored workloads. Concurrent PDB and workload writes can race because admission across separate resources is not atomic. Monitor actual PDB status and workload health when planning maintenance.

Security

Configure seccompProfiles and appArmorProfiles under enforce.workloads.security. The workload-level targets field selects the Pod or container locations to check.

seccompProfiles and appArmorProfiles allow, deny, or audit profile choices for Linux workloads. The Seccomp and AppArmor sections below describe each mechanism and its local-profile matching. Use security mutations to supply Pod defaults before enforcement. These checks complement namespace Pod Security labels; they skip Pods and templates declaring spec.os.name: windows.

Both matchers use these fields and the usual rule order:

Field	Meaning
`types`	Required list of 1–3 entries containing `RuntimeDefault`, `Localhost`, or `Unconfined`.
`localhostProfiles`	Optional list of up to 64 `exact`, `exp`, and `negate` expressions for local profile paths or names. Requires `Localhost` in `types`.

Each enforcement rule supports up to 64 seccompProfiles matchers and up to 64 appArmorProfiles matchers. Oversized lists are rejected when the policy is saved, limiting nested matching work during workload admission.

Multiple matchers are alternatives. The expressions in localhostProfiles constrain only Localhost; other types in the same matcher still match normally. Omitting these expressions permits any non-empty Localhost path or name. Audit reports matching admission choices through Capsule events; it does not configure kernel syscall logging or AppArmor learning modes.

The shared workload targets select where checks apply. Omitted targets check effective profiles for regular, init, and ephemeral containers. Explicit pod checks only the Pod default, so it does not prevent container overrides. Controller targets inspect effective profiles in their templates, including CronJobs’ nested templates. Targeted templates must declare allowed profiles themselves because profile mutation applies only to Pod creation.

Validation runs on Pod and controller create/update requests, and on pods/ephemeralcontainers updates. It does not run on status, resize, or delete requests and does not reconfigure existing containers. Policy changes apply to subsequent admissions, so a previously admitted Pod may fail a later update under a stricter policy.

Seccomp

enforce.workloads.security.seccompProfiles evaluates a container’s explicit securityContext.seccompProfile, falling back to the Pod’s spec.securityContext.seccompProfile. Regular containers, init containers (including restartable sidecars), and ephemeral containers use this resolution. A Pod default is optional when every selected container declares an allowed profile of its own.

Effective type	What the matcher checks
`RuntimeDefault`	The workload selects the container runtime’s default seccomp profile.
`Localhost`	The workload selects a local seccomp file. `localhostProfiles` matches its relative path.
`Unconfined`	The workload explicitly disables seccomp, or the selected container is privileged.

Privileged containers are evaluated as Unconfined even if their manifest names a different profile. When neither the container nor Pod supplies a profile, Capsule evaluates it as Unset. An allow-list rejects this value; Capsule does not infer the kubelet’s seccomp default. Unset appears only in diagnostics and cannot be listed in types. A deny rule for Unconfined alone therefore permits an unset profile. Pair an allow-list with seccomp mutation when a declared profile is required.

Match local seccomp paths

For Localhost, localhostProfiles matches the value of localhostProfile, such as profiles/solar.json, relative to the kubelet’s seccomp directory. It does not match an absolute node path or inspect the file’s contents.

This complete Tenant allows RuntimeDefault, the exact local path profiles/solar.json, and files matching ^profiles/shared/[a-z0-9-]+\.json$ in the selected namespaces. A local path outside those matches, Unconfined, or an unset effective profile is denied. It does not add defaults; use seccomp mutation for that.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          security-profile: confined
      enforce:
        action: allow
        workloads:
          security:
            seccompProfiles:
              - types: [RuntimeDefault, Localhost]
                localhostProfiles:
                  - exact: [profiles/solar.json]
                  - exp: '^profiles/shared/[a-z0-9-]+\.json$'

Admission checks the reference, while nodes apply the actual profile. Install approved files on eligible nodes before use. See the Kubernetes seccomp tutorial.

AppArmor

enforce.workloads.security.appArmorProfiles checks effective AppArmor profiles in this order: the container’s structured securityContext.appArmorProfile, a legacy per-container AppArmor annotation, then the Pod’s spec.securityContext.appArmorProfile. This also covers init, restartable sidecar, and ephemeral containers. A container override therefore remains subject to enforcement even when the Pod default is allowed.

Effective type	What the matcher checks
`RuntimeDefault`	The workload selects the container runtime’s default AppArmor profile.
`Localhost`	The workload selects a loaded AppArmor profile. `localhostProfiles` matches its name.
`Unconfined`	The workload explicitly disables AppArmor, or the selected container is privileged.

A missing effective profile is reported as Unset and fails an allow-list. Capsule does not infer a runtime’s implicit AppArmor behavior. A deny rule matching only Unconfined permits an unset profile. Use AppArmor mutation with an allow-list to require a declared profile. Privileged containers evaluate as Unconfined regardless of the profile written in the manifest.

Match loaded AppArmor names

localhostProfiles matches a loaded profile name such as solar-confined. The value is not the filesystem path where the AppArmor definition is stored. For example, exact: [solar-confined] checks the name in localhostProfile; exp: '^solar-[a-z0-9-]+$' would permit any matching name.

This complete Tenant allows RuntimeDefault and only the named local profile in selected namespaces. Other Localhost names, Unconfined, and unset effective profiles are rejected. It supplies no defaults by itself.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          security-profile: apparmor
      enforce:
        action: allow
        workloads:
          security:
            appArmorProfiles:
              - types: [RuntimeDefault, Localhost]
                localhostProfiles:
                  - exact: [solar-confined]

Capsule does not verify that AppArmor is supported, that the named profile is loaded, or that its contents are identical on all nodes. Configure eligible nodes before using these profiles. See the Kubernetes AppArmor guide.

Security reference

This complete Tenant defaults both profiles in selected namespaces. It allows RuntimeDefault and one approved Localhost profile for each mechanism. The same checks apply to explicitly targeted Deployment templates, which must supply their own profile configuration. Apply this example only with eligible Linux nodes supporting AppArmor and with the custom profiles installed before use.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          security-profile: confined
      mutate:
        - action: merge
          workloads:
            security:
              seccompProfile:
                type: RuntimeDefault
              appArmorProfile:
                type: RuntimeDefault
      enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
            - pod/initcontainers
            - pod/ephemeralcontainers
            - deployment
          security:
            seccompProfiles:
              - types: [RuntimeDefault, Localhost]
                localhostProfiles:
                  - exact: [profiles/solar.json]
            appArmorProfiles:
              - types: [RuntimeDefault, Localhost]
                localhostProfiles:
                  - exact: [solar-confined]

Container choice	Result
No override; Pod default is RuntimeDefault	Allowed through inheritance.
Approved Localhost profile	Allowed and preserved by merge.
Unapproved Localhost profile	Denied.
Explicit Unconfined or privileged container	Denied.
No profile at either level in a targeted template	Denied.
Ephemeral container overriding the Pod default with Unconfined	Denied.

Registries

Registry enforcement allows administrators to allow, deny, or audit Pod image references. Registry matchers are evaluated against the full OCI reference string, including registry, repository path, image name, tag, or digest.

Registry rules are configured under enforce.workloads.registries. The shared Workload targets field under enforce.workloads.targets selects image references in Pods or controller templates. See Pod targets for container and image-volume scopes and examples.

Registry matchers use the common match expression structure:

registries:
  - exact:
      - harbor/platform/debian:latest
      - harbor/platform/busybox:latest
  - exp: "harbor/platform/.*"

Use exact for a fixed list of complete references and exp for path or registry patterns. A single matcher may contain both fields:

registries:
  - exact:
      - harbor/platform/debian:latest
    exp: "harbor/shared/.*"

This matcher succeeds for harbor/platform/debian:latest or any reference matching harbor/shared/.*.

The following example allows Harbor images by default, denies a more specific customer path for regular containers and image volumes, allows and audits regular container images from an audit registry, and allows a production image path only for namespaces matching env=prod:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  ...
  rules:
    - enforce:
        action: allow
        workloads:
          registries:
            - exp: "harbor/.*"

    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
            - pod/volumes
          registries:
            - exp: "harbor/customer/.*"

    - enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
          registries:
            - exp: "audit/.*"

    - enforce:
        action: audit
        workloads:
          targets:
            - pod/containers
          registries:
            - exp: "audit/.*"

    - namespaceSelector:
        matchExpressions:
          - key: env
            operator: In
            values: ["prod"]
      enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
            - pod/volumes
          registries:
            - exp: "harbor/customer/prod-image/.*"
              policy: ["Always"]

Apply the following Pod in namespace solar-test, which does not match the env=prod selector:

apiVersion: v1
kind: Pod
metadata:
  name: image-volume
spec:
  containers:
    - name: shell
      command: ["sleep", "infinity"]
      imagePullPolicy: IfNotPresent
      image: harbor/customer/test-image/debian:latest
      volumeMounts:
        - name: volume
          mountPath: /volume
  volumes:
    - name: volume
      image:
        reference: quay.io/crio/artifact:v2
        pullPolicy: IfNotPresent

The request is denied:

kubectl apply -f pod.yaml -n solar-test

Error from server (Forbidden): error when creating "pod.yaml": admission webhook "pods.projectcapsule.dev" denied the request: containers[0] reference "harbor/customer/test-image/debian:latest" is denied by registry rule "harbor/customer/.*"

The Pod is denied because the regular container image matches both harbor/.* and harbor/customer/.*. Since the deny rule is declared later, it has higher precedence.

The image volume reference is not denied by the shown deny rule because it does not match harbor/customer/.*. If the image volume used a matching reference, for example harbor/customer/volume-artifact:v1, the same deny rule would apply because it targets both pod/containers and pod/volumes.

In a namespace matching env=prod, the more specific production allow rule is also considered:

apiVersion: v1
kind: Pod
metadata:
  name: prod-image
spec:
  containers:
    - name: shell
      command: ["sleep", "infinity"]
      imagePullPolicy: Always
      image: harbor/customer/prod-image/debian:latest

The request is allowed because the namespace-specific rule matches later and allows harbor/customer/prod-image/.* with imagePullPolicy: Always.

Target-specific registry rules allow different behavior for different parts of the same Pod. For example, this rule denies the registry only for init containers:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/initcontainers
          registries:
            - exp: "harbor/init-only/.*"

A matching reference under spec.initContainers is denied. The same reference under spec.containers is ignored by this rule.

PullPolicy

Define the allowed image pull policies for a matching registry rule. Supported policies are:

Always: The image is always pulled.
IfNotPresent: The image is pulled only if it is not already present on the node.
Never: The image is never pulled. If the image is not present on the node, the Pod fails to start.

The policy field is optional. If no policy is specified, all image pull policies are accepted for the matching registry rule.

To set the policy before validation, use mutate[].workloads.registries.imagePullPolicy. The configured mutation must produce a policy accepted by the matching enforcement rule.

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  ...
  rules:
    - enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
          registries:
            - exp: "harbor/v2/customer-registry/.*"
              policy: ["IfNotPresent", "Always"]

If the final matching registry decision is allow and that matching registry rule defines policy, the Pod must use one of the configured pull policies. For example, this rule allows the registry but only with Always:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
          registries:
            - exp: "harbor/v2/customer-registry/.*"
              policy: ["Always"]

A Pod using imagePullPolicy: Never for that registry is rejected:

Error from server (Forbidden): error when creating "pod.yaml": admission webhook "pods.projectcapsule.dev" denied the request: containers[0] reference "harbor/v2/customer-registry/debian:latest" uses pullPolicy=Never which is not allowed (allowed: Always)

Policy is checked only after the final registry decision is allow. A final deny decision always denies the request, regardless of the configured pull policy.

Negation

A registry matcher can be negated with negate: true. Negation applies to the final result of the matcher, including both exact and exp.

For example, the following rule denies every regular container image that is not from the trusted registry path:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  ...
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
          registries:
            - exp: "trusted/.*"
              negate: true

With this rule:

trusted/backend/api:1.0.0 is allowed in this deny-only example because it does not match the negated deny rule and no registry allow-list is configured.
docker.io/library/nginx:latest is denied because it does not match trusted/.*, so the negated matcher evaluates to true.

Negation also applies to exact values:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
          registries:
            - exact:
                - trusted/backend/api:1.0.0
                - trusted/frontend/web:1.0.0
              negate: true

This rule denies every explicit container image except the two exact references listed, as long as no separate registry allow-list requires an explicit allow. If an allow rule is configured for the same matcher scope, the excepted references must also match an allow rule.

You can combine exact values, regular expressions, negation, namespace selectors, and action precedence. For example, deny all untrusted container images by default, but allow a controlled exception in production namespaces:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        workloads:
          targets:
            - pod/containers
          registries:
            - exact:
                - trusted/base/debian:latest
              exp: "trusted/platform/.*"
              negate: true

    - enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
          registries:
            - exact:
                - trusted/base/debian:latest
              exp: "trusted/platform/.*"

    - namespaceSelector:
        matchLabels:
          env: prod
      enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
          registries:
            - exp: "partner-registry/prod-approved/.*"

The second rule explicitly allows the trusted references that were excluded from the negated deny rule, which is required when registry allow-list behavior is active. In a namespace labeled env=prod, partner-registry/prod-approved/app:1.0.0 is allowed because the later matching allow rule overrides the earlier negated deny rule.

OCI Examples

Registry exact match examples

Use exact when you want to allow or deny a fixed set of complete image references:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          targets:
            - pod/containers
          registries:
            - exact:
                - harbor/platform/debian:latest
                - harbor/platform/busybox:1.36

A Pod using harbor/platform/debian:latest or harbor/platform/busybox:1.36 is admitted. A Pod using harbor/platform/nginx:latest is denied because an allow rule exists for registry enforcement but does not match that reference.

You can combine exact and exp in the same registry matcher:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        workloads:
          registries:
            - exact:
                - harbor/platform/debian:latest
              exp: "harbor/shared/.*"

This rule allows the exact Debian image and any image under harbor/shared/*.

Reference

This complete Tenant combines workload kind restrictions, resource policies, image rules, QoS, all five placement policies, and PDB constraints. It applies only to its namespaces labeled example.com/profile: restricted; other namespaces in the Tenant keep their own profiles. Replace solar-owner with your owner identity.

The kind rule rejects DaemonSets. Resource, image, QoS, and placement policies use Pod targets and run when Pods are admitted, including controller-created Pods. The PDB rule also selects Deployment and StatefulSet templates, covering PDB writes and replica changes. See Workload targets to expand the scope of individual policies.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    # Keep kind restrictions separate from property policies.
    - namespaceSelector:
        matchLabels:
          example.com/profile: restricted
      enforce:
        action: deny
        workloads:
          targets: [daemonset]
    # Manage resources on regular and init containers.
    - namespaceSelector:
        matchLabels:
          example.com/profile: restricted
      enforce:
        action: deny
        workloads:
          targets: [pod/containers, pod/initcontainers]
          resources:
            requests:
              cpu:
                policy: Default
                value: 100m
              memory:
                policy: Default
                value: 128Mi
            limits:
              cpu:
                policy: Remove
              memory:
                policy: Ratio
                value: "2"
    # Include the whole Pod for placement and its parts for image checks.
    - namespaceSelector:
        matchLabels:
          example.com/profile: restricted
      enforce:
        action: allow
        workloads:
          targets:
            - pod
            - pod/containers
            - pod/initcontainers
            - pod/ephemeralcontainers
            - pod/volumes
          registries:
            - exp: '^registry\.example\.com/solar/.+$'
              policy: [Always, IfNotPresent]
          qosClasses: [Burstable, Guaranteed]
          placement:
            schedulers:
              - exact: [default-scheduler, batch-scheduler]
            nodeSelector:
              - key: {exact: [kubernetes.io/os]}
                values: {exact: [linux]}
              - key: {exact: [infrastructure.example.com/pool]}
                values: {exact: [shared, batch]}
            tolerations:
              - key: {exact: [infrastructure.example.com/dedicated]}
                operators: [Equal]
                values: {exact: [shared]}
                effects: [NoSchedule]
              - key:
                  exact:
                    - node.kubernetes.io/not-ready
                    - node.kubernetes.io/unreachable
                operators: [Exists]
                effects: [NoExecute]
                tolerationSeconds:
                  max: 600
                  allowUnlimited: false
              - key: {exact: [node.kubernetes.io/memory-pressure]}
                operators: [Exists]
                effects: [NoSchedule]
            topologySpreadConstraints:
              - topologyKey:
                  exact: [topology.kubernetes.io/zone, kubernetes.io/hostname]
                whenUnsatisfiable: [DoNotSchedule, ScheduleAnyway]
                maxSkew: {min: 1, max: 3}
                labelSelector:
                  required: true
                  requirements:
                    - key: {exact: [app.kubernetes.io/part-of]}
                      operators: [In]
                      values: {exact: [solar]}
            affinity:
              - types: [nodeAffinity]
                modes: [required, preferred]
                requirements:
                  - key: {exact: [topology.kubernetes.io/zone]}
                    operators: [In]
                    values: {exact: [zone-a, zone-b]}
              - types: [podAffinity, podAntiAffinity]
                modes: [preferred]
                topologyKey:
                  exact: [topology.kubernetes.io/zone, kubernetes.io/hostname]
                namespaceScope: SameNamespace
                weight: {min: 1, max: 100}
                labelSelector:
                  required: true
                  requirements:
                    - key: {exact: [app.kubernetes.io/part-of]}
                      operators: [In]
                      values: {exact: [solar]}
    # Keep selected workloads compatible with voluntary maintenance.
    - namespaceSelector:
        matchLabels:
          example.com/profile: restricted
      enforce:
        action: allow
        workloads:
          targets: [pod, deployment, statefulset]
          disruptionBudgets:
            allowOverlap: false
            evictableReplicas:
              min: 1
              max: 2
            unhealthyPodEvictionPolicies:
              - AlwaysAllow
    # Override the broader registry allow rule for this path.
    - namespaceSelector:
        matchLabels:
          example.com/profile: restricted
      enforce:
        action: deny
        workloads:
          registries:
            - exp: '^registry\.example\.com/solar/blocked/.+$'
    - namespaceSelector:
        matchLabels:
          example.com/profile: restricted
      enforce:
        action: audit
        workloads:
          registries:
            - exp: '^registry\.example\.com/solar/legacy/.+$'

With this configuration:

DaemonSet creation and main-resource updates are denied in selected namespaces.
On Pod creation, missing regular and init container requests default to 100m CPU and 128Mi memory. CPU limits are removed. Missing memory limits become twice the corresponding request; explicit limits above that ratio are denied. Resource admission lifecycle describes update behavior.
Every selected image reference must use registry.example.com/solar/ with an allowed pull policy. The later deny rule blocks the solar/blocked/ path. Images under solar/legacy/ are audited while the other checks still apply.
The resulting Pod must have Burstable or Guaranteed QoS and use default-scheduler or batch-scheduler. A custom scheduler must already be installed to schedule Pods assigned to it.
Supplied node selectors, tolerations, spread constraints, and affinity terms must match the configured shapes. These allow-lists do not require those properties to be present or populate them. Use workload mutation to establish placement settings before enforcement.
The toleration list includes common Kubernetes-injected entries. Add any additional tolerations required by your selected workloads or cluster.
Matching PDBs must not overlap and must use AlwaysAllow for unhealthy Pod eviction. Each selected Deployment or StatefulSet must have one to two configured evictions available at its desired replica count when a PDB covers it; zero replicas are exempt. Remove max: 2 if only a minimum is needed. These checks neither require a PDB nor guarantee live drain progress.

6.2.2 - Metadata

Metadata Enforcement

Metadata enforcement is configured under spec.rules[].enforce.metadata. Use Metadata Targets to select resource kinds, then configure labels and annotations, including defaults, managed values, required keys, and value validation. Migration covers the legacy Namespace, Pod, and Service metadata options.

Rules use the shared namespace selection, audiences, conditions, and actions and order. See Reference for a complete Tenant example.

Rule Composition

Actions and order

Metadata rules are evaluated in declaration order after namespace selection, audience, and condition filtering. The last matching allow or deny wins for each metadata key and value.

Action	Behavior
`allow`	Creates an allow-list for the evaluated metadata key. A present value with no matching allow or deny decision is rejected when an allow-list is active.
`deny`	Rejects matching metadata values. A later matching allow rule can override that decision.
`audit`	Emits Kubernetes events and admission warnings without allowing or denying the request. Audit matches never satisfy an allow-list.

If action is omitted, it defaults to deny. Each label and annotation key is evaluated independently; allowing one key does not authorize another.

Admission scope and limits

Metadata rules are evaluated during create and update admission. On creation, all present controlled keys are checked. On update, unchanged values skip value matching, while required keys must still be present. Keep the following scope and limits in mind:

Behavior	Explanation
Namespaced resources and explicitly selected Namespaces are evaluated	Metadata rules normally target resources inside Tenant namespaces. `Namespace` is the only supported cluster-scoped kind, and it must be selected explicitly with `kinds: ["Namespace"]`.
Managed objects remain subject to enforcement	Setting `managed-by=controller` does not bypass metadata validation. Capsule-managed resources retain their separate protection against unauthorized edits.
Capsule-managed metadata is ignored	Built-in Capsule labels and annotations are treated as managed metadata and are ignored by metadata validation. Do not rely on metadata rules to validate Capsule-owned keys.
Managed annotation prefixes are ignored	Capsule-managed annotation prefixes such as resource quota and resource usage annotations are ignored.
Missing optional metadata is ignored	If `required: false`, the key is only evaluated when it is present.
`required` applies to allow rules	`required: true` enforces presence for `action: allow`. `deny` and `audit` rules match values; they do not require missing keys to exist.
Empty metadata values are valid values	A label or annotation with an empty string value is still present and can be matched with `exact: [""]`.
Labels and annotations are independent	A matching annotation does not satisfy a required label with the same key, and a matching label does not satisfy a required annotation.
Empty `apiGroups` means core `v1`	Omitted `apiGroups`, an empty list, or an empty entry selects the core Kubernetes `v1` API. Use `apiGroups: ["*"]` to match every API group and version.
`kinds` must be set	Use `kinds: ["*"]` to match all namespaced kinds. A wildcard does not implicitly include `Namespace`.

Capsule-managed labels include labels used to track Tenant ownership, resource pools, freeze and cordon state, promotion state, Capsule ownership, and generated namespace resources. Capsule-managed annotations include release and reconciliation annotations, available class and registry annotations, forbidden namespace metadata annotations, protected Tenant annotations, and resource quota or resource usage annotation prefixes.

Because these keys are owned by Capsule, metadata rules that reference them are ignored by default. Use application-specific labels and annotations for Tenant policy enforcement.

Metadata Targets

Each metadata rule defines which resource kinds it applies to:

Field	Description
`apiGroups`	List of API group or group/version selectors. Empty or omitted means core `v1`; `apps` matches every version in that group; `apps/v1` matches that exact group/version; and `"*"` matches all groups and versions.
`kinds`	List of Kubernetes kind selectors. `"*"` and partial wildcards match namespaced kinds, but `Namespace` must always appear as a separate literal entry to include it.

Examples:

metadata:
  - kinds:
      - ConfigMap
      - Service

This targets core v1 ConfigMap and Service resources because apiGroups is omitted.

metadata:
  - apiGroups:
      - apps/v1
    kinds:
      - Deployment
      - StatefulSet

This targets only apps/v1 Deployment and StatefulSet resources.

metadata:
  - apiGroups:
      - "*"
    kinds:
      - "*"

This targets all namespaced resources handled by the generic metadata webhook. It does not target Namespace, despite both selectors being wildcards.

Partial wildcards are also supported:

metadata:
  - apiGroups:
      - "apps/*"
    kinds:
      - "*Set"

This can match resources such as apps/v1 ReplicaSet and apps/v1 StatefulSet.

Namespace

Namespace is the only cluster-scoped resource supported by metadata rules. It is deliberately opt-in: the kinds list must contain the literal, case-sensitive value Namespace. This prevents a broad rule intended for resources inside Tenant namespaces from accidentally changing or rejecting the Namespace object itself.

The Namespace GVK is core v1, Kind=Namespace. The apiGroups selector must therefore match core v1. The clearest form is:

metadata:
  - apiGroups:
      - "v1"
    kinds:
      - "Namespace"

Because omitted apiGroups defaults to core v1, this shorter form is equivalent:

metadata:
  - kinds:
      - Namespace

An API-group wildcard may also match core v1, but it still does not remove the explicit-kind requirement. For example, this targets all namespaced kinds and Namespace:

metadata:
  - apiGroups:
      - "*"
    kinds:
      - "*"
      - Namespace

The following selectors do not target Namespace:

# A full kind wildcard is not an explicit Namespace opt-in.
metadata:
  - apiGroups:
      - "*"
    kinds:
      - "*"

# A partial kind wildcard is not an explicit Namespace opt-in either,
# even when its pattern would otherwise match the word "Namespace".
  - apiGroups:
      - "v1"
    kinds:
      - "Name*"

In short, both conditions must be true: apiGroups must match core v1, and kinds must contain a dedicated Namespace entry.

API group selection

Omitted or empty apiGroups does not mean all API groups and versions. It means the core Kubernetes API version v1.

For example:

metadata:
  - kinds:
      - Deployment

This does not match apps/v1 Deployment, because omitted apiGroups is interpreted as core v1.

To match apps/v1 deployments, set the API group/version selector explicitly:

metadata:
  - apiGroups:
      - apps/v1
    kinds:
      - Deployment

To match deployments across all API groups and versions, use "*":

metadata:
  - apiGroups:
      - "*"
    kinds:
      - Deployment

Labels and Annotations

The following rule requires an approved Tenant label and validates an optional cost-center annotation on ConfigMaps and Services:

rules:
  - enforce:
      action: allow
      metadata:
        - apiGroups:
            - "*"
          kinds:
            - ConfigMap
            - Service
          labels:
            corp.com/tenant:
              required: true
              values:
                - exact:
                    - prod
                    - test
          annotations:
            example.corp/cost-center:
              required: false
              values:
                - exp: "^INV-[0-9]{4}$"
                  exact:
                    - prod
                    - test

Label rules

Label rules are configured under metadata[].labels. Each map key is the label key to validate.

rules:
  - enforce:
      action: allow
      metadata:
        - kinds:
            - ConfigMap
          labels:
            env:
              required: true
              values:
                - exact:
                    - prod
                    - test

With this rule, a matching ConfigMap must contain metadata.labels["env"], and its value must be either prod or test.

This object is admitted:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  labels:
    env: prod
data:
  key: value

This object is denied because the required label is missing:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  key: value

This object is denied because the label value does not match the allow-list:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  labels:
    env: stage
data:
  key: value

Example rejection:

Error from server (Forbidden): error when creating "configmap.yaml": admission webhook "rules.generic.projectcapsule.dev" denied the request: metadata label "env" is required at metadata.labels["env"]

Annotation rules

Annotation rules are configured under metadata[].annotations. Each map key is the annotation key to validate.

rules:
  - audience:
      - kind: "Group"
        name: "system:authenticated"
      - kind: "Custom"
        name: "CapsuleUser"
      - kind: "Custom"
        name: "Administrator"
      - kind: "Custom"
        name: "TenantOwner"
    enforce:
      action: allow
      metadata:
        - apiGroups:
            - "v1"
          kinds:
            - Namespace
          annotations:
            example.corp/cost-center:
              required: false
              values:
                - exp: "^INV-[0-9]{4}$"
              # Overwrites anything, even if the user has set a value, Should be applied using SSA by the rulestatus controller, if removed also removes (one fieldmanager per rulestatus which controlles all managed metadata). Also enforce at admission
              managed: "INV-10"
            example.corp/cost-center-2:
              values:
                - exp: "II-10"
              default: "{{$.tenant.spec.data.costCenter}}"

With this rule, the annotation is optional. If the object does not contain metadata.annotations["example.corp/cost-center"], Capsule ignores the rule. If the annotation is present, its value must match the configured expression.

This object is admitted because the annotation is absent and required is false:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  key: value

This object is admitted because the annotation value matches:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  annotations:
    example.corp/cost-center: INV-1234
data:
  key: value

This object is denied because the annotation is present but does not match:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  annotations:
    example.corp/cost-center: BAD-1234
data:
  key: value

Default

The default field provides a value for coressponding field should no value be provided by the user. This is only applied at admission time and does not enforce the value to be present in the object.

default is meaningful for action: allow. deny and audit rules are value matchers; they do not require missing metadata to exist.

rules:
  - enforce:
      action: allow
      metadata:
        - kinds:
            - ConfigMap
          labels:
           cost-center:
              default: "internal"

Default values still validate against the configured values matchers. If the default value does not match any allow or deny rule, the request is denied.

Managed

Managed values set metadata to the configured value during admission, overwriting an existing value. For background reconciliation behavior, see managed metadata and conditions.

rules:
  - enforce:
      action: allow
      metadata:
        - apiGroups:
            - "v1"
          kinds:
            - Namespace
          annotations:
            example.corp/cost-center:
              required: false
              values:
                - exp: "^INV-[0-9]{4}$"
              # Overwrites anything, even if the user has set a value, Should be applied using SSA by the rulestatus controller, if removed also removes (one fieldmanager per rulestatus which controlles all managed metadata). Also enforce at admission
              managed: "INV-10"

Required

The required field controls whether the metadata key must be present.

`required`	Behavior
`true`	For `action: allow`, the key must be present on matching objects.
`false`	The key is optional. If it is missing, Capsule ignores it. If it is present, configured values are evaluated.

required is meaningful for action: allow. deny and audit rules are value matchers; they do not require missing metadata to exist.

Presence-only enforcement is possible by setting required: true and omitting values:

rules:
  - enforce:
      action: allow
      metadata:
        - kinds:
            - ConfigMap
          labels:
            tenant-approved:
              required: true

With this rule, matching ConfigMap resources must contain the tenant-approved label, but any value is accepted.

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  labels:
    tenant-approved: "true"
data:
  key: value

If the label is missing, the request is denied.

Validation

The values field uses the common match expression structure with exact, exp, and optional negate.

values:
  - exact:
      - prod
      - test
  - exp: "^sandbox-[0-9]+$"

A metadata value matches if any configured value matcher matches.

exact and exp can be combined in the same matcher:

values:
  - exact:
      - prod
      - test
    exp: "^dev-[0-9]+$"

This matcher allows prod, test, and values matching ^dev-[0-9]+$.

negate: true inverts the final matcher result:

rules:
  - enforce:
      action: deny
      metadata:
        - apiVersion: "*"
          kinds:
            - ConfigMap
          labels:
            team:
              values:
                - exp: "^trusted-.*"
                  negate: true

With this rule:

team=trusted-platform is not denied by this deny-only rule.
team=untrusted is denied.

If an allow-list also exists for the same metadata key, values excluded from a negated deny rule still need a matching allow rule.

Allow-list behavior for metadata

An allow rule creates an allow-list for the specific metadata key it controls.

rules:
  - enforce:
      action: allow
      metadata:
        - kinds:
            - ConfigMap
          labels:
            env:
              required: true
              values:
                - exact:
                    - prod
                    - test

With this rule:

Object label	Result
`env=prod`	Allowed
`env=test`	Allowed
`env=stage`	Denied
missing `env`	Denied because `required: true`

If required is false, missing metadata is ignored:

rules:
  - enforce:
      action: allow
      metadata:
        - kinds:
            - ConfigMap
          labels:
            env:
              required: false
              values:
                - exact:
                    - prod
                    - test

With this rule:

Object label	Result
`env=prod`	Allowed
`env=test`	Allowed
`env=stage`	Denied
missing `env`	Allowed

Allow-list behavior is evaluated per metadata key. A matching value for one key does not satisfy another required key.

For example:

rules:
  - enforce:
      action: allow
      metadata:
        - kinds:
            - ConfigMap
          labels:
            env:
              required: true
              values:
                - exact:
                    - prod
            team:
              required: true
              values:
                - exact:
                    - platform

The object must contain both env=prod and team=platform.

Deny metadata values

Use action: deny to reject specific metadata values.

rules:
  - enforce:
      action: deny
      metadata:
        - kinds:
            - ConfigMap
          labels:
            environment:
              values:
                - exact:
                    - deprecated

This ConfigMap is denied:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  labels:
    environment: deprecated
data:
  key: value

A later matching allow rule can override an earlier deny rule:

rules:
  - enforce:
      action: deny
      metadata:
        - kinds:
            - ConfigMap
          labels:
            environment:
              values:
                - exact:
                    - deprecated

  - namespaceSelector:
      matchLabels:
        allow-deprecated: "true"
    enforce:
      action: allow
      metadata:
        - kinds:
            - ConfigMap
          labels:
            environment:
              required: true
              values:
                - exact:
                    - deprecated

In namespaces labeled allow-deprecated=true, environment=deprecated is admitted because the later namespace-specific allow rule matches.

Audit metadata values

Use action: audit to observe metadata usage without blocking the request.

rules:
  - enforce:
      action: audit
      metadata:
        - apiGroups:
            - "*"
          kinds:
            - ConfigMap
            - Service
          labels:
            example.corp/audit:
              values:
                - exp: "^audit-.*"

A matching object is admitted in this audit-only example, but Capsule emits an audit event and returns an admission warning.

If an allow-list also exists for the same metadata key, audit does not satisfy that allow-list. The metadata value must still match an allow rule.

Multiple resource kinds

A single metadata rule can target multiple kinds:

rules:
  - enforce:
      action: allow
      metadata:
        - apiGroups:
            - "*"
          kinds:
            - ConfigMap
            - Service
          labels:
            corp.com/tenant:
              required: true
              values:
                - exact:
                    - prod
                    - test

With this rule, both matching ConfigMap and Service objects must contain corp.com/tenant=prod or corp.com/tenant=test.

Namespace-specific metadata rules

Metadata enforcement supports namespaceSelector like other namespace rules.

rules:
  - namespaceSelector:
      matchLabels:
        environment: prod
    enforce:
      action: allow
      metadata:
        - kinds:
            - ConfigMap
          annotations:
            example.corp/approval:
              required: true
              values:
                - exact:
                    - approved

This rule only applies to namespaces labeled environment=prod. In those namespaces, matching ConfigMap objects must contain example.corp/approval=approved.

Migration

The legacy metadata fields under Tenant.spec can be migrated to metadata rules. Rules use the same policy structure for Namespaces, Pods, Services, and other resources, and allow validation, admission defaults, and reconciled managed values to be combined.

Migrate one resource type at a time. When old and new configuration overlap, keep their values identical until the rule has been verified.

Migrate Namespace Metadata

The legacy Namespace metadata options map to rules as follows:

requiredMetadata becomes an allow rule with required: true and a values expression.
additionalMetadata and additionalMetadataList become managed values. Managed values are applied during admission and reconciled on existing Namespaces. Use default instead when a value should only be added during admission and remain user-editable afterwards.
An additionalMetadataList[].namespaceSelector moves to the rules[].namespaceSelector at the same rule level. For a Namespace target, the selector is evaluated against that Namespace’s labels.
Exact entries in forbiddenLabels.denied and forbiddenAnnotations.denied become deny rules that match any value for the concrete metadata key.

Namespace must be selected explicitly. Use the concrete core v1 API and the literal Namespace kind, especially when the rule contains managed values:

apiGroups:
  - "v1"
kinds:
  - Namespace

The following rules replace the requiredMetadata, additionalMetadataList, and exact forbidden-key examples from the legacy page:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - name: alice
      kind: User
  rules:
    # Require user-provided metadata and manage common metadata on every
    # Namespace in the Tenant.
    - enforce:
        action: allow
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - Namespace
            labels:
              env:
                required: true
                values:
                  - exp: "^(prod|test|dev)$"
              projectcapsule.dev/backup:
                managed: "true"
              templated-label:
                managed: "{{ .namespace.metadata.name }}"
            annotations:
              example.corp/cost-center:
                required: true
                values:
                  - exp: "^INV-[0-9]{4}$"
              storagelocationtype:
                managed: s3
              templated-annotation:
                managed: "{{ .tenant.metadata.name }}"

    # Apply the baseline security profile unless the Namespace explicitly
    # opts into the low-security profile.
    - namespaceSelector:
        matchExpressions:
          - key: projectcapsule.dev/low_security_profile
            operator: NotIn
            values:
              - "true"
      enforce:
        action: allow
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - Namespace
            labels:
              pod-security.kubernetes.io/enforce:
                managed: baseline

    # The selectors are mutually exclusive, so only one rule manages the Pod
    # Security Admission label for a Namespace.
    - namespaceSelector:
        matchExpressions:
          - key: projectcapsule.dev/low_security_profile
            operator: In
            values:
              - "true"
      enforce:
        action: allow
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - Namespace
            labels:
              pod-security.kubernetes.io/enforce:
                managed: privileged

    # Deny the presence of these concrete labels or annotations, regardless
    # of their values.
    - enforce:
        action: deny
        metadata:
          - apiGroups:
              - "v1"
            kinds:
              - Namespace
            labels:
              foo.acme.net:
                values:
                  - exp: ".*"
              bar.acme.net:
                values:
                  - exp: ".*"
            annotations:
              foo.acme.net:
                values:
                  - exp: ".*"
              bar.acme.net:
                values:
                  - exp: ".*"

With these rules:

Tenant owners must provide a valid env label and cost-center annotation.
Common and templated metadata is enforced on new and existing Namespaces.
The Pod Security Admission profile follows the Namespace selector.
foo.acme.net and bar.acme.net are denied as both labels and annotations, including when their value is an empty string.

Legacy options without a direct equivalent

Metadata rules are keyed by concrete label and annotation names. They do not currently provide a direct replacement for these key-wide legacy behaviors:

managedMetadataOnly: true, which rejects all user metadata not managed by Capsule; and
forbiddenLabels.deniedRegex or forbiddenAnnotations.deniedRegex, which match metadata keys by regular expression.

Enumerate known sensitive keys with deny rules as shown above. If the policy must reject every unlisted key or a changing family of keys, retain the legacy option during migration or enforce that part with another admission policy. Value regular expressions remain supported through values[].exp; the limitation only applies to regular expressions over metadata key names.

Rollout

Add the equivalent rules while the legacy values are still configured. Ensure overlapping managed values are identical.
Check the generated RuleStatus objects and verify both creation and update admission with a test Namespace.
Confirm that managed metadata has been reconciled onto existing Namespaces.
Remove the migrated requiredMetadata, additionalMetadata, additionalMetadataList, and exact forbidden-key entries.
Keep managedMetadataOnly or key-regex restrictions until an alternative policy covers them.

Migrate Pod Metadata

Use the following rule to migrate the old spec.podOptions.additionalMetadata API to the new spec.rules[].enforce.metadata API:

rules:
  - enforce:
      action: allow
      metadata:
        - kinds:
            - Pod
          annotations:
            storagelocationtype:
              managed: s3
          labels:
            projectcapsule.dev/backup:
              managed: "true"

Migrate Service Metadata

Use the following rule to migrate the old spec.serviceOptions.additionalMetadata API to the new spec.rules[].enforce.metadata API:

rules:
  - enforce:
      action: allow
      metadata:
        - kinds:
            - Service
            - Endpoints
          annotations:
            customer.corp/routable:
              managed: "true"
          labels:
            customer.corp/network-tenant:
              managed: "{{ .tenant.metadata.name }}"
        - apiGroups:
            - "discovery.k8s.io/v1"
          kinds:
            - EndpointSlice
          annotations:
            customer.corp/routable:
              managed: "true"
          labels:
            customer.corp/network-tenant:
              managed: "{{ .tenant.metadata.name }}"

Reference

This complete Tenant combines required labels, optional annotations, multiple kinds, audit rules, deny rules, and namespace-specific exceptions. Replace solar-owner with your owner identity.

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        metadata:
          - apiGroups:
              - "*"
            kinds:
              - ConfigMap
              - Service
            labels:
              corp.com/tenant:
                required: true
                values:
                  - exact:
                      - prod
                      - test
            annotations:
              example.corp/cost-center:
                required: false
                values:
                  - exp: "^INV-[0-9]{4}$"
                  - exact:
                      - prod
                      - test

    - enforce:
        action: deny
        metadata:
          - kinds:
              - ConfigMap
            labels:
              environment:
                values:
                  - exact:
                      - deprecated

    - enforce:
        action: audit
        metadata:
          - apiGroups:
              - "*"
            kinds:
              - ConfigMap
              - Service
            labels:
              example.corp/audit:
                values:
                  - exp: "^audit-.*"

    - namespaceSelector:
        matchLabels:
          environment: prod
      enforce:
        action: allow
        metadata:
          - kinds:
              - ConfigMap
            annotations:
              example.corp/approval:
                required: true
                values:
                  - exact:
                      - approved

With this configuration:

ConfigMap and Service objects must contain corp.com/tenant=prod or corp.com/tenant=test.
example.corp/cost-center is optional, but if present it must match ^INV-[0-9]{4}$, prod, or test.
ConfigMap objects with environment=deprecated are denied unless a later matching allow rule overrides the decision.
Objects with example.corp/audit values matching ^audit-.* emit audit events.
In namespaces labeled environment=prod, ConfigMap objects must also contain example.corp/approval=approved.

6.2.3 - Network

Enforcement for NetworkPolicies, Services, and ingress hostnames

Network enforcement covers three policy groups under spec.rules[].enforce:

Policy	Configuration path
Network Policies	`network.policies.ingress.cidrs`, `network.policies.egress.cidrs`
Services	`services`
Ingress	`ingress`

Use these policies to control ingress and egress CIDR grants, Service types and addresses, and hostnames on ingress resources. Reference provides a complete Tenant example for each policy group.

Rules use the shared namespace selection, audiences, conditions, and actions and order.

Rule Composition

Actions and order

Rules are evaluated in declaration order after namespace selection, audience, and condition filtering. The last matching allow or deny wins for each evaluated value. If action is omitted, it defaults to deny.

Action	Behavior
`allow`	Creates an allow-list for the evaluated matcher. Values outside the allow-list are denied unless another applicable allow or deny rule matches.
`deny`	Rejects matching values. A later matching allow rule can override that decision.
`audit`	Reports matches without allowing or denying the request. It never satisfies an allow-list.

The three policy groups evaluate different values. Every applicable check must pass; an allow decision in one group does not bypass another.

Policy	Unit of evaluation	Audit reporting
NetworkPolicy ingress and egress CIDRs	Every address granted after subtracting `ipBlock.except`. All granted addresses must pass.	`NamespaceRuleAudit` Kubernetes events; no admission warnings.
Services	Each Service type, IP address, ExternalName hostname, and node port covered by a matcher. Type and value allow-lists are independent.	Kubernetes events and admission warnings.
Ingress	Each hostname on the selected resource types, including both routing and TLS hosts on an Ingress.	Kubernetes events, including reports of missing hostname fields.

For NetworkPolicies, ordering applies per address and direction. A later allow for 10.20.0.0/17 overrides an earlier deny for 10.20.0.0/16 only within the smaller range. Several allowed CIDRs can together cover one grant. See Combine CIDR rules. Ingress and egress have independent allow-lists: an allow in one direction cannot override a deny or satisfy an allow-list in the other. When an enforcement block includes both directions, its action and conditions apply to both.

For Services, type-specific constraints do not grant the Service type. If a services.types allow-list is active, it must allow the type independently of any matching address or port rule. See Combining Service Rules.

Admission scope and limits

These policies validate creation and updates in the selected namespaces. They do not install network configuration or repair existing resources when a rule or namespace label changes. Those changes affect subsequent admissions.

NetworkPolicies

These policies check ingress source addresses and egress destination addresses. They do not constrain podSelector, namespaceSelector, or ports.
An omitted direction in the Capsule rule imposes no constraint in that direction. Existing egress-only rules keep their behavior.
An empty NetworkPolicy ingress or egress list contains no grants in that direction and passes its CIDR check. Only directions enabled by the NetworkPolicy’s policyTypes are evaluated, including Kubernetes defaults.
A rule with omitted peers (ingress: [{}] or egress: [{}]) grants every IPv4 and IPv6 address. An empty peer (from: [{}] or to: [{}]) does too. Restricting ports does not reduce this address grant.
Policy updates are checked against the new object, including metadata changes that affect enforcement conditions. This check does not block deletion.
CIDRs must include a prefix length and be valid IPv4 or IPv6 prefixes. IPv4-mapped IPv6 prefixes are unsupported. Host bits are normalized to the containing network. Each list accepts up to 64 unique CIDRs.

This is admission validation of NetworkPolicy grants. It does not itself install a policy or enforce packet filtering. Traffic behavior still depends on the NetworkPolicies present and the cluster’s network implementation.

Services

Behavior	Explanation
`services.types` is the type gate	Type-specific sections do not automatically grant the Service type. Include the Service type in `services.types` when an allow-list for Service types is active.
Value constraints create allow-lists for their values	If `loadBalancers.cidrs`, `externalIPs.cidrs`, `externalNames.hostnames`, or `nodePorts.ports` is configured with `action: allow`, non-matching values are denied.
External IP rules do not require a value	`externalIPs.cidrs` is evaluated only when a Service supplies `spec.externalIPs`; every supplied address is evaluated independently.
An empty external IP deny rule denies all external IPs	`action: deny` with `externalIPs: {}` or `externalIPs.cidrs: []` denies any Service that supplies an external IP.
`loadBalancers.cidrs` requires explicit values	When CIDR constraints are configured, `LoadBalancer` Services must set `spec.loadBalancerIP` or `spec.loadBalancerSourceRanges`.
`nodePorts.ports` requires explicit node ports	When port constraints are configured, `NodePort` Services and LoadBalancer Services with node port allocation enabled must set `spec.ports[].nodePort`.
LoadBalancer node port allocation matters	`LoadBalancer` Services are subject to NodePort range checks unless `spec.allocateLoadBalancerNodePorts: false` is set.
Negation applies to the whole matcher	`negate: true` inverts the result of both `exact` and `exp`.

Ingress

Ingress rules are evaluated during create and update admission. A rule only participates when its types list contains the incoming resource kind. Other resource types are unaffected. For example, types: [Ingress, HTTPRoute] applies the hostname policy to those two kinds; it does not prohibit Gateways or other omitted kinds. The action applies to the selected resources' hostnames, not to the resource kinds themselves.

Ingress Types lists the API versions handled by Capsule. Gateway API and OpenShift resources require their respective APIs to be installed in the cluster.

When an allow or deny hostname rule targets a resource type, its expected hostname fields must be non-empty. Audit-only rules report missing values without making hostnames mandatory. See Missing hostnames.

Network Policies

NetworkPolicy enforcement applies to native networking.k8s.io/v1 NetworkPolicies in the namespaces selected by the rule.

Ingress CIDRs

Configure ingress source constraints under spec.rules[].enforce.network.policies.ingress.cidrs. Capsule checks the addresses granted by spec.ingress[].from[].ipBlock during NetworkPolicy creation and updates, after subtracting ipBlock.except.

This controls source addresses in NetworkPolicies. Hostname enforcement for Ingress, Gateway API, and OpenShift Route resources is covered under Ingress.

Disallow CIDR sources

To prevent users from granting ingress from any CIDR source, deny both address families. This rule applies to every namespace in the Tenant:

rules:
  - enforce:
      action: deny
      network:
        policies:
          ingress:
            cidrs:
              - 0.0.0.0/0
              - ::/0

With this rule, the following NetworkPolicy is rejected:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-cidr-ingress
spec:
  podSelector: {}
  policyTypes: [Ingress]
  ingress:
    - from:
        - ipBlock:
            cidr: 192.0.2.0/24

A rejection identifies the source grant, field path, and matching rule:

networkPolicy ingress CIDR "192.0.2.0/24" at spec.ingress[0].from[0].ipBlock.cidr matches deny rule for CIDR 0.0.0.0/0

NetworkPolicy ingress grant	Result with the rule above
Any IPv4 or IPv6 `ipBlock` granting addresses	Denied
`ingress: [{}]`, including a rule with only ports	Denied: omitted `from` grants every source address
`from: [{}]`	Denied: the empty peer grants every source address
Peers using `podSelector` or `namespaceSelector`	Passes the CIDR check
`ingress: []`	Passes: no ingress addresses are granted
An IP block whose exceptions exclude every address	Passes: the block grants no addresses

Setting ingress: {} or ingress.cidrs: [] in the Capsule rule applies no source constraint. The rule above leaves egress unchanged; configure network.policies.egress.cidrs separately to constrain destinations.

Allow approved source ranges

Allow ingress CIDR grants only from approved source networks in selected namespaces, then exclude a reserved subnet with a later deny:

rules:
  - namespaceSelector:
      matchLabels:
        network-profile: internal
    enforce:
      action: allow
      network:
        policies:
          ingress:
            cidrs:
              - 10.20.0.0/16
              - fd00:1234::/48
  - namespaceSelector:
      matchLabels:
        network-profile: internal
    enforce:
      action: deny
      network:
        policies:
          ingress:
            cidrs:
              - 10.20.66.0/24

NetworkPolicy source	Exceptions	Result in a selected namespace
`10.20.1.0/24` or `fd00:1234::/64`	None	Allowed
`10.20.0.0/16`	None	Denied: includes the reserved subnet
`10.20.0.0/16`	`10.20.66.0/24`	Allowed: the reserved subnet is fully excluded
`10.0.0.0/8`	None	Denied: extends outside the approved sources
`192.0.2.0/24`	None	Denied: outside the allow-list
Omitted `from`	None	Denied: grants all IPv4 and IPv6 sources

Namespaces without network-profile: internal are unaffected by these rules. Partial exceptions leave any remaining prohibited addresses subject to denial. Multiple allowed ranges can together cover a source grant, using the same CIDR composition as egress. Use action: audit to report matching ingress grants without allowing or denying them; audit-only matches emit NamespaceRuleAudit events and do not satisfy an allow-list.

Egress CIDRs

Configure egress CIDR constraints under spec.rules[].enforce.network.policies.egress.cidrs. Capsule checks these grants during NetworkPolicy creation and updates.

Disallow CIDR destinations

To prevent users from granting egress to any CIDR destination, deny both address families. 0.0.0.0/0 covers every IPv4 address; ::/0 covers every IPv6 address. This example applies to every namespace in the Tenant:

rules:
  - enforce:
      action: deny
      network:
        policies:
          egress:
            cidrs:
              - 0.0.0.0/0
              - ::/0

With this rule, the following NetworkPolicy is rejected:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-cidr-egress
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
    - to:
        - ipBlock:
            cidr: 192.0.2.0/24

An egress rule with no to destinations, such as egress: [{}], grants access to all destinations. Capsule checks it against both address families and rejects it too. Restricting such a rule to particular ports does not bypass the CIDR restriction.

Selector-based peers remain permitted by this check. For example, this policy allows egress to matching Pods in the same namespace:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-selected-pods
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: backend

Use both universal CIDRs when blocking all CIDR destinations. Denying only 0.0.0.0/0 leaves explicit IPv6 grants unaffected. Setting egress: {} or cidrs: [] in the Capsule rule applies no CIDR constraint.

This check evaluates the addresses a policy grants, rather than the presence of the ipBlock field. If except excludes every address in an IP block, that block grants nothing and passes this check.

Disallow selected CIDRs

List specific CIDRs to protect only those ranges. This rule applies only to namespaces labeled network-profile: restricted:

rules:
  - namespaceSelector:
      matchLabels:
        network-profile: restricted
    enforce:
      action: deny
      network:
        policies:
          egress:
            cidrs:
              - 10.20.0.0/16
              - fd00:1234::/48

Matching uses address overlap, including subnets and broader ranges. Capsule subtracts ipBlock.except before evaluating the remaining grant:

NetworkPolicy destination	Exceptions	Result with the rule above
`10.20.4.0/24`	None	Denied: inside the protected range
`10.0.0.0/8`	None	Denied: includes the protected range
`0.0.0.0/0`	None	Denied: includes the protected range
`10.0.0.0/8`	`10.20.0.0/16`	Allowed: the protected range is fully excluded
`10.0.0.0/8`	`10.20.0.0/17`	Denied: part of the protected range remains
`10.0.0.0/8`	`10.20.0.0/17`, `10.20.128.0/17`	Allowed: the exceptions together exclude the protected range
`192.0.2.0/24`	None	Allowed: no overlap
`::/0`	None	Denied: includes the protected IPv6 range
`::/0`	`fd00:1234::/48`	Allowed: the protected IPv6 range is fully excluded

For example, this policy passes the CIDR check:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: exclude-protected-range
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
    - to:
        - ipBlock:
            cidr: 10.0.0.0/8
            except:
              - 10.20.0.0/16

A rejection identifies the destination, its field path, and the matched rule:

networkPolicy egress CIDR "10.0.0.0/8" at spec.egress[0].to[0].ipBlock.cidr matches deny rule for CIDR 10.20.0.0/16

Combine CIDR rules

For example, these two allowed ranges together cover 192.0.2.0/24:

rules:
  - enforce:
      action: allow
      network:
        policies:
          egress:
            cidrs:
              - 192.0.2.0/25
              - 192.0.2.128/25

A grant to 192.0.2.0/24 passes. A grant to 0.0.0.0/0 fails because it includes addresses outside the allow-list. An unrestricted egress rule requires complete coverage of both IPv4 and IPv6 address space.

To observe matching grants without blocking them, use an audit-only rule:

rules:
  - enforce:
      action: audit
      network:
        policies:
          egress:
            cidrs:
              - 10.20.0.0/16

This rule admits a grant to 10.20.0.0/16 and emits an audit event. NetworkPolicy CIDR audit rules report through events; they do not add admission warnings.

Services

Service enforcement is configured under spec.rules[].enforce.services. Use Service Types to control which types can be created, then constrain LoadBalancer addresses, External IPs, ExternalName hostnames, and NodePort ranges.

rules:
  - enforce:
      action: allow
      services:
        types:
          - ClusterIP
          - NodePort
          - LoadBalancer
          - ExternalName
        loadBalancers:
          cidrs:
            - 10.0.0.2/32
        externalIPs:
          cidrs:
            - 10.20.0.0/16
        externalNames:
          hostnames:
            - exp: ".*\\.example\\.com"
              exact:
                - internal.git.com
        nodePorts:
          ports:
            - from: 30000
              to: 32767

Service Types

The services.types field controls which Kubernetes Service types are allowed, denied, or audited by a rule.

Supported values are:

Type	Description
`ClusterIP`	Allows, denies, or audits Services of type `ClusterIP`.
`NodePort`	Allows, denies, or audits Services of type `NodePort`.
`LoadBalancer`	Allows, denies, or audits Services of type `LoadBalancer`.
`ExternalName`	Allows, denies, or audits Services of type `ExternalName`.

Allow selected types

Allow only ClusterIP Services:

rules:
  - enforce:
      action: allow
      services:
        types:
          - ClusterIP

With this rule, a ClusterIP Service is admitted:

apiVersion: v1
kind: Service
metadata:
  name: internal-api
spec:
  type: ClusterIP
  ports:
    - name: http
      port: 8080
      targetPort: 8080

A Service of another type, for example ExternalName, is denied because an allow-list exists for Service types and ExternalName is not listed:

apiVersion: v1
kind: Service
metadata:
  name: external-api
spec:
  type: ExternalName
  externalName: internal.git.com
  ports:
    - name: http
      port: 443
      targetPort: 443

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: service type "ExternalName" at spec.type is not allowed by namespace rule: value did not match any allowed rule. Allowed service types: ClusterIP

Deny selected types

Deny LoadBalancer Services:

rules:
  - enforce:
      action: deny
      services:
        types:
          - LoadBalancer

Allow ClusterIP and ExternalName, but deny ExternalName again for selected namespaces:

rules:
  - enforce:
      action: allow
      services:
        types:
          - ClusterIP
          - ExternalName

  - namespaceSelector:
      matchLabels:
        external-services: blocked
    enforce:
      action: deny
      services:
        types:
          - ExternalName

Because later matching allow or deny decisions win, namespaces labeled external-services=blocked cannot create ExternalName Services, while other matching namespaces can.

Combine types and value constraints

The services.types field is the Service capability gate. Constraint sections such as loadBalancers, externalIPs, externalNames, and nodePorts do not automatically allow a Service type by themselves. The externalIPs constraint is type-independent and applies to any allowed Service that specifies spec.externalIPs.

For example, this rule restricts LoadBalancer CIDRs, but it does not by itself allow LoadBalancer Services if another type allow-list exists that excludes LoadBalancer:

rules:
  - enforce:
      action: allow
      services:
        types:
          - ClusterIP

  - enforce:
      action: allow
      services:
        loadBalancers:
          cidrs:
            - 10.0.0.2/32

In this example, a LoadBalancer Service is denied by the Service type allow-list because LoadBalancer is not included in services.types.

To allow and constrain LoadBalancer Services, configure both:

rules:
  - enforce:
      action: allow
      services:
        types:
          - LoadBalancer
        loadBalancers:
          cidrs:
            - 10.0.0.2/32

LoadBalancer

LoadBalancer rules allow administrators to restrict the IPs and source ranges used by Services of type LoadBalancer.

LoadBalancer constraints are configured under enforce.services.loadBalancers.cidrs.

Capsule evaluates the following Service fields:

Field	Description
`spec.loadBalancerIP`	Explicit LoadBalancer IP requested by the Service.
`spec.loadBalancerSourceRanges[]`	Source CIDR ranges allowed to access the LoadBalancer.

Allow LoadBalancer Services only with a specific IP:

rules:
  - enforce:
      action: allow
      services:
        types:
          - LoadBalancer
        loadBalancers:
          cidrs:
            - 10.0.0.2/32

This Service is admitted:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  loadBalancerIP: 10.0.0.2
  ports:
    - name: http
      port: 80
      targetPort: 8080

This Service is denied because the requested IP is outside the allowed CIDR:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  loadBalancerIP: 10.0.171.239
  ports:
    - name: http
      port: 80
      targetPort: 8080

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: loadBalancer CIDR "10.0.171.239" at spec.loadBalancerIP is not allowed by namespace rule: value did not match any allowed rule. Allowed CIDRs: 10.0.0.2/32

Allow a LoadBalancer IP range:

rules:
  - enforce:
      action: allow
      services:
        types:
          - LoadBalancer
        loadBalancers:
          cidrs:
            - 10.0.1.0/24

The following Service is admitted because 10.0.1.44 is contained in 10.0.1.0/24:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  loadBalancerIP: 10.0.1.44
  ports:
    - name: http
      port: 80
      targetPort: 8080

Restrict loadBalancerSourceRanges:

rules:
  - enforce:
      action: allow
      services:
        types:
          - LoadBalancer
        loadBalancers:
          cidrs:
            - 10.0.1.0/24

This Service is admitted because the requested source range is fully contained in the allowed CIDR:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  loadBalancerSourceRanges:
    - 10.0.1.0/25
  ports:
    - name: http
      port: 80
      targetPort: 8080

This Service is denied because the requested source range is not fully contained in the allowed CIDR:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  loadBalancerSourceRanges:
    - 10.0.1.0/23
  ports:
    - name: http
      port: 80
      targetPort: 8080

Required LoadBalancer fields when CIDRs are configured

If any matching rule configures loadBalancers.cidrs, then a LoadBalancer Service must explicitly set at least one of:

spec.loadBalancerIP
spec.loadBalancerSourceRanges

This is intentional. If CIDR restrictions are configured, Capsule requires the Service request to provide a value that can be evaluated.

For example, this Service is denied when loadBalancers.cidrs is configured:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  ports:
    - name: http
      port: 80
      targetPort: 8080

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: loadBalancer service requires spec.loadBalancerIP or spec.loadBalancerSourceRanges because loadBalancer CIDR constraints are enforced by namespace rule

If no loadBalancers.cidrs constraint is configured, Capsule does not require these fields. In that case, a LoadBalancer Service can be admitted as long as the Service type itself is allowed.

Denying selected LoadBalancer CIDRs

You can also deny specific LoadBalancer CIDRs:

rules:
  - enforce:
      action: allow
      services:
        types:
          - LoadBalancer
        loadBalancers:
          cidrs:
            - 10.0.0.0/8

  - enforce:
      action: deny
      services:
        loadBalancers:
          cidrs:
            - 10.0.66.0/24

A Service using 10.0.66.10 is denied because the later deny rule matches:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: loadBalancer CIDR "10.0.66.10" at spec.loadBalancerIP is denied by namespace rule: 10.0.66.10 is contained in 10.0.66.0/24

A later namespace-specific allow rule can override an earlier allow miss or deny decision:

rules:
  - enforce:
      action: allow
      services:
        types:
          - LoadBalancer
        loadBalancers:
          cidrs:
            - 10.0.0.2/32

  - namespaceSelector:
      matchLabels:
        environment: prod
    enforce:
      action: allow
      services:
        loadBalancers:
          cidrs:
            - 10.0.171.0/24

In namespaces labeled environment=prod, a Service using 10.0.171.239 is admitted. In other namespaces, it is denied because it does not match the default allowed CIDR.

External IPs

External IP rules allow administrators to restrict the addresses supplied in spec.externalIPs.

External IP constraints are configured under enforce.services.externalIPs.cidrs. They are admission rules only: Capsule does not allocate addresses, add entries to spec.externalIPs, or configure routing for them.

The constraint applies to any Service type that specifies spec.externalIPs. Individual addresses in a rule are treated as host CIDRs:

an IPv4 address without a prefix is treated as /32;
an IPv6 address without a prefix is treated as /128.

Allow external IPs from selected networks:

rules:
  - enforce:
      action: allow
      services:
        types:
          - ClusterIP
        externalIPs:
          cidrs:
            - 10.20.0.0/16
            - 192.168.1.2
            - 2001:db8::/32

This Service is admitted because both requested addresses match an allowed CIDR:

apiVersion: v1
kind: Service
metadata:
  name: internal-api
spec:
  type: ClusterIP
  externalIPs:
    - 10.20.1.44
    - 192.168.1.2
  ports:
    - name: http
      port: 8080
      targetPort: 8080

Every entry in spec.externalIPs must satisfy the rule evaluation. The following Service is denied because 8.8.8.8 is outside the allowed CIDRs:

apiVersion: v1
kind: Service
metadata:
  name: public-dns
spec:
  type: ClusterIP
  externalIPs:
    - 10.20.1.44
    - 8.8.8.8
  ports:
    - name: dns
      protocol: UDP
      port: 53
      targetPort: 5353

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: external IP "8.8.8.8" at spec.externalIPs[1] is not allowed by namespace rule: value did not match any allowed rule. Allowed CIDRs: 10.20.0.0/16, 192.168.1.2, 2001:db8::/32

Unlike LoadBalancer CIDR and NodePort range constraints, an external IP rule does not require the Service to specify a value. A Service without spec.externalIPs is admitted if it satisfies the other applicable Service rules.

Denying selected external IPs

You can combine a broad allow-list with a later deny rule:

rules:
  - enforce:
      action: allow
      services:
        externalIPs:
          cidrs:
            - 10.20.0.0/16

  - enforce:
      action: deny
      services:
        externalIPs:
          cidrs:
            - 10.20.66.0/24

An address such as 10.20.1.44 is admitted, while 10.20.66.4 is denied because the later deny rule matches.

Denying all external IPs

For a deny rule, an empty cidrs array matches every value in spec.externalIPs:

rules:
  - enforce:
      action: deny
      services:
        externalIPs:
          cidrs: []

The equivalent short form is:

rules:
  - enforce:
      action: deny
      services:
        externalIPs: {}

Both forms deny any Service that supplies one or more external IPs. Services that omit spec.externalIPs remain unaffected. An empty cidrs array in an allow or audit rule does not create an external IP restriction.

ExternalName

ExternalName rules allow administrators to restrict spec.externalName for Services of type ExternalName.

ExternalName constraints are configured under enforce.services.externalNames.hostnames.

Each hostname matcher uses the common match expression structure with exact, exp, and optional negate.

Allow selected ExternalName hostnames:

rules:
  - enforce:
      action: allow
      services:
        types:
          - ExternalName
        externalNames:
          hostnames:
            - exact:
                - internal.git.com
            - exp: ".*\\.example\\.com"

The following Services are admitted:

apiVersion: v1
kind: Service
metadata:
  name: git
spec:
  type: ExternalName
  externalName: internal.git.com
  ports:
    - name: https
      port: 443
      targetPort: 443

apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  type: ExternalName
  externalName: api.example.com
  ports:
    - name: https
      port: 443
      targetPort: 443

A non-matching hostname is denied:

apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  type: ExternalName
  externalName: api.bad.com
  ports:
    - name: https
      port: 443
      targetPort: 443

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: externalName hostname "api.bad.com" at spec.externalName is not allowed by namespace rule: value did not match any allowed rule. Allowed hostnames: exact: internal.git.com, exp: .*\.example\.com

Use exact and exp together in the same matcher:

rules:
  - enforce:
      action: allow
      services:
        types:
          - ExternalName
        externalNames:
          hostnames:
            - exact:
                - combined.internal.git.com
              exp: "combined\\..*\\.example\\.com"

This matcher allows both:

combined.internal.git.com
hostnames matching combined\\..*\\.example\\.com

Negation for ExternalName hostnames

negate: true inverts the final matcher result. This applies to both exact and exp.

Deny every ExternalName except trusted hostnames:

rules:
  - enforce:
      action: deny
      services:
        externalNames:
          hostnames:
            - exp: "trusted\\..*"
              negate: true

  - enforce:
      action: allow
      services:
        types:
          - ExternalName
        externalNames:
          hostnames:
            - exp: "trusted\\..*"

With these rules:

trusted.api is admitted.
api.example.com is denied by the negated deny rule.

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: externalName hostname "api.example.com" at spec.externalName is denied by namespace rule: "api.example.com" matched hostname rule not exp: trusted\..*

Important: when an allow-list exists for ExternalName hostnames, values excluded from a negated deny rule still need a matching allow rule. The deny rule prevents untrusted values, while the allow rule satisfies allow-list behavior for trusted values.

Namespace-specific ExternalName rules

You can use namespaceSelector to apply ExternalName restrictions only to selected namespaces:

rules:
  - enforce:
      action: allow
      services:
        types:
          - ExternalName
        externalNames:
          hostnames:
            - exp: ".*\\.example\\.com"

  - namespaceSelector:
      matchLabels:
        external-policy: restricted
    enforce:
      action: deny
      services:
        externalNames:
          hostnames:
            - exact:
                - blocked.example.com

In namespaces labeled external-policy=restricted, blocked.example.com is denied. Other hostnames matching .*\\.example\\.com remain allowed.

NodePort

NodePort rules allow administrators to restrict explicitly requested spec.ports[].nodePort values.

NodePort constraints are configured under enforce.services.nodePorts.ports.

Each port range contains:

Field	Description
`from`	First allowed or denied port in the range.
`to`	Last allowed or denied port in the range.

The from value must be lower than or equal to to. Equal values are valid and represent a single port.

Allow selected NodePort ranges:

rules:
  - enforce:
      action: allow
      services:
        types:
          - NodePort
        nodePorts:
          ports:
            - from: 30000
              to: 30100
            - from: 30500
              to: 30500

This Service is admitted because 30080 is in the allowed range:

apiVersion: v1
kind: Service
metadata:
  name: tenant-api
spec:
  type: NodePort
  ports:
    - name: http
      port: 8080
      targetPort: 8080
      nodePort: 30080

This Service is also admitted because 30500 matches the single-port range:

apiVersion: v1
kind: Service
metadata:
  name: tenant-api-single
spec:
  type: NodePort
  ports:
    - name: http
      port: 8080
      targetPort: 8080
      nodePort: 30500

This Service is denied because 32080 is outside the allowed ranges:

apiVersion: v1
kind: Service
metadata:
  name: tenant-api
spec:
  type: NodePort
  ports:
    - name: http
      port: 8080
      targetPort: 8080
      nodePort: 32080

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: nodePort "32080" at spec.ports[0].nodePort is not allowed by namespace rule: value did not match any allowed rule. Allowed ranges: 30000-30100, 30500

Required explicit nodePort when ranges are configured

If any matching rule configures nodePorts.ports, then a NodePort Service must explicitly set spec.ports[].nodePort.

This is intentional. Kubernetes can allocate a node port automatically when the field is omitted, but the validating webhook cannot know the allocated value at admission time. To enforce configured port ranges reliably, Capsule requires the requested node port to be explicit.

The following Service is denied when nodePorts.ports is configured:

apiVersion: v1
kind: Service
metadata:
  name: tenant-api
spec:
  type: NodePort
  ports:
    - name: http
      port: 8080
      targetPort: 8080

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: service requires explicit spec.ports[*].nodePort because nodePort ranges are enforced by namespace rule

If no nodePorts.ports constraint is configured, Capsule does not require explicit nodePort values. In that case, a NodePort Service can be admitted as long as the Service type itself is allowed.

Denying selected NodePorts

You can allow a broad range and deny a specific port afterwards:

rules:
  - enforce:
      action: allow
      services:
        types:
          - NodePort
        nodePorts:
          ports:
            - from: 30000
              to: 30100

  - enforce:
      action: deny
      services:
        nodePorts:
          ports:
            - from: 30090
              to: 30090

A Service using 30080 is admitted. A Service using 30090 is denied because the later deny rule also matches.

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: nodePort "30090" at spec.ports[0].nodePort is denied by namespace rule: nodePort 30090 is within allowed range 30090

Although the detail says the port is within the matched range, the rule action is deny, so the request is rejected.

LoadBalancer Services and NodePorts

Kubernetes LoadBalancer Services may allocate node ports unless spec.allocateLoadBalancerNodePorts is explicitly set to false.

Therefore, NodePort range enforcement also applies to LoadBalancer Services when node port allocation is enabled.

This rule allows LoadBalancer Services, restricts the LoadBalancer IP, and restricts the allocated node port:

rules:
  - enforce:
      action: allow
      services:
        types:
          - LoadBalancer
        loadBalancers:
          cidrs:
            - 10.0.0.2/32
        nodePorts:
          ports:
            - from: 30000
              to: 30100

This Service is admitted because the LoadBalancer IP and node port are both allowed:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  loadBalancerIP: 10.0.0.2
  ports:
    - name: http
      port: 80
      targetPort: 8080
      nodePort: 30080

This Service is denied because the explicit node port is outside the allowed range:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  loadBalancerIP: 10.0.0.2
  ports:
    - name: http
      port: 80
      targetPort: 8080
      nodePort: 32080

When nodePorts.ports is configured and LoadBalancer node port allocation is enabled, Capsule requires explicit spec.ports[].nodePort values:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  loadBalancerIP: 10.0.0.2
  ports:
    - name: http
      port: 80
      targetPort: 8080

Example rejection:

Error from server (Forbidden): error when creating "svc.yaml": admission webhook "services.validating.projectcapsule.dev" denied the request: service requires explicit spec.ports[*].nodePort because nodePort ranges are enforced by namespace rule

To avoid node port enforcement for a LoadBalancer Service, disable node port allocation explicitly:

apiVersion: v1
kind: Service
metadata:
  name: public-api
spec:
  type: LoadBalancer
  allocateLoadBalancerNodePorts: false
  loadBalancerIP: 10.0.0.2
  ports:
    - name: http
      port: 80
      targetPort: 8080

With allocateLoadBalancerNodePorts: false, Capsule does not require or validate spec.ports[].nodePort for that LoadBalancer Service. The Service must still satisfy any configured LoadBalancer CIDR rules.

Auditing Services

Use action: audit to observe Service usage without directly blocking the request. Audit rules emit Kubernetes events and return admission warnings, but they do not allow or deny the request.

Audit ExternalName usage:

rules:
  - enforce:
      action: audit
      services:
        types:
          - ExternalName
        externalNames:
          hostnames:
            - exp: "audit\\..*"

A matching Service is admitted in this audit-only example because no Service type or hostname allow-list is configured:

apiVersion: v1
kind: Service
metadata:
  name: audited-external
spec:
  type: ExternalName
  externalName: audit.internal
  ports:
    - name: https
      port: 443
      targetPort: 443

If an allow-list is also configured, audit does not satisfy it:

rules:
  - enforce:
      action: audit
      services:
        externalNames:
          hostnames:
            - exp: "audit\\..*"

  - enforce:
      action: allow
      services:
        types:
          - ExternalName
        externalNames:
          hostnames:
            - exp: "allowed\\..*"

With these rules, audit.internal emits an audit event but is still denied because it does not match the allowed hostname rule.

Combining Service Rules

Service rules can be split across multiple rule blocks. This is useful when type permissions, external IP rules, LoadBalancer CIDR rules, hostname rules, and NodePort ranges should be managed independently.

For example:

rules:
  - enforce:
      action: allow
      services:
        types:
          - ClusterIP
          - ExternalName

  - enforce:
      action: allow
      services:
        externalNames:
          hostnames:
            - exp: ".*\\.example\\.com"

This configuration:

allows ClusterIP Services;
allows ExternalName Services as a type;
allows only ExternalName hostnames matching .*\\.example\\.com.

A Service of type ExternalName with externalName: api.example.com is admitted. A Service of type ExternalName with externalName: api.bad.com is denied by the hostname allow-list.

A later deny rule can override an earlier allow rule:

rules:
  - enforce:
      action: allow
      services:
        types:
          - ExternalName
        externalNames:
          hostnames:
            - exp: ".*\\.example\\.com"

  - enforce:
      action: deny
      services:
        externalNames:
          hostnames:
            - exact:
                - blocked.example.com

Here, api.example.com is allowed, but blocked.example.com is denied because the later deny rule matches.

A later allow rule can override an earlier deny rule:

rules:
  - enforce:
      action: deny
      services:
        nodePorts:
          ports:
            - from: 30080
              to: 30080

  - namespaceSelector:
      matchLabels:
        allow-special-nodeport: "true"
    enforce:
      action: allow
      services:
        types:
          - NodePort
        nodePorts:
          ports:
            - from: 30080
              to: 30080

In namespaces labeled allow-special-nodeport=true, a NodePort Service using 30080 is admitted because the namespace-specific allow rule matches later.

Ingress

Ingress enforcement is configured under spec.rules[].enforce.ingress. Use Ingress Types to select Kubernetes Ingresses, OpenShift Routes, and Gateway API resources, then configure Hostnames to allow, deny, or audit their hostname values.

Ingress Types

The types list selects the resource kinds whose hostnames the rule evaluates. Configure at least one type and one hostname expression together. A rule with only types or only hostnames is rejected at admission.

Capsule supports the following resource types and hostname fields:

Type	API	Evaluated fields
`Ingress`	`networking.k8s.io/v1`	`spec.rules[].host` and `spec.tls[].hosts[]`
`Route`	`route.openshift.io/v1`	`spec.host`
`Gateway`	`gateway.networking.k8s.io/v1`	`spec.listeners[].hostname`
`ListenerSet`	`gateway.networking.k8s.io/v1`	`spec.listeners[].hostname`
`HTTPRoute`	`gateway.networking.k8s.io/v1`	`spec.hostnames[]`
`TLSRoute`	`gateway.networking.k8s.io/v1`	`spec.hostnames[]`
`GRPCRoute`	`gateway.networking.k8s.io/v1`	`spec.hostnames[]`

Hostnames

The hostnames list contains match expressions. Each entry supports:

Field	Matching behavior
`exact`	Match one of the listed hostname strings exactly.
`exp`	Match a regular expression. Use `^` and `$` to match the complete hostname.
`negate`	Invert the entry’s match result. Defaults to `false`.

An entry can use exact, exp, or both. When both are set, either match is sufficient before applying negate. Multiple entries provide alternative matches for the rule’s action.

Each hostname on a targeted resource is evaluated independently. The entire request is denied if any hostname is denied or does not satisfy an active allow-list. For an Ingress, this includes both routing hosts and TLS hosts, so all values in spec.rules[].host and spec.tls[].hosts[] must satisfy the policy.

Allow selected hostnames

The following rule allows one exact hostname and any single-label hostname under example.com for Kubernetes Ingress and Gateway API HTTPRoute resources:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        ingress:
          types:
            - Ingress
            - HTTPRoute
          hostnames:
            - exact:
                - internal.example.com
            - exp: "^[a-z0-9-]+\\.example\\.com$"

This Ingress is admitted because both its routing hostname and TLS hostname match the allow-list:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tenant-api
spec:
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: tenant-api
                port:
                  number: 8080
  tls:
    - hosts:
        - api.example.com
      secretName: tenant-api-tls

Changing either occurrence to api.example.net denies the request. A rejection for the routing hostname includes the object path and configured allow-list:

ingress hostname "api.example.net" at spec.rules[0].host is not allowed by namespace rule: value did not match any allowed rule. Allowed hostnames: exact: internal.example.com, exp: ^[a-z0-9-]+\.example\.com$

An Ingress TLS entry is not exempt from enforcement. For example, the following object is denied even though its routing hostname is allowed, because legacy.example.net in spec.tls[0].hosts[0] is not allowed:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mixed-hostnames
spec:
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: tenant-api
                port:
                  number: 8080
  tls:
    - hosts:
        - legacy.example.net
      secretName: tenant-api-tls

Gateway API and OpenShift Route examples

A single rule can target several supported resource shapes:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        ingress:
          types:
            - Route
            - Gateway
            - ListenerSet
            - HTTPRoute
            - TLSRoute
            - GRPCRoute
          hostnames:
            - exp: "^([a-z0-9-]+\\.)*apps\\.example\\.com$"

For an HTTPRoute, Capsule evaluates every entry in spec.hostnames:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: store
spec:
  hostnames:
    - store.apps.example.com
    - checkout.apps.example.com
  rules:
    - backendRefs:
        - name: store
          port: 8080

Both values match the expression, so the request is admitted. If one hostname does not match, Capsule denies the entire HTTPRoute.

For a Gateway or ListenerSet, Capsule evaluates the hostname of every listener:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: tenant-gateway
spec:
  gatewayClassName: shared
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      hostname: gateway.apps.example.com
      tls:
        mode: Terminate
        certificateRefs:
          - name: gateway-tls

For an OpenShift Route, the same rule evaluates spec.host:

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: tenant-api
spec:
  host: api.apps.example.com
  to:
    kind: Service
    name: tenant-api

Deny hostnames and add exceptions

This Tenant allows a hostname family, denies a reserved hostname with a later rule, and allows that hostname again in namespaces selected by the final rule:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        ingress:
          types: [Ingress, HTTPRoute]
          hostnames:
            - exp: '^[a-z0-9-]+\.example\.com$'

    - enforce:
        action: deny
        ingress:
          types: [Ingress, HTTPRoute]
          hostnames:
            - exact: [admin.example.com]

    - namespaceSelector:
        matchLabels:
          ingress-admin: "true"
      enforce:
        action: allow
        ingress:
          types: [Ingress, HTTPRoute]
          hostnames:
            - exact: [admin.example.com]

api.example.com is admitted in every namespace in this Tenant. admin.example.com is admitted only in namespaces labeled ingress-admin=true; the later deny rule blocks it in other namespaces. Keep permission to change exception-selecting labels with the administrators who manage the policy.

Negated matches

You can also use negation to deny every hostname outside a trusted suffix:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        ingress:
          types:
            - Ingress
          hostnames:
            - exp: "^([a-z0-9-]+\\.)*example\\.com$"
              negate: true

This deny rule matches hostnames that do not match the expression. Because it does not create an allow-list, matching example.com hostnames pass unless another rule denies them.

Audit hostname usage

Use action: audit to observe selected hostnames without blocking them:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: audit
        ingress:
          types:
            - Ingress
            - HTTPRoute
          hostnames:
            - exp: "^preview-.*\\.example\\.com$"

A matching hostname is admitted in this audit-only example, and Capsule emits a Kubernetes event for it. If an allow-list is also configured and the hostname does not match an allow rule, the request is still denied; the audit rule does not grant access.

Missing hostnames

As soon as at least one allow or deny hostname rule targets a resource type, every expected hostname field on that resource must contain a non-empty value. Omitted, empty, and whitespace-only values are treated as missing.

Examples of missing values include:

an Ingress with no routing or TLS hostname, an Ingress rule without host, or a TLS entry without hosts;
a Route without spec.host;
a Gateway or ListenerSet listener without hostname;
an HTTPRoute, TLSRoute, or GRPCRoute without spec.hostnames.

For example, this Gateway is denied when an allow or deny hostname rule targets Gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: hostless
spec:
  gatewayClassName: shared
  listeners:
    - name: http
      protocol: HTTP
      port: 80

The rejection identifies the missing field:

hostname is required at spec.listeners[0].hostname because hostname rules target Gateway

If no ingress rule targets the resource type, Capsule does not apply this hostname requirement.

Audit-only rules do not make hostnames mandatory. When an expected hostname is missing, Capsule admits the request and emits an audit event that identifies the empty field, for example:

empty hostname detected at spec.listeners[0].hostname for Gateway by audit namespace rule

If audit and allow or deny rules target the same resource type, Capsule emits the empty-hostname audit event and enforces the non-audit rule, which denies the request.

Reference

The following Tenant examples combine the policies in each group. Use the configuration paths at the start of this chapter to combine them in one Tenant.

Network Policies

This complete Tenant blocks CIDR-based ingress and egress grants by default, permits approved ranges in selected namespaces, and denies a reserved subnet in those namespaces. Replace solar-owner with your owner identity.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: deny
        network:
          policies:
            ingress:
              cidrs:
                - 0.0.0.0/0
                - ::/0
            egress:
              cidrs:
                - 0.0.0.0/0
                - ::/0

    - namespaceSelector:
        matchLabels:
          network-profile: approved-cidrs
      enforce:
        action: allow
        network:
          policies:
            ingress:
              cidrs:
                - 10.20.0.0/16
                - fd00:1234::/48
            egress:
              cidrs:
                - 10.20.0.0/16
                - fd00:1234::/48

    - namespaceSelector:
        matchLabels:
          network-profile: approved-cidrs
      enforce:
        action: deny
        network:
          policies:
            ingress:
              cidrs:
                - 10.20.66.0/24
            egress:
              cidrs:
                - 10.20.66.0/24

Namespace profile	NetworkPolicy grant in either direction	Result
Without `network-profile: approved-cidrs`	Any CIDR granting addresses	Denied
`network-profile: approved-cidrs`	`10.20.1.0/24` or `fd00:1234::/64`	Allowed by the later allow rule
`network-profile: approved-cidrs`	`10.20.66.0/24`	Denied by the final deny rule
`network-profile: approved-cidrs`	`10.20.0.0/16` without exceptions	Denied: includes the reserved subnet
`network-profile: approved-cidrs`	`10.20.0.0/16` except `10.20.66.0/24`	Allowed
`network-profile: approved-cidrs`	`192.0.2.0/24`	Denied: outside the approved ranges
Any namespace in this Tenant	Ingress or egress rule with omitted peers	Denied
Any namespace in this Tenant	Selector-based peers or no grants in that direction	Passes the CIDR check

Other applicable enforcement rules must also pass. Rules from this Tenant do not apply to namespaces belonging to another Tenant.

Services

This complete Tenant combines type enforcement, external IP restrictions, LoadBalancer CIDR restrictions, ExternalName hostname restrictions, NodePort range restrictions, audit rules, and namespace-specific exceptions. Replace solar-owner with your owner identity.

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        services:
          types:
            - ClusterIP
            - NodePort
            - LoadBalancer
            - ExternalName

    - enforce:
        action: allow
        services:
          externalIPs:
            cidrs:
              - 10.20.0.0/16
              - 192.168.1.2

    - enforce:
        action: allow
        services:
          loadBalancers:
            cidrs:
              - 10.0.0.2/32
              - 10.0.1.0/24

    - enforce:
        action: allow
        services:
          externalNames:
            hostnames:
              - exact:
                  - internal.git.com
              - exp: ".*\\.example\\.com"

    - enforce:
        action: allow
        services:
          nodePorts:
            ports:
              - from: 30000
                to: 30100
              - from: 30500
                to: 30500

    - enforce:
        action: deny
        services:
          nodePorts:
            ports:
              - from: 30090
                to: 30090

    - enforce:
        action: deny
        services:
          externalIPs:
            cidrs:
              - 10.20.66.0/24

    - enforce:
        action: deny
        services:
          loadBalancers:
            cidrs:
              - 10.0.66.0/24

    - enforce:
        action: audit
        services:
          externalNames:
            hostnames:
              - exp: "audit\\..*"

    - namespaceSelector:
        matchLabels:
          environment: prod
      enforce:
        action: allow
        services:
          loadBalancers:
            cidrs:
              - 10.0.171.0/24

With this configuration:

ClusterIP, NodePort, LoadBalancer, and ExternalName Services are valid Service types.
External IPs must be contained in 10.20.0.0/16 or match 192.168.1.2.
External IPs in 10.20.66.0/24 are denied even though they are inside the broader allowed range.
LoadBalancer IPs must be contained in 10.0.0.2/32 or 10.0.1.0/24.
Namespaces labeled environment=prod can also use LoadBalancer IPs in 10.0.171.0/24.
ExternalName hostnames must be internal.git.com or match .*\\.example\\.com.
Explicit node ports must be in 30000-30100 or equal to 30500.
Node port 30090 is denied even though it is inside the broader allowed range.
ExternalName hostnames matching audit\\..* emit audit events and warnings.
Audit matches do not allow values that fail the allow-list.

Ingress

This complete Tenant allows a hostname family, denies a reserved hostname, audits preview hostnames, and grants a namespace-specific exception. Replace solar-owner with your owner identity. It covers every resource kind listed under Ingress Targets, including Gateway API and OpenShift resources when those APIs are installed.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - enforce:
        action: allow
        ingress:
          types: [Ingress, Route, Gateway, ListenerSet, HTTPRoute, TLSRoute, GRPCRoute]
          hostnames:
            - exact: [internal.example.com]
            - exp: '^[a-z0-9-]+\.solar\.example\.com$'

    - enforce:
        action: deny
        ingress:
          types: [Ingress, Route, Gateway, ListenerSet, HTTPRoute, TLSRoute, GRPCRoute]
          hostnames:
            - exact: [admin.solar.example.com]

    - enforce:
        action: audit
        ingress:
          types: [Ingress, Route, Gateway, ListenerSet, HTTPRoute, TLSRoute, GRPCRoute]
          hostnames:
            - exp: '^preview-[a-z0-9-]+\.solar\.example\.com$'

    - namespaceSelector:
        matchLabels:
          ingress-admin: "true"
      enforce:
        action: allow
        ingress:
          types: [Ingress, Route, Gateway, ListenerSet, HTTPRoute, TLSRoute, GRPCRoute]
          hostnames:
            - exact: [admin.solar.example.com]

Each routing, TLS, or listener hostname is checked independently:

Hostname	Namespace	Result
`api.solar.example.com` or `internal.example.com`	Any namespace in this Tenant	Allowed
`admin.solar.example.com`	Without `ingress-admin: "true"`	Denied by the later deny rule
`admin.solar.example.com`	With `ingress-admin: "true"`	Allowed by the final exception
`preview-api.solar.example.com`	Any namespace in this Tenant	Allowed and audited
`api.example.net`	Any namespace in this Tenant	Denied because it does not match the allow-list
A required hostname is missing	Any namespace in this Tenant	Denied because non-audit hostname rules target the resource type

Every hostname on a resource must pass. For example, an Ingress with an allowed routing host and a disallowed TLS host is denied. If you narrow the types lists, omitted resource kinds remain unaffected by these rules.

6.2.4 - Storage

Default PersistentVolume isolation and additional volume access through namespace rules.

Storage enforcement is configured under spec.rules[].enforce.storage. Persistent Volumes covers default Tenant isolation and additional volume access for PVs without a Tenant label.

Rules use the shared namespace selection, audiences, conditions, and actions and order. See Reference for a complete Tenant example.

Rule Composition

Actions and order

Storage rules use the existing ordered action evaluation, but affect only additional access to PVs without a Tenant label:

Action	Effect on additional access
`allow`	Grants access when the selector, audience, namespace selection, and all conditions match.
`deny`	Rejects the exception when it is the last matching allow/deny rule. This is the default action.
`audit`	Emits a Kubernetes event for a match; never grants access.

The last matching allow or deny wins. If none grants access, the existing missing-Tenant-label rejection remains. An audit-only rule therefore does not permit an unlabeled PV. Other applicable admission checks must still pass after a storage rule allows access.

Same-Tenant volume access remains available without these rules. A matching rule cannot authorize another Tenant’s labeled PV or a PV whose Tenant label has an empty value. See Ownership and label protection.

Admission scope and limits

Checks apply to PVC creation and updates of an unbound PVC with an explicit spec.volumeName, including the PATCH that sets it during a restore.
The referenced PV must exist. A PV being deleted cannot receive an additional grant.
PVCs without spec.volumeName do not use these additional rules. They do not change dynamic provisioning or how Kubernetes selects a PV.
Already-bound PVC updates keep their existing behavior. These rules do not reauthorize or revoke an established binding.
The grant authorizes the PVC admission request. It does not rewrite the PV’s claimRef, restore labels, move data, or complete binding. RBAC, StorageClass restrictions, other admission policies, and Kubernetes binding requirements continue to apply.

PVC selector mutation runs only on creation. All updates preserve the existing selector, including updates while the claim is still Pending. The PV ownership check still applies when an unbound PVC is updated to reference a volume. See PVC selectors and dynamic provisioning.

Persistent Volumes

Capsule isolates PersistentVolumes (PVs) between Tenants by default. Storage rules let administrators grant additional access to selected PVs that do not yet carry Capsule’s Tenant ownership label. Configure these additional grants under spec.rules[].enforce.storage.volumes.

This supports workflows that patch a destination PersistentVolumeClaim (PVC) before changing the restored PV’s claim reference from a temporary claim to the destination claim. The default isolation described below continues to apply.

Default PersistentVolume isolation

Tenant owners can create PVCs using the StorageClasses available to their Tenant. When a PV’s spec.claimRef points to a PVC in a Tenant namespace, Capsule’s PV controller labels the volume with capsule.clastix.io/tenant: <tenant-name>. The controller resolves the Tenant from the claim’s namespace; PVs themselves are cluster-scoped. This behavior does not require any storage.volumes rules.

For example, a PV claimed in solar-production, a namespace belonging to the solar Tenant, has the following relevant fields. This is an excerpt of the existing PV, not a complete volume definition:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: solar-data
  labels:
    capsule.clastix.io/tenant: solar
spec:
  storageClassName: standard
  persistentVolumeReclaimPolicy: Retain
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    namespace: solar-production
    name: data

Reusing retained volumes

What happens after a PVC is deleted depends on the PV’s reclaim policy. With Retain, the volume and its data remain for manual reclamation; it is not automatically ready for another claim. Follow the Kubernetes reclamation procedure before reusing the storage.

Once the PV is available for reuse, Capsule permits claims from namespaces belonging to solar to reference it. Kubernetes must still be able to bind the claim: the requested capacity, access modes, StorageClass, and other binding requirements must be satisfied. Deleting the old PVC does not remove the PV’s Tenant label or authorize a different Tenant to use it.

A claim from green-energy, a namespace belonging to another Tenant named green, cannot reference solar-data:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data
  namespace: green-energy
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi
  volumeName: solar-data

Capsule rejects that request with a cross-Tenant mount error:

admission webhook "pvc.validating.projectcapsule.dev" denied the request: Preventing a cross-tenant mount for PersistentVolume solar-data

PVC selectors and dynamic provisioning

On PVC creation, Capsule adds the owning Tenant requirement to an existing spec.selector, or creates a selector when spec.volumeName is set. It preserves unrelated selector requirements and replaces conflicting requirements for capsule.clastix.io/tenant with the correct Tenant value.

A new PVC with neither spec.selector nor spec.volumeName receives no Tenant selector, preserving dynamic provisioning. On update, Capsule leaves the selector unchanged, even while the PVC is Pending: Kubernetes treats the selector as immutable after creation. This allows a restore component to set spec.volumeName without Capsule adding a selector change that Kubernetes would reject.

For an explicit spec.volumeName, Capsule also checks the referenced PV’s Tenant label during PVC admission. A PV labeled for the same Tenant passes this ownership check; another Tenant’s PV is rejected. Without an additional grant, a PV with no Tenant label is also rejected. The rules below extend this last case while retaining the default ownership checks.

Additional volume access

Each entry in storage.volumes has a required Kubernetes label selector and an optional name for admission and audit messages. The selector matches labels on the PV named by the PVC’s spec.volumeName.

rules:
  - namespaceSelector:
      matchLabels:
        storage-profile: shared-imports
    enforce:
      action: allow
      storage:
        volumes:
          - name: approved-imports
            selector:
              matchLabels:
                storage.example.com/pool: imports
              matchExpressions:
                - key: storage.example.com/quarantined
                  operator: DoesNotExist

Place this rules block under Tenant.spec. It grants additional access to PVs with the imports pool label and no quarantine label, only from selected namespaces. Without an audience, the rule applies to every caller in that scope.

All requirements in one selector must match. Entries in volumes are alternatives: any matching entry matches that enforcement rule.
selector: {} explicitly matches every eligible PV, including a PV with no labels. Use conditions and an audience to scope a restore exception.
Omitting selector, or setting it to null, is invalid. An omitted or empty volumes list grants no additional access.
A rule supports at most 64 volume entries. Optional names must be unique within the list and use DNS labels of up to 63 characters.

Ownership and label protection

Capsule checks the PV’s capsule.clastix.io/tenant label before evaluating any additional grant:

PV ownership label	Volume access decision
Matches the destination Tenant	Existing access is retained, even if no storage rule matches or a storage rule says `deny`.
Names another Tenant	Denied; no selector or CEL condition can override ownership.
Key exists with an empty value	Denied; this is not treated as an absent label.
Key is absent	Requires an applicable matching `allow` rule.

Selectors only read PV labels. They never add, remove, or overwrite PV labels, and are never copied into PVC.spec.selector. Labels supplied on the PVC cannot satisfy a PV selector. Capsule’s existing PVC Tenant selector enforcement remains in place.

Labels used for authorization must be controlled by administrators or trusted storage components. Tenant actors must not receive RBAC permissions to edit PVs or the rules granting access. A name prefix or a label alone does not establish Tenant ownership.

Conditions

Use enforce.conditions to inspect both the PVC being admitted and the referenced PV. There is no separate conditions field on a volume entry.

CEL variable	Value during additional volume access evaluation
`object`	The incoming PVC, including its destination namespace and `spec.volumeName`.
`request`	Admission metadata, including operation and authenticated caller.
`volume`	A read-only snapshot of the existing PV referenced by `spec.volumeName`.

Check optional fields with has(...), for example volume != null && has(volume.spec.claimRef). Outside additional volume access evaluation, volume is null, including in metadata conditions on a PVC. Keep conditions that depend on volume in a separate storage enforcement block when other policies need independent conditions.

All conditions must be true. A false condition skips the enforcement block; it does not grant access. Evaluation errors reject admission unless another condition in that block is false, following the normal condition evaluation rules.

Reference

Restore through a temporary namespace

For a restore workflow using temporary claims in openshift-adp, the destination PVC may need to reference a PV whose claimRef still names a temporary claim. PVs are cluster-scoped; the namespace in spec.claimRef belongs to that PVC. If that PV has no Tenant label, the default Capsule ownership check rejects the PVC patch. An additional storage rule can authorize this handoff without relaxing access to another Tenant’s labeled PVs.

The following Tenant grants the exception in namespaces labeled storage-profile: restore. It allows service accounts from openshift-adp to update a PVC only when the PV’s temporary claim is in that namespace and its name starts with the destination namespace followed by -.

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  owners:
    - kind: User
      name: solar-owner
  rules:
    - namespaceSelector:
        matchLabels:
          storage-profile: restore
      audience:
        - kind: Group
          name: system:serviceaccounts:openshift-adp
      enforce:
        action: allow
        conditions:
          - name: restore-handoff
            expression: >-
              request.operation == 'UPDATE' &&
              volume != null &&
              has(volume.spec.claimRef) &&
              volume.spec.claimRef.namespace == 'openshift-adp' &&
              volume.spec.claimRef.name.startsWith(object.metadata.namespace + '-')              
        storage:
          volumes:
            - name: temporary-restore-volume
              selector: {}

Replace the owner, staging namespace, and claim naming condition with values from your restore workflow. This example grants the audience to all service accounts in openshift-adp. To scope it to one restoring component, use an audience with kind: ServiceAccount and its full system:serviceaccount:<namespace>:<name> username. Match the identity making the PVC patch, which may differ from the user who submitted a NonAdminRestore.

The prefix check is a naming convention, not proof of ownership. The trusted restore component remains responsible for handing the correct volume to the destination claim. Use the actual temporary claim naming scheme; Capsule does not derive it from a NonAdminRestore object.

If the temporary PV carries an administrator-controlled pool label, replace selector: {} with a matchLabels selector to narrow the grant further. Capsule does not add that label on your behalf.

After the PVC patch is admitted, the restore component can finish the PV handoff. Capsule’s existing PV labeling behavior still applies when the claim reference points to a Tenant namespace. The rule itself does not change labels or claim references.

Request or PV state	Result with this example
Trusted restore caller, selected namespace, matching temporary claim, no Tenant label	Additional access is allowed.
Tenant owner without the restore audience	No additional access.
Namespace without `storage-profile: restore`	No additional access.
PVC creation rather than update	No additional access; the condition requires `UPDATE`.
Missing claim reference or a different temporary namespace/name prefix	No additional access.
PV labeled for another Tenant	Denied regardless of the rule.
PV already labeled for `solar`	Existing access is retained; other checks still apply.

7 - Permits

Request, review, and manage Kubernetes resources and permissions through reusable templates.

Capsule’s Resource Permit API provides approved access to Kubernetes resources. A user creates a namespaced ResourcePermit, Capsule renders the referenced template, an authorized reviewer or the template’s automatic approval policy approves it, and Capsule manages the resulting resources for the approved lifetime.

Resource Permits can create any discoverable Kubernetes type, including cluster-scoped objects. Typical uses include temporary Role and RoleBinding objects for break-the-glass access, access to a custom resource, or a managed service provisioned for a tenant. An effective duration gives the permit an expiry time; without one, it stays active until explicitly expired. Each resource group’s deletion policy determines whether its resources are removed or retained when the permit ends.

API objects

The feature uses three APIs in capsule.clastix.io/v1beta2:

Kind	Short name	Scope	Purpose
`ResourcePermit`	`rp`	Namespace	A user’s request, parameters, reason, schedule, review, and rendered-resource status
`ResourcePermitTemplate`	`rpt`	Namespace	A template available to requests in the same namespace
`GlobalResourcePermitTemplate`	`grpt`	Cluster	A reusable template optionally restricted to selected namespaces

A request explicitly identifies the kind and name of its template:

template:
  kind: GlobalResourcePermitTemplate
  name: temporary-widget-editor

Admission rejects missing templates, local templates from another namespace and global templates that do not select the request namespace. The reference cannot be changed after the request is created.

Lifecycle

flowchart LR
  create[Create] --> prepare[Created: prepare and dry-run]
  prepare -->|Manual approval| review[Requested: review]
  prepare -->|Automatic approval| approved
  prepare -->|Dry-run error| failed[Failed]
  failed -->|Retry| retrying[Retrying]
  retrying -->|Preflight recovered| review
  retrying -->|Activation recovered| approved
  retrying -->|Error| failed
  review -->|Approve| approved[Approved: wait for start time]
  approved --> active[Active]
  approved -->|Activation error| failed
  review -->|Deny| denied[Denied]
  review -->|Cancel| deleted[Deleted]
  active --> expired[Expired]
  expired --> deleted

Pending is reserved for a request held by a system workflow. Automatic approval skips manual review; rendering errors remain visible in Created, while dry-run or activation failures enter the retryable Failed phase. The complete paths are shown in Resource Permits.

When a request becomes active, Capsule uses server-side apply to manage the exact manifests stored in the request status. On expiry it removes or orphans those resources according to their policies. A finalizer provides cascading cleanup even for cluster-scoped targets, for which a namespaced request cannot be an owner.

Minimal example

The following global template grants a named user read-only Pod access in the namespace where the request is created. Save it as template.yaml.

Use a Capsule build that includes these CRDs and configure request and review RBAC. The request namespace must already exist. The template’s execution identity needs permission to manage Roles and RoleBindings there and to grant the listed Pod permissions.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalResourcePermitTemplate
metadata:
  name: temporary-pod-reader
spec:
  defaultDuration: 30m
  maxDuration: 2h
  resources:
    - targets:
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: Role
          metadata:
            name: '{{ .request.name }}-pod-reader'
          rules:
            - apiGroups: [""]
              resources: ["pods", "pods/log"]
              verbs: ["get", "list", "watch"]
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: RoleBinding
          metadata:
            name: '{{ .request.name }}-pod-reader'
          roleRef:
            apiGroup: rbac.authorization.k8s.io
            kind: Role
            name: '{{ .request.name }}-pod-reader'
          subjects:
            - kind: User
              apiGroup: rbac.authorization.k8s.io
              name: '{{ .user }}'
  paramSchema:
    type: object
    required: [user]
    additionalProperties: false
    properties:
      user:
        type: string
        minLength: 1

A user can then request the permit. Save this as request.yaml:

apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePermit
metadata:
  name: investigate-pods
  namespace: solar-uat
spec:
  template:
    kind: GlobalResourcePermitTemplate
    name: temporary-pod-reader
  params:
    user: alice
  reason: Investigate incident INC-1042
  duration: 20m

kubectl apply -f template.yaml
kubectl wait --for=condition=Ready globalresourcepermittemplate/temporary-pod-reader --timeout=60s
kubectl apply -f request.yaml
kubectl wait --for=jsonpath='{.status.phase}'=Requested resourcepermit/investigate-pods -n solar-uat --timeout=60s
kubectl capsule rp review investigate-pods -n solar-uat --approve
kubectl get resourcepermit investigate-pods -n solar-uat

Run template installation as a template administrator, request creation as the requester, and review as an authorized reviewer. Generated Role and RoleBinding names include the permit name so separate permits can coexist in the same namespace.

Capsule overwrites spec.requester with the authenticated user’s name and groups; clients should not rely on supplying that field themselves.

Security model

The Resource Permit API combines Kubernetes authorization with Capsule admission checks:

Kubernetes RBAC controls who may create requests or update their status.
Template approvers and CEL conditions further restrict approval.
The rendered manifests, reviewer identity, template version and resolved ServiceAccount are controller-owned status fields and cannot be forged.
Managed-resource protection can block direct update and delete operations while a request is active.
Context reads and resource operations use one recorded ServiceAccount identity.
Form hints never grant access and must list resources with the requesting user’s credentials.

The template author remains responsible for defining safe parameters, conditions, resource policies, and RBAC for the execution identity.

Continue reading

Use Templates to define inputs, rendering, context, and execution ServiceAccounts. Resource Permits covers lifecycle, approval, commands, and operations. Reusable behavior is documented under concepts:

7.1 - Resource Permit Templates

Define reusable resources, inputs, execution identities, readiness, and limits.

A template describes what Capsule renders for a ResourcePermit, who executes it, and how long the resulting resources may remain active.

Choose the template scope

Use ResourcePermitTemplate when the workflow and its execution identity belong to one namespace. Use GlobalResourcePermitTemplate when platform administrators provide the same workflow to several namespaces.

Behavior	`ResourcePermitTemplate`	`GlobalResourcePermitTemplate`
Scope	Namespaced	Cluster
Request location	Same namespace	Any allowed namespace
ServiceAccount reference	Name only, in the template namespace	Name and namespace
Namespace selectors/status	Not applicable	Supported
Ready condition and observed generation	Supported	Supported

A minimal local template renders its namespaced targets into the ResourcePermit namespace when metadata.namespace is omitted:

apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePermitTemplate
metadata:
  name: temporary-config
  namespace: solar-uat
spec:
  paramSchema:
    type: object
    required: [owner]
    properties:
      owner:
        type: string
  resources:
    - targets:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: temporary-config
          data:
            requestedBy: '{{ .owner }}'

For a global template, namespaceSelectors is an OR list. Omitting it allows requests from every namespace.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalResourcePermitTemplate
metadata:
  name: production-debug
spec:
  namespaceSelectors:
    - matchLabels:
        environment: production
    - matchExpressions:
        - key: resource-permit.example.com/enabled
          operator: In
          values: ["true"]
  resources:
    - targets:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: production-debug

Capsule publishes matching namespaces in .status.namespaces; an unrestricted global template reports "*". Admission rejects a reference when the template is missing, its current generation has not reconciled, or the request namespace is not selected.

Template readiness

Both template kinds expose .status.observedGeneration and a Ready condition in .status.conditions. Successful reconciliation reports Ready=True with reason Succeeded; a failure reports Ready=False with reason Failed and an explanatory message. The condition’s observedGeneration identifies the template generation that was checked. Compare it with .metadata.generation after editing a template.

Template readiness covers static validation of the approval policy, duration limits, parameter schema, and resource definitions. Global templates also resolve namespace selection. On failure, their namespace list is cleared so it cannot grant access from an earlier successful reconciliation. A ready template still requires each permit’s parameters, context reads, rendering, execution permissions, and dry-run to succeed.

For example, an unrestricted global template reports:

status:
  observedGeneration: 1
  namespaces: ["*"]
  conditions:
    - type: Ready
      status: "True"
      reason: Succeeded
      message: reconciled
      observedGeneration: 1
      lastTransitionTime: "2026-09-09T12:00:00Z"

A namespaced template has the same readiness fields without namespaces. Both kinds show Ready and Status columns in kubectl get output:

kubectl get rpt -n solar-uat
kubectl get grpt
kubectl wait --for=condition=Ready rpt/temporary-config -n solar-uat --timeout=60s
kubectl wait --for=condition=Ready grpt/production-debug --timeout=60s

The controller exports readiness gauges for both kinds; see Metrics for names, labels, and example queries.

Define inputs and rendered resources

spec.paramSchema validates ResourcePermit.spec.params before context is loaded or resources are rendered:

paramSchema:
  type: object
  additionalProperties: false
  required: [name, crd]
  properties:
    name:
      type: string
      minLength: 1
      maxLength: 63
    crd:
      type: string

See Parameter Schemas and Dynamic Forms for JSON Schema 2020-12, cross-field CEL, and Kubernetes resource selectors used by clients such as Headlamp.

Every entry in spec.resources is one resource group. Its policy applies to all manifests produced by that entry. A group may contain structured targets, a multi-document template, or both:

resources:
  - policy:
      creation: Owner
      protect: true
      force: false
      deletion: Remove
    targets:
      - apiVersion: v1
        kind: ConfigMap
        metadata:
          name: '{{ .name }}'
        data:
          requestedBy: '{{ .request.username }}'
    template: |
      ---
      apiVersion: v1
      kind: Secret
      metadata:
        name: {{ $.params.name }}
      stringData:
        requestedBy: {{ $.request.username }}

Direct targets expose parameters and loaded context indexes at the root. Generated templates use a structured root:

Data	Direct target	Generated template
Request parameters	`.name`	`$.params.name`
Loaded context	`index .crdInfo 0`	`index $.context.resources.crdInfo 0`
Trusted request data	`.request.username`	`$.request.username`

The trusted request object contains:

Field	Meaning
`name`	ResourcePermit name
`username`	Authenticated requester
`groups`	Authenticated groups as a string array
`timestamp`	Stable ResourcePermit creation time in UTC RFC3339

Use join "," .request.groups for a direct string field and join "," $.request.groups in a generator. The request key is reserved; parameters and context indexes cannot replace these trusted values. Missing template keys are errors, and environment-reading functions such as env are not exposed.

The policy shown above uses all defaults. Its ownership, conflict, protection, and cleanup semantics are documented once in Managed Resources and Server-Side Apply.

Load Kubernetes context

Context references load arbitrary Kubernetes GVKs before every resource group is rendered. Each result is an array, even when selecting one named object. Context indexes must be distinct from parameter names and the reserved request key. Here, crd is the input parameter and crdInfo holds the loaded CRD.

context:
  resources:
    - apiVersion: apiextensions.k8s.io/v1
      kind: CustomResourceDefinition
      name: '{{ .crd }}'
      index: crdInfo
      optional: false
    - apiVersion: v1
      kind: ConfigMap
      namespace: '{{ .managementNamespace }}'
      selector:
        matchLabels:
          access.example.com/source: resource-permit
      index: management

name, namespace, and selector values may use declared parameters. For a namespaced GVK, an omitted namespace means the ResourcePermit namespace; cluster-scoped GVKs ignore it. The execution ServiceAccount performs all reads, and Capsule sanitizes objects before exposing them to templates.

Access a ConfigMap value whose key contains punctuation with index:

{{ index (index .management 0).data "team-a" }}
{{ index (index $.context.resources.management 0).data "team-a" }}

The first expression is for a direct target and the second for a generated template.

Choose the execution identity and limits

Capsule resolves one ServiceAccount for context reads, discovery, apply, reconciliation, dry-run, and cleanup. It always records the result in .status.request.impersonation, including when the controller ServiceAccount is used.

Template	Explicit identity	Configuration fallback	Final fallback
Local	`spec.impersonation.name` in the template namespace	`tenantDefaultServiceAccount` in that namespace	Capsule controller ServiceAccount
Global	`spec.impersonation.name` and `.namespace`	`globalDefaultServiceAccount` in `globalDefaultServiceAccountNamespace`	Capsule controller ServiceAccount

Local and global references respectively look like this:

# ResourcePermitTemplate
spec:
  impersonation:
    name: resource-permit-runner
---
# GlobalResourcePermitTemplate
spec:
  impersonation:
    name: resource-permit-global-runner
    namespace: capsule-system

Configure fallbacks centrally:

apiVersion: capsule.clastix.io/v1beta2
kind: CapsuleConfiguration
metadata:
  name: default
spec:
  impersonation:
    tenantDefaultServiceAccount: resource-permit-runner
    globalDefaultServiceAccount: resource-permit-global-runner
    globalDefaultServiceAccountNamespace: capsule-system

The ServiceAccount needs get for named context objects or list for selected context, plus the lifecycle verbs required by every rendered GVK. This usually means get, create, patch, and update, plus delete for resources that Capsule may remove. Conditional targets need get even when the expression is always false. Cleanup also needs get on each target Namespace; the conditional example limits this with resourceNames. Cluster-scoped inputs and outputs require appropriately scoped RBAC. Keep this execution identity separate from approval identities: the approver authorizes the snapshot; the ServiceAccount executes it.

Capsule performs a server-side dry-run with this identity before the request becomes reviewable. The ServiceAccount is protected from deletion while it is referenced by a ResourcePermit that has not expired, a TenantResource, or a GlobalTenantResource.

Duration and retention limits are also template properties:

defaultDuration: 30m
maxDuration: 2h
keepFor: 7d

defaultDuration applies when the request omits a duration, maxDuration limits the approved value, and keepFor retains an expired request for auditing. Without an effective duration, access remains until explicitly expired.

Conditional resources

Both template kinds support spec.resources[].policy.condition. The shared apply-condition contract defines CEL inputs (object and now), validation, and skipped-target behavior. Conditions run after rendering, independently for each destination.

Field	Decision	Inputs
`spec.approvals.conditions[]`	Approval eligibility; conditions are ORed	`request`, `requester`, `reviewer`
`spec.resources[].policy.condition`	Whether to apply one target’s rendered content	`object`, `now`

A false apply condition is a successful skip. It does not deny approval, keep the request pending, or schedule another attempt once Active. A permit can become Active even when every target is skipped. Use approval conditions to gate the whole request. Apply expressions are plain CEL and cannot reference template parameters or loaded context directly; a false expression cannot suppress a rendering error.

For recurring time-based updates, use a TenantResource or GlobalTenantResource, such as the age-key rotation example. See conditional target lifecycle for preflight, activation, snapshot immutability, retries, and cleanup.

Namespaced example

These examples assume a solar Tenant with a solar-system namespace. Run the ServiceAccount, RBAC, and template setup as a platform administrator. Submit each ResourcePermit as a tenant owner allowed to create requests in that namespace. Use controller and CRD versions that support policy.condition.

Create an execution identity that can manage ConfigMaps only in solar-system. Cleanup also needs permission to read that Namespace; the ClusterRole limits that read to its exact name. Requester permissions remain separate from the execution identity’s permissions.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: condition-runner
  namespace: solar-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: condition-runner
  namespace: solar-system
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: condition-runner
  namespace: solar-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: condition-runner
subjects:
  - kind: ServiceAccount
    name: condition-runner
    namespace: solar-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: solar-condition-namespace-reader
rules:
  - apiGroups: [""]
    resources: ["namespaces"]
    resourceNames: ["solar-system"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: solar-condition-namespace-reader
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: solar-condition-namespace-reader
subjects:
  - kind: ServiceAccount
    name: condition-runner
    namespace: solar-system

Create this template as the administrator, wait for Ready=True, then submit the ResourcePermit as a tenant owner. Save the template and request in separate files so they can be applied with their respective identities. Automatic approval is intentional in this limited ConfigMap example; configure approvers for workflows that need review.

apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePermitTemplate
metadata:
  name: initialize-profile
  namespace: solar-system
spec:
  impersonation:
    name: condition-runner
  approvals:
    auto: true
  defaultDuration: 5m
  keepFor: 10m
  resources:
    - policy:
        creation: Owner
        force: false
        protect: true
        deletion: Remove
        condition: "object == null"
      targets:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: conditional-profile
          data:
            profile: temporary
    - policy:
        condition: "false"
      targets:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: skipped-profile
          data:
            profile: disabled
---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePermit
metadata:
  name: initialize-profile
  namespace: solar-system
spec:
  template:
    kind: ResourcePermitTemplate
    name: initialize-profile

With no preexisting targets, conditional-profile is created and skipped-profile stays absent. If conditional-profile already exists, it is skipped without adoption or modification. Both outcomes allow the request to become Active. keepFor retains the expired request for inspection; it does not extend the target’s five-minute lifetime.

Omitting a target namespace defaults namespaced resources to the permit’s namespace. A namespaced template can only be referenced from its own namespace. The local impersonation reference always resolves there.

Global example

A global template uses the same policy, but its execution identity includes a namespace. namespaceSelectors restrict where a permit can reference it; they do not grant the execution identity permission to read or write resources. The example below reuses the previous ServiceAccount and RBAC and is available only in solar-system.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalResourcePermitTemplate
metadata:
  name: initialize-global-profile
spec:
  impersonation:
    name: condition-runner
    namespace: solar-system
  namespaceSelectors:
    - matchLabels:
        kubernetes.io/metadata.name: solar-system
  approvals:
    auto: true
  defaultDuration: 5m
  keepFor: 10m
  resources:
    - policy:
        creation: Owner
        force: false
        protect: true
        deletion: Remove
        condition: "object == null"
      targets:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: conditional-global-profile
          data:
            profile: temporary
---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePermit
metadata:
  name: initialize-global-profile
  namespace: solar-system
spec:
  template:
    kind: GlobalResourcePermitTemplate
    name: initialize-global-profile

Wait for solar-system in the global template’s status.namespaces before submitting its request. A request from another tenant namespace is rejected by the template’s namespace selection.

Complete CRD access example

This global template loads a CRD and creates a dedicated Role and RoleBinding for the requester. The namespaced manifests omit metadata.namespace, so Capsule places them in the ResourcePermit namespace.

Install the target CRD and label the request namespace before using this example:

kubectl label namespace solar-uat resource-permit.example.com/enabled=true

The execution identity must be able to read CRDs and manage and grant the rendered RBAC permissions. A reviewer must belong to platform-on-call and have permission to update the permit’s status.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalResourcePermitTemplate
metadata:
  name: temporary-crd-editor
spec:
  defaultDuration: 30m
  maxDuration: 4h
  keepFor: 14d
  namespaceSelectors:
    - matchLabels:
        resource-permit.example.com/enabled: "true"
  approvals:
    approvers:
      - kind: Group
        name: platform-on-call
  paramSchema:
    type: object
    additionalProperties: false
    required: [name, crd]
    properties:
      name:
        type: string
      crd:
        type: string
  context:
    resources:
      - apiVersion: apiextensions.k8s.io/v1
        kind: CustomResourceDefinition
        name: '{{ .crd }}'
        index: crdInfo
        optional: false
  resources:
    - policy:
        creation: Owner
        protect: true
        force: false
        deletion: Remove
      template: |
        ---
        apiVersion: rbac.authorization.k8s.io/v1
        kind: Role
        metadata:
          name: {{ $.params.name }}
        rules:
          - apiGroups:
              - {{ (index $.context.resources.crdInfo 0).spec.group }}
            resources:
              - {{ (index $.context.resources.crdInfo 0).spec.names.plural }}
            verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
        ---
        apiVersion: rbac.authorization.k8s.io/v1
        kind: RoleBinding
        metadata:
          name: {{ $.request.name }}
        roleRef:
          apiGroup: rbac.authorization.k8s.io
          kind: Role
          name: {{ $.params.name }}
        subjects:
          - kind: User
            apiGroup: rbac.authorization.k8s.io
            name: {{ $.request.username }}

See Resource Permits for approval, review, activation, expiry, and troubleshooting.

7.2 - Resource Permits

Create, review, activate, expire, and monitor permits for Kubernetes resources.

A ResourcePermit is namespaced. Its namespace is the authorization boundary for creating the request and the default destination for namespaced resources rendered by its template. A request may also create cluster-scoped resources.

Lifecycle

The normal lifecycle is deliberately small:

Phase	Meaning
`Created`	Accepted by admission; Capsule is resolving and rendering the request
`Requested`	Ready for manual review
`Pending`	Reserved for a system-held request
`Denied`	A reviewer denied the request
`Approved`	Approved and waiting for activation
`Active`	The rendered resources are being managed
`Failed`	A resource preflight or activation failed and may be retried
`Retrying`	A caller requested another preflight or activation attempt
`Expired`	Access ended; cleanup and retention apply

Creation

flowchart LR
  create[Create] --> admission[Admission checks]
  admission -->|Rejected| rejected[Not created]
  admission -->|Accepted| created[Created]
  created --> render[Resolve and render]
  render -->|Error| createdError[Created; Ready = False]
  render --> dryrun[Dry-run as execution identity]
  dryrun -->|Manual| requested[Requested]
  dryrun -->|Automatic| approved[Approved]
  dryrun -->|Error| failed[Failed]

Admission first verifies that the referenced template exists, is available in the request namespace, and accepts the parameters, requested duration, start time, and automatic-approval conditions. Capsule then loads context, resolves the execution ServiceAccount, and renders the resources. It performs a server-side dry-run with the resolved execution identity before review. Template lookup, context, or rendering failures remain in Created with Ready=False; a dry-run failure enters Failed. Nothing is applied before approval.

Review

flowchart LR
  requested[Requested] -->|Approve| approved[Approved]
  requested -->|Deny| denied[Denied]
  requested -->|Cancel| deleted[Deleted]

Pending has the same review and cancellation paths, but is reserved for system workflows. With automatic approval, Capsule moves directly from successful rendering to Approved and records a system reviewer.

Access and cleanup

flowchart LR
  approved[Approved] --> active[Active]
  approved -->|Apply error| failed[Failed]
  failed -->|Retry| retrying[Retrying]
  retrying -->|Recovered| resume[Requested or Approved]
  retrying -->|Error| failed
  approved -->|Expire early| expired[Expired]
  active -->|Duration ends or expire| expired[Expired]
  denied[Denied] -->|Expire| expired
  expired --> cleanup[Remove or orphan resources]
  cleanup --> retention[Retention if configured]
  retention --> deleted[Deleted]

Capsule writes Active only after every resource applies successfully. It uses a finalizer for cleanup because a namespaced request may manage cluster-scoped objects. The detailed apply, ownership, and cleanup behavior is described in Managed Resources and Server-Side Apply.

Normal-user deletion follows the lifecycle:

Current state	Deletion
`Created`, `Requested`, or `Pending`	Allowed as cancellation
`Denied`, `Approved`, `Active`, `Failed`, or `Retrying`	Expire first
`Expired` before `keepUntil`	Wait until retention ends
`Expired` without retention, or after `keepUntil`	Allowed

Configured Capsule administrators may request deletion in any phase. Finalizer cleanup still runs for resources Capsule is responsible for removing.

Create and inspect a request

apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePermit
metadata:
  name: gateway-editor-incident-1042
  namespace: solar-uat
spec:
  template:
    kind: GlobalResourcePermitTemplate
    name: temporary-crd-editor
  params:
    name: incident-1042-gateway-editor
    crd: httproutes.gateway.networking.k8s.io
  reason: Restore the public API during INC-1042
  duration: 45m

template.kind must be ResourcePermitTemplate or GlobalResourcePermitTemplate. A local template is resolved in the request namespace; a global template must select that namespace. The reference is immutable.

This request uses the complete CRD access template. It grants access to the authenticated requester through $.request.username; the template does not accept a separate user parameter.

params must satisfy the template schema. duration and startTime are optional, and a start time supplied during creation must be in the future. Capsule replaces spec.requester with the authenticated username and groups, so callers cannot claim a different identity. See Parameter Schemas and Dynamic Forms for JSON Schema, CEL, and Kubernetes-backed form fields.

Before review, Capsule records:

the template kind, name, and resourceVersion;
the resolved execution ServiceAccount, including the controller ServiceAccount when it is the fallback;
the template approval policy used for this request;
the effective duration and start time; and
the concrete rendered manifests and their resource-group policies.

The rendered snapshot in .status.request.resources is what reviewers inspect and what Capsule later applies. Updating the source template does not change an existing request’s resource or approval-policy snapshot.

The following abridged status omits the rendered manifests and processed items:

status:
  request:
    template:
      kind: GlobalResourcePermitTemplate
      name: temporary-crd-editor
      resourceVersion: "18422"
    impersonation:
      name: resource-permit-runner
      namespace: capsule-system
    approvals:
      auto: false
      approvers:
        - kind: Group
          name: platform-on-call
    duration: 45m0s
    startTime: "2026-08-31T10:00:00Z"
    resources: []
  phase: Requested
  transitions:
    - type: Created
      timestamp: "2026-08-31T09:58:00Z"
      actor:
        name: alice
        type: User
      reason: CreatedByUser
    - type: Requested
      timestamp: "2026-08-31T09:58:01Z"
      actor:
        name: alice
        type: User
      reason: PendingReview
  conditions:
    - type: Ready
      status: "True"
      reason: Succeeded
      message: rendered resources are ready for review
      observedGeneration: 1
      lastTransitionTime: "2026-08-31T09:58:01Z"
  processedItems: []
  size: 0

status.transitions is the chronological lifecycle audit trail. Each entry records the entered state as type, its timestamp, compact actor identity (name and type), reason, and optional message. Group claims are deliberately not duplicated into transitions; requester groups remain in spec.requester.groups. conditions describe operational state such as Ready, rather than lifecycle history.

Inspect the complete snapshot and conditions with:

kubectl get resourcepermit gateway-editor-incident-1042 -n solar-uat -o yaml

If template lookup, context loading, rendering, impersonation, dry-run, or apply fails, Capsule sets Ready=False with a reason and message. Typical reasons are TemplateResolutionFailed, TemplateContextFailed, TemplateRenderingFailed, ImpersonationFailed, ResourcesNotReady, ResourceDryRunFailed, and ResourceApplyFailed. A rendering failure remains Created and cannot be approved. A dry-run or activation failure enters Failed and records its stage, retry phase, reason, and message under .status.failure.

kubectl get resourcepermit gateway-editor-incident-1042 -n solar-uat \
  -o jsonpath='{range .status.conditions[*]}{.type}{"="}{.status}{"  "}{.reason}{": "}{.message}{"\n"}{end}'

Review and control a permit

Build the kubectl plugin from the Capsule source tree and put it on PATH:

make kubectl-capsule
export PATH="$PWD/bin:$PATH"

resource-permit, resourcepermit, rp, and permit are equivalent command names. Review interactively, or provide a decision directly:

kubectl capsule rp review gateway-editor-incident-1042 -n solar-uat

kubectl capsule rp review gateway-editor-incident-1042 \
  -n solar-uat --approve \
  --message "Approved for INC-1042" \
  --duration 30m \
  --keep-for 14d

kubectl capsule rp review gateway-editor-incident-1042 \
  -n solar-uat --deny --message "Use the read-only template"

The review table shows each concrete manifest as a row and its resource-group policy in the left column. Reviewers may adjust effective duration, start time, and retention within the template limits.

For scheduled access, supply a future RFC3339 timestamp through spec.startTime or the review command’s --start-time flag. Omit it to allow activation after approval. The CLI’s review command accepts permits in Requested; system workflows managing Pending permits must use the status API.

Approval rules belong to the referenced template:

spec:
  approvals:
    auto: false
    approvers:
      - kind: User
        name: Timmy
      - kind: Group
        name: platform-on-call
    conditions:
      - "request.spec.reason.startsWith('INC-')"
      - "'platform-engineering' in requester.groups"

For manual approval, a non-empty approvers list must match the authenticated reviewer in addition to Kubernetes RBAC. Supported kinds are User, Group, and ServiceAccount. An empty list permits any RBAC-authorized reviewer. The approver list restricts approval, not denial.

Conditions form an OR list: at least one must evaluate to true; an empty list is unconditional. The approver check and condition block are ANDed. Expressions can use the ResourcePermit as request, the authenticated requester as requester, and the authenticated manual reviewer as reviewer.

With auto: true, approvers is ignored and conditions are checked during creation. A non-matching request is rejected rather than left for manual review. Automatic approval without conditions is unconditional and should only be used when every caller who can create the request may safely receive the rendered access.

Clients submit only the desired phase change. Admission reconstructs reviewer identity and lifecycle fields while preserving the requester, rendered snapshot, resolved template version, execution ServiceAccount, and activation timestamps. A denied request cannot later be approved, and Expired is terminal.

Activation normally happens automatically when the approved start time is reached. Authorized callers can also request activation or end access early:

kubectl capsule rp activate gateway-editor-incident-1042 -n solar-uat
kubectl capsule rp expire gateway-editor-incident-1042 -n solar-uat
kubectl capsule rp retry gateway-editor-incident-1042 -n solar-uat

retry is valid only for a Failed request. After the underlying ServiceAccount, RBAC, admission, or target problem is corrected, Capsule repeats the failed preflight or activation path. The requester may request retry or early expiry when Kubernetes RBAC permits the status update.

The commands support Kubernetes impersonation:

kubectl capsule rp review gateway-editor-incident-1042 \
  -n solar-uat --as alice --as-group platform-on-call

Repeat --as-group for multiple groups. A group cannot be supplied without a user unless the loaded kubeconfig already impersonates one.

Conditional target lifecycle

Capsule renders the request and records the concrete targets and policies in status.request.resources.
Preflight evaluates each condition against the live destination and dry-runs allowed writes using the resolved execution ServiceAccount. No target is persisted by preflight.
Activation reevaluates against the live destination and current now, then applies allowed targets. A result can change between preflight and activation.
Active permits keep the reviewed snapshot. Editing the template does not replace that snapshot, and Active permits do not periodically rerender or reevaluate. Explicit retries of a Failed permit use its captured snapshot. During an activation retry, a false condition can still reconcile protection metadata on targets already applied by that permit, using the captured policy. This does not make template policy edits affect an existing permit.
Expiration/deletion cleans up applied targets according to policy.deletion. A target that was never applied by this permit is left untouched. A target applied on an earlier attempt remains eligible for cleanup even if a later attempt skips it.

A changed apply expression requires a new request to capture the updated template. Retrying a Failed permit retains its reviewed expression and targets. Template policy edits do not change an existing request’s snapshot. Examples for both template kinds are in Conditional resources.

Inspect conditions without printing rendered target data, which may contain Secrets:

kubectl get resourcepermit REQUEST -n NAMESPACE -o json \
  | jq '.status | {phase, failure, processedItems}'

Applied targets have a lastApply timestamp. Skipped targets report ConditionNotMet: apply skipped; previously applied targets retain their last successful timestamp, while never-applied targets have none. The condition never suppresses expiration cleanup of targets the permit previously applied.

Operations and troubleshooting

Resource Permits depend on their CRDs, controllers, admission webhooks, and Kubernetes RBAC. Chart RBAC for creating ResourcePermits is opt-in:

helm upgrade --install capsule projectcapsule/capsule \
  --namespace capsule-system \
  --set rbac.resourcepermits.create=true \
  --set-string 'rbac.resourcepermits.labels.rbac\.authorization\.k8s\.io/aggregate-to-admin=true'

The generated role covers resourcepermits and its status subresource. Kubernetes RBAC still determines the namespaces in which a user may create or review requests.

Keep the mutating and validating webhooks for resourcepermits, resourcepermittemplates, and globalresourcepermittemplates enabled. The generic Resource Permit webhook also protects managed resources when a group uses protect: true.

kubectl get validatingwebhookconfigurations,mutatingwebhookconfigurations
kubectl logs -n capsule-system deployment/capsule-controller-manager

Common problems can be narrowed down quickly:

Symptom	Check
Request creation is forbidden	Enable chart RBAC and bind the user in the request namespace
Template reference is rejected	Check kind/name, local namespace, global namespace selection, and observed generation
Template is not ready	Inspect its `.status.conditions`; correct static validation errors or global namespace selection failures
Request is not ready	Inspect `.status.conditions`, `.status.request`, and processed items
Status transition is denied	Use the CLI or a minimal status patch; verify reviewer, CEL, readiness, and duration
Request is `Failed`	Inspect `.status.failure`, correct the reported execution problem, then issue `rp retry` or `rp expire`
Request deletion is denied	Cancel only `Created`/`Requested`/`Pending`; otherwise expire and wait for retention
Managed object cannot be changed	Check whether its resource group uses `protect: true`
Template rejected at `spec.resources[i].policy.condition`	Correct CEL syntax, undeclared variables, or a non-boolean expression such as `42`
Preflight fails with `ResourceDryRunFailed` and `ConditionEvaluationFailed`	Guard missing fields and check timestamp formats; preflight has not persisted targets
Activation fails	Inspect failure and processed-item status; other independent targets may already have been applied
Conditional target read is forbidden	Grant the execution ServiceAccount `get` on that target, even for a false expression
Namespace read is forbidden during cleanup	Retain `get` on the target Namespace until cleanup finishes
Permit is Active but targets were skipped	A false condition is a successful skip; Active permits do not reevaluate when time passes or a template changes

Useful diagnostics:

kubectl get resourcepermit REQUEST -n NAMESPACE -o json \
  | jq '.status | {phase, request, review, failure, transitions, conditions, processedItems}'
kubectl get events -n NAMESPACE \
  --field-selector involvedObject.kind=ResourcePermit,involvedObject.name=REQUEST
kubectl get globalresourcepermittemplate TEMPLATE -o yaml
kubectl get resourcepermittemplate TEMPLATE -n NAMESPACE -o yaml
kubectl get namespace NAMESPACE --show-labels

Configured Capsule administrators may delete ResourcePermits regardless of phase or retention. This is an emergency escape hatch; cleanup still runs when a finalizer is present.

A ResourcePermit never keeps its namespace in Terminating. Once namespace deletion starts, Capsule allows the request to be deleted regardless of lifecycle phase or archive retention, prunes resources covered by the Remove deletion policy, and releases the request finalizer without waiting for keepUntil.

Capsule emits lifecycle events when a request becomes Requested, Approved, Denied, Active, or Expired. Every transition event carries the actor as labels:

event.projectcapsule.dev/actor: alice
event.projectcapsule.dev/actor-kind: User

This makes events selectable by actor, for example:

kubectl get events -n NAMESPACE \
  -l event.projectcapsule.dev/actor=alice,event.projectcapsule.dev/actor-kind=User

Kubernetes label values cannot contain characters such as @ or : and have a 63-character limit. Capsule converts such actor names to a stable, label-safe value with a short hash suffix. The unmodified identity is always available in .status.transitions[].actor. Transition entries remain the durable in-object audit trail and include eventTime after Capsule emits the corresponding event.

For audit purposes, retain the trusted requester, reason and parameters, request snapshot, review decision, transitions, conditions, and managed-resource results. Configure keepFor on templates that require an in-cluster audit window after access ends.

Metrics

The Capsule metrics endpoint exposes the following gauges:

Metric	Labels	Meaning
`capsule_resourcepermit_phase`	`name`, `target_namespace`, `status`	Value `1` for the permit’s current lifecycle phase; `status` contains the phase name
`capsule_resourcepermittemplate_condition`	`name`, `target_namespace`, `condition`	Template readiness; `condition="Ready"` is `1` when true and `0` otherwise
`capsule_globalresourcepermittemplate_condition`	`name`, `condition`	Global template readiness with the same values

The permit phase gauge removes the previous phase series when the phase changes. Template gauges describe the Ready condition, independently of permit lifecycle phases. Their series are removed when the template is deleted and restored when existing templates reconcile after a controller restart. A missing series does not mean a template is ready.

For example, find failed permits and templates that are not ready:

capsule_resourcepermit_phase{status="Failed"} == 1

capsule_resourcepermittemplate_condition{condition="Ready"} == 0

capsule_globalresourcepermittemplate_condition{condition="Ready"} == 0

8 - Replications

Replicate resources across tenants or namespaces

Capsule provides two dedicated Custom Resource Definitions for propagating Kubernetes resources across Tenant Namespaces, covering both the cluster administrator and Tenant owner personas:

GlobalTenantResource: cluster-scoped, managed by cluster administrators. Selects Tenants by label and replicates resources into all matching Tenant Namespaces.
TenantResource: namespace-scoped, managed by Tenant owners. Replicates resources across the Namespaces within a single Tenant.

Both CRDs follow the same structure: resources are defined in spec.resources blocks, reconciled on a configurable resyncPeriod, and support Go-template-based generators for dynamic resource creation.

Per-resource policies

Each spec.resources[] block accepts the same policy as a ResourcePermit template. The policy applies to every rawItems, namespacedItems, and generators result in that block. These are the defaults:

spec:
  resources:
    - policy:
        creation: Owner
        force: false
        protect: true
        deletion: Remove
      rawItems:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: namespace-profile
          data:
            environment: production

An optional policy.condition gates content apply for each destination using CEL. A false condition keeps existing content while policy updates still reconcile on managed targets. See the shared policy reference for ownership, protection, cleanup, conditions, and status, and the age-key rotation example for recurring updates that retain old data.

Deprecated settings and migration

spec.settings.adopt, spec.settings.force, and spec.pruningOnDelete remain accepted for compatibility but are deprecated. The replicationDefaults admission hook converts them on create and update, adding a policy only to blocks that do not already have one:

Deprecated setting	Per-resource policy
`settings.adopt: true`	`creation: Merge`
`settings.adopt: false` or omitted	`creation: Owner`
`settings.force`	`force`, defaulting to `false`
`pruningOnDelete: false`	`deletion: Orphan`
`pruningOnDelete: true` or omitted	`deletion: Remove`

Conversion uses the shared protect: true default, including adopted resources. Set protect: false explicitly when another actor must modify a managed target. The deprecated fields remain on the object; conversion does not remove them.

An explicit policy replaces the deprecated settings for its entire block. After conversion, change spec.resources[].policy to update behavior: changing the old settings does not overwrite an existing policy. Defaults fill omitted fields in an explicit policy rather than inheriting those fields from spec.settings.

Stored objects that have not passed through admission again keep a runtime fallback to their legacy settings, including legacy protection and pruning behavior. Their missing policies are converted the next time they are admitted. Cleanup uses the last successfully reconciled policy even after a block or its parent is removed.

8.1 - GlobalTenantResources

Replicate resources across tenants or namespaces as Cluster Administrator.

Overview

GlobalTenantResource is a cluster-scoped CRD designed for cluster administrators. It lets you automatically replicate Kubernetes resources - such as Secrets, ConfigMaps, or custom resources - into the Namespaces of selected Tenants. Tenant owners cannot create GlobalTenantResource objects; for tenant-scoped replication, see TenantResource.

The diagram below shows that an Administrator can create a GlobalTenantResource. In the GlobalTenantResource spec, an Administrator specifies which resource they would like to replicate, and where this resource should be replicated to. When applied, this resource gets automatically distributed across all Namespaces of the Tenants that are selected in the GlobalTenantResource.

Global Tenant Resource Replication overview

OpenShift: etcd Encryption

If you are running on OpenShift with etcd encryption enabled and replicating ConfigMaps or Secrets, you must exclude the OpenShift storage version migrator from the replication webhook. Without this, the migrator cannot rotate encryption keys. See the OpenShift installation guide for the required configuration.

A common use case is distributing image pull secrets to all Tenants that must use a specific container registry. In the following example, Bill labels two Tenants and then creates a GlobalTenantResource to push the corresponding pull secret into each of their Namespaces automatically.

$ kubectl label tnt/solar energy=renewable
tenant solar labeled

$ kubectl label tnt/green energy=renewable
tenant green labeled

The pull secret already exists in the harbor-system namespace, labelled accordingly:

$ kubectl -n harbor-system get secret --show-labels
NAME                    TYPE     DATA   AGE   LABELS
imagePullSecret   Opaque   1      28s   tenant=renewable

Without automation, these credentials would need to be distributed manually - against the self-service principle of Capsule. Bill solves this with a single GlobalTenantResource:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: renewable-pull-secrets
spec:
  tenantSelector:
    matchLabels:
      energy: renewable
  resyncPeriod: 60s
  resources:
    - namespacedItems:
        - apiVersion: v1
          kind: Secret
          namespace: harbor-system
          selector:
            matchLabels:
              tenant: renewable

Capsule selects all Tenants matching tenantSelector, then replicates every item in namespacedItems into each Namespace belonging to those Tenants. The controller reconciles on the interval defined by resyncPeriod.

Objects managed by this controller can be either created (new objects) or adopted (existing objects). See Object Management in the Advanced section for full details.

Basic Usage

TenantSelector

A block that describes which Tenants the resource should be replicated to. matchLabels and matchExpressions can be used to select the desired Tenants. To select all tenants with the label energy: renewable, use:

  tenantSelector:
    matchLabels:
      energy: renewable

TenantSelector is an optional field. If not set, the resources will be replicated to all tenants.

Resources

Each block accepts a resource policy for creation, protection, SSA conflicts, cleanup, and optional CEL conditions.

A resource block defines what to replicate. Multiple blocks can be stacked in the resources array, each using one or more of the strategies below.

NamespaceSelector

The namespaceSelector field restricts replication to Namespaces matching a label selector. Capsule also protects selected resources from modification by Tenant users via its webhook.

AdditionalMetadata

Use additionalMetadata to attach extra labels and annotations to every generated object. Fast Template values are supported:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-cluster-rbac
spec:
  scope: Tenant
  resources:
    - additionalMetadata:
        labels:
          k8s.company.com/tenant: "{{tenant.name}}"
        annotations:
          k8s.company.com/cost-center: "inv-120"
      generators:
        - missingKey: error
          template: |
            ---
            apiVersion: rbac.authorization.k8s.io/v1
            kind: ClusterRole
            metadata:
              name: tenant:{{$.tenant.metadata.name}}:priority
              labels:
                k8s.company.com/tenant: "test"
            rules:
              - apiGroups: ["scheduling.k8s.io"]
                verbs: ["get"]
                resources: ["priorityclasses"]

When the same label key appears in both additionalMetadata and the template, additionalMetadata takes priority.

The following labels are always stripped because they are reserved for the controller:

capsule.clastix.io/resources
projectcapsule.dev/created-by
capsule.clastix.io/managed-by
projectcapsule.dev/managed-by

NamespacedItems

Reference existing resources for replication across Tenant Namespaces. The controller validates that any resource kind listed here is namespace-scoped; cluster-scoped kinds are rejected with an error.

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 60s
  resources:
    - namespacedItems:

      # Replicate all Configmaps labeled with projectcapsule.dev/replicate: "true"
      - apiVersion: v1
        kind: ConfigMap
        selector:
          matchLabels:
            projectcapsule.dev/replicate: "true"

      # Replicate all Configmaps labeled with projectcapsule.dev/replicate: "true" and in namespace capsule-system
      - apiVersion: v1
        kind: ConfigMap
        namespace: capsule-system
        selector:
          matchLabels:
            projectcapsule.dev/replicate: "true"

      # Replicate Configmap named "logging-config" in namespace capsule-system labeled with projectcapsule.dev/replicate: "true" and in namespace capsule-system
      - apiVersion: v1
        kind: ConfigMap
        name: logging-config
        namespace: capsule-system

Note: Resources with the label projectcapsule.dev/created-by: resources are ignored by namespacedItems to prevent reconciliation loops.

If you try to define a cluster-scoped resource under namespacedItems, the reconciliation will fail immediately:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 60s
  - namespacedItems:
    - apiVersion: addons.projectcapsule.dev/v1alpha1
      kind: SopsProvider
      name: infrastructure-provider
      optional: true

status:
  conditions:
  - lastTransitionTime: "2026-01-15T21:04:15Z"
    message: cluster-scoped kind addons.projectcapsule.dev/v1alpha1/SopsProvider is
      not allowed
    reason: Failed
    status: "False"
    type: Ready

Name

Providing name triggers a GET request for that single resource rather than a LIST. You must also specify namespace when using name in a GlobalTenantResource:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-resource-replications
spec:
  resources:
  - namespacedItems:
    - apiVersion: v1
      kind: ConfigMap
      name: config-namespace
      optional: true
status:
  conditions:
  - lastTransitionTime: "2026-01-15T21:10:17Z"
    message: 'failed to get ConfigMap/config-namespace: an empty namespace may not
      be set when a resource name is provided'
    reason: Failed
    status: "False"
    type: Ready

Namespace

Providing only namespace performs a LIST of all resources of that kind in that namespace:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 60s
  resources:
    - namespacedItems:
      # Fetches all configmaps in the namespace tenants-system
      - apiVersion: v1
        kind: ConfigMap
        namespace: "tenants-system"

      # Fetches specific configmaps matching the selector in the namespaces tenants-system
      - apiVersion: v1
        kind: ConfigMap
        namespace: "tenants-system"
        selector:
          matchLabels:
            projectcapsule.dev/replicate: "true"

Fast Templates are supported for namespace:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 60s
  resources:
    - namespacedItems:
      # Fetch ConfigMaps labeled with the tenant name and replicate them into each Tenant Namespace
      - apiVersion: v1
        kind: Secret
        namespace: "{{tenant.name}}-system"

Note: When using TenantResource instead of GlobalTenantResource, the namespace field has no effect - resources can only be referenced from the Namespace where the TenantResource object was created.

Selector

When using selector, the selector labels are stripped from the replicated objects. This prevents the replicated copy from also matching the source selector, which would cause a circular reconciliation loop.

Source ConfigMap:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  labels:
    projectcapsule.dev/replicate: "true"
  namespace: wind-test
data:
  player_initial_lives: "3"
  ui_properties_file_name: "user-interface.properties"

TenantResource:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: app-config
spec:
  resources:
    - namespacedItems:
      - apiVersion: v1
        kind: ConfigMap
        selector:
          matchLabels:
            projectcapsule.dev/replicate: "true"

Resulting object in wind-prod (notice the absence of projectcapsule.dev/replicate):

apiVersion: v1
data:
  player_initial_lives: "3"
  ui_properties_file_name: "user-interface.properties"
kind: ConfigMap
metadata:
  labels:
    projectcapsule.dev/created-by: resources
    projectcapsule.dev/managed-by: resources
  name: app-config
  namespace: wind-prod
  resourceVersion: "784529"
  uid: 5f10a3f3-863e-4f45-9454-cff8f5bce86a

Fast Templates are supported for selector:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 60s
  resources:
    - namespacedItems:
      # Fetch ConfigMaps labeled with the tenant name and replicate them into each Tenant Namespace
      - apiVersion: v1
        kind: ConfigMap
        selector:
          matchLabels:
            company.com/replicate-for: "{{tenant.name}}"

Raw

Raw items let you define resources inline as standard Kubernetes manifests. Use this when the resource does not yet exist in the cluster, or when you want to define it directly in the spec. Fast Templates are supported.

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 300s
  resources:
    - rawItems:
      - apiVersion: v1
        kind: LimitRange
        metadata:
          name: "{{tenant.name}}-{{namespace}}-resource-constraint"
        spec:
          limits:
          - default: # this section defines default limits
              cpu: 500m
            defaultRequest: # this section defines default requests
              cpu: 500m
            max: # max and min define the limit range
              cpu: "1"
            min:
              cpu: 100m
            type: Container

The following example creates a SopsProvider for each Tenant:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-sops-providers
spec:
  resyncPeriod: 600s
  scope: Tenant
  resources:
    - rawItems:
        - apiVersion: addons.projectcapsule.dev/v1alpha1
          kind: SopsProvider
          metadata:
            name: "{{tenant.name}}-secrets"
          spec:
            keys:
            - namespaceSelector:
                matchLabels:
                  capsule.clastix.io/tenant: "{{tenant.name}}"
            sops:
            - namespaceSelector:
                matchLabels:
                  capsule.clastix.io/tenant: "{{tenant.name}}"

Because Server-Side Apply is used, you only need to specify the fields you want to manage - the full resource spec is not required.

For more advanced templating, consider Generators.

Generators

Generators render one or more Kubernetes objects from a Go template string. The template content must be valid YAML; multi-document output separated by --- is supported. The template engine is based on go-sprout - see available functions.

A simple example that creates a ClusterRole per Tenant:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-cluster-rbac
spec:
  scope: Tenant
  resources:
    - generators:
        - missingKey: error
          template: |
            apiVersion: rbac.authorization.k8s.io/v1
            kind: ClusterRole
            metadata:
              name: tenant:{{$.tenant.metadata.name}}:reader
            rules:
            - apiGroups: [""]
              resources: ["secrets"]
              verbs: ["get", "watch", "list"]

Templates can also produce multiple objects using flow control:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-priority-rbac
spec:
  scope: Tenant
  resources:
    - generators:
        - missingKey: error
          template: |
            {{- range $.tenant.status.classes.priority }}
            ---
            apiVersion: rbac.authorization.k8s.io/v1
            kind: ClusterRole
            metadata:
              name: tenant:{{$.tenant.metadata.name}}:priority:{{.}}
            rules:
              - apiGroups: ["scheduling.k8s.io"]
                resources: ["priorityclasses"]
                resourceNames: ["{{.}}"]
                verbs: ["get"]
            {{- end }}

See Base Context for available template variables. To load additional resources into the template context, see Context in the Advanced section.

Template Snippets

Some snippets that might be useful for certain cases.

Names

Extract the Tenant name:

{{ $.tenant.metadata.name }}

Extract the Namespace name:

{{ $.namespace.metadata.name }}

Foreach Owner

Iterate over all owners of a Tenant:

  {{- range $.tenant.status.owners }}
    {{ .kind }}: {{ .name }}
  {{- end }}

MissingKey

Controls template behaviour when a referenced context key is absent.

Invalid

Continues execution silently. Missing keys render as the string "<no value>".

This definition with the missing context:

kind: GlobalTenantResource
metadata:
  name: missing-key
spec:
  resources:
  - generators:
    - missingKey: invalid
      template: |
        ---
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: show-key
        data:
          value: {{ $.custom.account.name }}

Turns into after templating:

apiVersion: v1
kind: ConfigMap
metadata:
  name: show-key
data:
  value: "<no value>"

Zero

This is the default behavior. Missing keys resolve to the zero value of their type (usually an empty string).

This definition with the missing context:

kind: GlobalTenantResource
metadata:
  name: missing-key
spec:
  resources:
  - generators:
    - missingKey: zero
      template: |
        ---
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: show-key
        data:
          value: {{ $.custom.account.name }}

Turns into after templating:

apiVersion: v1
kind: ConfigMap
metadata:
  name: show-key
data:
  value: ""

Error

Stops execution immediately with an error when a required key is missing.

This definition with the missing context:

kind: GlobalTenantResource
metadata:
  name: missing-key
spec:
  resources:
  - generators:
    - missingKey: error
      template: |
        ---
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: show-key
        data:
          value: {{ $.custom.account.name }}

Will error the GlobalTenantResources:

NAME                    ITEMS   READY   STATUS                                                                                                               AGE
missing-key   6       False   error running generator: template: tpl:8:7: executing "tpl" at <$.namespace.name>: map has no entry for key "name"   9m5s

Reconciliation

Period

GlobalTenantResources reconcile on the interval defined by resyncPeriod. The default is 60s. Capsule does not watch source resources for changes; it reconciles periodically. A very short interval on large clusters with many Tenants and Namespaces can cause performance issues - tune this value accordingly.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: renewable-pull-secrets
spec:
  resyncPeriod: 300s # 5 minutes
  resources:
    - namespacedItems:
        - apiVersion: v1
          kind: Secret
          namespace: harbor-system
          selector:
            matchLabels:
              tenant: renewable

Manual

To trigger an immediate reconciliation, add the reconcile.projectcapsule.dev/requestedAt annotation. The annotation is removed once reconciliation completes, making the process repeatable.

kubectl annotate globaltenantresource renewable-pull-secrets \
  reconcile.projectcapsule.dev/requestedAt="$(date -Iseconds)"

Scope

By default, a GlobalTenantResource replicates resources into every Namespace of the selected Tenants. Setting scope: Tenant changes this to replicate once per Tenant instead.

Possible values:

None: Replicate based on items generated within generators. Essentially not reconciling based on items but running once.
Tenant: Replicate once per Tenant.
Namespace: Replicate into each Namespace of the selected Tenants. (Default)

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-sops-providers
spec:
  resyncPeriod: 60s
  scope: Tenant
  resources:
    - rawItems:
        - apiVersion: addons.projectcapsule.dev/v1alpha1
          kind: SopsProvider
          metadata:
            name: {{tenant.name}}-secrets
          spec:
            keys:
            - namespaceSelector:
                matchLabels:
                  capsule.clastix.io/tenant: {{tenant.name}}
            sops:
            - namespaceSelector:
                matchLabels:
                  capsule.clastix.io/tenant: {{tenant.name}}

Using the scope: Tenant is mainly useful when you want to deploy a cluster-scoped resource once per tenant, such as the SopsProvider above.

Note: When scope: Tenant is set, namespacedItems entries are not processed, since there is no target Namespace in that scope.

Impersonation

GlobalTenantResource has no default ServiceAccount and replicates with the Capsule controller ServiceAccount until you configure one. Impersonation runs the operations under a specific ServiceAccount identity instead, providing a proper audit trail and limiting privilege exposure. You can check which ServiceAccount is currently in use via the object’s status:

kubectl get globaltenantresource custom-cm -o jsonpath='{.status.serviceAccount}' | jq
{
  "name": "capsule",
  "namespace": "capsule-system"
}

To use a different ServiceAccount, set the serviceAccount field on the object:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-resource-replications
spec:
  serviceAccount:
    name: "default"
    namespace: "kube-system"
  resources:
    - namespacedItems:
      - apiVersion: v1
        kind: ConfigMap
        name: "config-namespace"

If the ServiceAccount lacks the required RBAC, replication will fail with a permission error:

  - kind: ConfigMap
    name: game-demo
    namespace: wind-prod
    status:
      created: true
      message: 'apply failed for item 0/raw-0: applying object failed: configmaps
        "game-demo" is forbidden: User "system:serviceaccount:kube-system:default"
        cannot patch resource "configmaps" in API group "" in the namespace "wind-prod"'
      status: "False"
      type: Ready
    tenant: wind
    version: v1

Grant the ServiceAccount the necessary permissions:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: capsule-tenant-replications
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["list", "get", "patch", "create", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: capsule-tenant-replications
subjects:
- kind: ServiceAccount
  name: default
  namespace: kube-system
roleRef:
  kind: ClusterRole
  name: capsule-tenant-replications
  apiGroup: rbac.authorization.k8s.io

Required Permissions

The following permissions are required for each resource type managed by the replication feature:

get (always required)
create (always required)
patch (always required)
delete (always required)
list (required for Namespaced Items and Context)

Missing any of these will cause replication to fail.

Default ServiceAccount

To ensure all GlobalTenantResource objects use a controlled identity by default, configure a cluster-wide default ServiceAccount in the Capsule manager options. Per-object serviceAccount fields override this default.

Read more about Impersonation. You must provide both the name and namespace of the ServiceAccount:

manager:
  options:
    impersonation:
      globalDefaultServiceAccount: "capsule-default-global"
      globalDefaultServiceAccountNamespace: "capsule-system"

The default ServiceAccount must have sufficient RBAC. The following example allows it to manage Secrets and LimitRanges across all Tenants:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: capsule-default-global
rules:
- apiGroups: [""]
  resources: ["limitranges", "secrets"]
  verbs: ["get", "patch", "create", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: capsule-default-global
subjects:
- kind: ServiceAccount
  name: capsule-default-global
  namespace: capsule-system
roleRef:
  kind: ClusterRole
  name: capsule-default-global
  apiGroup: rbac.authorization.k8s.io

If a GlobalTenantResource attempts to manage a resource type not covered by the default ServiceAccount’s ClusterRole, replication will fail with a permissions error:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: default-sa-replication
spec:
  resyncPeriod: 60s
  resources:
    - rawItems:
      - apiVersion: v1
        kind: ConfigMap
        metadata:
          name: game-demo
        data:
          player_initial_lives: "3"
          ui_properties_file_name: "user-interface.properties"

Advanced

This section covers more advanced features of the Replication setup.

Object Management

Set spec.resources[].policy separately for each resource block. The policy applies to all destinations produced by that block, while namespace selectors determine which namespace profiles receive them. See the shared SSA policy reference for the complete contract and migration guidance for existing manifests using spec.settings.

Create

creation: Owner is the default. Capsule creates an absent object and applies future changes to objects created by the applying controller. An unrelated existing target is an ownership error. Capsule records whether each processed target was created or adopted; protect: true blocks direct user updates and deletion while it is managed.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: namespace-profile
spec:
  resources:
    - policy:
        creation: Owner
        force: false
        protect: true
        deletion: Remove
      rawItems:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: namespace-profile
          data:
            environment: production

SSA tracks ownership of individual fields. Multiple managers can contribute non-conflicting fields to a shared object. A conflict is reported through processed-item status; use Force only when Capsule should take ownership of those fields.

Adopt

Use creation: Merge to manage fields on an existing object, or create it if it is absent. For example, this block adds a data entry to an existing ConfigMap:

spec:
  resources:
    - policy:
        creation: Merge
        force: false
        protect: false
        deletion: Remove
      rawItems:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: shared-config
          data:
            capsule-profile: production

This example explicitly disables protection so other actors can continue managing the ConfigMap. Adoption alone does not disable protection; the default remains protect: true. Existing fields owned by other managers remain unless ownership conflicts are intentionally resolved with force.

Pruning

Each block’s policy.deletion controls cleanup when its targets leave scope, including namespace deselection, block removal, or parent deletion:

Policy	Created target	Adopted target
`Remove`	Delete the object	Relinquish Capsule’s applied fields and tracking
`Orphan`	Retain the object and content; remove lifecycle metadata	Retain the object and content; remove lifecycle metadata

Cleanup uses the last successfully reconciled policy. A false apply condition does not suppress cleanup, and changing deletion while the condition is false updates the policy used for subsequent cleanup of already managed targets.

Conditional apply

Use policy.condition to control when rendered content is applied. It is evaluated independently for every destination, with object set to the existing resource or null, and now set to the evaluation timestamp:

policy:
  condition: "object == null"

This creates a target only when absent. Rendering still happens before evaluation. A false result reports ConditionNotMet: apply skipped, preserves content and its last apply timestamp, and keeps already managed targets tracked. Policy changes still take effect: protection metadata is reconciled and the current cleanup policy is retained. Previously unowned targets remain untouched. See apply conditions for validation, error handling, and field-retention semantics, and conditional age-key rotation for a complete stateful generator. A TenantResource can use the same resource block within its tenant.

DependsOn

A GlobalTenantResource can declare dependencies on other GlobalTenantResource objects using dependsOn. The controller will not reconcile the resource until all declared dependencies are in Ready state.

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: gitops-owners
spec:
  resyncPeriod: 60s
  dependsOn:
    - name: custom-cm
  resources:
    - additionalMetadata:
        labels:
          projectcapsule.dev/tenant: "{{tenant.name}}"
      rawItems:
        - apiVersion: capsule.clastix.io/v1beta2
          kind: TenantOwner
          metadata:
            name: "{{tenant.name}}-{{namespace}}"
          spec:
            clusterRoles:
              - capsule-namespace-deleter
              - admin
            kind: ServiceAccount
            name: "system:serviceaccount:{{namespace}}:gitops-reconciler"

We can observe the status of the GlobalTenantResource reflecting, that it depends GlobalTenantResource is not yet ready.

kubectl get globaltenantresource

NAME                           ITEM COUNT   READY   STATUS                            AGE
custom-cm                      6            False   applying of 6 resources failed    12h
gitops-owners                  6            False   dependency custom-cm-2 not found   8h

If a dependency does not exist, we can observe a similar status message when describing the GlobalTenantResource object.

kubectl get globaltenantresource gitops-owners

NAME                           ITEM COUNT   READY   STATUS                            AGE
gitops-owners                  6            False   dependency custom-cm-2 not found   8h

Dependencies are evaluated in the order they are declared in the dependsOn array.

Force

Setting spec.resources[].policy.force: true instructs Capsule to force-apply changes on Server-Side Apply conflicts, claiming field ownership even if another manager already holds it.

This option should generally be avoided. Forcing ownership over a field managed by another operator will almost certainly cause a reconcile war. Only use it in scenarios where you intentionally want Capsule to win ownership disputes.

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-technical-accounts
spec:
  resources:
    - policy:
        creation: Owner
        force: true
        protect: true
        deletion: Remove
      generators:
        - template: |
            ---
            apiVersion: v1
            kind: ConfigMap
            metadata:
              name: shared-config
            data:
              common.conf: |
                {{ toYAML $.tenant.metadata | nindent 4 }}

Context

The context field lets you load additional Kubernetes resources into the template rendering context. This is useful when you need to iterate over existing objects as part of your template logic. To inspect the full context available to a template, you can create a ConfigMap that dumps it:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: tenant-sops-providers
spec:
  resyncPeriod: 600s
  resources:
    - context:
        resources:
          - index: secrets
            apiVersion: v1
            kind: Secret
            namespace: "{{.namespace}}"
            selector:
              matchLabels:
                pullsecret.company.com: "true"
          - index: sa
            apiVersion: v1
            kind: ServiceAccount
            namespace: "{{.namespace}}"

      generators:
        - template: |
            ---
            apiVersion: v1
            kind: ConfigMap
            metadata:
              name: show-context
            data:
              context.yaml: |
                {{- toYAML $ | nindent 4 }}

A useful use case for this can be to add all imagePullSecrets to the default service account, so users don’t have to add these manually to their deployment spec. In the example below, all the secrets with the label energy: non-renewable are selected from the namespace system-imagepullsecrets. The names of these secrets are added to the default service account’s .imagePullSecrets for all matching tenants. Note also the dependsOn, which is a dependency to a different GlobalTenantResource which replicates the secret itself into each tenant namespace:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: imagepullsecrets-default-sa-non-renewable
spec:
  tenantSelector:
    matchLabels:
      energy: non-renewable
  dependsOn:
    - name: replicate-imagepullsecrets-non-renewable
  resyncPeriod: 600s
  resources:
    - policy:
        creation: Merge
        force: false
        protect: false
        deletion: Remove
      context:
        resources:
          - index: secrets
            apiVersion: v1
            kind: Secret
            namespace: "system-imagepullsecrets"
            selector:
              matchLabels:
                imagePullSecret: "true"
                energy: non-renewable
      generators:
        - template: |
            ---
            apiVersion: v1
            kind: ServiceAccount
            metadata:
              name: default
            {{- if $.secrets }}
            imagePullSecrets:
            {{- range $.secrets }}
              - name: {{ .metadata.name }}
            {{- end }}
            {{- end }}

Base Context

The following context is always available in generator templates. The tenant key is always present. The namespace key is only available when the scope is Namespace (the default); it is absent when scope: Tenant is set.

tenant:
    apiVersion: capsule.clastix.io/v1beta2
    kind: Tenant
    metadata:
        creationTimestamp: "2026-02-06T09:54:30Z"
        generation: 1
        labels:
            kubernetes.io/metadata.name: wind
        name: wind
        resourceVersion: "4038"
        uid: 93992a2b-cba4-4d33-9d09-da8fc0bfe93c
    spec:
        additionalRoleBindings:
            - clusterRoleName: view
              subjects:
                - apiGroup: rbac.authorization.k8s.io
                  kind: Group
                  name: wind-users
        owners:
            - clusterRoles:
                - admin
                - capsule-namespace-deleter
              kind: User
              name: joe
        permissions:
            matchOwners:
                - matchLabels:
                    team: devops
                - matchLabels:
                    tenant: wind
    status:
        classes:
            priority:
                - system-cluster-critical
                - system-node-critical
            storage:
                - standard
        conditions:
            - lastTransitionTime: "2026-02-06T09:54:30Z"
              message: reconciled
              reason: Succeeded
              status: "True"
              type: Ready
            - lastTransitionTime: "2026-02-06T09:54:30Z"
              message: not cordoned
              reason: Active
              status: "False"
              type: Cordoned
        namespaces:
            - wind-prod
            - wind-test
        owners:
            - clusterRoles:
                - admin
                - capsule-namespace-deleter
              kind: Group
              name: oidc:org:devops
            - clusterRoles:
                - admin
                - capsule-namespace-deleter
              kind: User
              name: joe
        size: 2
        spaces:
            - conditions:
                - lastTransitionTime: "2026-02-06T09:54:30Z"
                  message: reconciled
                  reason: Succeeded
                  status: "True"
                  type: Ready
                - lastTransitionTime: "2026-02-06T09:54:30Z"
                  message: not cordoned
                  reason: Active
                  status: "False"
                  type: Cordoned
              metadata: {}
              name: wind-test
              uid: 24bb3c33-6e93-4191-8dc6-24b3df7cb1ed
            - conditions:
                - lastTransitionTime: "2026-02-06T09:54:30Z"
                  message: reconciled
                  reason: Succeeded
                  status: "True"
                  type: Ready
                - lastTransitionTime: "2026-02-06T09:54:30Z"
                  message: not cordoned
                  reason: Active
                  status: "False"
                  type: Cordoned
              metadata: {}
              name: wind-prod
              uid: b3f3201b-8527-47c4-928b-ad6ae610e707
        state: Active
namespace:
    apiVersion: v1
    kind: Namespace
    metadata:
        creationTimestamp: "2026-02-06T09:54:30Z"
        labels:
            capsule.clastix.io/tenant: wind
            kubernetes.io/metadata.name: wind-test
        name: wind-test
        ownerReferences:
            - apiVersion: capsule.clastix.io/v1beta2
              kind: Tenant
              name: wind
              uid: 93992a2b-cba4-4d33-9d09-da8fc0bfe93c
        resourceVersion: "3977"
        uid: 24bb3c33-6e93-4191-8dc6-24b3df7cb1ed
    spec:
        finalizers:
            - kubernetes
    status:
        phase: Active

Monitoring

Currently mainly the conditions of GlobalTenantResources are exposed as metrics:

# HELP capsule_global_resource_condition The current condition status of a global tenant resource.
# TYPE capsule_global_resource_condition gauge
capsule_global_resource_condition{condition="Cordoned",name="templated-forbidden-namespace"} 0
capsule_global_resource_condition{condition="Ready",name="templated-forbidden-namespace"} 1

Examples

Different use cases for GlobalTenantResource objects.

Conditional age-key rotation

This namespace profile creates <tenant>-age-keys in every selected namespace and retains each previous private identity in its own Secret data entry. It rotates on the first successful reconciliation at least 30 days after the last rotation. 720h is a fixed 30-day duration, not a calendar month; 30d is not supported by resyncPeriod or CEL’s duration().

The example uses deletion: Orphan, so removing the GTR, its block, or namespace selection retains the Secret and keys while removing Capsule’s lifecycle metadata.

Create the execution identity and grant it Secret access before applying the GTR. This cluster-wide example is for platform administrators; scope the binding to the intended namespaces when distributing only to a fixed set of namespaces.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: gtr-reconciler
  namespace: capsule-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: capsule-age-key-reconciler
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: [""]
    resources: ["namespaces"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: capsule-age-key-reconciler
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: capsule-age-key-reconciler
subjects:
  - kind: ServiceAccount
    name: gtr-reconciler
    namespace: capsule-system

Apply this GTR, then opt in a namespace belonging to a Tenant:

---
# Enable this namespace profile with:
# kubectl label namespace <namespace> projectcapsule.dev/age-keys=enabled
# Uses the gtr-reconciler ServiceAccount and Secret permissions from rbac.yaml.
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: age-key-rotation
spec:
  scope: Namespace
  resyncPeriod: 720h
  serviceAccount:
    name: gtr-reconciler
    namespace: capsule-system
  resources:
    - namespaceSelector:
        matchLabels:
          projectcapsule.dev/age-keys: enabled
      policy:
        creation: Owner
        force: false
        protect: true
        deletion: Orphan
        # First creation, then the first successful reconciliation after 30 days.
        # A missing marker initializes rotation; an invalid marker blocks writes.
        condition: |
          object == null ||
          !has(object.metadata.annotations) ||
          !('keys.example.org/rotated-at' in object.metadata.annotations) ||
          now >= timestamp(object.metadata.annotations['keys.example.org/rotated-at']) + duration('720h')          
      context:
        resources:
          - index: existing
            apiVersion: v1
            kind: Secret
            # Fast-template syntax; the namespace defaults to the selected namespace.
            name: "{{tenant.name}}-age-keys"
            optional: true
      generators:
        - missingKey: error
          template: |
            {{- $name := printf "%s-age-keys" .tenant.metadata.name -}}
            {{- $existing := getResourceByName $name (index . "existing" | default (list)) -}}
            {{- $key := generateAgeKey -}}
            apiVersion: v1
            kind: Secret
            metadata:
              name: {{ $name | quote }}
              annotations:
                keys.example.org/rotated-at: {{ now | date "2006-01-02T15:04:05Z07:00" | quote }}
            type: Opaque
            data:
              # Keep every previous entry under its original name and value.
              {{- range $field, $value := get "data" $existing | default (dict) }}
              {{- if ne $field "recipient" }}
              {{ $field | quote }}: {{ $value | quote }}
              {{- end }}
              {{- end }}
              # Each identity gets a unique entry named after its public recipient.
              {{ printf "%s.agekey" $key.Recipient | quote }}: {{ $key.Identity | b64enc | quote }}
              recipient: {{ $key.Recipient | b64enc | quote }}

kubectl label namespace solar-uat projectcapsule.dev/age-keys=enabled

Context reference names use fast templates: "{{tenant.name}}-age-keys". Generator bodies use Go templates: .tenant.metadata.name. An omitted context namespace resolves to the selected namespace, and optional: true handles the initial missing Secret. Context reads use the execution ServiceAccount. When a conditional generator reads its destination as context, Capsule checks the observed resource version before applying content; concurrent changes trigger a fresh render.

Each identity is stored under <public-recipient>.agekey, preserving its name and value across rotations. data.recipient holds the newest public recipient. When mounted as a volume, each identity becomes a separate file. The generator explicitly loads and renders the previous data entries: a condition does not change SSA field retention. History has no automatic limit and must fit Kubernetes’ Secret size limit.

Rendering precedes condition evaluation, so a candidate key may be generated and discarded on a skip. A missing rotation marker initializes rotation; a malformed timestamp is an evaluation error and prevents the target write. A false condition keeps keys, the rotation marker, and the last content-apply timestamp unchanged. Protection and cleanup-policy changes still reconcile on an already managed Secret.

resyncPeriod controls how often eligibility is checked, with reconciliation jitter; the condition defines the minimum rotation interval. The example uses 720h for both. Use a shorter resync for more frequent eligibility checks. For a quick local demo, use resyncPeriod: 10s and duration('5m') in the condition.

Inspect the result without printing private identities:

kubectl get gtr age-key-rotation -o json \
  | jq '.status | {selectedTenants, size, processedItems, conditions}'
kubectl get secret solar-age-keys -n solar-uat -o json \
  | jq '{rotatedAt: .metadata.annotations["keys.example.org/rotated-at"],
         identityEntries: [.data | keys[] | select(endswith(".agekey"))]}'

A fresh Secret has one .agekey entry. Before the interval elapses, the timestamp and entries stay unchanged. The next eligible successful reconciliation advances the timestamp and adds one entry while preserving every previous entry’s name and value. Each selected namespace, including namespaces in different tenants, has an independent Secret and rotation timestamp.

Ready=True with size: 0 and empty processedItems can mean no namespace matched. selectedTenants alone does not prove that a namespace was selected. Check tenant membership and the namespace’s projectcapsule.dev/age-keys=enabled label. A condition skip on a rendered destination instead appears in processed-item status as ConditionNotMet: apply skipped.

Generate ServiceAccount Tenant Owner per Tenant

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: gitops-reconciler
spec:
  resyncPeriod: 60s
  resources:
    - rawItems:
        - apiVersion: capsule.clastix.io/v1beta2
          kind: TenantOwner
          metadata:
            name: "{{tenant.name}}-{{namespace}}"
          spec:
            clusterRoles:
              - capsule-namespace-deleter
              - admin
            kind: ServiceAccount
            name: "system:serviceaccount:{{namespace}}:gitops-reconciler"

Manage Global Proxy Settings per Tenant

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: capsule-proxy-settings
spec:
  scope: Tenant
  resyncPeriod: 30s
  resources:
    - generators:
        - missingKey: zero
          template: |
            ---
            apiVersion: capsule.clastix.io/v1beta1
            kind: GlobalProxySettings
            metadata:
              name: {{ $.tenant.metadata.name }}-proxy-settings
            spec:
              rules:
              - subjects:
                {{- range $.tenant.status.owners }}
                - kind: {{ .kind }}
                  name: {{ .name }}
                {{- end }}
                clusterResources:
                - apiGroups:
                  - "capsule.clastix.io"
                  resources:
                  - "globalcustomquotas"
                  operations:
                  - List
                  selector:
                    matchLabels:
                      company.com/tenant: {{ $.tenant.metadata.name }}

Collect HTTPRoutes within per Tenant and aggregate to managed gateway

The following example solves a common problem with Gateway-API and Certificate management. Assume you have a managed gateway with a managed cluster-issuer. In this case we can load all the HTTPRoutes and template the corresponding tenant. The following example also allows to modify the EnvoyProxy instance with Tenant Data

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: managed-envoy-gateway
spec:
  scope: Tenant
  resyncPeriod: 30s
  resources:
    - context:
        resources:
          - apiVersion: gateway.networking.k8s.io/v1
            kind: HTTPRoute
            index: https
            selector:
              matchLabels:
                projectcapsule.dev/tenant: "{{tenant.name}}"
      generators:
        - missingKey: zero
          template: |
            {{- $ingressBandwidth := dig "spec" "data" "networking" "ingress" "bandwidth" "" $.tenant }}
            {{- $egressBandwidth := dig "spec" "data" "networking" "egress" "bandwidth" "" $.tenant }}
            {{- $loadBalancerIP := dig "spec" "data" "networking" "ingress" "loadbalancer" "" $.tenant }}
            ---
            apiVersion: gateway.envoyproxy.io/v1alpha1
            kind: EnvoyProxy
            metadata:
              name: tenant-{{ $.tenant.metadata.name }}-gateway
              namespace: tenant-{{ $.tenant.metadata.name }}-system
            spec:
              logging:
                level:
                  default: info
              provider:
                type: Kubernetes
                kubernetes:
                  envoyDeployment:
                    replicas: 2
                    pod:
                      priorityClassName: tenant-critical
                      {{- if or $ingressBandwidth $egressBandwidth }}
                      annotations:
                        {{- with $ingressBandwidth }}
                        kubernetes.io/ingress-bandwidth: {{ . }}
                        {{- end }}
                        {{- with $egressBandwidth }}
                        kubernetes.io/egress-bandwidth: {{ . }}
                        {{- end }}
                      {{- end }}
                  {{- with $loadBalancerIP }}
                  envoyService:
                    loadBalancerIP: {{ . }}
                  {{- end }}            

        - missingKey: zero
          template: |
            ---
            apiVersion: gateway.networking.k8s.io/v1
            kind: Gateway
            metadata:
              name: tenant-{{ $.tenant.metadata.name }}-gateway
              namespace: tenant-{{ $.tenant.metadata.name }}-system
              annotations:
                cert-manager.io/cluster-issuer: managed-cluster-issuer
                cert-manager.io/private-key-size: "4096"
                cert-manager.io/private-key-algorithm: RSA
            spec:
              gatewayClassName: tenants
              infrastructure:
                parametersRef:
                  group: gateway.envoyproxy.io
                  kind: EnvoyProxy
                  name: tenant-{{ $.tenant.metadata.name }}-gateway
              listeners:
                - name: http-challenge
                  port: 80
                  protocol: HTTP
                  allowedRoutes:
                    namespaces:
                      from: Selector
                      selector:
                        matchLabels:
                          capsule.clastix.io/tenant: "{{ $.tenant.metadata.name }}"
                {{- range $_, $http := $.https }}
                  {{- range $i, $hostname := $http.spec.hostnames }}
                - name: {{ $http.metadata.namespace }}-{{ $http.metadata.name }}-{{ $i }}
                  port: 443
                  protocol: HTTPS
                  hostname: {{ $hostname}}
                  tls:
                    mode: Terminate
                    certificateRefs:
                      - group: ''
                        kind: Secret
                        name: {{ $http.metadata.namespace }}-{{ $http.metadata.name }}-{{ $i }}-tls
                  allowedRoutes:
                    namespaces:
                      from: Selector
                      selector:
                        matchLabels:
                           kubernetes.io/metadata.name: "{{ $http.metadata.namespace }}"
                  {{- end }}
                {{- end }}

Generate Cortex/Mimir Overrides per Tenant

This example shows the case when you need to populate content in a subkey, where Server-Side Apply is not sufficient, since it cannot manage specific fields of an object, but only the whole object itself. In this case, we can use a generator to generate the whole content of the subkey based on the Tenant’s data.

With Cortex/Mimir, you can use the overrides field to specify tenant-specific configuration (limits). This example generates a ConfigMap for each Tenant containing its overrides, based on the Tenant’s data, if provided. This is a case, where the scope is set to None, since we don’t want to replicate the generated ConfigMap into each Namespace, but just have it available in a single location for consumption by an external system:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: mimir-overrides-cluster
spec:
  scope: None
  resyncPeriod: 30s
  resources:
    - context:
        resources:
          - apiVersion: capsule.clastix.io/v1beta2
            kind: Tenant
            index: tnts
      generators:
        - missingKey: zero
          template: |
            ---
            apiVersion: v1
            kind: ConfigMap
            metadata:
              name: mimir-overrides
              namespace: observability-system
            data:
              overrides.yaml: |
                overrides:
                {{- range $i, $tnt := $.tnts }}
                  {{ $tnt.metadata.name }}:
                    ingestion_rate: {{ $tnt | dig "spec" "data" "limits" "ingestionRate" 10000 }}
                    ingestion_burst_size: {{ $tnt | dig "spec" "data" "limits" "ingestionBurstSize" 20000 }}
                {{- end }}

8.2 - TenantResources

Replicate resources across a Tenant’s Namespaces as Tenant Owner

Overview

TenantResource is a namespace-scoped CRD that lets Tenant owners automatically replicate Kubernetes resources across all Namespaces in their Tenant - without manual distribution or custom automation. It is the tenant-level counterpart to GlobalTenantResource, which is reserved for cluster administrators.

The diagram below shows that an Administrator or a Tenant Owner can create a TenantResource inside a Tenant. In the TenantResource spec, a user specifies which resource they would like to replicate across the Tenant. When applied, this resource gets automatically distributed across all Namespaces that are part of the Tenant.

Tenant Resource Replication overview

OpenShift: etcd Encryption

Prerequisites

Tenant owners must have RBAC permission to create, update, and delete TenantResource objects. The following ClusterRole aggregates to the admin role, granting all holders permission to manage TenantResource instances:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: allow-tenant-resources
  labels:
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
- apiGroups: ["capsule.clastix.io"]
  resources: ["tenantresources"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

Example

Alice, the project lead for the solar tenant, wants to provision a PostgreSQL database for each production Namespace automatically:

$ kubectl get namespaces -l capsule.clastix.io/tenant=solar --show-labels
NAME           STATUS   AGE   LABELS
solar-1        Active   59s   capsule.clastix.io/tenant=solar,environment=production,kubernetes.io/metadata.name=solar-1,name=solar-1
solar-2        Active   58s   capsule.clastix.io/tenant=solar,environment=production,kubernetes.io/metadata.name=solar-2,name=solar-2
solar-system   Active   62s   capsule.clastix.io/tenant=solar,kubernetes.io/metadata.name=solar-system,name=solar-system

She creates a TenantResource in solar-system:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: solar-db
  namespace: solar-system
spec:
  resyncPeriod: 60s
  resources:
    - additionalMetadata:
        labels:
          "replicated-by": "capsule"
      namespaceSelector:
        matchLabels:
          environment: production
      rawItems:
        - apiVersion: postgresql.cnpg.io/v1
          kind: Cluster
          metadata:
            name: "postgres-{{namespace}}"
          spec:
            description: PostgreSQL cluster for the {{tenant.name}} Project
            instances: 3
            postgresql:
              pg_hba:
                - hostssl app all all cert
            primaryUpdateStrategy: unsupervised
            storage:
              size: 1Gi

Capsule replicates the Cluster resource into every Namespace matching the namespaceSelector. The Namespace where the TenantResource itself lives (solar-system) is automatically excluded, and Capsule injects labels to prevent the TenantResource from propagating into unowned Namespaces.

$ kubectl get clusters.postgresql.cnpg.io -A
NAMESPACE   NAME              AGE   INSTANCES   READY   STATUS                     PRIMARY
solar-1     postgres-solar-1  80s   3           3       Cluster in healthy state   postgresql-1
solar-2     postgres-solar-2  80s   3           3       Cluster in healthy state   postgresql-1

Objects managed by this controller can be either created (new objects) or adopted (existing objects). See Object Management in the Advanced section for full details.

Basic Usage

Resources

Each block accepts a resource policy for creation, protection, SSA conflicts, cleanup, and optional CEL conditions.

A resource block defines what to replicate. Multiple blocks can be stacked in the resources array, each using one or more of the strategies below.

NamespaceSelector

The namespaceSelector field restricts replication to Namespaces matching a label selector. Capsule also protects selected resources from modification by Tenant users via its webhook.

AdditionalMetadata

Use additionalMetadata to attach extra labels and annotations to every generated object. Fast Template values are supported:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-cluster-rbac
spec:
  scope: Tenant
  resources:
    - additionalMetadata:
        labels:
          k8s.company.com/tenant: "{{tenant.name}}"
        annotations:
          k8s.company.com/cost-center: "inv-120"
      generators:
        - missingKey: error
          template: |
            ---
            apiVersion: rbac.authorization.k8s.io/v1
            kind: ClusterRole
            metadata:
              name: tenant:{{$.tenant.metadata.name}}:priority
              labels:
                k8s.company.com/tenant: "test"
            rules:
              - apiGroups: ["scheduling.k8s.io"]
                verbs: ["get"]
                resources: ["priorityclasses"]

When the same label key appears in both additionalMetadata and the template, additionalMetadata takes priority.

The following labels are always stripped because they are reserved for the controller:

capsule.clastix.io/resources
projectcapsule.dev/created-by
capsule.clastix.io/managed-by
projectcapsule.dev/managed-by

NamespacedItems

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 60s
  resources:
    - namespacedItems:

      # Replicate all Configmaps labeled with projectcapsule.dev/replicate: "true"
      - apiVersion: v1
        kind: ConfigMap
        selector:
          matchLabels:
            projectcapsule.dev/replicate: "true"

      # Replicate all Configmaps labeled with projectcapsule.dev/replicate: "true" and in namespace capsule-system
      - apiVersion: v1
        kind: ConfigMap
        namespace: capsule-system
        selector:
          matchLabels:
            projectcapsule.dev/replicate: "true"

      # Replicate Configmap named "logging-config" in namespace capsule-system labeled with projectcapsule.dev/replicate: "true" and in namespace capsule-system
      - apiVersion: v1
        kind: ConfigMap
        name: logging-config
        namespace: capsule-system

Note: Resources with the label projectcapsule.dev/created-by: resources are ignored by namespacedItems to prevent reconciliation loops.

If you try to define a cluster-scoped resource under namespacedItems, the reconciliation will fail immediately:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 60s
  resources:
    - namespacedItems:
      - apiVersion: addons.projectcapsule.dev/v1alpha1
        kind: SopsProvider
        name: infrastructure-provider
        optional: true

status:
  conditions:
  - lastTransitionTime: "2026-01-15T21:04:15Z"
    message: cluster-scoped kind addons.projectcapsule.dev/v1alpha1/SopsProvider is
      not allowed
    reason: Failed
    status: "False"
    type: Ready

Name

Providing name triggers a GET request for that single resource rather than a LIST:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-resource-replications
  namespace: wind-test
spec:
  resyncPeriod: 60s
  resources:
    - namespacedItems:
      # Fetch ConfigMaps labeled with the tenant name and replicate them into each Tenant Namespace
      - apiVersion: v1
        kind: ConfigMap
        name: "logging-config"

This distributes the ConfigMap named logging-config to all other Namespaces of the Tenant that wind-test belongs to.

Fast Templates are supported for name, namespace, and selector.

Namespace

Providing only namespace performs a LIST of all resources of that kind in that namespace:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-resource-replications
spec:
  resources:
  - namespacedItems:
    - apiVersion: v1
      kind: ConfigMap
      name: config-namespace
      optional: true

Fast Templates are supported for the namespace property:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 60s
  resources:
    - namespacedItems:
      # Fetch ConfigMaps labeled with the tenant name and replicate them into each Tenant Namespace
      - apiVersion: v1
        kind: Secret
        namespace: "{{tenant.name}}-system"

Selector

Source ConfigMap:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  labels:
    projectcapsule.dev/replicate: "true"
  namespace: wind-test
data:
  player_initial_lives: "3"
  ui_properties_file_name: "user-interface.properties"

TenantResource:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: app-config
spec:
  resources:
    - namespacedItems:
      - apiVersion: v1
        kind: ConfigMap
        selector:
          matchLabels:
            projectcapsule.dev/replicate: "true"

Resulting object in wind-prod (notice the absence of projectcapsule.dev/replicate):

apiVersion: v1
data:
  player_initial_lives: "3"
  ui_properties_file_name: "user-interface.properties"
kind: ConfigMap
metadata:
  labels:
    projectcapsule.dev/created-by: resources
    projectcapsule.dev/managed-by: resources
  name: app-config
  namespace: wind-prod
  resourceVersion: "784529"
  uid: 5f10a3f3-863e-4f45-9454-cff8f5bce86a

Fast Templates are supported for selector:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 60s
  resources:
    - namespacedItems:
      # Fetch ConfigMaps labeled with the tenant name and replicate them into each Tenant Namespace
      - apiVersion: v1
        kind: ConfigMap
        selector:
          matchLabels:
            company.com/replicate-for: "{{tenant.name}}"

Raw

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-resource-replications
spec:
  resyncPeriod: 300s
  resources:
    - rawItems:
      - apiVersion: v1
        kind: LimitRange
        metadata:
          name: "{{tenant.name}}-{{namespace}}-resource-constraint"
        spec:
          limits:
          - default: # this section defines default limits
              cpu: 500m
            defaultRequest: # this section defines default requests
              cpu: 500m
            max: # max and min define the limit range
              cpu: "1"
            min:
              cpu: 100m
            type: Container

The following example creates a SopsProvider for each Tenant:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-sops-providers
spec:
  resyncPeriod: 600s
  scope: Tenant
  resources:
    - rawItems:
        - apiVersion: addons.projectcapsule.dev/v1alpha1
          kind: SopsProvider
          metadata:
            name: "{{tenant.name}}-secrets"
          spec:
            keys:
            - namespaceSelector:
                matchLabels:
                  capsule.clastix.io/tenant: "{{tenant.name}}"
            sops:
            - namespaceSelector:
                matchLabels:
                  capsule.clastix.io/tenant: "{{tenant.name}}"

Because Server-Side Apply is used, you only need to specify the fields you want to manage - the full resource spec is not required.

For more advanced templating, consider Generators.

Generators

A simple example that creates a ClusterRole per Tenant:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-cluster-rbac
spec:
  resources:
    - generators:
        - missingKey: error
          template: |
            apiVersion: rbac.authorization.k8s.io/v1
            kind: ClusterRole
            metadata:
              name: tenant:{{$.tenant.metadata.name}}:reader
            rules:
            - apiGroups: [""]
              resources: ["secrets"]
              verbs: ["get", "watch", "list"]

Templates can also produce multiple objects using flow control:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-priority-rbac
spec:
  scope: Tenant
  resources:
    - generators:
        - missingKey: error
          template: |
            {{- range $.tenant.status.classes.priority }}
            ---
            apiVersion: rbac.authorization.k8s.io/v1
            kind: ClusterRole
            metadata:
              name: tenant:{{$.tenant.metadata.name}}:priority:{{.}}
            rules:
              - apiGroups: ["scheduling.k8s.io"]
                resources: ["priorityclasses"]
                resourceNames: ["{{.}}"]
                verbs: ["get"]
            {{- end }}

See Base Context for available template variables. To load additional resources into the template context, see Context in the Advanced section.

Template Snippets

Some snippets that might be useful for certain cases.

Names

Extract the Tenant name:

{{ $.tenant.metadata.name }}

Extract the Namespace name:

{{ $.namespace.metadata.name }}

Foreach Owner

Iterate over all owners of a Tenant:

  {{- range $.tenant.status.owners }}
    {{ .kind }}: {{ .name }}
  {{- end }}

MissingKey

Controls template behaviour when a referenced context key is absent.

Invalid

Continues execution silently. Missing keys render as the string "<no value>".

This definition with the missing context:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: missing-key
spec:
  resources:
  - generators:
    - missingKey: invalid
      template: |
        ---
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: show-key
        data:
          value: {{ $.custom.account.name }}

Turns into after templating:

apiVersion: v1
kind: ConfigMap
metadata:
  name: show-key
data:
  value: "<no value>"

Zero

This is the default behavior. Missing keys resolve to the zero value of their type (usually an empty string).

This definition with the missing context:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: missing-key
spec:
  resources:
  - generators:
    - missingKey: zero
      template: |
        ---
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: show-key
        data:
          value: {{ $.custom.account.name }}

Turns into after templating:

apiVersion: v1
kind: ConfigMap
metadata:
  name: show-key
data:
  value: ""

Error

Stops execution immediately with an error when a required key is missing.

This definition with the missing context:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: missing-key
spec:
  resources:
  - generators:
    - missingKey: error
      template: |
        ---
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: show-key
        data:
          value: {{ $.custom.account.name }}

Will error the TenantResources:

NAME                    ITEMS   READY   STATUS                                                                                                               AGE
missing-key   6       False   error running generator: template: tpl:7:13: executing "tpl" at <$.custom.account.name>: map has no entry for key "custom"   9m5s

Reconciliation

Period

TenantResources reconcile on the interval defined by resyncPeriod. The default is 60s. Capsule does not watch source resources for changes; it reconciles periodically. A very short interval on large clusters with many Tenants and Namespaces can cause performance issues - tune this value accordingly.

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: renewable-pull-secrets
  namespace: wind-test
spec:
  resyncPeriod: 300s # 5 minutes
  resources:
    - namespacedItems:
        - apiVersion: v1
          kind: Secret
          namespace: harbor-system
          selector:
            matchLabels:
              tenant: renewable

Manual

To trigger an immediate reconciliation, add the reconcile.projectcapsule.dev/requestedAt annotation. The annotation is removed once reconciliation completes, making the process repeatable.

kubectl annotate tenantresource renewable-pull-secrets -n wind-test \
  reconcile.projectcapsule.dev/requestedAt="$(date -Iseconds)"

Impersonation

Replication runs as the default ServiceAccount of the TenantResource Namespace. Impersonation gives the operations a proper audit trail and limits privilege exposure. You can check which ServiceAccount is currently in use via the object’s status:

kubectl get tenantresource custom-cm -n wind-test -o jsonpath='{.status.serviceAccount}' | jq
{
  "name": "default",
  "namespace": "wind-test"
}

To use a different ServiceAccount, set the serviceAccount field on the object. For TenantResource, only the name is required - the namespace is always inferred from the Namespace the TenantResource resides in:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-resource-replications
  namespace: wind-test
spec:
  serviceAccount:
    name: "tenant-replicator"
  resources:
    - namespacedItems:
      - apiVersion: v1
        kind: ConfigMap
        name: "config-namespace"

If the ServiceAccount lacks the required RBAC, replication will fail with a permission error:

  - kind: ConfigMap
    name: game-demo
    namespace: wind-test
    status:
      created: true
      message: 'apply failed for item 0/raw-0: applying object failed: configmaps
        "game-demo" is forbidden: User "system:serviceaccount:wind-test:tenant-replicator"
        cannot patch resource "configmaps" in API group "" in the namespace "wind-test"'
      status: "False"
      type: Ready
    tenant: wind
    version: v1

Grant the ServiceAccount the necessary permissions:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: capsule-tenant-replications
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["list", "get", "patch", "create", "delete"]
---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-replicator-rbac
spec:
  resyncPeriod: 60s
  resources:
    - rawItems:
      - apiVersion: rbac.authorization.k8s.io/v1
        kind: RoleBinding
        metadata:
          name: wind-replication
        subjects:
          - kind: ServiceAccount
            name: tenant-replicator
            namespace: wind-test
        roleRef:
          kind: ClusterRole
          name: capsule-tenant-replications
          apiGroup: rbac.authorization.k8s.io

Required Permissions

The following permissions are required for each resource type managed by the replication feature:

get (always required)
create (always required)
patch (always required)
delete (always required)
list (required for Namespaced Items and Context)

Missing any of these will cause replication to fail.

Default ServiceAccount

tenantDefaultServiceAccount gives every TenantResource a controlled identity and is set to default. Per-object serviceAccount fields override it. Only the name is required; the namespace is always the one the TenantResource resides in.

Advanced

This section covers more advanced features of the Replication setup.

Object Management

Create

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: namespace-profile
  namespace: solar-uat
spec:
  resources:
    - policy:
        creation: Owner
        force: false
        protect: true
        deletion: Remove
      rawItems:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: namespace-profile
          data:
            environment: production

Adopt

Use creation: Merge to manage fields on an existing object, or create it if it is absent. For example, this block adds a data entry to an existing ConfigMap:

spec:
  resources:
    - policy:
        creation: Merge
        force: false
        protect: false
        deletion: Remove
      rawItems:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: shared-config
          data:
            capsule-profile: production

Pruning

Each block’s policy.deletion controls cleanup when its targets leave scope, including namespace deselection, block removal, or parent deletion:

Policy	Created target	Adopted target
`Remove`	Delete the object	Relinquish Capsule’s applied fields and tracking
`Orphan`	Retain the object and content; remove lifecycle metadata	Retain the object and content; remove lifecycle metadata

Conditional apply

policy:
  condition: "object == null"

DependsOn

A TenantResource can declare dependencies on other TenantResource objects in the same Namespace using dependsOn. The controller will not reconcile the resource until all declared dependencies are in Ready state.

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: gitops-secret
  namespace: wind-test
spec:
  resyncPeriod: 60s
  dependsOn:
    - name: custom-cm
  resources:
    - additionalMetadata:
        labels:
          projectcapsule.dev/tenant: "{{tenant.name}}"
      rawItems:
        - apiVersion: v1
          kind: Secret
          metadata:
            name: myregistrykey
            namespace: awesomeapps
          data:
            .dockerconfigjson: UmVhbGx5IHJlYWxseSByZWVlZWVlZWVlZWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWxsbGxsbGxsbGxsbGxsbGxsbGxsbGxsbGxsbGxsbGx5eXl5eXl5eXl5eXl5eXl5eXl5eSBsbGxsbGxsbGxsbGxsbG9vb29vb29vb29vb29vb29vb29vb29vb29vb25ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubmdnZ2dnZ2dnZ2dnZ2dnZ2dnZ2cgYXV0aCBrZXlzCg==
          type: kubernetes.io/dockerconfigjson

The status reflects whether a dependency is not yet ready:

kubectl get tenantresource -n wind-test

NAME                           ITEM COUNT   READY   STATUS                            AGE
custom-cm                      6            False   applying of 6 resources failed    12h
gitops-secret                  6            False   dependency custom-cm not ready    8h

If a dependency does not exist:

kubectl get tenantresource gitops-secret -n wind-test

NAME                           ITEM COUNT   READY   STATUS                            AGE
gitops-secret                  6            False   dependency custom-cm not found    8h

Dependencies are evaluated in the order they are declared in the dependsOn array.

Force

Setting spec.resources[].policy.force: true instructs Capsule to force-apply changes on Server-Side Apply conflicts, claiming field ownership even if another manager already holds it.

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-technical-accounts
  namespace: wind-test
spec:
  resources:
    - policy:
        creation: Owner
        force: true
        protect: true
        deletion: Remove
      generators:
        - template: |
            ---
            apiVersion: v1
            kind: ConfigMap
            metadata:
              name: shared-config
            data:
              common.conf: |
                {{ toYAML $.tenant.metadata | nindent 4 }}

Context

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-sops-providers
  namespace: wind-test
spec:
  resyncPeriod: 600s
  resources:
    - context:
        resources:
          - index: secrets
            apiVersion: v1
            kind: Secret
            namespace: "{{.namespace}}"
            selector:
              matchLabels:
                pullsecret.company.com: "true"
          - index: sa
            apiVersion: v1
            kind: ServiceAccount
            namespace: "{{.namespace}}"

      generators:
        - template: |
            ---
            apiVersion: v1
            kind: ConfigMap
            metadata:
              name: show-context
            data:
              context.yaml: |
                {{- toYAML $ | nindent 4 }}

A useful use case for this can be to add all imagePullSecrets to the default service account, so users don’t have to add these manually to their deployment spec. In the example below, all the secrets with the label imagePullSecret: "true" are selected from the current Namespace. The names of these secrets are added to the default service account’s .imagePullSecrets for all Namespaces of the Tenant:

---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: imagepullsecrets-default-sa-green
  namespace: wind-test
spec:
  resyncPeriod: 600s
  resources:
    - policy:
        creation: Merge
        force: false
        protect: false
        deletion: Remove
      context:
        resources:
          - index: secrets
            apiVersion: v1
            kind: Secret
            namespace: "{{.namespace}}"
            selector:
              matchLabels:
                imagePullSecret: "true"
      generators:
        - template: |
            ---
            apiVersion: v1
            kind: ServiceAccount
            metadata:
              name: default
            {{- if $.secrets }}
            imagePullSecrets:
            {{- range $.secrets }}
              - name: {{ .metadata.name }}
            {{- end }}
            {{- end }}

Base Context

The following context is always available in generator templates. The tenant key is always present.

tenant:
    apiVersion: capsule.clastix.io/v1beta2
    kind: Tenant
    metadata:
        creationTimestamp: "2026-02-06T09:54:30Z"
        generation: 1
        labels:
            kubernetes.io/metadata.name: wind
        name: wind
        resourceVersion: "4038"
        uid: 93992a2b-cba4-4d33-9d09-da8fc0bfe93c
    spec:
        additionalRoleBindings:
            - clusterRoleName: view
              subjects:
                - apiGroup: rbac.authorization.k8s.io
                  kind: Group
                  name: wind-users
        owners:
            - clusterRoles:
                - admin
                - capsule-namespace-deleter
              kind: User
              name: joe
        permissions:
            matchOwners:
                - matchLabels:
                    team: devops
                - matchLabels:
                    tenant: wind
    status:
        classes:
            priority:
                - system-cluster-critical
                - system-node-critical
            storage:
                - standard
        conditions:
            - lastTransitionTime: "2026-02-06T09:54:30Z"
              message: reconciled
              reason: Succeeded
              status: "True"
              type: Ready
            - lastTransitionTime: "2026-02-06T09:54:30Z"
              message: not cordoned
              reason: Active
              status: "False"
              type: Cordoned
        namespaces:
            - wind-prod
            - wind-test
        owners:
            - clusterRoles:
                - admin
                - capsule-namespace-deleter
              kind: Group
              name: oidc:org:devops
            - clusterRoles:
                - admin
                - capsule-namespace-deleter
              kind: User
              name: joe
        size: 2
        spaces:
            - conditions:
                - lastTransitionTime: "2026-02-06T09:54:30Z"
                  message: reconciled
                  reason: Succeeded
                  status: "True"
                  type: Ready
                - lastTransitionTime: "2026-02-06T09:54:30Z"
                  message: not cordoned
                  reason: Active
                  status: "False"
                  type: Cordoned
              metadata: {}
              name: wind-test
              uid: 24bb3c33-6e93-4191-8dc6-24b3df7cb1ed
            - conditions:
                - lastTransitionTime: "2026-02-06T09:54:30Z"
                  message: reconciled
                  reason: Succeeded
                  status: "True"
                  type: Ready
                - lastTransitionTime: "2026-02-06T09:54:30Z"
                  message: not cordoned
                  reason: Active
                  status: "False"
                  type: Cordoned
              metadata: {}
              name: wind-prod
              uid: b3f3201b-8527-47c4-928b-ad6ae610e707
        state: Active


namespace:
    apiVersion: v1
    kind: Namespace
    metadata:
        creationTimestamp: "2026-02-06T09:54:30Z"
        labels:
            capsule.clastix.io/tenant: wind
            kubernetes.io/metadata.name: wind-test
        name: wind-test
        ownerReferences:
            - apiVersion: capsule.clastix.io/v1beta2
              kind: Tenant
              name: wind
              uid: 93992a2b-cba4-4d33-9d09-da8fc0bfe93c
        resourceVersion: "3977"
        uid: 24bb3c33-6e93-4191-8dc6-24b3df7cb1ed
    spec:
        finalizers:
            - kubernetes
    status:
        phase: Active

Monitoring

Currently mainly the conditions of TenantResources are exposed as metrics:

# HELP capsule_resource_condition The current condition status of a tenant resource.
# TYPE capsule_resource_condition gauge
capsule_resource_condition{condition="Cordoned",name="templated-forbidden-namespace",target_namespace="e2e-tenantresource-ssa-system"} 0
capsule_resource_condition{condition="Ready",name="templated-forbidden-namespace",target_namespace="e2e-tenantresource-ssa-system"} 1

9 - Resource Management

Manage resources across tenants and namespaces, including replication, quotas, and limits with custom quota mechanisms.

Capsule provides three complementary resource-management systems. Choose the system based on whether you want to limit actual usage, allocate capacity before it is used, or calculate a custom quantity from Kubernetes objects.

	GlobalResourceQuota	ResourcePool	CustomQuota / GlobalCustomQuota
Main purpose	Enforce one immediate, shared ceiling on actual usage.	Allocate a fixed amount of capacity to namespaces through claims.	Enforce a limit on a custom quantity calculated from selected objects.
Accounting model	Native Kubernetes `ResourceQuotaSpec`, including compute, storage, object counts, scopes, and scope selectors.	Native Kubernetes resource quantities distributed through `ResourcePoolClaim` objects.	One or more sources using `add`, `sub`, or `count`; values can be extracted with JSONPath and filtered with selectors.
Namespace distribution	Dynamic. All selected namespaces compete for the same available budget.	Explicit. Each namespace receives the capacity assigned by its claims.	`CustomQuota` covers one namespace; `GlobalCustomQuota` aggregates across selected namespaces.
Admission behavior	Atomically reserves usage and denies requests that would exceed the shared hard limit.	Claims reserve pool capacity; the resulting native `ResourceQuota` enforces workload usage in the namespace.	Atomically reserves the calculated usage when the calculation webhook is enabled; otherwise it is observational only.
Best fit	Kubernetes-native compute, memory, storage, and object-count quotas shared across namespaces.	Delegated self-service allocation where teams claim and release portions of platform capacity.	Quantities not covered by native ResourceQuota accounting, custom resource fields, or advanced calculations across multiple sources.

Start with GlobalResourceQuota when the requirement can be expressed with Kubernetes ResourceQuota resources and semantics. It directly limits aggregate usage and safely handles concurrent requests across namespaces.

Use ResourcePool when capacity allocation should be separate from workload consumption. Platform administrators define the available pool, while namespace users claim and release their portion of it.

Use CustomQuota or GlobalCustomQuota when the quantity cannot be represented by native ResourceQuota, or when you need a custom calculation. Custom quotas can extract Kubernetes quantities from arbitrary namespaced resources or CRDs through JSONPath, combine multiple sources, and apply add, sub, or count operations. Use CustomQuota for one namespace and GlobalCustomQuota for an aggregate limit across selected namespaces.

9.1 - GlobalResourceQuota

Share one atomic Kubernetes resource budget across multiple namespaces.

GlobalResourceQuota provides a single Kubernetes ResourceQuota budget shared by a selected set of namespaces.

A standard Kubernetes ResourceQuota is namespaced. Copying the same quota into multiple namespaces gives every namespace the full limit, so the combined usage can grow with the number of namespaces. GlobalResourceQuota keeps the familiar Kubernetes quota API while enforcing one aggregate hard limit across all selected namespaces.

The resource is cluster-scoped and can select any namespace. Selected namespaces do not have to belong to a Capsule Tenant.

Concept

A GlobalResourceQuota consists of:

One or more namespace label selectors.
One native Kubernetes ResourceQuotaSpec.
Aggregate usage and availability in its status.
Per-namespace usage in its status.
An internal QuantityLedger used to serialize concurrent admission.

Capsule creates a managed native ResourceQuota in every selected namespace. The native quotas provide Kubernetes-compatible accounting and status. Capsule’s validating admission webhook evaluates all matching GlobalResourceQuota objects and reserves new usage atomically before allowing the request.

This combination prevents concurrent requests in different namespaces from exceeding the shared limit.

Benefits

Share one hard limit across any number of namespaces.
Use native Kubernetes resource names, scopes, and scope selectors.
Select Tenant namespaces, application namespaces, environments, or arbitrary namespace groups.
Prevent oversubscription during concurrent admission.
Inspect aggregate and per-namespace usage from one cluster-scoped resource.
Monitor limits, usage, availability, conditions, and namespace consumption with Prometheus.
Generate quotas from Capsule Tenant rules where desired.

GlobalResourceQuota

The following quota shares CPU and memory across all namespaces belonging to the green Tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalResourceQuota
metadata:
  name: green-shared-compute
spec:
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: green
  quota:
    hard:
      limits.cpu: "8"
      limits.memory: 16Gi
      requests.cpu: "8"
      requests.memory: 16Gi

Each GlobalResourceQuota represents one native ResourceQuotaSpec. Create multiple GlobalResourceQuota objects when different scopes, selectors, or independently managed limits are required.

Namespace selection

Namespaces are selected through .spec.namespaceSelectors.

Requirements inside one selector are combined with AND. Multiple entries in namespaceSelectors are combined with OR. A namespace matching any entry is included only once.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalResourceQuota
metadata:
  name: production-and-staging
spec:
  namespaceSelectors:
    - matchLabels:
        company.example/environment: production
      matchExpressions:
        - key: company.example/quota-enabled
          operator: In
          values: ["true"]
    - matchLabels:
        company.example/environment: staging
  quota:
    hard:
      requests.cpu: "20"
      requests.memory: 40Gi

Selection behavior:

Omitting namespaceSelectors selects no namespaces.
A selector entry containing an empty matchLabels/matchExpressions selector matches every namespace.
A selector with a nil label selector is ignored.
Terminating namespaces are not included.
Namespace membership is recalculated when namespace labels or the quota selectors change.

Non-Tenant Namespaces

While you can define namespaceSelectors to select any namespace, by default only namespaced resources in Namespaces which are part of a Tenant are only sent to admission. If you want to use GlobalResourceQuota for Namespaces that are not part of a Tenant, you must mainly change the admission configuration to also sent namespaced resources to the GlobalResourceQuota webhook:

webhooks:
  hooks:
    globalresourcequotas:
      enabled: true
      namespaceSelector:
        matchExpressions:
          - key: capsule.clastix.io/tenant
            operator: Exists
          - key: kubernetes.io/metadata.name
            operator: In
            values:
              - tenants-shared-system

While this may be cumbersome, it was the best solution to avoid implementing a potential cluster DoS attack vector. Only then calculations are sent to the admission webhook for Namespaces that are not part of a Tenant.

spec:
  namespaceSelectors:
    - matchExpressions:
        - key: kubernetes.io/metadata.name
          operator: In
          values:
            - tenants-shared-system

Selecting Tenant namespaces

Capsule labels Tenant namespaces with the Tenant name. The compatibility label can be used directly:

spec:
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar

Multiple Tenants can share one quota:

spec:
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
    - matchLabels:
        capsule.clastix.io/tenant: wind

Quota resources

.spec.quota is a Kubernetes ResourceQuotaSpec. The same hard-resource names and scope rules used by Kubernetes apply.

Compute resources

spec:
  namespaceSelectors:
    - matchLabels:
        company.example/team: payments
  quota:
    hard:
      requests.cpu: "16"
      requests.memory: 32Gi
      limits.cpu: "32"
      limits.memory: 64Gi
      pods: "100"

Pod requests and limits are calculated using the Kubernetes pod resource helpers. This includes:

Regular containers.
Init containers, including restartable sidecars.
Pod overhead.
Pod-level CPU and memory resources when supported by the Kubernetes API server.
Pod lifecycle behavior, such as excluding terminal Pods from compute usage while retaining count/pods.

Pod-level resources can be placed on a Pod template:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: solar-production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      resources:
        requests:
          cpu: 100m
          memory: 256Mi
        limits:
          cpu: "1"
          memory: 1Gi
      containers:
        - name: nginx
          image: nginx:1.27

Kubernetes must support and enable Pod-level resources. If the API server removes the field because the feature is unavailable, quota validation falls back to Kubernetes’ historical per-container CPU and memory requirements.

Ephemeral storage

Ephemeral-storage requests and limits are accounted like native Kubernetes Pod quotas:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalResourceQuota
metadata:
  name: shared-ephemeral-storage
spec:
  namespaceSelectors:
    - matchLabels:
        company.example/storage-budget: shared
  quota:
    hard:
      ephemeral-storage: 100Gi
      requests.ephemeral-storage: 100Gi
      limits.ephemeral-storage: 200Gi

Define ephemeral-storage requests and limits on containers:

apiVersion: v1
kind: Pod
metadata:
  name: worker
  namespace: solar-production
spec:
  containers:
    - name: worker
      image: example.com/worker:latest
      resources:
        requests:
          ephemeral-storage: 2Gi
        limits:
          ephemeral-storage: 4Gi
      volumeMounts:
        - name: cache
          mountPath: /cache
  volumes:
    - name: cache
      emptyDir:
        sizeLimit: 8Gi

An emptyDir.sizeLimit is a volume limit, not a Pod resource request. It does not add 8Gi to ResourceQuota usage. Configure container requests.ephemeral-storage when storage must be reserved and counted.

Object counts

Both legacy core-resource names and generic object-count names are supported.

spec:
  quota:
    hard:
      pods: "100"
      services: "20"
      secrets: "50"
      count/configmaps: "50"
      count/deployments.apps: "30"
      count/horizontalpodautoscalers.autoscaling: "10"

Generic count syntax is:

count/<plural-resource-name>.<api-group>

The API group is omitted for core resources:

count/configmaps
count/pods

For grouped or custom resources, include the group:

count/deployments.apps
count/jobs.batch
count/widgets.platform.example.com

Object creation is reserved through the same atomic ledger as compute resources. For example, if services: 5 is shared by two namespaces and ten Services are created concurrently, only five admissions can succeed.

Persistent storage

PVC accounting includes the native Kubernetes quota resource names:

spec:
  quota:
    hard:
      persistentvolumeclaims: "20"
      requests.storage: 500Gi
      fast.storageclass.storage.k8s.io/persistentvolumeclaims: "10"
      fast.storageclass.storage.k8s.io/requests.storage: 250Gi

PVC storage-class resources, allocated storage, resize status, and supported VolumeAttributesClass scopes follow Kubernetes ResourceQuota behavior.

Scopes and scope selectors

Native quota scopes can be used without Capsule-specific syntax:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalResourceQuota
metadata:
  name: high-priority-compute
spec:
  namespaceSelectors:
    - matchExpressions:
        - key: capsule.clastix.io/tenant
          operator: Exists
  quota:
    hard:
      requests.cpu: "20"
      requests.memory: 40Gi
    scopeSelector:
      matchExpressions:
        - scopeName: PriorityClass
          operator: In
          values: ["high"]

Pod scopes such as Terminating, NotTerminating, BestEffort, NotBestEffort, PriorityClass, and CrossNamespacePodAffinity are evaluated before usage is reserved.

Operating

Status

The status exposes the selected namespaces and observed usage:

status:
  observedGeneration: 1
  namespaceCount: 2
  namespaces:
    - solar-production
    - solar-staging
  total:
    hard:
      requests.cpu: "8"
      requests.memory: 16Gi
    used:
      requests.cpu: "3"
      requests.memory: 6Gi
    available:
      requests.cpu: "5"
      requests.memory: 10Gi
  namespaceUsage:
    solar-production:
      used:
        requests.cpu: "2"
        requests.memory: 4Gi
    solar-staging:
      used:
        requests.cpu: "1"
        requests.memory: 2Gi
  conditions:
    - type: Ready
      status: "True"
      reason: Succeeded
      message: reconciled

status.total.used is observed native usage. Very recent admissions may still exist only as ledger reservations, so admission can correctly reject new work before the observed status or metrics reach the hard limit.

Useful commands:

kubectl get globalresourcequotas
kubectl get globalquota green-shared-compute -o yaml
kubectl get resourcequotas -A \
  -l projectcapsule.dev/global-resource-quota=green-shared-compute
kubectl get quantityledgers -n capsule-system \
  -l projectcapsule.dev/global-resource-quota=green-shared-compute \
  -o yaml

Managed native ResourceQuota objects and QuantityLedgers are implementation details. Do not edit or delete them directly.

Atomic admission

Native ResourceQuota.status.used is eventually consistent. Reading the status and then allowing a request is insufficient because multiple requests can observe the same available capacity.

Global quota admission uses reservations:

The webhook finds every GlobalResourceQuota matching the request namespace.
The incoming object is decoded and evaluated once.
Only resources present in each quota’s hard list are considered.
For updates, only the positive usage delta is reserved.
The webhook atomically adds the delta to the quota’s QuantityLedger using Kubernetes optimistic concurrency.
Admission is denied if observed usage plus active reservations would exceed any hard limit.
The controller removes reservations as native ResourceQuota.status.used catches up.

This process is applied to every matching GlobalResourceQuota. If any matching quota rejects the request, admission is denied and reservations already made for that request are rolled back.

Dry-run requests are evaluated but do not persist reservations.

Readiness and failure behavior

Admission fails closed while a matching quota is not ready. The ledger must:

Refer to the current GlobalResourceQuota UID.
Have observed the current quota generation.
Contain the request namespace.
Be initialized from every selected native ResourceQuota.

The chart enables the webhook with failurePolicy: Fail by default:

webhooks:
  hooks:
    globalresourcequotas:
      enabled: true
      failurePolicy: Fail

The webhook must observe namespaced create and update requests across API groups. Narrowing its webhook rules, namespace selector, object selector, or match conditions can create accounting gaps.

Reservations expire after two minutes if native usage never appears, for example when a later admission plugin rejects the object. A ledger accepts at most 1024 active reservations at one time.

Updating a quota

Hard limits can be increased normally.

A hard resource cannot be reduced below currently allocated usage. It also cannot be removed while observed usage or active reservations for that resource are non-zero. Release the resources first, wait for status reconciliation, and then reduce or remove the limit.

When selectors or hard limits change, admission waits for the ledger to observe the new generation and selected namespace set. This prevents requests from being admitted against stale quota state.

If several GlobalResourceQuota objects select the same namespace, all matching quotas apply. A request must satisfy every matching hard limit and scope.

Monitoring

The following Prometheus metrics are exposed:

Metric	Labels	Description
`capsule_global_resource_quota_condition`	`global_resource_quota`, `condition`	Current condition status, where true is `1`.
`capsule_global_resource_quota_limit`	`global_resource_quota`, `resource`	Shared hard limit.
`capsule_global_resource_quota_usage`	`global_resource_quota`, `resource`	Observed aggregate usage.
`capsule_global_resource_quota_available`	`global_resource_quota`, `resource`	Observed available capacity.
`capsule_global_resource_quota_usage_percentage`	`global_resource_quota`, `resource`	Aggregate usage as a percentage of the hard limit.
`capsule_global_resource_quota_namespace_usage`	`global_resource_quota`, `target_namespace`, `resource`	Observed usage per namespace.
`capsule_global_resource_quota_namespace_usage_percentage`	`global_resource_quota`, `target_namespace`, `resource`	Namespace usage as a percentage of the shared limit.

Example alerts:

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: capsule-global-resource-quota-alerts
spec:
  groups:
    - name: capsule-global-resource-quotas.rules
      rules:
        - alert: CapsuleGlobalResourceQuotaHighUsage
          expr: capsule_global_resource_quota_usage_percentage > 90
          for: 10m
          labels:
            severity: warning
          annotations:
            summary: Global resource quota usage is high
            description: >-
              Resource {{ $labels.resource }} in GlobalResourceQuota
              {{ $labels.global_resource_quota }} is at {{ $value }} percent.              

        - alert: CapsuleGlobalResourceQuotaNotReady
          expr: >-
            capsule_global_resource_quota_condition{condition="Ready"} == 0            
          for: 10m
          labels:
            severity: warning
          annotations:
            summary: Global resource quota is not ready
            description: >-
              GlobalResourceQuota {{ $labels.global_resource_quota }}
              has not been ready for ten minutes.

Migration

Troubleshooting

`QuantityLedger ... is not initialized`

The controller is waiting for every selected native ResourceQuota to report its hard and used status. Check:

kubectl get globalquota <name> -o yaml
kubectl get resourcequotas -A \
  -l projectcapsule.dev/global-resource-quota=<name>
kubectl get quantityledgers -n capsule-system \
  -l projectcapsule.dev/global-resource-quota=<name> \
  -o yaml

Also verify that Capsule can list the selected namespaces and reconcile ResourceQuota status.

`resource exceeds GlobalResourceQuota`

The denial lists only the resource limits that the request would exceed. Quantities use the same canonical formatting as Kubernetes, for example:

resource exceeds GlobalResourceQuota "solar-shared-compute": limits.cpu (requested=1, current=8, projected=9, hard=8, exceededBy=1)

requested is the additional usage introduced by this admission request.
current is the usage already accounted for, including active admission reservations.
projected is current + requested.
hard is the configured limit.
exceededBy is projected - hard.

Because current includes in-flight reservations, it can temporarily be higher than status.total.used.

`must specify requests.cpu ... for: <container>`

Kubernetes historically requires every regular and init container to declare CPU or memory resources when those resources are tracked by quota.

When supported Pod-level resources are present, Capsule follows Kubernetes and skips that legacy per-container requirement. Inspect the Pod or the controller’s stored Pod template:

kubectl get deployment <name> -n <namespace> -o yaml
kubectl get replicaset -n <namespace> -l app=<label> -o yaml

An older ReplicaSet can continue reporting failed Pod creation events after a Deployment is updated with Pod-level resources. Confirm that the ReplicaSet named in the event actually contains .spec.template.spec.resources.

A deleted object still appears in usage

Deletes release capacity after native ResourceQuota status observes the deletion. The delay is normally short but is eventually consistent.

A namespace is unexpectedly selected

Inspect all entries under namespaceSelectors. Entries are ORed, and an empty label selector matches every namespace.

9.2 - Resource Pools

Resource Pools are our answer to manage resources in a multi-tenant Kubernetes cluster. Strategies on granting quotas on tenant-basis

Concept

resourcepools

Benefits

Shifting left now comes to Resource-Management. From the perspective of Cluster-Administrators you just define the Quantity and the Audience for Resources. The rest is up to users managing these namespaces (audience).
Better automation options and integrations. One important aspect for us is, how we still can be beneficial with concepts like VClusters (VCluster/K3K) or CPs as pods (Kamaji). We think with this solution we have found a way to make capsule still beneficial and even open new use-cases for larger Kubernetes platforms.
Enables more use-cases and provides more flexibility than standard ResourceQuotas or our previous ResourceQuota-Implementation. Autobalancing is no longer given by default, however can be implemented according to your platform’s needs see future ideas.

ResourcePool

ResourcePools allow you to define a set of resources, similar to how ResourceQuotas work. ResourcePools are defined at the cluster scope and should be managed by cluster administrators. However, they provide an interface where cluster administrators can specify from which namespaces resources in a ResourcePool can be claimed. Claiming is done via a namespaced CRD called ResourcePoolClaim.

It is then up to the group of users within those namespaces to manage the resources they consume per namespace. Each ResourcePool provisions a ResourceQuota into all the selected namespaces. Essentially, when ResourcePoolClaims are assigned to a ResourcePool, they stack additional resources on top of that ResourceQuota, based on the namespace from which the ResourcePoolClaim was created.

You can create any number of ResourcePools for any kind of namespace; they do not need to be part of a Tenant. Note that the usual ResourceQuota mechanisms apply when, for example, the same resources are defined in multiple ResourcePools for the same namespaces (e.g., the lowest defined quota for a resource is always considered).

apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: example
spec:
  quota:
    hard:
      limits.cpu: "2"
      limits.memory: 2Gi
      requests.cpu: "2"
      requests.memory: 2Gi
      requests.storage: "5Gi"
  selectors:
  - matchLabels:
      capsule.clastix.io/tenant: example

Selection

The selection of namespaces is done via labels, you can define multiple independent LabelSelectors for a ResourcePool. This gives you a lot of flexibility if you want to span over different kind of namespaces (eg. all namespaces of multiple Tenants, System Namespaces, stages of Tenants etc.)

Here’s an example of a simple Selector:

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: solar
spec:
  quota:
    hard:
      limits.cpu: "2"
      limits.memory: 2Gi
  selectors:
  - matchLabels:
      capsule.clastix.io/tenant: solar

This will select all the namespaces, which are part of the Tenant solar. Each statement under selectors is treated independent, so for example this is how you can select multiple Tenant’s namespaces:

apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: green
spec:
  quota:
    hard:
      limits.cpu: "2"
      limits.memory: 2Gi
  selectors:
  - matchLabels:
      capsule.clastix.io/tenant: solar
  - matchLabels:
      capsule.clastix.io/tenant: wind

Quota

Nothing special here, just all the fields you know from ResourceQuotas. The amount defined in quota.hard represents the total resources which can be claimed from the selected namespaces. Through claims the ResourceQuota is then increased or decreased. Note the following:

You can’t decrease the .spec.quota.hard if the current allocation from claims is greater than the new decreased number. You must first release claims, to free up that space.
You can decrease or remove resources, if they are unused (0)

Other than that, you can use all the fields from ResourceQuotas

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: best-effort-pool
spec:
  selectors:
  - matchExpressions:
    - { key: capsule.clastix.io/tenant, operator: Exists }
  quota:
    hard:
      cpu: "1000"
      memory: "200Gi"
      pods: "10"
    scopeSelector:
      matchExpressions:
      - operator: In
        scopeName: PriorityClass
        values:
        - "best-effort"

Each ResourcePool is representative for one ResourceQuota. In contrast to the old implementation, where multiple ResourceQuotas could have been defined in a slice. So if you eg. want to use different scopeSelectors or similar, you should create a new ResourcePool for each.

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: gold-storage
spec:
  selectors:
  - matchExpressions:
    - { key: company.com/env, operator: In, values: [prod, pre-prod] }
  quota:
    hard:
      requests.storage: "10Gi"
      persistentvolumeclaims: "10"
    scopeSelector:
      matchExpressions:
      - operator: In
        scopeName: VolumeAttributesClass
        values: ["gold"]

Defaults

Defaults can contain resources, which are not mentioned in the Quota of a ResourcePool. This is mainly to allow you, to block resources for example:

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: example
spec:
  defaults:
    requests.storage: "0Gi"
  quota:
    hard:
      limits.cpu: "2"
      limits.memory: 2Gi
      requests.cpu: "2"
      requests.memory: 2Gi
      requests.storage: "5Gi"
  selectors:
  - matchLabels:
      capsule.clastix.io/tenant: example

This results in a ResourceQuota from this pool in all selected, which blocks the allocation of requests.storage:

NAME                     AGE   REQUEST                 LIMIT
capsule-pool-example     3s    requests.storage: 0/0

If no Defaults are defined, the ResourceQuota for the ResourcePool is still provisioned but it’s .spec.hard is empty.

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: example
spec:
  quota:
    hard:
      limits.cpu: "2"
      limits.memory: 2Gi
      requests.cpu: "2"
      requests.memory: 2Gi
      requests.storage: "5Gi"
  selectors:
  - matchLabels:
      capsule.clastix.io/tenant: example

This allows users to essentially schedule anything in the namespace:

NAME                     AGE     REQUEST           LIMIT
capsule-pool-exmaple     2m47s

To prevent this, you might consider using the DefaultsZero option. This option can also be combined with setting other defaults, not part of the .spec.quota.hard. Here we are additionally restricting the creation of persistentvolumeclaims:

apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: example
spec:
  defaults:
    "count/persistentvolumeclaims": 3
  config:
    defaultsZero: true
  quota:
    hard:
      limits.cpu: "2"
      limits.memory: 2Gi
      requests.cpu: "2"
      requests.memory: 2Gi
      requests.storage: "5Gi"
  selectors:
  - matchLabels:
      capsule.clastix.io/tenant: example

Results in:

NAME                     AGE   REQUEST                                                                                             LIMIT
capsule-pool-example     10h   count/persistentvolumeclaims: 0/3, requests.cpu: 0/0, requests.memory: 0/0, requests.storage: 0/0   limits.cpu: 0/0, limits.memory: 0/0

Options

Options that can be defined on a per-ResourcePool basis and influence the general behavior of the ResourcePool.

OrderedQueue

When ResourcePoolClaims are allocated to a pool, they are placed in a queue. The pool attempts to allocate claims in the order of their creation timestamps. However, even if a claim was created earlier, if it requests more resources than are currently available, it will remain in the queue. Meanwhile, a lower-priority claim that fits within the available resources may still be allocated, despite its lower priority.

Enabling this option enforces strict ordering: claims cannot be skipped, even if they block other claims from being fulfilled due to resource exhaustion. The CreationTimestamp is strictly respected, meaning that once a claim is queued, no subsequent claim can bypass it, even if it requires fewer resources.

Default: false

DefaultsZero

Sets the default values for the ResourceQuota created for the ResourcePool. When enabled, all resources in the quota are initialized to zero. This is useful in scenarios where users should not be able to consume any resources without explicitly creating claims. In such cases, it makes sense to initialize all available resources in the ResourcePool to 0.

Default: false

DeleteBoundResources

By default, when a ResourcePool is deleted, any ResourcePoolClaims bound to it are only disassociated, not deleted. Enabling this option ensures that all ResourcePoolClaims in a bound state are deleted when the corresponding ResourcePool is deleted.

Default: false

LimitRanges

When defining ResourcePools you might want to consider distributing LimitRanges via Tenant Replications:

apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: example
  namespace: solar-system
spec:
  resyncPeriod: 60s
  resources:
    - namespaceSelector:
        matchLabels:
          capsule.clastix.io/tenant: example
      rawItems:
        - apiVersion: v1
          kind: LimitRange
          metadata:
            name: cpu-resource-constraint
          spec:
            limits:
            - default: # this section defines default limits
                cpu: 500m
              defaultRequest: # this section defines default requests
                cpu: 500m
              max: # max and min define the limit range
                cpu: "1"
              min:
                cpu: 100m
              type: Container

ResourcePoolClaims

ResourcePoolClaims declared claims of resources from a single ResourcePool. When a ResourcePoolClaim is successfully bound to a ResourcePool, it’s requested resources are stacked to the ResourceQuota from the ResourcePool in the corresponding namespaces, where the ResourcePoolClaim was declared. So the declaration of a ResourcePoolClaim is very simple:

apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePoolClaim
metadata:
  name: get-me-cpu
  namespace: solar-test
spec:
  pool: "sample"
  claim:
    requests.cpu: "2"
    requests.memory: 2Gi

ResourcePoolClaims are decoupled from the lifecycle of ResourcePools. If a ResourcePool is deleted where a ResourcePoolClaim was bound to, the ResourcePoolClaim becomes unassigned, but is not deleted.

Allocation

The Connection between ResourcePools and ResourcePoolClaims is done via the .spec.pool field. With that field you must be very specific, from which ResourcePool a ResourcePoolClaim claims resources. On the counter-part, the ResourcePool, the namespace from the ResourcePoolClaim must be allowed to claim resources from the ResourcePool.

If you are trying to allocate a Pool which does not exist or is not allowed to be claimed from, from the namespace the ResourcePoolClaim was made, you will get a failed Assigned status:

solar-test   get-me-cpu          Assigned   Failed   ResourcePool.capsule.clastix.io "sample" not found   12s

Similar errors may occur if you are trying to claim resources from a pool, where the given resources are not claimable.

Auto-Assignment

If no .spec.pool was delivered a Webhook will try to evaluate a matching ResourcePool for the ResourcePoolClaim. In that process of evaluation the following criteria are considered:

A ResourcePool has all the resources in their definition available the ResourcePoolClaim is trying to claim.

If no Pool can be auto-assigned, the ResourcePoolClaim will enter an Unassigned state. Where it remains until ResourcePools considering the namespaces the ResourcePoolClaim is deployed in have more resources or a new ResourcePool is defined manually.

The Auto-Assignment Process is only executed, when .spec.pool is unset on Create or Update operations.

Bound

A ResourcePoolClaim is considered Bound, when the requested resources from the claim were successfully allocated from the ResourcePool. And the resources are actually used by any ResourceQuota in the namespace the claim was created in. If the resources are not used yet, the ResourcePoolClaims is considered Unused and can be deleted, change to a different ResourcePool or released without any further actions. However when it’s resources are used, the claim is Bound and can not be modified or deleted until the resources are released (not longer in use).

The selection of which ResourcePoolClaim is Bound is based on a greedy pattern. Meaning we sort the ResourcePoolClaims by their CreationTimestamp and try to allocate them one by one until no more resources are available from the ResourcePool.

Let’s see this in action. We can see that both claims are unused and can be released.

kubectl get resourcepoolclaim -n solar-test

NAME             POOL         READY   MESSAGE      BOUND   REASON            AGE
get-me-solar     solar-pool   True    reconciled   False   claim is unused   9h
get-me-solar-2   solar-pool   True    reconciled   False   claim is unused   9h


kubectl get resourcequota  -n solar-test

NAME                         REQUEST                                       LIMIT   AGE
capsule-pool-solar-pool      requests.cpu: 4/4, requests.memory: 4Gi/4Gi           7m53s

We now create a pod to consume the amount of resources provided by the claim get-me-solar (cpu: 2 and memory: 2Gi). We can see that half of the claim is now used:

kubectl get resourcepoolclaim -n solar-test

NAME             POOL         READY   MESSAGE      BOUND   REASON            AGE
get-me-solar     solar-pool   True    reconciled   True    claim is used     12m
get-me-solar-2   solar-pool   True    reconciled   False   claim is unused   12m

kubectl get resourcequota  -n solar-test

NAME                         REQUEST                                       LIMIT   AGE
capsule-pool-solar-pool      requests.cpu: 2/4, requests.memory: 2Gi/4Gi           11m

We can remove get-me-solar-2, as it’s still unused:

kubectl delete resourcepoolclaim -n solar-test get-me-solar-2

resourcepoolclaim.capsule.clastix.io "get-me-solar-2" deleted

However interactions with get-me-solar are now limited, as it’s Bound:

kubectl delete resourcepoolclaim -n solar-test get-me-solar

Error from server (Forbidden): admission webhook "resourcepoolclaims.projectcapsule.dev" denied the request: cannot delete the pool while claim is used in resourcepool solar-pool

If we remove the pod again, the ResourcePoolClaim becomes unused again and can be deleted or modified.

kubectl get resourcepoolclaim -n solar-test

NAME           POOL         READY   MESSAGE      BOUND   REASON            AGE
get-me-solar   solar-pool   True    reconciled   False   claim is unused   16m


kubectl get resourcequota  -n solar-test

NAME                         REQUEST                                     LIMIT   AGE
capsule-pool-solar-pool      requests.cpu: 0/2, requests.memory: 0/2Gi           17m

Release

If a ResourcePoolClaim is deleted, the resources are released back to the ResourcePool. This means that the resources are no longer reserved for the claim and can be used by other claims.

By deleting the ResourcePoolClaim object (Recommended).
By annotating the ResourcePoolClaim with projectcapsule.dev/release: "true". This will release the ResourcePoolClaim from the ResourcePool without deleting the object itself and instantly requeue.

Both these actions can only be performed if the ResourcePoolClaim is in a Bound state False (not used currently). Otherwise your first have to free the resources used by the claim in order to release it. You can verify the Bound state for all ResourcePoolClaims in a namespace with.

Immutable

Once a ResourcePoolClaim has successfully claimed resources from a ResourcePool, the claim is immutable. This means that the claim cannot be modified or deleted until the resources have been released back to the ResourcePool. This means ResourcePoolClaim can not be expanded or shrunk, without releasing.

Queue

ResourcePoolClaims can always be created, even if the targeted ResourcePool does not have enough resources available at the time. In that case ResourcePoolClaims are put into a Queue-State, where they wait until they can claim the resources they are after. They following describes the different exhaustion indicators and what they mean, in case a ResourcePoolClaim gets scheduled.

When a ResourcePoolClaims is in Queued-State it is still mutable. So Resources and Pool-Assignment can still be changed.

Exhaustions

There are different types of exhaustions which may occur when attempting to allocate a claim. They Status of each claim indicates

PoolExhausted

The requested resources are not available on the ResourcePool. Until other resources release resources or the pool size is increased the ResourcePoolClaim is queued. In this example the ResourcePoolClaim is trying to claim requests.memory=2Gi. However only requests.memory=1Gi are still available to be claimed from the ResourcePool

NAMESPACE    NAME         POOL      STATUS   REASON          MESSAGE                                                          AGE
solar-test   get-mem      sampler   Bound    QueueExhausted   requested: requests.memory=2Gi, queued: requests.memory=1Gi   9m19s

In this case you have the following options:

Request less resources for claiming - requests.memory=1Gi
Wait until resources become from the ResourcePool. When 1Gi of requests.memory gets released, the ResourcePoolClaim will be able to bind requests.memory=2Gi.
Release another ResourcePoolClaim which might free up requests.memory

However, claims which are requesting less than the ResourcePoolClaim solar-test, will be able to allocate their resources. Let’s say we have this second ResourcePoolClaim:

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePoolClaim
metadata:
  name: skip-the-line
  namespace: solar-test
spec:
  pool: "sampler"
  claim:
    requests.memory: 512Mi

Applying this ResourcePoolClaim leads to it being able to bind these resources. This behavior can be controlled with orderedQueue.

NAMESPACE    NAME            POOL      STATUS   REASON          MESSAGE                                                            AGE
solar-test   get-me-cpu      sampler   Bound    PoolExhausted   requested: requests.memory=2Gi, available: requests.memory=512Mi   16m
solar-test   skip-the-line   sampler   Bound    Succeeded       Claimed resources                                                  2s

If orderedQueue is enabled, only the first item that exhausted a resource from the ResourcePool get the PoolExhausted state. Following claims fro the same resources get QueueExhausted.

QueueExhausted

A ResourcePoolClaim with higher priority is trying to allocate these resources, but is exhausting the ResourcePool. The ResourcePool has orderedQueue enabled, meaning that the ResourcePoolClaim with the highest priority must first schedule it’s resources, before any other ResourcePoolClaim can claim further resources. This queue is resource based (eg. requests.memory), ResourcePoolClaim with lower priority may still be Bound, if they are not trying to allocate resources which are being exhausted by another ResourcePoolClaim with highest priority.

NAMESPACE    NAME         POOL      STATUS   REASON          MESSAGE                                                          AGE
solar-test   get-mem      sampler   Bound    QueueExhausted   requested: requests.memory=2Gi, queued: requests.memory=1Gi   9m19s

The above means, that as ResourcePoolClaim with higher priority is trying to allocate requests.memory=1Gi but that already leads to an PoolExhausted for that ResourcePoolClaim.

Priority

The Priority of how the claims are processed, is deterministic defined based on the following order of attributes from each claim:

CreationTimestamp - Oldest first
Name - Tiebreaker
Namespace - Tiebreaker

Tiebreaker: If two claims have the same CreationTimestamp, they are then sorted alphabetically by their Name. If two claims have the same CreationTimestamp and Name, they are then sorted alphabetically by their Namespace. This means that if two claims are created at the same time, and have the same name, the claim with the lexicographically smaller Name will be processed first. If two claims have the same CreationTimestamp, Name, and Namespace, then the namespace is tiebreaking. This may be relevant in GitOps setups.

Operating

Monitoring

Dashboards can be deployed via helm-chart, enable the following values:

monitoring:
  dashboards:
    enabled: true

Dashboard which grants a detailed overview over the ResourcePools

Resourcepool Dashboard

Rules

Example rules to give you some idea, what’s possible.

Alert on ResourcePools usage

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: capsule-resourcepools-alerts
spec:
groups:
  - name: capsule-resourcepools.rules
    rules:
      - alert: CapsuleResourcePoolHighUsageWarning
        expr: |
          capsule_pool_usage_percentage > 90          
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: High resource usage in Resourcepool
          description: |
            Resource {{ $labels.resource }} in pool {{ $labels.pool }} is at {{ $value }}% usage for the last 10 minutes.            

      - alert: CapsuleResourcePoolHighUsageCritical
        expr: |
          capsule_pool_usage_percentage > 95          
        for: 10m
        labels:
          severity: critical
        annotations:
          summary: Critical resource usage in Resourcepool
          description: |
            Resource {{ $labels.resource }} in pool {{ $labels.pool }} has exceeded 95% usage for the last 10 minutes.            

      - alert: CapsuleResourcePoolExhausted
        expr: |
          capsule_pool_condition{condition="Exhausted"} == 1          
        for: 60m
        labels:
          severity: critical
        annotations:
          summary: Resource pool exhausted
          description: |
            Pool {{ $labels.pool }} has been Exhausted for more than 60 minutes.            

      - alert: CapsuleResourcePoolNotReady
        expr: |
          capsule_pool_condition{condition="Ready"} == 0          
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: Resource pool not ready
          description: |
            Pool {{ $labels.pool }} has not been Ready for more than 10 minutes.            


  - name: capsule-resourcepoolclaims.rules
    rules:
      - alert: CapsuleResourcePoolClaimExhausted
        expr: |
          capsule_claim_condition{condition="Exhausted"} == 1          
        for: 24h
        labels:
          severity: critical
        annotations:
          summary: ResourcePoolClaim exhausted
          description: |
            ResourcePoolClaim {{ $labels.name }}/{{ $labels.target_namespace }} has been Exhausted for more than 24 hours.            

      - alert: CapsuleResourcePoolClaimNotReady
        expr: |
          capsule_claim_condition{condition="Ready"} == 0          
        for: 60m
        labels:
          severity: warning
        annotations:
          summary: ResourcePoolClaim not ready
          description: |
            ResourcePoolClaim {{ $labels.name }}/{{ $labels.target_namespace }}  has not been Ready for more than 60 minutes.

Metrics

The following Metrics are exposed and can be used for monitoring:


# HELP capsule_claim_condition The current condition status of a claim.
# TYPE capsule_claim_condition gauge
capsule_claim_condition{condition="Bound",name="get-me-customer",target_namespace="solar-test"} 1
capsule_claim_condition{condition="Bound",name="get-me-solar",target_namespace="solar-test"} 1
capsule_claim_condition{condition="Bound",name="get-me-solar-2",target_namespace="solar-test"} 0
capsule_claim_condition{condition="Exhausted",name="get-me-customer",target_namespace="solar-test"} 0
capsule_claim_condition{condition="Exhausted",name="get-me-solar",target_namespace="solar-test"} 0
capsule_claim_condition{condition="Exhausted",name="get-me-solar-2",target_namespace="solar-test"} 1
capsule_claim_condition{condition="Ready",name="get-me-customer",target_namespace="solar-test"} 1
capsule_claim_condition{condition="Ready",name="get-me-solar",target_namespace="solar-test"} 1
capsule_claim_condition{condition="Ready",name="get-me-solar-2",target_namespace="solar-test"} 1

# HELP capsule_claim_pool The current assigned pool of a claim.
# TYPE capsule_claim_pool gauge
capsule_claim_pool{name="get-me-solar",pool="solar-compute",target_namespace="solar-test"} 1
capsule_claim_pool{name="get-me-solar-2",pool="solar-compute",target_namespace="solar-test"} 1

# HELP capsule_claim_resource The given amount of resources from the claim
# TYPE capsule_claim_resource gauge
capsule_claim_resource{name="compute",resource="limits.cpu",target_namespace="solar-prod"} 0.375
capsule_claim_resource{name="compute",resource="limits.memory",target_namespace="solar-prod"} 4.02653184e+08
capsule_claim_resource{name="compute",resource="requests.cpu",target_namespace="solar-prod"} 0.375
capsule_claim_resource{name="compute",resource="requests.memory",target_namespace="solar-prod"} 4.02653184e+08
capsule_claim_resource{name="compute-10",resource="limits.memory",target_namespace="solar-prod"} 1.073741824e+10
capsule_claim_resource{name="compute-2",resource="limits.cpu",target_namespace="solar-prod"} 0.5
capsule_claim_resource{name="compute-2",resource="limits.memory",target_namespace="solar-prod"} 5.36870912e+08
capsule_claim_resource{name="compute-2",resource="requests.cpu",target_namespace="solar-prod"} 0.5
capsule_claim_resource{name="compute-2",resource="requests.memory",target_namespace="solar-prod"} 5.36870912e+08
capsule_claim_resource{name="compute-3",resource="requests.cpu",target_namespace="solar-prod"} 0.5
capsule_claim_resource{name="compute-4",resource="requests.cpu",target_namespace="solar-test"} 0.5
capsule_claim_resource{name="compute-5",resource="requests.cpu",target_namespace="solar-test"} 0.5
capsule_claim_resource{name="compute-6",resource="requests.cpu",target_namespace="solar-test"} 5
capsule_claim_resource{name="pods",resource="pods",target_namespace="solar-test"} 3

# HELP capsule_pool_available Current resource availability for a given resource in a resource pool
# TYPE capsule_pool_available gauge
capsule_pool_available{pool="solar-compute",resource="limits.cpu"} 1.125
capsule_pool_available{pool="solar-compute",resource="limits.memory"} 1.207959552e+09
capsule_pool_available{pool="solar-compute",resource="requests.cpu"} 0.125
capsule_pool_available{pool="solar-compute",resource="requests.memory"} 1.207959552e+09
capsule_pool_available{pool="solar-size",resource="pods"} 4


# HELP capsule_pool_condition Current conditions for a given resource in a resource pool
# TYPE capsule_pool_condition gauge
capsule_pool_condition{condition="Exhausted",pool="solar-size"} 0
capsule_pool_condition{condition="Exhausted",pool="solar-compute"} 1
capsule_pool_condition{condition="Ready",pool="solar-size"} 1
capsule_pool_condition{condition="Ready",pool="solar-compute"} 1

# HELP capsule_pool_exhaustion Resources become exhausted, when there's not enough available for all claims and the claims get queued
# TYPE capsule_pool_exhaustion gauge
capsule_pool_exhaustion{pool="solar-compute",resource="limits.memory"} 1.073741824e+10
capsule_pool_exhaustion{pool="solar-compute",resource="requests.cpu"} 5.5

# HELP capsule_pool_exhaustion_percentage Resources become exhausted, when there's not enough available for all claims and the claims get queued (Percentage)
# TYPE capsule_pool_exhaustion_percentage gauge
capsule_pool_exhaustion_percentage{pool="solar-compute",resource="limits.memory"} 788.8888888888889
capsule_pool_exhaustion_percentage{pool="solar-compute",resource="requests.cpu"} 4300

# HELP capsule_pool_limit Current resource limit for a given resource in a resource pool
# TYPE capsule_pool_limit gauge
capsule_pool_limit{pool="solar-compute",resource="limits.cpu"} 2
capsule_pool_limit{pool="solar-compute",resource="limits.memory"} 2.147483648e+09
capsule_pool_limit{pool="solar-compute",resource="requests.cpu"} 2
capsule_pool_limit{pool="solar-compute",resource="requests.memory"} 2.147483648e+09
capsule_pool_limit{pool="solar-size",resource="pods"} 7

# HELP capsule_pool_namespace_usage Current resources claimed on namespace basis for a given resource in a resource pool for a specific namespace
# TYPE capsule_pool_namespace_usage gauge
capsule_pool_namespace_usage{pool="solar-compute",resource="limits.cpu",target_namespace="solar-prod"} 0.875
capsule_pool_namespace_usage{pool="solar-compute",resource="limits.memory",target_namespace="solar-prod"} 9.39524096e+08
capsule_pool_namespace_usage{pool="solar-compute",resource="requests.cpu",target_namespace="solar-prod"} 1.375
capsule_pool_namespace_usage{pool="solar-compute",resource="requests.cpu",target_namespace="solar-test"} 0.5
capsule_pool_namespace_usage{pool="solar-compute",resource="requests.memory",target_namespace="solar-prod"} 9.39524096e+08
capsule_pool_namespace_usage{pool="solar-size",resource="pods",target_namespace="solar-test"} 3

# HELP capsule_pool_namespace_usage_percentage Current resources claimed on namespace basis for a given resource in a resource pool for a specific namespace (percentage)
# TYPE capsule_pool_namespace_usage_percentage gauge
capsule_pool_namespace_usage_percentage{pool="solar-compute",resource="limits.cpu",target_namespace="solar-prod"} 43.75
capsule_pool_namespace_usage_percentage{pool="solar-compute",resource="limits.memory",target_namespace="solar-prod"} 43.75
capsule_pool_namespace_usage_percentage{pool="solar-compute",resource="requests.cpu",target_namespace="solar-prod"} 68.75
capsule_pool_namespace_usage_percentage{pool="solar-compute",resource="requests.cpu",target_namespace="solar-test"} 25
capsule_pool_namespace_usage_percentage{pool="solar-compute",resource="requests.memory",target_namespace="solar-prod"} 43.75
capsule_pool_namespace_usage_percentage{pool="solar-size",resource="pods",target_namespace="solar-test"} 42.857142857142854

# HELP capsule_pool_resource Type of resource being used in a resource pool
# TYPE capsule_pool_resource gauge
capsule_pool_resource{pool="solar-compute",resource="limits.cpu"} 1
capsule_pool_resource{pool="solar-compute",resource="limits.memory"} 1
capsule_pool_resource{pool="solar-compute",resource="requests.cpu"} 1
capsule_pool_resource{pool="solar-compute",resource="requests.memory"} 1
capsule_pool_resource{pool="solar-size",resource="pods"} 1

# HELP capsule_pool_usage Current resource usage for a given resource in a resource pool
# TYPE capsule_pool_usage gauge
capsule_pool_usage{pool="solar-compute",resource="limits.cpu"} 0.875
capsule_pool_usage{pool="solar-compute",resource="limits.memory"} 9.39524096e+08
capsule_pool_usage{pool="solar-compute",resource="requests.cpu"} 1.875
capsule_pool_usage{pool="solar-compute",resource="requests.memory"} 9.39524096e+08
capsule_pool_usage{pool="solar-size",resource="pods"} 3

# HELP capsule_pool_usage_percentage Current resource usage for a given resource in a resource pool (percentage)
# TYPE capsule_pool_usage_percentage gauge
capsule_pool_usage_percentage{pool="solar-compute",resource="limits.cpu"} 43.75
capsule_pool_usage_percentage{pool="solar-compute",resource="limits.memory"} 43.75
capsule_pool_usage_percentage{pool="solar-compute",resource="requests.cpu"} 93.75
capsule_pool_usage_percentage{pool="solar-compute",resource="requests.memory"} 43.75
capsule_pool_usage_percentage{pool="solar-size",resource="pods"} 42.857142857142854

Migration

We highly recommend to reassess how you are allocating resources to your tenants and if you want to change the scope of the ResourcePools to be more granular or wider.

To Migrate from the old ResourceQuota to ResourcePools, you can follow the steps below. This guide assumes you want to port the old ResourceQuota to the new ResourcePools in exactly the same capacity and scope.

The steps shown are an example to migrate a single Tenants ResourceQuota to a ResourcePool.

1. Overview

We are working with the following tenant. Asses the Situation of resourceQuotas. This guide is mainly relevant if the scope is Tenant:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  labels:
    kubernetes.io/metadata.name: migration
  name: migration
spec:
  owners:
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: User
    name: bob
  preventDeletion: false
  resourceQuotas:
    items:
    - hard:
        limits.cpu: "2"
        limits.memory: 2Gi
        requests.cpu: "2"
        requests.memory: 2Gi
    - hard:
        pods: "7"
    scope: Tenant
status:
  namespaces:
  - migration-dev
  - migration-prod
  - migration-test
  size: 3
  state: Active

2. Abstracting to ResourcePools

Warning

Do not apply the resourcepools yet, this may lead to workloads not being able to schedule!

We are now abstracting. For each item, we are creating a ResourcePool with the same values. The ResourcePool will be scoped to the Tenant and will be used for all namespaces in the tenant. Let’s first migrate the first item:

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: migration-compute
spec:
  config:
    defaultsZero: true
  selectors:
  - matchLabels:
      capsule.clastix.io/tenant: migration
  quota:
    hard:
      limits.cpu: "2"
      limits.memory: 2Gi
      requests.cpu: "2"
      requests.memory: 2Gi

The naming etc. is up to you. Important, we again select all namespaces from the migration tenant with the selector capsule.clastix.io/tenant: migration. The defined config is what we deem to be most compatible with the old ResourceQuota behavior. You may change these according to your requirements.

The same process can be repeated for the second item (or each of your items). The ResourcePool will be scoped to the Tenant and will be used for all namespaces in the tenant. Let’s migrate the second item:

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: migration-size
spec:
  config:
    defaultsZero: true
  selectors:
  - matchLabels:
      capsule.clastix.io/tenant: migration
  quota:
    hard:
      pods: "7"

3. Create ResourcePoolClaims

Now we need to create the ResourcePoolClaims for the ResourcePools. The ResourcePoolClaims are used to claim resources from the ResourcePools to the respective namespaces. Let’s start with the namespace migration-dev:

kubectl get resourcequota -n migration-dev

NAME                  AGE     REQUEST                                                   LIMIT
capsule-migration-0   5m21s   requests.cpu: 375m/1500m, requests.memory: 384Mi/1536Mi   limits.cpu: 375m/1500m, limits.memory: 384Mi/1536Mi
capsule-migration-1   5m21s   pods: 3/3

Our goal is now to port the current usage into ResourcePoolClaims. Here you must make sure, that you might need to allocate more resources to your claims, than currently is needed (eg. to allow RollingUpdates etc.).. For the example we are porting the current usage over 1:1 to ResourcePoolClaims

We created the ResourcePool named migration-compute, where we are going to claim the resources from (for capsule-migration-0). This results in the following ResourcePoolClaim:

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePoolClaim
metadata:
  name: compute
  namespace: migration-dev
spec:
  pool: "migration-compute"
  claim:
    requests.cpu: 375m
    requests.memory: 384Mi
    limits.cpu: 375m
    limits.memory: 384Mi

The same can be done for the capsule-migration-1 ResourceQuota.

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePoolClaim
metadata:
  name: pods
  namespace: migration-dev
spec:
  pool: "migration-size"
  claim:
    pods: "3"

You can create the claims, they will remain in failed state until we apply the ResourcePools:

kubectl get resourcepoolclaims -n migration-dev

NAME      POOL   STATUS     REASON   MESSAGE                                                         AGE
compute          Assigned   Failed   ResourcePool.capsule.clastix.io "migration-compute" not found   2s
pods             Assigned   Failed   ResourcePool.capsule.clastix.io "migration-size" not found      2s

4. Applying and Verifying the ResourcePools

You may now apply the ResourcePools prepared in step 2:

kubectl apply -f pools.yaml
resourcepool.capsule.clastix.io/migration-compute created
resourcepool.capsule.clastix.io/migration-size created

After applying, you should instantly see, that the ResourcePoolClaims in the migration-dev namespace could be Bound to the corresponding ResourcePools:

kubectl get resourcepoolclaims -n migration-dev
NAME      POOL                STATUS   REASON      MESSAGE             AGE
compute   migration-compute   Bound    Succeeded   Claimed resources   4m9s
pods      migration-size      Bound    Succeeded   Claimed resources   4m9s

Now you can verify the new ResourceQuotas in the migration-dev namespace:

kubectl get resourcequota -n migration-dev
NAME                             AGE    REQUEST                                                   LIMIT
capsule-migration-0              23m    requests.cpu: 375m/1500m, requests.memory: 384Mi/1536Mi   limits.cpu: 375m/1500m, limits.memory: 384Mi/1536Mi
capsule-migration-1              23m    pods: 3/3

capsule-pool-migration-compute   110s   requests.cpu: 375m/375m, requests.memory: 384Mi/384Mi     limits.cpu: 375m/375m, limits.memory: 384Mi/384Mi
capsule-pool-migration-size      110s   pods: 3/3

That looks already super promising. Now You need to repeat these steps for migration-prod and migration-test. (Script Contributions are welcome).

5. Removing old ResourceQuotas

Once we have migrated all resources over the ResourcePoolClaims, we can remove the ResourceQuota system. First of all, we are removing the .spec.resourceQuotas entirely. Currently it will again add the .spec.resourceQuotas.scope field, important is, that no more .spec.resourceQuotas.items exist:

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  labels:
    kubernetes.io/metadata.name: migration
  name: migration
spec:
  owners:
  - clusterRoles:
    - admin
    - capsule-namespace-deleter
    kind: User
    name: bob
  resourceQuotas: {}

This will remove all ResourceQuotas from namespace, verify like:

kubectl get resourcepoolclaims -n migration-dev

capsule-pool-migration-compute   130m   requests.cpu: 375m/375m, requests.memory: 384Mi/384Mi   limits.cpu: 375m/375m, limits.memory: 384Mi/384Mi
capsule-pool-migration-size      130m   pods: 3/3

Success 🍀

Why this is our answer

This part should provide you with a little bit of back story, as to why this implementation was done the way it currently is. Let’s start.

Since the beginning of capsule we are struggling with a concurrency problem regarding ResourcesQuotas, this was already early detected in Issue 49. Let’s quickly recap what really the problem is with the current ResourceQuota centric approach.

With the current ResourceQuota with Scope: Tenant we encounter the problem, that resourcequotas spread across multiple namespaces referring to one tenant quota can be overprovisioned, if an operation is executed in parallel (eg. total is services/count: 3, in each namespace you could then create 3 services, leading to a possible overprovision of hard * amount-namespaces). The Problem in this approach is, that we are not doing anything with Webhooks, therefore we rely on the speed of the controller, where this entire construct becomes a matter of luck and racing conditions.

So, there needs to be change. But times have also changed and we have listened to our users, so the new approach to ResourceQuotas should:

Not exclusively be scoped to one Tenant. Often scenarios include granting resources to multiple Tenants eg.
- When a application has multiple stages split into multiple stages
- An Application-Team owns multiple Tenants
- You want to share resources amongst applications of the same stage.
Select based on namespaces, even if they are not part of the Tenant ecosystem. Often the requirement to control resources for operators, which make up your distribution, must also be guardlined across n-namespaces.
Supplement new generation technology like Kamaji, vCluster or K3K. All these tool abstract Kubernetes into Pods. We also want to provide a solution which still proves capsule relevant in combination with such modern tools.
Shifting Resource-Management to Tenant-Owners while Cluster-Administrators orchestrate a greater Pool of resources.
Consistency!!

Our initial Idea for a redesign was simple: What if we just intercepted operations on the resourcequota/status subresource and calculate the offsets (or essentially what still can fit) on a Admission-Webhook. If another operation would have taken place the client operation would have thrown a conflict and rejected the admission, until it retries. Makes sense, right?

Here we have the problem, that even if we would block resourcequota status updates and wait until the actual quantity was added to the total, the resources have already been scheduled. The reason for that, is that the status for resourcequotas is eventually consistent, but what really matters at that moment is the hard spec (see this response from a maintainer kubernetes/kubernetes#123434 (comment)). So essentially no matter the status, you can always provision as much resources, as the .spec.hard of a ResourceQuota indicates. This makes perfect sense, if your ResourceQuota is acting in a single namespace. However in our scenario, we have the same ResourceQuota in n-namespaces. So the overprovisioning problem still persists.

Thinking of other ways: So the next idea was essentially increasing the ResourceQuota.spec.hard based on the workloads which are added to a namespaces (essentially a reversed approach). The workflow for this would look like something like this:

All resourcequotas get for their hard spec 0
New resource is requested (Evaluation what’s needed at Admission)
Controller gives the requested resources to the quota (by adding it to the total and updating the hard)

This way it’s only possible to scheduled “ordered”. In conclusion this would also downscale the resourcequota when the resources are no longer needed. This is how ResourceQuotas from the Kubernetes Core-API reject workload, when you try to allocate a Quantity in a namespaces, but the ResourceQuota does not have enough space.

But there are some problems with this approach as well:

if you eg. schedule a pod and the quota is count/0 there’s no admission call on the resourcequota, which would be the easiest. So we would need to find a way to know, there’s something new requesting resources. For example Rancher works around this problem with namespaced DefaultLimits. But this is not the agile approach we would like to offer.
The only indication that I know of is that we get an Event, which we can intercept with admission (ResourceQuota Denied), regarding quotaoverprovision.

If you eg update the resource quota that a pod now has space, it takes some time until that’s registered and actually scheduled (just tested it for pods). I guess the timing depends on the kube-controller-manager flag --concurrent-resource-quota-syncs and/or `–resource-quota-sync-period

So it’s really really difficult to increase quotas by the resources which are actually requested, especially the adding new resources process is where the performance would take a heavy hit.

Still thinking on this idea, the optimal solution would have been to calculate everything at admission and keep the usage vs available state on a global resources but not provisioning namespaced ResourceQuotas. This would have taken a bit pressure from the entire operation, as the resources would not have to be calculated twice (For our GlobalResourceQuota and the core ResourceQuota). In addition we should have added

So that’s when we discarded everything and came up with the concept of ResourcePools and ResourcePoolClaims.

Future Ideas

We want to keep this API as lightweight as possible. But we have already identified use-cases with customers, which make heavy use of ResourcePools:

JIT-Claiming: Every Workload queues it’s own claim when being submitted to admission. The respective claims are bound to the lifecycle of the provisioned resource.
Node-Population: Populate the Quantity of a ResourcePool based on selected nodes.

9.3 - Custom Quotas

CustomQuotas let you define and enforce arbitrary, label-scoped limits for any Kubernetes resource kind or CRD, at namespace or cluster scope.

Quickstart

If you just want to get common quotas up and running without reading the full model first, this section gives you copy-paste-ready GlobalCustomQuota examples for the most common dimensions: CPU, memory, Pod count, and storage. Skip ahead to Concept when you want to understand how sources, selectors, and JSONPath expressions work in depth.

1. Enable the Admission Webhook

Without the admission webhook enabled, quotas are observational only: usage is calculated and reported in status, but requests are never denied. Enable and configure it via Helm values:

webhooks:
  hooks:
    calculations:
      enabled: true
      namespaceSelector:
        matchExpressions:
          - key: capsule.clastix.io/tenant
            operator: Exists
      rules:
        - apiGroups:
            - ""
          apiVersions:
            - "v1"
          operations:
            - CREATE
            - UPDATE
            - DELETE
          resources:
            - "pods"
            - "pods/status"
            - "persistentvolumeclaims"
          scope: Namespaced
      matchConditions:
        - name: ignore-subresources-except-pod-status-phase
          expression: >-
            !has(request.subResource) || request.subResource == "" ||
            (request.subResource == "status" && request.resource.resource == "pods" && object.status.phase != oldObject.status.phase)

Including pods/status keeps quota usage accurate the moment a Pod’s phase changes to Succeeded or Failed, instead of waiting for another triggering event (e.g. a different Pod being created or deleted) to notice that a Pod no longer counts towards the quota. The ignore-subresources-except-pod-status-phase matchCondition combines two things into one expression:

it excludes all subresource requests by default (matching the intent of ignore-subresources in the more advanced example shown later in Admission)
except for the pods status subresource, which is only let through when .status.phase actually changed, filtering out irrelevant status updates.

2. Apply default GlobalCustomQuotas

Adjust .spec.limit and the .spec.namespaceSelectors matchLabels below to match your own tenant labeling, then apply. The CPU and memory examples cover both containers and initContainers, and exclude terminal Pods (Succeeded/Failed/Unknown) via fieldSelectors, matching the default behavior of native Kubernetes ResourceQuota (see Not-Equals for details):

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: solar-cpu-requests
spec:
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.containers[*].resources.requests.cpu
      selectors:
        - fieldSelectors:
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.initContainers[*].resources.requests.cpu
      selectors:
        - fieldSelectors:
            - '.spec.initContainers'
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'
  limit: "2"
---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: solar-cpu-limits
spec:
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.containers[*].resources.limits.cpu
      selectors:
        - fieldSelectors:
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.initContainers[*].resources.limits.cpu
      selectors:
        - fieldSelectors:
            - '.spec.initContainers'
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'
  limit: "4"
---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: solar-memory-requests
spec:
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.containers[*].resources.requests.memory
      selectors:
        - fieldSelectors:
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.initContainers[*].resources.requests.memory
      selectors:
        - fieldSelectors:
            - '.spec.initContainers'
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'
  limit: "32Gi"
---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: solar-memory-limits
spec:
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.containers[*].resources.limits.memory
      selectors:
        - fieldSelectors:
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.initContainers[*].resources.limits.memory
      selectors:
        - fieldSelectors:
            - '.spec.initContainers'
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'
  limit: "32Gi"
---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: solar-pod-count
spec:
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: v1
      kind: Pod
      op: count
  limit: 300
---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: solar-storage
spec:
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: v1
      kind: PersistentVolumeClaim
      op: add
      path: .spec.resources.requests.storage
  limit: "50Gi"

3. Verify

kubectl get globalcustomquota

kubectl get globalcustomquota solar-pod-count -o yaml

status:
  conditions:
  - reason: Succeeded
    status: "True"
    type: Ready
  namespaces:
  - solar-test
  usage:
    available: "297"
    used: "3"

Concept

CustomQuotas complement Kubernetes ResourceQuota by enforcing limits on custom usage metrics extracted from objects themselves.

Capsule provides two quota resources:

CustomQuota: namespaced CRD that limits usage inside one namespace. It can only target namespaced resources.
GlobalCustomQuota: cluster-scoped CRD that aggregates usage across a set of namespaces selected by label selectors.

A quota is defined by:

a limit
one or more sources
optional selectors to restrict which objects are counted

Each matching object contributes a quantity to the quota. Capsule persists the current aggregate in status.usage.used and keeps the list of counted objects in status.claims.

Namespace Scope

GlobalCustomQuota and CustomQuota can operate in any namespace and do not have to be part of a Capsule tenant. This means that you can define a CustomQuota in any namespace, even if it’s not part of a tenant, and it will still be enforced for objects in that namespace. Similarly, you can define a GlobalCustomQuota that selects namespaces based on labels, regardless of whether those namespaces are part of a tenant or not.

Calculation

CustomQuotas are calculated in two cooperating parts:

Admission webhook: performs enforcement during CREATE, UPDATE, and DELETE
Controller reconcile loop: rebuilds the quota status from the actual cluster state and keeps it authoritative

Admission

The admission webhook intercepts operations for the configured resource kinds and evaluates whether the change would violate any matching quota.

For each matching quota it:

matches the object against the quota source GVK
evaluates source selectors
computes the requested usage for the operation
creates or updates a short-lived reservation in the corresponding QuantityLedger
denies the request if persistedUsed + inflightReserved + requested > limit

This makes quota enforcement safe even during bursts of concurrent requests.

Without the Admission Webhook enabled, CustomQuotas are observational only. The controllers still rebuild and report usage, but requests are not denied.

By default, no objects are sent to this webhook. You must explicitly enable it and configure matching rules.

Example: enable calculations for all Pod create/update/delete operations in tenant namespaces:

webhooks:
  hooks:
    calculations:
      enabled: true
      namespaceSelector:
        matchExpressions:
          - key: capsule.clastix.io/tenant
            operator: Exists
      rules:
          - apiGroups:
              - ""
            apiVersions:
              - ""
            operations:
              - CREATE
              - UPDATE
              - DELETE
            resources:
              - "pods"
            scope: Namespaced

Make sure to configure this webhook carefully, as it can impact cluster performance and availability if it matches a large number of operations. Start with a narrow scope (e.g., specific GVKs and namespace labels) and monitor the impact before expanding it. Also make sure to exclude system critical components or namespaces to avoid accidental disruptions.

webhooks:
  hooks:
    calculations:
      enabled: true
      namespaceSelector:
        matchExpressions:
          - key: name
            operator: NotIn
            values: ["kube-system", "kube-public", "kube-node-lease"]
      rules:
        - apiGroups:
            - ""
          apiVersions:
            - ""
          operations:
            - CREATE
            - UPDATE
            - DELETE
          resources:
            - "pods"
          scope: Namespaced
      matchConditions:
        # Exclude Event and Subresource requests to avoid performance issues and disruptions in case of issues with the webhook (Example).
        - name: ignore-subresources
          expression: '!has(request.subResource) || request.subResource == ""'
        - name: ignore-events
          expression: 'request.resource.resource != "events"'

        # Exclude Entities which never count towards quotas to avoid performance issues and disruptions in case of issues with the webhook (Example).
        - name: 'exclude-kubelet-requests'
          expression: '!("system:nodes" in request.userInfo.groups)'
        - name: 'exclude-kube-system'
          expression: '!("system:serviceaccounts:kube-system" in request.userInfo.groups)'

Without the Admission Webhook enabled, CustomQuotas are purely observational and do not enforce limits. You can use this mode to monitor usage and understand the impact before enabling enforcement.

JSONPath

The Custom Quota system relies on JSONPath expressions to extract numeric values from objects. The spec.sources[*].path field defines the JSONPath to the value that should be counted towards the quota. This allows you to define quotas based on any numeric field in any Kubernetes resource, including custom resources.

The following constraints apply to the JSONPath:

Expressions must start with a dot (.) and use standard JSONPath syntax. (valid .spec.storage.usage).
Paths can not be empty.
The maximum length of the path is 1024 characters.
Expressions can not contain any of the following characters:
- \n (newline)
- \r (carriage return)
- \t (tab)
Values can resolve to array results, which are then summed up. (For example, .spec.containers[*].resources.limits.cpu would sum the CPU limits of all containers in a Pod.)
Missing fields result in an error: when a matched resource (e.g. a Pod or PVC) does not have the field defined in path, it is denied by the admission controller (if enabled) and skipped by the reconciler with an error, setting the quota’s Ready condition to False. If you want to add a quota on a non-mandatory field (e.g. initContainers), add an existence check to fieldSelectors as well (see FieldSelectors), for example with .spec.initContainers as a truthy fieldSelector.

Matching Strategies

This section describes how JSONPath expressions are evaluated and how their results are interpreted for conditional matching.

Truthy

When a fieldSelectors entry does not contain a top-level = or ==, Capsule treats it as a JSONPath expression.

The selector matches when the JSONPath result is truthy.

Truthy evaluation rules:

empty result: false
false, case-insensitive: false
0: false
any other non-empty result: true

Example:

spec:
  sources:
    - apiVersion: v1
      kind: PersistentVolumeClaim
      op: add
      path: .spec.resources.requests.storage
      selectors:
        - fieldSelectors:
            - '.spec.accessModes[?(@=="ReadWriteOnce")]'
            - '.status.phase'

This selector matches only if:

.spec.accessModes[?(@=="ReadWriteOnce")] returns a non-empty result
.status.phase returns a non-empty result

For example, this matches a PVC with:

spec:
  accessModes:
    - ReadWriteOnce
status:
  phase: Bound

Equality

When an entry contains a top-level = or == (not nested JP expressions), Capsule treats it as an equality comparison. The left side is evaluated as a JSONPath expression. The right side is compared as a string.

spec:
  sources:
    - apiVersion: v1
      kind: Service
      op: count
      selectors:
        - fieldSelectors:
            - '.spec.type=ClusterIP'

The following forms are equivalent:

fieldSelectors:
  - '.spec.type=ClusterIP'
  - '.spec.type==ClusterIP'
  - '.spec.type=="ClusterIP"'
  - ".spec.type=='ClusterIP'"

A == inside a JSONPath filter is still treated as part of the JSONPath expression, not as Capsule equals matching.

For example:

fieldSelectors:
  - '.spec.accessModes[?(@=="ReadWriteOnce")]'

This is interpreted as a truthy JSONPath selector, not as an equals selector.

Use JSONPath filters for arrays:

fieldSelectors:
  - '.spec.accessModes[?(@=="ReadWriteOnce")]'

Use equals matching for scalar fields:

fieldSelectors:
  - '.spec.type=ClusterIP'

Not-Equals

When an entry contains a top-level !=, Capsule treats it as a not-equals comparison. The left side is evaluated as a JSONPath expression. The right side is compared as a string. The selector matches when the field value does not equal the specified string.

spec:
  sources:
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.containers[*].resources.limits.cpu
      selectors:
        - fieldSelectors:
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'

Recommended for CPU and memory resources
Kubernetes built-in ResourceQuota excludes terminal Pods (those in Succeeded, Failed, or Unknown phase) from CPU and memory accounting by default, because those Pods no longer hold resources on the kubelet. When defining CustomQuotas or GlobalCustomQuotas for CPU or memory resources, it is strongly recommended to use != field selectors to exclude terminal Pods and match this behavior.

Quota Matches

As it’s the case with native ResourceQuotas, when a request is made, Capsule evaluates all existing CustomQuotas and GlobalCustomQuotas to determine which ones match the request. Always the smallest quantity of quotas is enforced, meaning that if multiple quotas match a request, the one with the least available capacity will be the one that determines whether the request is allowed or denied.

Let’s look at this example. We have a GlobalCustomQuota targeting all namespaces of the tenant solar with a limit of 6 Pods, and a CustomQuota in the namespace solar-test (part of tenant solar) with a limit of 3 Pods:

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: pod-count-limit
spec:
  limit: 6
  namespaceSelectors:
  - matchLabels:
      capsule.clastix.io/tenant: solar
  sources:
  - apiVersion: v1
    kind: Pod
    op: count
---
apiVersion: capsule.clastix.io/v1beta2
kind: CustomQuota
metadata:
  name: pod-count-limit
  namespace: solar-test
spec:
  limit: 3
  sources:
  - apiVersion: v1
    kind: Pod
    op: count

When we now try to create 6 Pods in the namespace solar-test, we can observe that the GlobalCustomQuota allows only 6 Pods in total across all namespaces of the tenant, while the CustomQuota allows only 3 Pods in the solar-test namespace:

kubectl get pod -n solar-test

NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-6ff89574f8-2jbvp   1/1     Running   0          4m20s
nginx-deployment-6ff89574f8-sdzvr   1/1     Running   0          4m20s
nginx-deployment-6ff89574f8-tvk74   1/1     Running   0          4m20s

We can see that requests are blocked because of the limits by the CustomQuota first, as it has the least available capacity (3 available vs 6 available in the GlobalCustomQuota):

115s        Warning   FailedCreate        replicaset/nginx-deployment-6ff89574f8   Error creating: admission webhook "calculation.custom-quotas.projectcapsule.dev" denied the request: creating resource exceeds limit for CustomQuota "pod-count-limit" (requested=1, currentUsed=4, available=0, limit=3, inflightReserved=1)

Race Conditions

GlobalCustomQuotas and CustomQuotas are only considered for enforcement once the target GVK has been posted to their status. If you quickly create workloads that match the GVK of a quota before the quota has been fully reconciled and posted to status, there is a possibility that those workloads are not counted towards the quota usage until the next reconciliation loop. This is because the admission webhook relies on the quota status to determine which quotas to enforce, and if the quota has not yet been reconciled and posted to status, it may not be considered during admission.

Sources

A quota may define one or many sources. Each source describes:

which objects are candidates (apiVersion, kind)
what value is extracted from them (path, if applicable)
how that value contributes to usage (op)
optional additional source-level selectors

In practice, a source answers:

“For objects of this kind, what should count toward the quota?”

Sources are evaluated independently and then aggregated into one total.

GVK

Each source must identify a Kubernetes resource type by apiVersion / Kind. Example for a core Kubernetes Pod:

apiVersion: v1
kind: Pod

Example for CRDs:

apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim

apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket

How matching works:

only objects whose GVK exactly matches the source are considered
for CustomQuota, the target resource must be namespaced
for GlobalCustomQuota, both namespaced Kubernetes resources and namespaced CRDs are supported across all selected namespaces
if a source refers to a GVK that is not installed or not discoverable, the controller reports a reconcile failure in the quota condition

A source does not automatically follow subresources, versions, or related objects. If you want to count two kinds, define two sources.

For example, to count both Pods and PVCs, use two sources:

spec:
  sources:
    - apiVersion: v1
      kind: Pod
      op: count
    - apiVersion: v1
      kind: PersistentVolumeClaim
      op: count

Path

path defines which value is extracted from a matching object.

Use path when the operation is add or sub and you want to sum up numeric fields from the objects, such as CPU requests or storage sizes. You can not use path with count.

The path expression leverages JSONPath syntax and must resolve to a numeric value or an array of numeric values. The resulting number is added to the quota usage according to the defined operation. Here some examples of paths:

Count requested PVC storage:

path: .spec.resources.requests.storage

Sum all container CPU requests in a Pod:

path: .spec.containers[*].resources.requests.cpu

Sum ephemeral volume claim sizes declared inside a Pod:

path: .spec.volumes[*].ephemeral.volumeClaimTemplate.spec.resources.requests.storage

Important notes:

the extracted value must be parseable as a Kubernetes Quantity
if the expression resolves to multiple values, Capsule sums them
if a matched resource has a missing field, it is denied by the admission controller (if enabled) and skipped by the reconciler. If you want to add a quota on a non-mandatory field (like an initContainer), add it to the fieldSelectors as well (see JSONPath for more information).

Operations

Each source has an op (operation) field that controls how each matching object contributes to the total usage:

add: used += extracted value
sub: used -= extracted value
count: used += 1 per matching object

On updates, usage is recalculated from the current object state and the authoritative quota status is rebuilt from scratch from all matching objects.

`add`

Adds the extracted quantity to the quota usage. Typical use cases:

CPU requests
memory limits
PVC storage
emptyDir or ephemeral storage sizes

This is the default behavior.

Example:

spec:
  sources:
    - apiVersion: v1
      kind: PersistentVolumeClaim
      op: add
      path: .spec.resources.requests.storage

`sub`

Subtracts the extracted quantity from the quota usage. This is useful when you want a source to offset or discount usage from another source.

`count`

Counts matching objects as 1 each. Rules for count:

path must not be set
each matching object contributes exactly 1

Example:

spec:
  sources:
    - apiVersion: v1
      kind: Pod
      op: count

Selectors

Each source can optionally include extra selectors to further restrict which objects contribute to usage. Capsule evaluates selectors after the object already matched the source GVK. Each entry will be aggregated with OR semantics, meaning that if an object matches any of the selector entries, it is counted. LabelSelectors and FieldSelectors can be combined within the same selector entry with AND semantics, meaning that an object must match both to be counted.

LabelSelectors

A source selector may contain Kubernetes-style matchLabels / matchExpressions against the object labels.

spec:
  sources:
    - apiVersion: v1
      kind: PersistentVolumeClaim
      op: add
      path: .spec.resources.requests.storage
      selectors:
        - matchExpressions:
            - key: "team"
              operator: In
              values: ["platform", "dev"]
        - matchLabels:
            - key: "team"
              operator: In
              values: ["platform", "dev"]

FieldSelectors

fieldSelectors are additional per-source filters. Each entry is a JSONPath expression evaluated against the candidate object.

View the available matching semantics for fieldSelectors.

spec:
  sources:
    - apiVersion: v1
      kind: PersistentVolumeClaim
      op: add
      path: .spec.resources.requests.storage
      selectors:
        - fieldSelectors:
          - '.spec.accessModes[?(@=="ReadWriteOnce")]'
          - '.status.phase'

the selector matches only if:

.spec.accessModes[?(@=="ReadWriteOnce")] returns a non-empty result, and
.status.phase returns a non-empty result

Within one selectors entry:

labelSelector AND all fieldSelectors

Across multiple selectors entries:

OR semantics

FieldSelectors are not Kubernetes API field selectors. They are evaluated by Capsule using JSONPath after the object has been listed.

Examples

Match PVCs that contain ReadWriteMany

selectors:
  - fieldSelectors:
      - '.spec.accessModes[?(@=="ReadWriteMany")]'

Match objects where a field exists

selectors:
  - fieldSelectors:
      - '.spec.storageClassName'

Match objects where a boolean field is true

selectors:
  - fieldSelectors:
      - '.spec.suspend'

If .spec.suspend resolves to true, it matches. If it resolves to false or is missing, it does not match.

Match objects with a specific condition present in an array

selectors:
  - fieldSelectors:
      - '.status.conditions[?(@.type=="Ready")]'

This matches if at least one Ready condition exists.

Match objects where a field does not equal a value

selectors:
  - fieldSelectors:
      - '.status.phase!=Succeeded'

This excludes objects whose .status.phase equals Succeeded.

GlobalCustomQuota

GlobalCustomQuota aggregates usage across multiple namespaces.

Sources

Sources can be distributed across many namespaces. Other than that, they follow the same Sources rules.

Selectors

Selectors pre-filter which items are considered for the quota. Only items matching the selectors are counted towards usage. Selectors from Sources are applied after the source GVK is matched, so they can be used to further filter which objects are counted based on their labels or fields. However they can’t select items which are not selected by the selectors on GlobalCustomQuota level. This means that if you want to select items across multiple namespaces, you need to use namespaceSelectors and not selectors.

NamespaceSelectors

Definition of spec.namespaceSelectors determines which namespaces are in scope. Only objects from matching namespaces are considered. This enforces a 500Gi cap on ObjectBucketClaim storage in namespaces labeled with capsule.clastix.io/tenant=solar:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: object-bucket-claim-storage
spec:
  limit: "500Gi"
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: objectbucket.io/v1alpha1
      kind: ObjectBucketClaim
      op: add
      path: .spec.additionalConfig.maxSize

The collected namespaces are also reported in status.namespaces and can be used for informational purposes or by external systems to understand which namespaces are contributing to the quota usage.

kubectl get globalcustomquota object-bucket-claim-storage -o yaml

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: object-bucket-claim-storage
spec:
  limit: "500Gi"
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: objectbucket.io/v1alpha1
      kind: ObjectBucketClaim
      op: add
      path: .spec.additionalConfig.maxSize
status:
  conditions:
  - lastTransitionTime: "2026-04-17T08:13:17Z"
    message: reconciled
    reason: Succeeded
    status: "True"
    type: Ready
  namespaces:
  - solar-prod
  targets:
    - version: v1alpha1
      kind: ObjectBucketClaim
      group: objectbucket.io
      op: add
      path: .spec.additionalConfig.maxSize
  usage:
    available: "500Gi"
    used: "0"

ScopeSelectors

Sources can be distributed across multiple namespaces. Other than that, they follow the same Sources rules. This enforces a 500Gi cap on ObjectBucketClaim storage in namespaces labeled with capsule.clastix.io/tenant=solar, counting only claims labeled with objectbucket.io/storage-class=gold:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: object-bucket-claim-storage
spec:
  limit: "500Gi"
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  scopeSelectors:
    - matchLabels:
        objectbucket.io/storage-class: gold
  sources:
    - apiVersion: objectbucket.io/v1alpha1
      kind: ObjectBucketClaim
      op: add
      path: .spec.additionalConfig.maxSize

Options

Additional options available for GlobalCustomQuota.

emitMetricPerClaimUsage

Additionally expose usage metrics for each claim contributing to the quota. This is disabled by default to avoid high cardinality in the metrics, but can be enabled for more granular monitoring and alerting. By default this option is disabled.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: object-bucket-claim-storage
spec:
  options:
    emitMetricPerClaimUsage: true
  ...

Example metrics:

# HELP capsule_global_custom_quota_resource_item_usage Claimed resources from given item
# TYPE capsule_global_custom_quota_resource_item_usage gauge
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-299zf",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-5hzp9",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-9zzzw",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-gnf8f",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-l68c5",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-lrzvd",target_namespace="solar-test"} 0.25

Examples

Feel free to contribute examples if you have found interesting use cases!

Calculate total requested CPU across tenant namespaces (Running Pods)

This pattern mirrors the default behavior of Kubernetes ResourceQuota, which excludes terminal Pods from CPU and memory accounting.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: tenant-cpu-limits-quota
spec:
  limit: "15"
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.containers[*].resources.limits.cpu
      selectors:
        - fieldSelectors:
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.initContainers[*].resources.limits.cpu
      selectors:
        - fieldSelectors:
            - '.spec.initContainers'
            - '.status.phase!=Succeeded'
            - '.status.phase!=Failed'
            - '.status.phase!=Unknown'

Limit total max storage across bucket claims for selected namespaces

This enforces a 500Gi cap on max storage requested by ObjectBucketClaims in all namespaces labeled with capsule.clastix.io/tenant=solar, but only counting those claims with the storage class label objectbucket.io/storage-class=gold.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: object-bucket-claim-storage
spec:
  limit: "500Gi"
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  scopeSelectors:
    - matchLabels:
        objectbucket.io/storage-class: gold
  sources:
    - apiVersion: objectbucket.io/v1alpha1
      kind: ObjectBucketClaim
      op: add
      path: .spec.additionalConfig.maxSize

Limit the number of LoadBalancer Services across tenant namespaces

This limits the total number of Services of type LoadBalancer across all namespaces of one tenant.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: customer-a-loadbalancers
spec:
  limit: 3
  namespaceSelectors:
    - matchLabels:
        customer: a
  sources:
    - apiVersion: v1
      kind: Service
      op: count
      selectors:
        - fieldSelectors:
            - '.spec.type=="LoadBalancer"'

Aggregate ephemeral and persistent storage

This policy combines:

storage requested by Pod ephemeral volume claim templates
storage requested by PVCs

---
apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: solar-storage-aggregate
spec:
  limit: 5Gi
  namespaceSelectors:
  - matchLabels:
      capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: v1
      kind: Pod
      op: add
      path: ".spec.volumes[*].ephemeral.volumeClaimTemplate.spec.resources.requests.storage"
    - apiVersion: v1
      kind: PersistentVolumeClaim
      op: add
      path: ".spec.resources.requests.storage"
      selectors:
        - fieldSelectors:
            - '.spec.accessModes[?(@=="ReadWriteOnce")]'

Count Crossplane Buckets across tenant namespaces

This limits how many Crossplane S3 buckets may exist across a tenant.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: tenant-crossplane-buckets
spec:
  limit: 5
  namespaceSelectors:
    - matchLabels:
        capsule.clastix.io/tenant: solar
  sources:
    - apiVersion: s3.aws.upbound.io/v1beta1
      kind: Bucket
      op: count

Monitoring

See how you can monitor GlobalCustomQuota usage via Prometheus metrics. The example metrics are based on this GlobalCustomQuota definition:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalCustomQuota
metadata:
  name: cpu-limits
spec:
  limit: 5
  namespaceSelectors:
  - matchLabels:
      capsule.clastix.io/tenant: solar
  sources:
  - apiVersion: v1
    kind: Pod
    op: add
    path: .spec.containers[*].resources.limits.cpu
    selectors:
      - fieldSelectors:
          - '.status.phase!=Succeeded'
          - '.status.phase!=Failed'
          - '.status.phase!=Unknown'
  - apiVersion: v1
    kind: Pod
    op: add
    path: .spec.initContainers[*].resources.limits.cpu
    selectors:
      - fieldSelectors:
          - '.spec.initContainers'
          - '.status.phase!=Succeeded'
          - '.status.phase!=Failed'
          - '.status.phase!=Unknown'
status:
  claims:
  - group: ""
    kind: Pod
    name: nginx-deployment-6ff89574f8-299zf
    namespace: solar-test
    uid: f7ff7d7c-7128-4f44-ad13-3c44882420f8
    usage: 250m
    version: v1
  - group: ""
    kind: Pod
    name: nginx-deployment-6ff89574f8-9zzzw
    namespace: solar-test
    uid: 24c1bdea-000d-4e10-8af6-eb23c44ceaa3
    usage: 250m
    version: v1
  - group: ""
    kind: Pod
    name: nginx-deployment-6ff89574f8-gnf8f
    namespace: solar-test
    uid: 25368dd6-b3e7-4cbd-9fc8-9082db50372e
    usage: 250m
    version: v1
  - group: ""
    kind: Pod
    name: nginx-deployment-6ff89574f8-l68c5
    namespace: solar-test
    uid: bb697ba6-6512-4d63-acf8-6d058364c9d4
    usage: 250m
    version: v1
  - group: ""
    kind: Pod
    name: nginx-deployment-6ff89574f8-lrzvd
    namespace: solar-test
    uid: 50556db5-0134-4f0a-a0b8-56235f2bdc59
    usage: 250m
    version: v1
  - group: ""
    kind: Pod
    name: nginx-deployment-6ff89574f8-5hzp9
    namespace: solar-test
    uid: 7c6d1252-f649-4106-bfae-22c558c798df
    usage: 250m
    version: v1
  conditions:
  - lastTransitionTime: "2026-04-17T08:29:15Z"
    message: reconciled
    reason: Succeeded
    status: "True"
    type: Ready
  namespaces:
  - solar-prod
  - solar-test
  targets:
  - group: ""
    kind: Pod
    op: add
    path: .spec.containers[*].resources.limits.cpu
    scope: namespace
    version: v1
  - group: ""
    kind: Pod
    op: add
    path: .spec.initContainers[*].resources.limits.cpu
    scope: namespace
    version: v1
  usage:
    available: 3500m
    used: 1500m

Metrics

The following metrics are exposed for each GlobalCustomQuota:

# HELP capsule_global_custom_quota_condition Provides per global custom quota condition status
# TYPE capsule_global_custom_quota_condition gauge
capsule_global_custom_quota_condition{condition="Ready",custom_quota="cpu-limits"} 1

# TYPE capsule_global_custom_quota_resource_limit gauge
capsule_global_custom_quota_resource_limit{custom_quota="cpu-limits"} 5

# TYPE capsule_global_custom_quota_resource_available gauge
capsule_global_custom_quota_resource_available{custom_quota="cpu-limits"} 3.5

# TYPE capsule_global_custom_quota_resource_usage gauge
capsule_global_custom_quota_resource_usage{custom_quota="cpu-limits"} 1.5

## -- Requires .spec.options.emitMetricPerClaimUsage to be enabled
## May cause high cardinality if many claims are present, use with caution.

# HELP capsule_global_custom_quota_resource_item_usage Claimed resources from given item
# TYPE capsule_global_custom_quota_resource_item_usage gauge
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-299zf",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-5hzp9",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-9zzzw",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-gnf8f",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-l68c5",target_namespace="solar-test"} 0.25
capsule_global_custom_quota_resource_item_usage{custom_quota="cpu-limits",group="",kind="Pod",name="nginx-deployment-6ff89574f8-lrzvd",target_namespace="solar-test"} 0.25

CustomQuota

CustomQuota is namespaced and only counts resources in the same namespace as the quota.

Sources

Sources can originate in the same Namespace as the CustomQuota is deployed in. Other than that, they follow the same Sources rules.

Selectors

ScopeSelectors

Sources can be distributed across multiple namespaces. Other than that, they follow the same Sources rules. This enforces a 500Gi cap on ObjectBucketClaim storage counting only claims labeled with objectbucket.io/storage-class=gold:

apiVersion: capsule.clastix.io/v1beta2
kind: CustomQuota
metadata:
  name: object-bucket-claim-storage
  namespace: solar-test
spec:
  limit: "500Gi"
  scopeSelectors:
    - matchLabels:
        objectbucket.io/storage-class: gold
  sources:
    - apiVersion: objectbucket.io/v1alpha1
      kind: ObjectBucketClaim
      op: add
      path: .spec.additionalConfig.maxSize

Options

Additional options available for CustomQuota.

emitMetricPerClaimUsage

apiVersion: capsule.clastix.io/v1beta2
kind: CustomQuota
metadata:
  name: pod-count-limit
  namespace: wind-test
spec:
  options:
    emitMetricPerClaimUsage: true
  ...

Example metrics:

# HELP capsule_custom_quota_resource_item_usage Claimed resources from given item
# TYPE capsule_custom_quota_resource_item_usage gauge
capsule_custom_quota_resource_item_usage{custom_quota="pod-count-limit",group="",kind="Pod",name="nginx-deployment-77bc6bd484-4qm4h",target_namespace="wind-test"} 1
capsule_custom_quota_resource_item_usage{custom_quota="pod-count-limit",group="",kind="Pod",name="nginx-deployment-77bc6bd484-bsnfz",target_namespace="wind-test"} 1
capsule_custom_quota_resource_item_usage{custom_quota="pod-count-limit",group="",kind="Pod",name="nginx-deployment-77bc6bd484-f8qcv",target_namespace="wind-test"} 1

Examples

Feel free to contribute examples if you have found interesting use cases!

Limit total PVC storage in one namespace

apiVersion: capsule.clastix.io/v1beta2
kind: CustomQuota
metadata:
  name: pvc-storage-limit
  namespace: team-a
spec:
  limit: "200Gi"
  scopeSelectors:
    - matchLabels:
        team: platform
  sources:
    - apiVersion: v1
      kind: PersistentVolumeClaim
      op: add
      path: .spec.resources.requests.storage

Limit the number of LoadBalancer Services in one namespace

apiVersion: capsule.clastix.io/v1beta2
kind: CustomQuota
metadata:
  name: namespace-loadbalancers
  namespace: team-a
spec:
  limit: 2
  sources:
    - apiVersion: v1
      kind: Service
      op: count
      selectors:
        - fieldSelectors:
            - '.spec.type=="LoadBalancer"'

Limit total memory requests of Pods in one namespace

apiVersion: capsule.clastix.io/v1beta2
kind: CustomQuota
metadata:
  name: pod-memory-requests
  namespace: team-a
spec:
  limit: 16Gi
  sources:
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.containers[*].resources.requests.memory
    - apiVersion: v1
      kind: Pod
      op: add
      path: .spec.initContainers[*].resources.requests.memory
      selectors:
        - fieldSelectors:
            - '.spec.initContainers'

Count Crossplane SQL instances in one namespace

apiVersion: capsule.clastix.io/v1beta2
kind: CustomQuota
metadata:
  name: sql-instances
  namespace: team-a
spec:
  limit: 3
  sources:
    - apiVersion: database.gcp.upbound.io/v1beta1
      kind: SQLDatabaseInstance
      op: count

Count only suspended CronJobs

apiVersion: capsule.clastix.io/v1beta2
kind: CustomQuota
metadata:
  name: suspended-cronjobs
  namespace: team-a
spec:
  limit: 5
  sources:
    - apiVersion: batch/v1
      kind: CronJob
      op: count
      selectors:
        - fieldSelectors:
            - '.spec.suspend'

Monitoring

See how you can monitor CustomQuota usage via Prometheus metrics. The example metrics are based on this CustomQuota definition:

apiVersion: capsule.clastix.io/v1beta2
kind: CustomQuota
metadata:
  name: pod-count-limit
  namespace: wind-test
spec:
  limit: 3
  options:
    emitMetricPerClaimUsage: false
  sources:
  - apiVersion: "v1"
    kind: Pod
    op: count
status:
  claims:
  - group: ""
    kind: Pod
    name: nginx-deployment-77bc6bd484-4qm4h
    namespace: wind-test
    uid: c6df70ce-f483-4b02-af65-c8c150d22ed2
    usage: "1"
    version: v1
  - group: ""
    kind: Pod
    name: nginx-deployment-77bc6bd484-f8qcv
    namespace: wind-test
    uid: eeae006b-5ce8-442b-b6c3-f208387545a7
    usage: "1"
    version: v1
  - group: ""
    kind: Pod
    name: nginx-deployment-77bc6bd484-bsnfz
    namespace: wind-test
    uid: 9e1135a7-b286-4768-becd-147b37c999f8
    usage: "1"
    version: v1
  conditions:
  - lastTransitionTime: "2026-04-23T09:21:29Z"
    message: reconciled
    reason: Succeeded
    status: "True"
    type: Ready
  targets:
  - group: ""
    kind: Pod
    op: count
    scope: namespace
    version: v1
  usage:
    available: "0"
    used: "3"

Metrics

The following metrics are exposed for each CustomQuota:

# HELP capsule_custom_quota_condition Provides per custom quota condition status
# TYPE capsule_custom_quota_condition gauge
capsule_custom_quota_condition{condition="Ready",custom_quota="pod-count-limit",target_namespace="wind-test"} 1

# HELP capsule_custom_quota_resource_available Available resources for given custom quota
# TYPE capsule_custom_quota_resource_available gauge
capsule_custom_quota_resource_available{custom_quota="pod-count-limit",target_namespace="wind-test"} 0

# HELP capsule_custom_quota_resource_limit Current resource limit for given custom quota
# TYPE capsule_custom_quota_resource_limit gauge
capsule_custom_quota_resource_limit{custom_quota="pod-count-limit",target_namespace="wind-test"} 3

# HELP capsule_custom_quota_resource_usage Current resource usage for given custom quota
# TYPE capsule_custom_quota_resource_usage gauge
capsule_custom_quota_resource_usage{custom_quota="pod-count-limit",target_namespace="wind-test"} 3


## -- Requires .spec.options.emitMetricPerClaimUsage to be enabled
## May cause high cardinality if many claims are present, use with caution.

# HELP capsule_custom_quota_resource_item_usage Claimed resources from given item
# TYPE capsule_custom_quota_resource_item_usage gauge
capsule_custom_quota_resource_item_usage{custom_quota="pod-count-limit",group="",kind="Pod",name="nginx-deployment-77bc6bd484-4qm4h",target_namespace="wind-test"} 1
capsule_custom_quota_resource_item_usage{custom_quota="pod-count-limit",group="",kind="Pod",name="nginx-deployment-77bc6bd484-bsnfz",target_namespace="wind-test"} 1
capsule_custom_quota_resource_item_usage{custom_quota="pod-count-limit",group="",kind="Pod",name="nginx-deployment-77bc6bd484-f8qcv",target_namespace="wind-test"} 1

10 - Proxy

Improve the User-Experience with the capsule-proxy

Capsule Proxy is an add-on for Capsule Operator addressing some RBAC issues when enabling multi-tenancy in Kubernetes since users cannot list the owned cluster-scoped resources. One solution to this problem would be to grant all users LIST permissions for the relevant cluster-scoped resources (eg. Namespaces). However, this would allow users to list all cluster-scoped resources, which is not desirable in a multi-tenant environment and may lead to security issues. Kubernetes RBAC cannot list only the owned cluster-scoped resources since there are no ACL-filtered APIs. For example:

Error from server (Forbidden): namespaces is forbidden:
User "alice" cannot list resource "namespaces" in API group "" at the cluster scope

The reason, as the error message reported, is that the RBAC list action is available only at Cluster-Scope and it is not granted to users without appropriate permissions.

To overcome this problem, many Kubernetes distributions introduced mirrored custom resources supported by a custom set of ACL-filtered APIs. However, this leads to radically change the user’s experience of Kubernetes by introducing hard customizations that make it painful to move from one distribution to another.

With Capsule, we took a different approach. As one of the key goals, we want to keep the same user experience on all the distributions of Kubernetes. We want people to use the standard tools they already know and love and it should just work.

Integrations

Capsule Proxy is a strong addition for Capsule and works well with other CNCF kubernetes based solutions. It allows any user facing solutions, which make impersonated requests as the users, to proxy the request and show the results Correctly.

10.1 - Setup

Install and configure the Capsule Proxy

10.1.1 - Installation

Installation guide for the capsule-proxy

Capsule Proxy is an optional add-on of the main Capsule Operator, so make sure you have a working instance of Capsule before attempting to install it. Use the capsule-proxy only if you want Tenant Owners to list their Cluster-Scope resources.

The capsule-proxy can be deployed in standalone mode, e.g. running as a pod bridging any Kubernetes client to the APIs server. Optionally, it can be deployed as a sidecar container in the backend of a dashboard.

We officially only support the installation of Capsule using the Helm chart. The chart itself handles the Installation/Upgrade of needed CustomResourceDefinitions. The following Artifacthub repository are official:

Perform the following steps to install the capsule Operator:

Add repository:

 helm repo add projectcapsule https://projectcapsule.github.io/charts

Install Capsule-Proxy:

 helm install capsule-proxy projectcapsule/capsule-proxy -n capsule-system --create-namespace

or (OCI)

 helm install capsule-proxy oci://ghcr.io/projectcapsule/charts/capsule-proxy -n capsule-system --create-namespace

Show the status:

 helm status capsule-proxy -n capsule-system

Upgrade the Chart

 helm upgrade capsule-proxy projectcapsule/capsule-proxy -n capsule-system

or (OCI)

 helm upgrade capsule-proxy oci://ghcr.io/projectcapsule/charts/capsule-proxy --version 0.13.0

Uninstall the Chart

 helm uninstall capsule-proxy -n capsule-system

GitOps

There are no specific requirements for using Capsule with GitOps tools like ArgoCD or FluxCD. You can manage Capsule resources as you would with any other Kubernetes resource.

ArgoCD

Visit the ArgoCD Integration for more options to integrate Capsule with ArgoCD.

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: capsule
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: system
  source:
    repoURL: ghcr.io/projectcapsule/charts
    targetRevision: 0.14.6
    chart: capsule
    helm:
      valuesObject:
        ...
        proxy:
          enabled: true
          webhooks:
            enabled: true
          certManager:
            generateCertificates: true
          options:
            generateCertificates: false
            oidcUsernameClaim: "email"
            extraArgs:
            - "--feature-gates=ProxyClusterScoped=true"
          serviceMonitor:
            enabled: true
            annotations:
              argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true

  destination:
    server: https://kubernetes.default.svc
    namespace: capsule-system

  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - ServerSideApply=true
    - CreateNamespace=true
    - PrunePropagationPolicy=foreground
    - PruneLast=true
    - RespectIgnoreDifferences=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m
---
apiVersion: v1
kind: Secret
metadata:
  name: capsule-repo
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  url: ghcr.io/projectcapsule/charts
  name: capsule
  project: system
  type: helm
  enableOCI: "true"

FluxCD

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: capsule
  namespace: flux-system
spec:
  serviceAccountName: kustomize-controller
  targetNamespace: "capsule-system"
  interval: 10m
  releaseName: "capsule"
  chart:
    spec:
      chart: capsule
      version: "0.14.6"
      sourceRef:
        kind: HelmRepository
        name: capsule
      interval: 24h
  install:
    createNamespace: true
  upgrade:
    remediation:
      remediateLastFailure: true
  driftDetection:
    mode: enabled
  values:
    proxy:
      enabled: true
      webhooks:
        enabled: true
      certManager:
        generateCertificates: true
      options:
        generateCertificates: false
        oidcUsernameClaim: "email"
        extraArgs:
        - "--feature-gates=ProxyClusterScoped=true"
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: capsule
  namespace: flux-system
spec:
  type: "oci"
  interval: 12h0m0s
  url: oci://ghcr.io/projectcapsule/charts

Considerations

Considerations when deploying capsule-proxy

Scalability

For large clusters you might need to consider adjusting values for the Capsule controller.

QPS/Burst

In order to handle a large number of tenants and resources, you may need to increase the QPS and Burst values for the Capsule-Proxy. This avoids the Proxy being throttled by the Kubernetes API server (Client Rate limited). You can set the following values in the Helm chart:

options:
  # -- QPS to use for interacting with Kubernetes API Server.
  clientConnectionQPS: 200
  # -- Burst to use for interacting with kubernetes API Server.
  clientConnectionBurst: 400

API Priority and Fairness (APF)

With APF enabled, the Capsule controller will be subject to the APF configuration of the cluster. If you are running a large cluster with many users/tenants, you may need to adjust the APF configuration to ensure that the Capsule controller has sufficient resources to operate effectively. For more information on APF, see Kubernetes API Priority and Fairness.

We provide a built-in APF configuration for the Capsule-Proxy, which provides API priority for all LIST operations and especially for subjectaccessreviews and tokenreviews. This configuration is applied automatically when you install Capsule-Proxy. To enable the built-in APF configuration, set the following value in the Helm chart:

apiPriorityAndFairness:
  # -- Change to `true` if you want to insulate the API calls made by Capsule admission controller activities.
  # This will help ensure Capsule stability in busy clusters.
  # Ref: https://kubernetes.io/docs/concepts/cluster-administration/flow-control/
  enabled: true
  # -- Only the first matching FlowSchema for a given request matters. If multiple FlowSchemas match a single inbound request, it will be assigned based on the one with the highest matchingPrecedence.
  # Ref: https://kubernetes.io/docs/concepts/cluster-administration/flow-control/#flowschema
  matchingPrecedence: 900
  # -- Priority level configuration.
  # The block is directly forwarded into the priorityLevelConfiguration, so you can use whatever specification you want.
  # ref: https://kubernetes.io/docs/concepts/cluster-administration/flow-control/#prioritylevelconfiguration
  priorityLevelConfigurationSpec:
    type: Limited
    limited:
        nominalConcurrencyShares: 100
        limitResponse:
          type: Queue
          queuing:
            queues: 64
            handSize: 6
            queueLengthLimit: 100

Ignore Groups

When tokens have a lot of groups, you can try to exclude groups which may never be used for authorization. This can be done by setting options.impersonationGroupRegexp and options.ignoredImpersonationGroups in the Helm chart:

options:
    # -- Regular expression to match the groups which are considered for impersonation
  impersonationGroupRegexp: "kubernetes:.*"
  # --  Names of the groups which are not used for impersonation (considered after impersonation-group-regexp)
  ignoredImpersonationGroups:
    - "specific:ignored"

This results in less SSAR requests to the Kubernetes API server and less memory usage for the Capsule-Proxy. This is especially useful when using OIDC providers which provide a lot of groups, but only a few are used for authorization.

Exposure

Depending on your environment, you can expose the capsule-proxy by:

Gateway API (Recommended)
Ingress (Recommended)
NodePort Service
LoadBalance Service
HostPort
HostNetwork

Gateway API

If you are using a Gateway API compliant Ingress Controller, you must first make the decision, how TLS is terminiated or rather what’s possible in your environment. We have two potential options.

Backend Termination (Recommended)

This is where the TLS termination is executed by the capsule-proxy, meaning that the Ingress Controller will just forward the encrypted traffic to the capsule-proxy, which will decrypt it and forward it to the Kubernetes API Server. In this way, the client certificate authentication will be preserved and reversed to the upstream.

On your Gateway, add a listener which allows to forward the encrypted traffic to the capsule-proxy (Pass-Through TLS):

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: service-gateway
  namespace: solar-system
spec:
  gatewayClassName: default
  listeners:
    - allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              kubernetes.io/metadata.name: capsule-system
      hostname: api.cluster-name.company.com
      name: https-capsule-proxy
      port: 443
      protocol: TLS
      tls:
        mode: Passthrough

Install a TLSRoute resource to forward the encrypted traffic to the capsule-proxy:

extraManifests:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: TLSRoute
    metadata:
      name: capsule-proxy-tls-route
      namespace: capsule-system
    spec:
      parentRefs:
        - name: service-gateway
          namespace: solar-system
          sectionName: https-capsule-proxy
      hostnames:
        - api.cluster-name.company.com
      rules:
        - backendRefs:
            - name: capsule-proxy
              port: 9001

Gateway Termination

Support

With Gateway Termination currently only Bearer Token authentication is supported, meaning that users providing tokens are always able to reach the APIs Server. Client certificate authentication is not supported in this scenario. If the Gateway must terminate, you must consider using Forwarded Client Certificate Authentication (XFCC) if supported by your Gateway Controller.

When the Gateway is terminating we must ensure, that users use the corresponding CA/Serving Certificate in their Kubeconfigs. You must ensure that the CA certificate of the Gateway is distributed to the users, so they can use it to verify the identity of the capsule-proxy. In this way, the client certificate authentication will be withdrawn and not reversed to the upstream.

Just create a listener on the Gateway and Reference the TLS certificate. The following example uses the cert-manager integration:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  annotations:
    cert-manager.io/cluster-issuer: cluster-service-issuer
    cert-manager.io/private-key-algorithm: RSA
    cert-manager.io/private-key-size: "4096"
  name: service-gateway
  namespace: solar-system
spec:
  gatewayClassName: default
  listeners:
    - allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              kubernetes.io/metadata.name: capsule-system
      hostname: api.cluster-name.company.com
      name: https-capsule-proxy
      port: 443
      protocol: HTTPS
      tls:
         certificateRefs:
         - group: ""
           kind: Secret
           name: capsule-proxy-tls
         mode: Terminate

Ingress

When using an Ingress Controller, you can expose the capsule-proxy through an Ingress resource. The Ingress Controller will handle the TLS termination and forward the requests to the capsule-proxy. The capsule-proxy will then forward the requests to the Kubernetes API Server.

                +-----------+          +-----------+         +-----------+
 kubectl ------>|:443       |--------->|:9001      |-------->|:6443      |
                +-----------+          +-----------+         +-----------+
                ingress-controller     capsule-proxy         kube-apiserver

You can use the Ingress Values provided in the Helm chart to configure the Ingress resource for the capsule-proxy:

ingress:
  enabled: true
  className: "nginx" # or your ingress class name
  hosts:
    - host: capsule-proxy.company.com
      paths:
        - "/"

User Authentication

The capsule-proxy intercepts all the requests from the kubectl client directed to the APIs Server. Users using a TLS client-based authentication with a certificate and key can talk with the API Server since it can forward client certificates to the Kubernetes APIs server. You can configure the capsule-proxy to use multiple authentication methods, for example, you can prefer Bearer Token authentication over TLS client-based authentication or use Forwarded Client Certificate Authentication (XFCC) if supported by your Ingress Controller. The following sections describe the supported authentication methods.

options:
  authPreferredTypes: "BearerToken,TLSCertificate,XForwardedClientCert"

Bearer Token Authentication

Bearer Token authentication is supported by default, meaning that users providing tokens are always able to reach the APIs Server. You can configure the capsule-proxy to prefer Bearer Token authentication over TLS client-based authentication:

options:
  authPreferredTypes: "BearerToken"

Client Certificate Authentication

It is possible to protect the capsule-proxy using a certificate provided by Let’s Encrypt. Keep in mind that, in this way, the TLS termination will be executed by the Ingress Controller, meaning that the authentication based on the client certificate will be withdrawn and not reversed to the upstream. For such cases you may want to rely on the token-based authentication, for example, OIDC or Bearer tokens. Users providing tokens are always able to reach the APIs Server or consider using the Forwarded Client Certificate Authentication (XFCC) if supported by your Ingress Controller.

options:
  authPreferredTypes: "TLSCertificate"

Forwarded Client Certificate Authentication (XFCC)

If your prerequisite is exposing capsule-proxy using an Ingress, you must rely on the token-based authentication, for example, OIDC or Bearer tokens. Users providing tokens are always able to reach the APIs Server.

options:
  authPreferredTypes: "XForwardedClientCert"

By default the HTTP-Header used for the client certificate is X-Forwarded-Client-Cert, but it can be customized using the --xfcc-header-name argument:

options:
  authPreferredTypes: "XForwardedClientCert"
  extraArgs:
    - "--xfcc-header-name=X-My-Custom-Client-Cert"

Trusted Sources

CIDR ranges of trusted proxies allowed to send forwarded client certificate headers:

options:
  trustedProxyCidrs:
    - "10.0.0.0/8"
    - "127.0.0.1/32"

Certificate Management

By default, Capsule delegates its certificate management to cert-manager. This is the recommended way to manage the TLS certificates for Capsule.This relates to certifiacates for the proxy and the admissions server. However, you can also use a job to generate self-signed certificates and store them in a Kubernetes Secret:

options:
  generateCertificates: true
certManager:
  generateCertificates: false

Distribute CA within the Cluster

The capsule-proxy requires the CA certificate to be distributed to the clients. The CA certificate is stored in a Secret named capsule-proxy in the capsule-system namespace, by default. In most cases the distribution of this secret is required for other clients within the cluster (e.g. the Tekton Dashboard). If you are using Ingress or any other endpoints for all the clients, this step is probably not required.

Here’s an example of how to distribute the CA certificate to the namespace tekton-pipelines by using kubectl and jq:

 kubectl get secret capsule-proxy -n capsule-system -o json \
 | jq 'del(.metadata["namespace","creationTimestamp","resourceVersion","selfLink","uid"])' \
 | kubectl apply -n tekton-pipelines -f -

This can be used for development purposes, but it’s not recommended for production environments. Here are solutions to distribute the CA certificate, which might be useful for production environments:

Kubernetes Reflector

External Secrets (ESO)

How to distribute the CA certificate using External Secrets Operator (ESO). In the following example we have Headlamp running in a different namespace and we want to distribute the CA certificate to it.

First allow ServiceAccount headlamp to read the Secret capsule-proxy in the capsule-system namespace:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: headlamp:capsule-proxy-ca-reader
  namespace: capsule-system
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["capsule-proxy"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: headlamp:capsule-proxy-ca-reader
  namespace: capsule-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: headlamp:capsule-proxy-ca-reader
subjects:
  - kind: ServiceAccount
    name: headlamp
    namespace: dashboard-system

Create ExternalSecret to sync the Secret capsule-proxy from the capsule-system namespace to the dashboard-system namespace:

---
apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: capsule-proxy-ca
  namespace: dashboard-system
spec:
  provider:
    kubernetes:
      remoteNamespace: capsule-system
      server:
        url: https://kubernetes.default.svc
        caProvider:
          type: ConfigMap
          name: kube-root-ca.crt
          key: ca.crt
      auth:
        serviceAccount:
          name: headlamp
---
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: capsule-proxy-ca
  namespace: dashboard-system
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: SecretStore
    name: capsule-proxy-ca
  target:
    name: capsule-proxy-ca
    creationPolicy: Owner
  data:
    - secretKey: ca.crt
      remoteRef:
        key: capsule-proxy
        property: ca.crt

HTTP Support

NOTE: kubectl will not work against a http server.

Capsule proxy supports https and http, although the latter is not recommended, we understand that it can be useful for some use cases (i.e. development, working behind a TLS-terminated reverse proxy and so on). As the default behaviour is to work with https, we need to use the flag –enable-ssl=false if we want to work under http.

After having the capsule-proxy working under http, requests must provide authentication using an allowed Bearer Token.

For example:

TOKEN=<type your TOKEN>
curl -H "Authorization: Bearer $TOKEN" http://localhost:9001/api/v1/namespaces

Metrics

Starting from the v0.3.0 release, Capsule Proxy exposes Prometheus metrics available at http://0.0.0.0:8080/metrics.

The offered metrics are related to the internal controller-manager code base, such as work queue and REST client requests, and the Go runtime ones.

Along with these, metrics capsule_proxy_response_time_seconds and capsule_proxy_requests_total have been introduced and are specific to the Capsule Proxy code-base and functionalities.

capsule_proxy_response_time_seconds offers a bucket representation of the HTTP request duration. The available variables for these metrics are the following ones:

path: the HTTP path of every single request that Capsule Proxy passes to the upstream capsule_proxy_requests_total counts the global requests that Capsule Proxy is passing to the upstream with the following labels.

path: the HTTP path of every single request that Capsule Proxy passes to the upstream status: the HTTP status code of the request

10.1.2 - Controller Options

Configure the Capsule Proxy Controller

You can customize the Capsule Proxy with the following configurations.

Controller Options

You can provide additional options via the helm chart:

options:
  extraArgs:
    - --disable-caching=true

Options are also available as dedicated configuration values:

# Controller Options
options:
  # -- Set the listening port of the capsule-proxy
  listeningPort: 9001
  # -- Set leader election to true if you are running n-replicas
  leaderElection: false
  # -- Set the log verbosity of the capsule-proxy with a value from 1 to 10
  logLevel: 4
  # -- Name of the CapsuleConfiguration custom resource used by Capsule, required to identify the user groups
  capsuleConfigurationName: default
  # -- Define which groups must be ignored while proxying requests
  ignoredUserGroups: []
  # --  Names of the groups which are not used for impersonation (considered after impersonation-group-regexp)
  ignoredImpersonationGroups: []
  # -- Regular expression to match the groups which are considered for impersonation
  impersonationGroupRegexp: ""
  # -- Specify if capsule-proxy will use SSL
  oidcUsernameClaim: preferred_username
  # -- Specify if capsule-proxy will use SSL
  enableSSL: true
  # -- Set the directory, where SSL certificate and keyfile will be located
  SSLDirectory: /opt/capsule-proxy
  # -- Set the name of SSL certificate file
  SSLCertFileName: tls.crt
  # -- Set the name of SSL key file
  SSLKeyFileName: tls.key
  # -- Specify if capsule-proxy will generate self-signed SSL certificates
  generateCertificates: false
  # -- Specify additional subject alternative names for the self-signed SSL
  additionalSANs: []
  # -- Specify an override for the Secret containing the certificate for SSL. Default value is empty and referring to the generated certificate.
  certificateVolumeName: ""
  # -- Set the role bindings reflector resync period, a local cache to store mappings between users and their namespaces. [Use a lower value in case of flaky etcd server connections.](https://github.com/projectcapsule/capsule-proxy/issues/174)
  rolebindingsResyncPeriod: 10h
  # -- Disable the go-client caching to hit directly the Kubernetes API Server, it disables any local caching as the rolebinding reflector.
  disableCaching: false
  # -- Enable the rolebinding reflector, which allows to list the namespaces, where a rolebinding mentions a user.
  roleBindingReflector: false
  # -- Authentication types to be used for requests. Possible Auth Types: [BearerToken, TLSCertificate, XForwardedClientCert]
  authPreferredTypes: "BearerToken,TLSCertificate"
  # -- QPS to use for interacting with Kubernetes API Server.
  clientConnectionQPS: 20
  # -- Burst to use for interacting with kubernetes API Server.
  clientConnectionBurst: 30
  # -- Enable Pprof for profiling
  pprof: false
  # -- CIDR ranges of trusted proxies allowed to make requests to the proxy
  trustedProxyCidrs: []

The following options are available for the Capsule Proxy Controller:

      --allowed-paths strings                 URL paths which are not inspected by capsule-proxy (still require valid authentication) (default [/api,/apis,/version])
      --auth-preferred-types string           Authentication types to be used for requests. Possible Auth Types: [BearerToken, TLSCertificate, XForwardedClientCert]
                                              First match is used and can be specified multiple times as comma separated values or by using the flag multiple times. (default "[TLSCertificate,BearerToken]")
      --capsule-configuration-name string     Name of the CapsuleConfiguration used to retrieve the Capsule user groups names (default "default")
      --client-connection-burst int32         Burst to use for interacting with kubernetes apiserver. (default 30)
      --client-connection-qps float32         QPS to use for interacting with kubernetes apiserver. (default 20)
      --disable-caching                       Disable the go-client caching to hit directly the Kubernetes API Server, it disables any local caching as the rolebinding reflector (default: false)
      --enable-leader-election                Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
      --enable-pprof                          Enables Pprof endpoint for profiling (not recommend in production)
      --enable-reflector                      Enable rolebinding reflector. The reflector allows to list the namespaces, where a rolebinding mentions a user
      --enable-ssl                            Enable the bind on HTTPS for secure communication (default: true) (default true)
      --feature-gates mapStringBool           A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
                                              AllAlpha=true|false (ALPHA - default=false)
                                              AllBeta=true|false (BETA - default=false)
                                              ProxyAllNamespaced=true|false (ALPHA - default=false)
                                              ProxyClusterScoped=true|false (ALPHA - default=false)
                                              SkipImpersonationReview=true|false (ALPHA - default=false)
      --ignored-impersonation-group strings   Names of the groups which are not used for impersonation (considered after impersonation-group-regexp)
      --ignored-user-group strings            Names of the groups which requests must be ignored and proxy-passed to the upstream server
      --impersonation-group-regexp string     Regular expression to match the groups which are considered for impersonation
      --listening-port uint                   HTTP port the proxy listens to (default: 9001) (default 9001)
      --metrics-addr string                   The address the metric endpoint binds to. (default ":8080")
      --oidc-username-claim string            The OIDC field name used to identify the user (default: preferred_username) (default "preferred_username")
      --rolebindings-resync-period duration   Resync period for rolebindings reflector (default 10h0m0s)
      --ssl-cert-path string                  Path to the TLS certificate (default: /opt/capsule-proxy/tls.crt)
      --ssl-key-path string                   Path to the TLS certificate key (default: /opt/capsule-proxy/tls.key)
      --trusted-proxy-cidrs strings           CIDR ranges of trusted proxies allowed to make requests to the proxy
      --webhook-port int                      The port the webhook server binds to. (default 9443)
      --xfcc-header-name string               Name of the header inspected for forwarded client certificates (default "X-Forwarded-Client-Cert")
      --zap-devel                             Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error)
      --zap-encoder encoder                   Zap log encoding (one of 'json' or 'console')
      --zap-log-level level                   Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', 'panic'or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
      --zap-stacktrace-level level            Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic').
      --zap-time-encoding time-encoding       Zap time encoding (one of 'epoch', 'millis', 'nano', 'iso8601', 'rfc3339' or 'rfc3339nano'). Defaults to 'epoch'.

Feature Gates

Feature Gates are a set of key/value pairs that can be used to enable or disable certain features of the Capsule Proxy. The following feature gates are available:

Feature Gate	Default Value	Description
`SkipImpersonationReview`	`false`	`SkipImpersonationReview` allows to skip the impersonation review for all requests containing impersonation headers (user and groups). DANGER: Enabling this flag allows any user to impersonate as any user or group essentially bypassing any authorization. Only use this option in trusted environments where authorization/authentication is offloaded to external systems.
`ProxyClusterScoped`	`false`	`ProxyClusterScoped` allows to proxy all clusterScoped objects for all tenant users. These can be defined via ProxySettings

10.2 - ProxySettings

Configure proxy settings for your tenants

The configuration for the Proxy is also declarative via CRDs. This allows both Administrators and Tenant Owners to create flexible rules.

GlobalProxySettings

GlobalProxySettings grants selected subjects read access to labelled, cluster-scoped resources through Capsule Proxy. A subject does not have to own a Tenant, which makes this resource useful for shared infrastructure and project-wide views of resources such as PersistentVolumes, StorageClasses, ResourcePools, and Capsule global quota resources.

The ProxyClusterScoped feature gate must be enabled on Capsule Proxy:

options:
  extraArgs:
  - --feature-gates=ProxyClusterScoped=true

These permissions exist only when a request is sent through Capsule Proxy. GlobalProxySettings does not create Kubernetes ClusterRole or ClusterRoleBinding objects, so the same subject may still receive Forbidden when connecting directly to the Kubernetes API server.

Subjects and resources

Every entry in spec.rules contains one or more subjects and clusterResources. Supported subject kinds are User, Group, and ServiceAccount. Subjects must also be admitted by the Capsule Proxy user and group configuration.

apiGroups and resources use plural resource names and follow the same general conventions as Kubernetes ClusterRole rules:

Use an empty string ("") for the core Kubernetes API group.
Use the API group name, such as capsule.clastix.io, for grouped APIs.
Use "*" to match every discovered API group or resource.
Only cluster-scoped resources can be selected. For example, persistentvolumes are supported, while namespaced persistentvolumeclaims are not.

Supported operations

GlobalProxySettings supports only List and Get. Mutation operations such as Create, Update, Patch, and Delete must be granted with native Kubernetes RBAC and are not added by Capsule Proxy.

The effective mappings are:

Configured `operations`	Effective Kubernetes verbs	Request shape
Omitted	`list`, `get`	Collection and named-resource reads
`[List]`	`list`, `get`	Collection and named-resource reads
`[Get]`	`get`	Named-resource reads only
`[List, Get]`	`list`, `get`	Collection and named-resource reads

List also includes Get for backward compatibility. Older v1beta1 configurations could only specify List, while Capsule Proxy already intercepted both collection and named-resource routes. Existing rules therefore do not need to be rewritten to retain named-resource access.

Although both operations use the HTTP GET method, Kubernetes treats them as different verbs:

GET /api/v1/persistentvolumes is the Kubernetes list verb.
GET /api/v1/persistentvolumes/<name> is the Kubernetes get verb.
GET /apis/capsule.clastix.io/v1beta2/tenantowners is list.
GET /apis/capsule.clastix.io/v1beta2/tenantowners/<name> is get.

For a collection request, Capsule Proxy combines the applicable selectors and adds them to the request as a label selector. For a named-resource request, the proxy first resolves the object and grants access only when its labels match an applicable selector. A rule never grants access to objects outside its selector.

Default LIST and GET example

When operations is omitted, both collection and named-resource reads are enabled. The following rule lets Alice and members of oidc:org:platform list solar PersistentVolumes and fetch a single matching PersistentVolume by name:

apiVersion: capsule.clastix.io/v1beta1
kind: GlobalProxySettings
metadata:
  name: solar-proxy-settings
spec:
  rules:
  - subjects:
    - kind: User
      name: alice
    - kind: Group
      name: oidc:org:platform
    clusterResources:
    - apiGroups:
      - ""
      resources:
      - persistentvolumes
      selector:
        matchLabels:
          capsule.clastix.io/tenant: solar

For example, both of these requests are covered:

kubectl get persistentvolumes
kubectl get persistentvolume pvc-0574499c-f01b-4a53-85f1-ddb002de9cae

The PersistentVolume itself must contain the selected label:

metadata:
  labels:
    capsule.clastix.io/tenant: solar

GET-only example

Use Get without List when a subject may retrieve a known object but must not enumerate the collection:

apiVersion: capsule.clastix.io/v1beta1
kind: GlobalProxySettings
metadata:
  name: solar-proxy-settings
spec:
  rules:
  - subjects:
    - kind: User
      name: alice
    clusterResources:
    - apiGroups:
      - capsule.clastix.io
      resources:
      - tenantowners
      operations:
      - Get
      selector:
        matchLabels:
          projectcapsule.dev/tenant: solar

This adds permission for kubectl get tenantowner alice when that object matches the selector, but it does not add permission for kubectl get tenantowners. Native Kubernetes RBAC may still grant additional access independently.

Multiple resources and selectors

A rule can select several resources, and a GlobalProxySettings can contain several rules. The following legacy-style List declarations enable both LIST and GET because of the compatibility mapping described above:

apiVersion: capsule.clastix.io/v1beta1
kind: GlobalProxySettings
metadata:
  name: global-proxy-settings
spec:
  rules:
  - subjects:
    - kind: User
      name: alice
    clusterResources:
    - apiGroups:
      - capsule.clastix.io
      resources:
      - resourcepools
      - globalcustomquotas
      - globalresourcequotas
      - tenantowners
      operations:
      - List
      selector:
        matchLabels:
          projectcapsule.dev/tenant: green
    - apiGroups:
      - kyverno.io
      resources:
      - "*"
      operations:
      - List
      - Get
      selector:
        matchLabels:
          app.kubernetes.io/type: dev

ProxySetting

A ProxySetting is created in a namespace of a tenant, if it’s not in a namespace of a tenant it’s not regarded as valid. With the ProxySetting Tenant Owners can further improve the experience for their fellow tenant users.

apiVersion: capsule.clastix.io/v1beta1
kind: ProxySetting
metadata:
  name: solar-proxy
  namespace: solar-prod
spec:
  subjects:
  - kind: User
    name: alice
    proxySettings:
    - kind: IngressClasses
      operations:
      - List

Primitives

This will be refactored

Namespaces are treated specially. A users can list the namespaces they own, but they cannot list all the namespaces in the cluster. You can’t define additional selectors.

The proxy setting kind is an enum accepting the supported resources:

Nodes: Based on the NodeSelector and the Scheduling Expressions nodes can be listed
StorageClasses: Perform actions on the allowed StorageClasses for the tenant
IngressClasses: Perform actions on the allowed IngressClasses for the tenant
PriorityClasses: Perform actions on the allowed PriorityClasses for the tenant PriorityClasses
RuntimeClasses: Perform actions on the allowed RuntimeClasses for the tenant
PersistentVolumes: Perform actions on the PersistentVolumes owned by the tenant

The following operations belong to the legacy proxySettings field shown in this section. They are separate from the GlobalProxySettings clusterResources.operations mappings documented above.

Each Resource kind can be granted with several verbs, such as:

List
Update
Delete

Special routes for kubectl describe

When issuing a kubectl describe node, some other endpoints are put in place:

api/v1/pods?fieldSelector=spec.nodeName%3D{name}
/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/{name}

These are mandatory to retrieve the list of the running Pods on the required node and provide info about its lease status.

10.3 - Reflection

Leverage the RoleBinding Reflector to grant LIST capabilities to non-owners across namespaces.

Namespace RoleBinding reflection allows users and groups to list every Namespace in which they are referenced by a RoleBinding. The user does not need to be a Tenant Owner.

Enable the reflector in the Helm values:

options:
  roleBindingReflector: true

Alternatively, pass the corresponding command-line argument:

  --enable-reflector=true

Namespaces

The functionality of allowing access to other Namespaces is achieved by enabling the RoleBinding Reflector, which allows the Capsule Proxy to list the namespaces where a rolebinding mentions a user. First to make use of this feature you must enable it for the proxy: For example, the following Role and RoleBinding allow alice to list Pods in the green-test Namespace. Because alice is referenced by the RoleBinding, Namespace reflection also allows her to see green-test when listing Namespaces through Capsule Proxy:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-reader
  namespace: green-test
rules:
  - apiGroups:
      - ""
    resources:
      - pods
    verbs:
      - list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: alice-pod-reader
  namespace: green-test
subjects:
  - kind: User
    name: alice
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Namespace reflection supports the following subject kinds:

User
Group
ServiceAccount

The RoleBinding does not require a special label for Namespace reflection. All RoleBindings mentioning the requesting subject are considered. The label reflection.proxy.projectcapsule.dev/enabled: “true” is only required when reflecting thereferenced Role or ClusterRole permissions to other namespaced resource LIST requests.

GlobalProxySettings

The same behavior can be achieved by using GlobalProxySettings to enable further listing of Namespace resources. Note that this must use the label "kubernetes.io/metadata.name". This provides the user alice the ability to list the Namespaces green-test and green-prod without being Tenant Owner or having any other permissions on the Tenant:

apiVersion: capsule.clastix.io/v1beta1
kind: GlobalProxySettings
metadata:
  name: green-tenant-proxy-settings
spec:
  rules:
    - subjects:
        - kind: User
          name: alice
      clusterResources:
        - apiGroups:
            - ""
          resources:
            - "namespaces"
          selector:
            matchExpressions:
              - key: "kubernetes.io/metadata.name"
                operator: In
                values:
                - green-test
                - green-prod

Namespaced Items

For all namespaced items it’s possible to grant users LIST permissions within any Tenant namespace. They don’t have to be a Tenant Owner or anything. This is useful for example for operators, where they might also want to see all pods but are not directly owner on any Tenant. Reflection is resolved based on RoleBindings and the associated Roles and ClusterRoles (recommended).

The Role or ClusterRole must provide LIST permissions to the allowed resource(s). The RoleBindings considered for reflection must always use the label reflection.proxy.projectcapsule.dev/enabled: "true" to be considered for reflection. The following example shows how to grant a user LIST permissions on all pods within any Tenant namespace:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: custom:pod-viewer
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    reflection.proxy.projectcapsule.dev/enabled: "true"
  name: custom-pod-viewer
  namespace: green-test
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: custom:pod-viewer
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: alice

This grants the user alice the ability to LIST all pods within any Tenant's namespace. The RoleBinding must be created in each namespace where you want to grant this permission, and it must reference a Role or ClusterRole that has the appropriate permissions.

With this ClusterRole you can grant LIST permission to all namespaced resources from the core API:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: custom:proxy-viewer
rules:
- apiGroups: [""]
  resources: ["*"]
  verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    reflection.proxy.projectcapsule.dev/enabled: "true"
  name: custom-viewer
  namespace: green-test
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: custom:proxy-viewer
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: alice

Provision RoleBindings

Provisioning the RoleBindings via Tenant Rules is the best way to ensure that the RoleBindings are created in all namespaces of a Tenant and also in any new namespace created within the Tenant. The following example shows how to provision a RoleBinding for the user joe in all namespaces of the solar tenant:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  rules:
    - permissions:
        bindings:
          - clusterRoleName: 'custom:proxy-viewer'
            subjects:
            - apiGroup: rbac.authorization.k8s.io
              kind: User
              name: joe
            labels:
              reflection.proxy.projectcapsule.dev/enabled: "true"

By default, any user can add further RoleBindings with the reflection.proxy.projectcapsule.dev/enabled: "true" label to any RoleBinding. To prevent that, you can deny it with another Enforcement rule:

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
spec:
  rules:
    - permissions:
        bindings:
          - clusterRoleName: 'custom:proxy-viewer'
            subjects:
            - apiGroup: rbac.authorization.k8s.io
              kind: User
              name: joe
            labels:
              reflection.proxy.projectcapsule.dev/enabled: "true"
      enforce:
        action: deny
        metadata:
          - apiGroups:
              - "rbac.authorization.k8s.io/v1"
            kinds:
              - "RoleBinding"
            labels:
              reflection.proxy.projectcapsule.dev/enabled:
                values:
                  - exp: ".*"

10.4 - Gangplank

Capsule Integration with Gangplank

Gangplank is a web application that allows users to authenticate with an OIDC provider and configure their kubectl configuration file with the OpenID Connect Tokens. Gangplank is based on Gangway, which is no longer maintained.

Prerequisites

For Authentication you will need a Confidential OIDC client configured in your OIDC provider, such as Keycloak, Dex, or Google Cloud Identity. By default the Kubernetes API only validates tokens against a Public OIDC client, so you will need to configure your OIDC provider to allow the Gangplank client to issue tokens. You must make use of the Kubernetes Authentication Configuration, which allows to define multiple audiences (clients). This way we can issue tokens for a gangplank client, which is Confidential, and a kubernetes client, which is Public. The Kubernetes API will validate the tokens against both clients. The Config might look like this:

apiVersion: apiserver.config.k8s.io/v1beta1
kind: AuthenticationConfiguration
jwt:
- issuer:
    url: https://keycloak/realms/realm-name
    audiences:
    - kubernetes
    - gangplank
    audienceMatchPolicy: MatchAny # This one is important
  claimMappings:
    username:
      claim: 'email'
      prefix: ""
    groups:
      claim: 'groups'
      prefix: ""

Integration

We provide the option to install Gangplank alongside the Capsule Proxy. This allows users to authenticate with their OIDC provider and configure their kubectl configuration file with the OpenID Connect Tokens, which are valid for the Capsule Proxy Ingress. This way users can access the Kubernetes API through the Capsule Proxy without having to worry about the authentication and token management. To install gangplank, you must enable it:

gangplank:
  enabled: true

Gangplank won’t just work out of the box. You will need to provide some configuration values, which are required for gangplank to work properly. These values are:

GANGPLANK_CONFIG_AUTHORIZE_URL: https://keycloak/realms/realm-name/protocol/openid-connect/auth
GANGPLANK_CONFIG_TOKEN_URL: https://keycloak/realms/realm-name/protocol/openid-connect/token
GANGPLANK_CONFIG_REDIRECT_URL: https://gangplank.example.com/callback
GANGPLANK_CONFIG_CLIENT_ID: gangplank
GANGPLANK_CONFIG_CLIENT_SECRET: <SECRET>
GANGPLANK_CONFIG_USERNAME_CLAIM: The JWT claim to use as the username. (we use email in the authentication config above, so this should also be email)
GANGPLANK_CONFIG_APISERVER_URL: The URL Capsule Proxy Ingress. Since the users probably want to access the Kubernetes API from outside the cluster, you should use the Capsule Proxy Ingress URL here.

When using the Helm chart, you can set these values in the values.yaml file:

gangplank:
  enabled: true
  config:
     clusterName: "tenant-cluster"
     apiServerURL: "https://capsule-proxy.company.com:443"
     scopes: ["openid", "profile", "email", "groups", "offline_access"]
     redirectURL: "https://gangplank.company.com/callback"
     usernameClaim: "email"
     clientID: "gangplank"
     authorizeURL: "https://keycloak/realms/realm-name/protocol/openid-connect/auth"
     tokenURL: "https://keycloak/realms/realm-name/protocol/openid-connect/token"

  # Mount The Client Secret as Environment Variables (GANGPLANK_CONFIG_CLIENT_SECRET)
  envFrom:
  - secretRef:
       name: gangplank-secrets

Now the only thing left to do is to change the CA certificate which is provided. By default the CA certificate is set to the Kubernetes API server CA certificate, which is not valid for the Capsule Proxy Ingress. For this we can simply override the CA certificate in the Helm chart. You can do this by creating a Kubernetes Secret with the CA certificate and mounting it as a volume in the Gangplank deployment.

gangplank:
  volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: token-ca
  volumes:
    - name: token-ca
      projected:
        sources:
        - serviceAccountToken:
            path: token
        - secret:
            name: proxy-ingress-tls
            items:
            - key: tls.crt
              path: ca.crt

Note: In this example we used the tls.crt key of the proxy-ingress-tls secret. This is a classic Cert-Manager TLS secret, which contains only the Certificate and Key for the Capsule Proxy Ingress. However the Certificate contains the CA certificate as well (Certificate Chain), so we can use it to verify the Capsule Proxy Ingress. If you use a different secret, make sure to adjust the key accordingly.

If you directly use the CA certificate generated by cert-manager, your values will look like this:

gangplank:
  enabled: true
  config:
    clusterCAPath: "/capsule-proxy/ca.crt"
  volumeMounts:
    - mountPath: /capsule-proxy/
      name: proxy-ca
  volumes:
    - name: proxy-ca
      projected:
        sources:
        - secret:
            name: capsule-proxy
            items:
            - key: ca.crt
              path: ca.crt

10.5 - API Reference

API Reference

Packages:

capsule.clastix.io/v1beta1

capsule.clastix.io/v1beta1

Resource Types:

GlobalProxySettings
ProxySetting

GlobalProxySettings

GlobalProxySettings is the Schema for the globalproxysettings API.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta1	true
kind	string	GlobalProxySettings	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	GlobalProxySettingsSpec defines the desired state of GlobalProxySettings.	false
status	object	GlobalProxySettingsStatus defines the observed state of GlobalProxySettings.	false

GlobalProxySettings.spec

GlobalProxySettingsSpec defines the desired state of GlobalProxySettings.

Name	Type	Description	Required
rules	[]object	Subjects that should receive additional permissions. The subjects are selected based on the oncoming requests. They don’t have to relate to an existing tenant. However they must be part of the capsule-user groups.	true

GlobalProxySettings.spec.rules[index]

Name	Type	Description	Required
subjects	[]object	Subjects that should receive additional permissions. The subjects are selected based on the oncoming requests. They don’t have to relate to an existing tenant. However they must be part of the capsule-user groups.	true
clusterResources	[]object	Cluster Resources for tenant Owner.	false

GlobalProxySettings.spec.rules[index].subjects[index]

Name	Type	Description	Required
kind	enum	Kind of tenant owner. Possible values are “User”, “Group”, and “ServiceAccount”. Enum: User, Group, ServiceAccount	true
name	string	Name of tenant owner.	true

GlobalProxySettings.spec.rules[index].clusterResources[index]

ClusterResource Specification

Name	Type	Description	Required
apiGroups	[]string	APIGroups is the name of the APIGroup that contains the resources. If multiple API groups are specified, any action requested against any resource listed will be allowed. ‘*’ represents all resources. Empty string represents v1 api resources.	true
resources	[]string	Resources is a list of resources this rule applies to. ‘*’ represents all resources.	true
selector	object	Select all cluster scoped resources with the given label selector. Defining a selector which does not match any resources is considered not selectable (eg. using operation NotExists).	true
operations	[]enum	Operations which can be executed on the selected resources. Only GET and LIST are supported. When omitted, both operations are enabled. LIST also enables GET for backward compatibility with existing v1beta1 rules. Enum: List, Get Default: [List Get]	false

GlobalProxySettings.spec.rules[index].clusterResources[index].selector

Select all cluster scoped resources with the given label selector. Defining a selector which does not match any resources is considered not selectable (eg. using operation NotExists).

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalProxySettings.spec.rules[index].clusterResources[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalProxySettings.status

GlobalProxySettingsStatus defines the observed state of GlobalProxySettings.

Name	Type	Description	Required
conditions	[]object	Conditions contains the reconciliation conditions for this GlobalProxySettings.	false
observedGeneration	integer	ObservedGeneration is the most recent generation observed by the controller. Format: int64	false

GlobalProxySettings.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

ProxySetting

ProxySetting is the Schema for the proxysettings API.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta1	true
kind	string	ProxySetting	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	ProxySettingSpec defines the additional Capsule Proxy settings for additional users of the Tenant. Resource is Namespace-scoped and applies the settings to the belonged Tenant.	false
status	object	ProxySettingStatus defines the observed state of ProxySetting.	false

ProxySetting.spec

ProxySettingSpec defines the additional Capsule Proxy settings for additional users of the Tenant. Resource is Namespace-scoped and applies the settings to the belonged Tenant.

Name	Type	Description	Required
subjects	[]object	Subjects that should receive additional permissions.	true

ProxySetting.spec.subjects[index]

Name	Type	Description	Required
kind	enum	Kind of tenant owner. Possible values are “User”, “Group”, and “ServiceAccount” Enum: User, Group, ServiceAccount	true
name	string	Name of tenant owner.	true
clusterResources	[]object	Deprecated: ClusterResources cannot be granted through a namespaced ProxySetting. Use administrator-managed GlobalProxySettings instead. This field is retained to decode existing objects, but is ignored by the proxy and must be omitted or empty in new settings.	false
proxySettings	[]object	Deprecated: Use Global Proxy Settings instead (https://projectcapsule.dev/docs/proxy/proxysettings/#globalproxysettings) Proxy settings for tenant owner.	false

ProxySetting.spec.subjects[index].clusterResources[index]

ClusterResource Specification

Name	Type	Description	Required
apiGroups	[]string	APIGroups is the name of the APIGroup that contains the resources. If multiple API groups are specified, any action requested against any resource listed will be allowed. ‘*’ represents all resources. Empty string represents v1 api resources.	true
resources	[]string	Resources is a list of resources this rule applies to. ‘*’ represents all resources.	true
selector	object	Select all cluster scoped resources with the given label selector. Defining a selector which does not match any resources is considered not selectable (eg. using operation NotExists).	true
operations	[]enum	Operations which can be executed on the selected resources. Only GET and LIST are supported. When omitted, both operations are enabled. LIST also enables GET for backward compatibility with existing v1beta1 rules. Enum: List, Get Default: [List Get]	false

ProxySetting.spec.subjects[index].clusterResources[index].selector

Select all cluster scoped resources with the given label selector. Defining a selector which does not match any resources is considered not selectable (eg. using operation NotExists).

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

ProxySetting.spec.subjects[index].clusterResources[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

ProxySetting.spec.subjects[index].proxySettings[index]

Name	Type	Description	Required
kind	enum	Enum: Nodes, StorageClasses, IngressClasses, PriorityClasses, RuntimeClasses, PersistentVolumes	true
operations	[]enum	Enum: List, Update, Delete	true

ProxySetting.status

ProxySettingStatus defines the observed state of ProxySetting.

Name	Type	Description	Required
conditions	[]object	Conditions contains the reconciliation conditions for this ProxySetting.	false
observedGeneration	integer	ObservedGeneration is the most recent generation observed by the controller. Format: int64	false

ProxySetting.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

11 - API Reference

API Reference

Packages:

capsule.clastix.io/v1beta2
capsule.clastix.io/v1beta1

capsule.clastix.io/v1beta2

Resource Types:

CapsuleConfiguration
CustomQuota
GlobalCustomQuota
GlobalResourceQuota
GlobalTenantResource
QuantityLedger
ResourcePoolClaim
ResourcePool
RuleStatus
TenantOwner
TenantResource
Tenant

CapsuleConfiguration

CapsuleConfiguration is the Schema for the Capsule configuration API.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	CapsuleConfiguration	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	CapsuleConfigurationSpec defines the Capsule configuration.	true
status	object	CapsuleConfigurationStatus defines the Capsule configuration status.	false

CapsuleConfiguration.spec

CapsuleConfigurationSpec defines the Capsule configuration.

Name	Type	Description	Required
cacheInvalidation	string	Define the period of time upon a cache invalidation is executed for all caches. Default: 24h	true
enableTLSReconciler	boolean	Toggles the TLS reconciler, the controller that is able to generate CA and certificates for the webhooks when not using an already provided CA and certificate, or when these are managed externally with Vault, or cert-manager. Default: false	true
rbac	object	Define Properties for managed ClusterRoles by Capsule Default: map[]	true
administrators	[]object	Define entities which can act as Administrators in the capsule construct These entities are automatically owners for all existing tenants. Meaning they can add namespaces to any tenant. However they must be specific by using the capsule label for interacting with namespaces. Because if that label is not defined, it’s assumed that namespace interaction was not targeted towards a tenant and will therefore be ignored by capsule.	false
admission	object	Configuration for dynamic Validating and Mutating Admission webhooks managed by Capsule.	false
allowServiceAccountPromotion	boolean	ServiceAccounts within tenant namespaces can be promoted to owners of the given tenant this can be achieved by labeling the serviceaccount and then they are considered owners. This can only be done by other owners of the tenant. However ServiceAccounts which have been promoted to owner can not promote further serviceAccounts. Default: false	false
events	object	Event (Audit) Configuration Default: map[namespace:default]	false
forceTenantPrefix	boolean	Enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash. This is useful to avoid Namespace name collision in a public CaaS environment. Default: false	false
ignoreUserWithGroups	[]string	Define groups which when found in the request of a user will be ignored by the Capsule this might be useful if you have one group where all the users are in, but you want to separate administrators from normal users with additional groups.	false
impersonation	object	Service Account Client configuration for impersonation properties	false
nodeMetadata	object	Allows to set the forbidden metadata for the worker nodes that could be patched by a Tenant. This applies only if the Tenant has an active NodeSelector, and the Owner have right to patch their nodes.	false
overrides	object	Allows to set different name rather than the canonical one for the Capsule configuration objects, such as webhook secret or configurations. Default: map[TLSSecretName:capsule-tls mutatingWebhookConfigurationName:capsule-mutating-webhook-configuration validatingWebhookConfigurationName:capsule-validating-webhook-configuration]	false
protectedNamespaceRegex	string	Disallow creation of namespaces, whose name matches this regexp	false
userGroups	[]string	Deprecated: use users property instead (https://projectcapsule.dev/docs/operating/setup/configuration/#users) Names of the groups considered as Capsule users.	false
userNames	[]string	Deprecated: use users property instead (https://projectcapsule.dev/docs/operating/setup/configuration/#users) Names of the users considered as Capsule users.	false
users	[]object	Define entities which are considered part of the Capsule construct Users not mentioned here will be ignored by Capsule	false

CapsuleConfiguration.spec.rbac

Define Properties for managed ClusterRoles by Capsule

Name	Type	Description	Required
administrationClusterRoles	[]string	The ClusterRoles applied for Administrators Default: [capsule-namespace-deleter]	false
deleter	string	Name for the ClusterRole required to grant Namespace Deletion permissions. Default: capsule-namespace-deleter	false
promotionClusterRoles	[]string	The ClusterRoles applied for ServiceAccounts which had owner Promotion Default: [capsule-namespace-provisioner capsule-namespace-deleter]	false
provisioner	string	Name for the ClusterRole required to grant Namespace Provision permissions. Default: capsule-namespace-provisioner	false

CapsuleConfiguration.spec.administrators[index]

Name	Type	Description	Required
kind	enum	Kind of entity. Possible values are “User”, “Group”, and “ServiceAccount” Enum: User, Group, ServiceAccount	true
name	string	Name of the entity.	true

CapsuleConfiguration.spec.admission

Configuration for dynamic Validating and Mutating Admission webhooks managed by Capsule.

Name	Type	Description	Required
mutating	object	Configure dynamic Mutating Admission for Capsule	false
serviceName	string	Service Name of the Admission Service Default: capsule-webhook-service	false
validating	object	Configure dynamic Validating Admission for Capsule	false

CapsuleConfiguration.spec.admission.mutating

Configure dynamic Mutating Admission for Capsule

Name	Type	Description	Required
client	object	Client defines how the Kubernetes API server reaches the admission webhook. Exactly one of URL or Service must be configured.	true
annotations	map[string]string	Annotations added to the Admission Webhook	false
labels	map[string]string	Labels added to the Admission Webhook	false
name	string	Name the Admission Webhook	false
webhooks	[]object	Define Dynamic Admission Webhooks	false

CapsuleConfiguration.spec.admission.mutating.client

Client defines how the Kubernetes API server reaches the admission webhook. Exactly one of URL or Service must be configured.

Name	Type	Description	Required
caBundle	string	caBundle is a PEM encoded CA bundle which will be used to validate the webhook’s server certificate. If unspecified, system trust roots on the apiserver are used. Format: byte	false
service	object	service is a reference to the service for this webhook. Either `service` or `url` must be specified. If the webhook is running within the cluster, then you should use `service`.	false
url	string	url gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some apiservers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. The scheme must be “https”; the URL must begin with “https://”. A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. Attempting to use a user or basic auth e.g. “user:password@” is not allowed. Fragments ("#…") and query parameters ("?…") are not allowed, either.	false

CapsuleConfiguration.spec.admission.mutating.client.service

service is a reference to the service for this webhook. Either service or url must be specified.

If the webhook is running within the cluster, then you should use service.

Name	Type	Description	Required
name	string	name is the name of the service. Required	true
namespace	string	namespace is the namespace of the service. Required	true
path	string	path is an optional URL path which will be sent in any request to this service.	false
port	integer	port is the port on the service that hosts the webhook. Default to 443 for backward compatibility. `port` should be a valid port number (1-65535, inclusive). Format: int32	false

CapsuleConfiguration.spec.admission.mutating.webhooks[index]

Name	Type	Description	Required
admissionReviewVersions	[]string	AdmissionReviewVersions is an ordered list of preferred `AdmissionReview` versions the Webhook expects. API server will try to use first version in the list which it supports. If none of the versions specified in this list supported by API server, validation will fail for this object. If a persisted webhook configuration specifies allowed versions and does not include any versions known to the API Server, calls to the webhook will fail and be subject to the failure policy.	true
name	string	The name of the admission webhook. Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where “imagepolicy” is the name of the webhook, and kubernetes.io is the name of the organization. Required.	true
path	string	`path` is the URL path which will be sent in any request to this service.	true
sideEffects	string	SideEffects states whether this webhook has side effects. Acceptable values are: None, NoneOnDryRun (webhooks created via v1beta1 may also specify Some or Unknown). Webhooks with side effects MUST implement a reconciliation system, since a request may be rejected by a future step in the admission chain and the side effects therefore need to be undone. Requests with the dryRun attribute will be auto-rejected if they match a webhook with sideEffects == Unknown or Some.	true
failurePolicy	string	FailurePolicy defines how unrecognized errors from the admission endpoint are handled - allowed values are Ignore or Fail. Defaults to Fail.	false
matchConditions	[]object	MatchConditions is a list of conditions that must be met for a request to be sent to this webhook. Match conditions filter requests that have already been matched by the rules, namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests. There are a maximum of 64 match conditions allowed. The exact matching logic is (in order): 1. If ANY matchCondition evaluates to FALSE, the webhook is skipped. 2. If ALL matchConditions evaluate to TRUE, the webhook is called. 3. If any matchCondition evaluates to an error (but none are FALSE): - If failurePolicy=Fail, reject the request - If failurePolicy=Ignore, the error is ignored and the webhook is skipped	false
matchPolicy	string	matchPolicy defines how the “rules” list is used to match incoming requests. Allowed values are “Exact” or “Equivalent”. - Exact: match a request only if it exactly matches a specified rule. For example, if deployments can be modified via apps/v1, apps/v1beta1, and extensions/v1beta1, but “rules” only included `apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"]`, a request to apps/v1beta1 or extensions/v1beta1 would not be sent to the webhook. - Equivalent: match a request if modifies a resource listed in rules, even via another API group or version. For example, if deployments can be modified via apps/v1, apps/v1beta1, and extensions/v1beta1, and “rules” only included `apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"]`, a request to apps/v1beta1 or extensions/v1beta1 would be converted to apps/v1 and sent to the webhook. Defaults to “Equivalent”	false
namespaceSelector	object	NamespaceSelector decides whether to run the webhook on an object based on whether the namespace for that object matches the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is another cluster scoped resource, it never skips the webhook. For example, to run the webhook on any objects whose namespace is not associated with “runlevel” of “0” or “1”; you will set the selector as follows: “namespaceSelector”: { “matchExpressions”: [ { “key”: “runlevel”, “operator”: “NotIn”, “values”: [ “0”, “1” ] } ] } If instead you want to only run the webhook on any objects whose namespace is associated with the “environment” of “prod” or “staging”; you will set the selector as follows: “namespaceSelector”: { “matchExpressions”: [ { “key”: “environment”, “operator”: “In”, “values”: [ “prod”, “staging” ] } ] } See https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ for more examples of label selectors. Default to the empty LabelSelector, which matches everything.	false
objectSelector	object	ObjectSelector decides whether to run the webhook based on if the object has matching labels. objectSelector is evaluated against both the oldObject and newObject that would be sent to the webhook, and is considered to match if either object matches the selector. A null object (oldObject in the case of create, or newObject in the case of delete) or an object that cannot have labels (like a DeploymentRollback or a PodProxyOptions object) is not considered to match. Use the object selector only if the webhook is opt-in, because end users may skip the admission webhook by setting the labels. Default to the empty LabelSelector, which matches everything.	false
opts	object	Capsule Custom Admission Options	false
reinvocationPolicy	string	reinvocationPolicy indicates whether this webhook should be called multiple times as part of a single admission evaluation. Allowed values are “Never” and “IfNeeded”. Never: the webhook will not be called more than once in a single admission evaluation. IfNeeded: the webhook will be called at least one additional time as part of the admission evaluation if the object being admitted is modified by other admission plugins after the initial webhook call. Webhooks that specify this option must be idempotent, able to process objects they previously admitted. Note: * the number of additional invocations is not guaranteed to be exactly one. * if additional invocations result in further modifications to the object, webhooks are not guaranteed to be invoked again. * webhooks that use this option may be reordered to minimize the number of additional invocations. * to validate an object after all mutations are guaranteed complete, use a validating admission webhook instead. Defaults to “Never”.	false
rules	[]object	Rules describes what operations on what resources/subresources the webhook cares about. The webhook cares about an operation if it matches any Rule. However, in order to prevent ValidatingAdmissionWebhooks and MutatingAdmissionWebhooks from putting the cluster in a state which cannot be recovered from without completely disabling the plugin, ValidatingAdmissionWebhooks and MutatingAdmissionWebhooks are never called on admission requests for ValidatingWebhookConfiguration and MutatingWebhookConfiguration objects.	false
timeoutSeconds	integer	TimeoutSeconds specifies the timeout for this webhook. After the timeout passes, the webhook call will be ignored or the API call will fail based on the failure policy. The timeout value must be between 1 and 30 seconds. Default to 10 seconds. Format: int32	false

CapsuleConfiguration.spec.admission.mutating.webhooks[index].matchConditions[index]

MatchCondition represents a condition which must by fulfilled for a request to be sent to a webhook.

Name	Type	Description	Required
expression	string	expression represents the expression which will be evaluated by CEL. Must evaluate to bool. CEL expressions have access to the contents of the AdmissionRequest and Authorizer, organized into CEL variables: ‘object’ - The object from the incoming request. The value is null for DELETE requests. ‘oldObject’ - The existing object. The value is null for CREATE requests. ‘request’ - Attributes of the admission request(/pkg/apis/admission/types.go#AdmissionRequest). ‘authorizer’ - A CEL Authorizer. May be used to perform authorization checks for the principal (user or service account) of the request. See https://pkg.go.dev/k8s.io/apiserver/pkg/cel/library#Authz ‘authorizer.requestResource’ - A CEL ResourceCheck constructed from the ‘authorizer’ and configured with the request resource. Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/ Required.	true
name	string	name is an identifier for this match condition, used for strategic merging of MatchConditions, as well as providing an identifier for logging purposes. A good name should be descriptive of the associated expression. Name must be a qualified name consisting of alphanumeric characters, ‘-’, ‘’ or ‘.’, and must start and end with an alphanumeric character (e.g. ‘MyName’, or ‘my.name’, or ‘123-abc’, regex used for validation is ‘([A-Za-z0-9][-A-Za-z0-9.]*)?[A-Za-z0-9]’) with an optional DNS subdomain prefix and ‘/’ (e.g. ’example.com/MyName’) Required.	true

CapsuleConfiguration.spec.admission.mutating.webhooks[index].namespaceSelector

NamespaceSelector decides whether to run the webhook on an object based on whether the namespace for that object matches the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is another cluster scoped resource, it never skips the webhook.

For example, to run the webhook on any objects whose namespace is not associated with “runlevel” of “0” or “1”; you will set the selector as follows: “namespaceSelector”: { “matchExpressions”: [ { “key”: “runlevel”, “operator”: “NotIn”, “values”: [ “0”, “1” ] } ] }

If instead you want to only run the webhook on any objects whose namespace is associated with the “environment” of “prod” or “staging”; you will set the selector as follows: “namespaceSelector”: { “matchExpressions”: [ { “key”: “environment”, “operator”: “In”, “values”: [ “prod”, “staging” ] } ] }

See https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ for more examples of label selectors.

Default to the empty LabelSelector, which matches everything.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

CapsuleConfiguration.spec.admission.mutating.webhooks[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

CapsuleConfiguration.spec.admission.mutating.webhooks[index].objectSelector

ObjectSelector decides whether to run the webhook based on if the object has matching labels. objectSelector is evaluated against both the oldObject and newObject that would be sent to the webhook, and is considered to match if either object matches the selector. A null object (oldObject in the case of create, or newObject in the case of delete) or an object that cannot have labels (like a DeploymentRollback or a PodProxyOptions object) is not considered to match. Use the object selector only if the webhook is opt-in, because end users may skip the admission webhook by setting the labels. Default to the empty LabelSelector, which matches everything.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

CapsuleConfiguration.spec.admission.mutating.webhooks[index].objectSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

CapsuleConfiguration.spec.admission.mutating.webhooks[index].opts

Capsule Custom Admission Options

Name	Type	Description	Required
administrators	boolean	If enabled, the request is only sent to admission if the user is mentioned As Part of the Capsule Administrators Default: false	true
capsuleUsers	boolean	If enabled, the request is only sent to admission if the user is mentioned As Part of the Capsule Users Default: false	true

CapsuleConfiguration.spec.admission.mutating.webhooks[index].rules[index]

RuleWithOperations is a tuple of Operations and Resources. It is recommended to make sure that all the tuple expansions are valid.

Name	Type	Description	Required
apiGroups	[]string	apiGroups is the API groups the resources belong to. ‘’ is all groups. If ‘’ is present, the length of the slice must be one. Required.	false
apiVersions	[]string	apiVersions is the API versions the resources belong to. ‘’ is all versions. If ‘’ is present, the length of the slice must be one. Required.	false
operations	[]string	operations is the operations the admission hook cares about - CREATE, UPDATE, DELETE, CONNECT or for all of those operations and any future admission operations that are added. If ‘’ is present, the length of the slice must be one. Required.	false
resources	[]string	resources is a list of resources this rule applies to. For example: ‘pods’ means pods. ‘pods/log’ means the log subresource of pods. ’’ means all resources, but not subresources. ‘pods/’ means all subresources of pods. ’/scale’ means all scale subresources. ’/*’ means all resources and their subresources. If wildcard is present, the validation rule will ensure resources do not overlap with each other. Depending on the enclosing object, subresources might not be allowed. Required.	false
scope	string	scope specifies the scope of this rule. Valid values are “Cluster”, “Namespaced”, and “" “Cluster” means that only cluster-scoped resources will match this rule. Namespace API objects are cluster-scoped. “Namespaced” means that only namespaced resources will match this rule. ”" means that there are no scope restrictions. Subresources match the scope of their parent resource. Default is “*”.	false

CapsuleConfiguration.spec.admission.validating

Configure dynamic Validating Admission for Capsule

Name	Type	Description	Required
client	object	Client defines how the Kubernetes API server reaches the admission webhook. Exactly one of URL or Service must be configured.	true
annotations	map[string]string	Annotations added to the Admission Webhook	false
labels	map[string]string	Labels added to the Admission Webhook	false
name	string	Name the Admission Webhook	false
webhooks	[]object	Define Dynamic Admission Webhooks	false

CapsuleConfiguration.spec.admission.validating.client

Client defines how the Kubernetes API server reaches the admission webhook. Exactly one of URL or Service must be configured.

Name	Type	Description	Required
caBundle	string	caBundle is a PEM encoded CA bundle which will be used to validate the webhook’s server certificate. If unspecified, system trust roots on the apiserver are used. Format: byte	false
service	object	service is a reference to the service for this webhook. Either `service` or `url` must be specified. If the webhook is running within the cluster, then you should use `service`.	false
url	string	url gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some apiservers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. The scheme must be “https”; the URL must begin with “https://”. A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. Attempting to use a user or basic auth e.g. “user:password@” is not allowed. Fragments ("#…") and query parameters ("?…") are not allowed, either.	false

CapsuleConfiguration.spec.admission.validating.client.service

service is a reference to the service for this webhook. Either service or url must be specified.

If the webhook is running within the cluster, then you should use service.

Name	Type	Description	Required
name	string	name is the name of the service. Required	true
namespace	string	namespace is the namespace of the service. Required	true
path	string	path is an optional URL path which will be sent in any request to this service.	false
port	integer	port is the port on the service that hosts the webhook. Default to 443 for backward compatibility. `port` should be a valid port number (1-65535, inclusive). Format: int32	false

CapsuleConfiguration.spec.admission.validating.webhooks[index]

Name	Type	Description	Required
admissionReviewVersions	[]string	AdmissionReviewVersions is an ordered list of preferred `AdmissionReview` versions the Webhook expects. API server will try to use first version in the list which it supports. If none of the versions specified in this list supported by API server, validation will fail for this object. If a persisted webhook configuration specifies allowed versions and does not include any versions known to the API Server, calls to the webhook will fail and be subject to the failure policy.	true
name	string	The name of the admission webhook. Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where “imagepolicy” is the name of the webhook, and kubernetes.io is the name of the organization. Required.	true
path	string	`path` is the URL path which will be sent in any request to this service.	true
sideEffects	string	SideEffects states whether this webhook has side effects. Acceptable values are: None, NoneOnDryRun (webhooks created via v1beta1 may also specify Some or Unknown). Webhooks with side effects MUST implement a reconciliation system, since a request may be rejected by a future step in the admission chain and the side effects therefore need to be undone. Requests with the dryRun attribute will be auto-rejected if they match a webhook with sideEffects == Unknown or Some.	true
failurePolicy	string	FailurePolicy defines how unrecognized errors from the admission endpoint are handled - allowed values are Ignore or Fail. Defaults to Fail.	false
matchConditions	[]object	MatchConditions is a list of conditions that must be met for a request to be sent to this webhook. Match conditions filter requests that have already been matched by the rules, namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests. There are a maximum of 64 match conditions allowed. The exact matching logic is (in order): 1. If ANY matchCondition evaluates to FALSE, the webhook is skipped. 2. If ALL matchConditions evaluate to TRUE, the webhook is called. 3. If any matchCondition evaluates to an error (but none are FALSE): - If failurePolicy=Fail, reject the request - If failurePolicy=Ignore, the error is ignored and the webhook is skipped	false
matchPolicy	string	matchPolicy defines how the “rules” list is used to match incoming requests. Allowed values are “Exact” or “Equivalent”. - Exact: match a request only if it exactly matches a specified rule. For example, if deployments can be modified via apps/v1, apps/v1beta1, and extensions/v1beta1, but “rules” only included `apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"]`, a request to apps/v1beta1 or extensions/v1beta1 would not be sent to the webhook. - Equivalent: match a request if modifies a resource listed in rules, even via another API group or version. For example, if deployments can be modified via apps/v1, apps/v1beta1, and extensions/v1beta1, and “rules” only included `apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"]`, a request to apps/v1beta1 or extensions/v1beta1 would be converted to apps/v1 and sent to the webhook. Defaults to “Equivalent”	false
namespaceSelector	object	NamespaceSelector decides whether to run the webhook on an object based on whether the namespace for that object matches the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is another cluster scoped resource, it never skips the webhook. For example, to run the webhook on any objects whose namespace is not associated with “runlevel” of “0” or “1”; you will set the selector as follows: “namespaceSelector”: { “matchExpressions”: [ { “key”: “runlevel”, “operator”: “NotIn”, “values”: [ “0”, “1” ] } ] } If instead you want to only run the webhook on any objects whose namespace is associated with the “environment” of “prod” or “staging”; you will set the selector as follows: “namespaceSelector”: { “matchExpressions”: [ { “key”: “environment”, “operator”: “In”, “values”: [ “prod”, “staging” ] } ] } See https://kubernetes.io/docs/concepts/overview/working-with-objects/labels for more examples of label selectors. Default to the empty LabelSelector, which matches everything.	false
objectSelector	object	ObjectSelector decides whether to run the webhook based on if the object has matching labels. objectSelector is evaluated against both the oldObject and newObject that would be sent to the webhook, and is considered to match if either object matches the selector. A null object (oldObject in the case of create, or newObject in the case of delete) or an object that cannot have labels (like a DeploymentRollback or a PodProxyOptions object) is not considered to match. Use the object selector only if the webhook is opt-in, because end users may skip the admission webhook by setting the labels. Default to the empty LabelSelector, which matches everything.	false
opts	object	Capsule Custom Admission Options	false
rules	[]object	Rules describes what operations on what resources/subresources the webhook cares about. The webhook cares about an operation if it matches any Rule. However, in order to prevent ValidatingAdmissionWebhooks and MutatingAdmissionWebhooks from putting the cluster in a state which cannot be recovered from without completely disabling the plugin, ValidatingAdmissionWebhooks and MutatingAdmissionWebhooks are never called on admission requests for ValidatingWebhookConfiguration and MutatingWebhookConfiguration objects.	false
timeoutSeconds	integer	TimeoutSeconds specifies the timeout for this webhook. After the timeout passes, the webhook call will be ignored or the API call will fail based on the failure policy. The timeout value must be between 1 and 30 seconds. Default to 10 seconds. Format: int32	false

CapsuleConfiguration.spec.admission.validating.webhooks[index].matchConditions[index]

MatchCondition represents a condition which must by fulfilled for a request to be sent to a webhook.

Name	Type	Description	Required
expression	string	expression represents the expression which will be evaluated by CEL. Must evaluate to bool. CEL expressions have access to the contents of the AdmissionRequest and Authorizer, organized into CEL variables: ‘object’ - The object from the incoming request. The value is null for DELETE requests. ‘oldObject’ - The existing object. The value is null for CREATE requests. ‘request’ - Attributes of the admission request(/pkg/apis/admission/types.go#AdmissionRequest). ‘authorizer’ - A CEL Authorizer. May be used to perform authorization checks for the principal (user or service account) of the request. See https://pkg.go.dev/k8s.io/apiserver/pkg/cel/library#Authz ‘authorizer.requestResource’ - A CEL ResourceCheck constructed from the ‘authorizer’ and configured with the request resource. Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/ Required.	true
name	string	name is an identifier for this match condition, used for strategic merging of MatchConditions, as well as providing an identifier for logging purposes. A good name should be descriptive of the associated expression. Name must be a qualified name consisting of alphanumeric characters, ‘-’, ‘’ or ‘.’, and must start and end with an alphanumeric character (e.g. ‘MyName’, or ‘my.name’, or ‘123-abc’, regex used for validation is ‘([A-Za-z0-9][-A-Za-z0-9.]*)?[A-Za-z0-9]’) with an optional DNS subdomain prefix and ‘/’ (e.g. ’example.com/MyName’) Required.	true

CapsuleConfiguration.spec.admission.validating.webhooks[index].namespaceSelector

See https://kubernetes.io/docs/concepts/overview/working-with-objects/labels for more examples of label selectors.

Default to the empty LabelSelector, which matches everything.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

CapsuleConfiguration.spec.admission.validating.webhooks[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

CapsuleConfiguration.spec.admission.validating.webhooks[index].objectSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

CapsuleConfiguration.spec.admission.validating.webhooks[index].objectSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

CapsuleConfiguration.spec.admission.validating.webhooks[index].opts

Capsule Custom Admission Options

Name	Type	Description	Required
administrators	boolean	If enabled, the request is only sent to admission if the user is mentioned As Part of the Capsule Administrators Default: false	true
capsuleUsers	boolean	If enabled, the request is only sent to admission if the user is mentioned As Part of the Capsule Users Default: false	true

CapsuleConfiguration.spec.admission.validating.webhooks[index].rules[index]

RuleWithOperations is a tuple of Operations and Resources. It is recommended to make sure that all the tuple expansions are valid.

Name	Type	Description	Required
apiGroups	[]string	apiGroups is the API groups the resources belong to. ‘’ is all groups. If ‘’ is present, the length of the slice must be one. Required.	false
apiVersions	[]string	apiVersions is the API versions the resources belong to. ‘’ is all versions. If ‘’ is present, the length of the slice must be one. Required.	false
operations	[]string	operations is the operations the admission hook cares about - CREATE, UPDATE, DELETE, CONNECT or for all of those operations and any future admission operations that are added. If ‘’ is present, the length of the slice must be one. Required.	false
resources	[]string	resources is a list of resources this rule applies to. For example: ‘pods’ means pods. ‘pods/log’ means the log subresource of pods. ’’ means all resources, but not subresources. ‘pods/’ means all subresources of pods. ’/scale’ means all scale subresources. ’/*’ means all resources and their subresources. If wildcard is present, the validation rule will ensure resources do not overlap with each other. Depending on the enclosing object, subresources might not be allowed. Required.	false
scope	string	scope specifies the scope of this rule. Valid values are “Cluster”, “Namespaced”, and “" “Cluster” means that only cluster-scoped resources will match this rule. Namespace API objects are cluster-scoped. “Namespaced” means that only namespaced resources will match this rule. ”" means that there are no scope restrictions. Subresources match the scope of their parent resource. Default is “*”.	false

CapsuleConfiguration.spec.events

Event (Audit) Configuration

Name	Type	Description	Required
namespace	string	Namespace where the events are logged for cluster scoped resources or deny events (default namespace) Default: default	false

CapsuleConfiguration.spec.impersonation

Service Account Client configuration for impersonation properties

Name	Type	Description	Required
caSecretKey	string	Key in the secret that holds the CA certificate (e.g., “ca.crt”) Default: ca.crt	false
caSecretName	string	Name of the secret containing the CA certificate	false
caSecretNamespace	string	Namespace where the CA certificate secret is located	false
endpoint	string	Kubernetes API Endpoint to use for impersonation	false
globalDefaultServiceAccount	string	Default ServiceAccount for global resources (GlobalTenantResource) When defined, users are required to use this ServiceAccount anywhere in the cluster unless they explicitly provide their own.	false
globalDefaultServiceAccountNamespace	string	Namespace of the default ServiceAccount for global resources (GlobalTenantResource) When defined, users are required to use this ServiceAccount anywhere in the cluster unless they explicitly provide their own.	false
skipTlsVerify	boolean	If true, TLS certificate verification is skipped (not recommended for production) Default: false	false
tenantDefaultServiceAccount	string	Default ServiceAccount for namespaced resources (TenantResource) When defined, users are required to use this ServiceAccount within the namespace where they deploy the resource, unless they explicitly provide their own.	false

CapsuleConfiguration.spec.nodeMetadata

Allows to set the forbidden metadata for the worker nodes that could be patched by a Tenant. This applies only if the Tenant has an active NodeSelector, and the Owner have right to patch their nodes.

Name	Type	Description	Required
forbiddenAnnotations	object	Define the annotations that a Tenant Owner cannot set for their nodes.	false
forbiddenLabels	object	Define the labels that a Tenant Owner cannot set for their nodes.	false

CapsuleConfiguration.spec.nodeMetadata.forbiddenAnnotations

Define the annotations that a Tenant Owner cannot set for their nodes.

Name	Type	Description	Required
denied	[]string		false
deniedRegex	string		false

CapsuleConfiguration.spec.nodeMetadata.forbiddenLabels

Define the labels that a Tenant Owner cannot set for their nodes.

Name	Type	Description	Required
denied	[]string		false
deniedRegex	string		false

CapsuleConfiguration.spec.overrides

Allows to set different name rather than the canonical one for the Capsule configuration objects, such as webhook secret or configurations.

Name	Type	Description	Required
TLSSecretName	string	Defines the Secret name used for the webhook server. Must be in the same Namespace where the Capsule Deployment is deployed. Default: capsule-tls	true
mutatingWebhookConfigurationName	string	Deprecated: use dynamic admission instead Name of the MutatingWebhookConfiguration which contains the dynamic admission controller paths and resources. Default: capsule-mutating-webhook-configuration	true
validatingWebhookConfigurationName	string	Deprecated: use dynamic admission instead Name of the ValidatingWebhookConfiguration which contains the dynamic admission controller paths and resources. Default: capsule-validating-webhook-configuration	true

CapsuleConfiguration.spec.users[index]

Name	Type	Description	Required
kind	enum	Kind of entity. Possible values are “User”, “Group”, and “ServiceAccount” Enum: User, Group, ServiceAccount	true
name	string	Name of the entity.	true

CapsuleConfiguration.status

CapsuleConfigurationStatus defines the Capsule configuration status.

Name	Type	Description	Required
conditions	[]object	Conditions holds the reconciliation conditions for this CapsuleConfiguration. Includes a Ready condition indicating whether the configuration was successfully validated and applied.	false
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false
tenants	[]string	Tenants is the sorted list of Tenant names currently present in the cluster. The total count is available via len(Tenants).	false
users	[]object	Users which are considered Capsule Users and are bound to the Capsule Tenant construct.	false

CapsuleConfiguration.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

CapsuleConfiguration.status.users[index]

Name	Type	Description	Required
kind	enum	Kind of entity. Possible values are “User”, “Group”, and “ServiceAccount” Enum: User, Group, ServiceAccount	true
name	string	Name of the entity.	true

CustomQuota

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	CustomQuota	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	CustomQuotaSpec.	true
status	object	CustomQuotaStatus defines the observed state of GlobalResourceQuota.	false

CustomQuota.spec

CustomQuotaSpec.

Name	Type	Description	Required
limit	int or string	Resource Quantity as limit	true
options	object	Additional Options for the CustomQuotaSpecification Default: map[emitMetricPerClaimUsage:false]	true
sources	[]object	Target resource	true
scopeSelectors	[]object	Select items governed by this quota	false

CustomQuota.spec.options

Additional Options for the CustomQuotaSpecification

Name	Type	Description	Required
emitMetricPerClaimUsage	boolean	Additionally expose usage metrics for each claim contributing to the quota. This is disabled by default to avoid high cardinality in the metrics, but can be enabled for more granular monitoring and alerting. Default: false	false

CustomQuota.spec.sources[index]

Name	Type	Description	Required
kind	string	Kind of the referent. Use “*” to match all kinds.	true
apiVersion	string	API version, API group, or API group/version selector of the referent. Empty APIVersion means the core Kubernetes API version “v1”. Use “” to explicitly match all API groups and versions. Examples: - "" means core “v1”. - “v1” means core “v1”. - “apps” means any version in the “apps” API group. - “apps/v1” means the “apps/v1” API group/version. - “apps/” means any version in the “apps” API group.	false
cel	string	CEL expression evaluated against the source object. The object is available as “object”. Must evaluate to kubernetes.Quantity or list<kubernetes.Quantity>. Mutually exclusive with path and must be empty when op is “count”.	false
op	enum	Operation used to evaluate usage. Enum: add, sub, count Default: add	false
path	string	Path on GVK where usage is evaluated. Must be empty when op is “count”. Required and non-empty for all other operations.	false
selectors	[]object	Provide more granular selectors for these sources The ScopeSelector and NamespaceSelector are always applied Allowing these selectors to make further selecting on the resulting subset.	false

CustomQuota.spec.sources[index].selectors[index]

Name	Type	Description	Required
celExpressions	[]string	Additional CEL expressions evaluated against the selected object. The object is available as “object”. All must evaluate to true for this selector to match. CEL expressions and fieldSelectors may be used together.	false
fieldSelectors	[]string	Additional boolean JSONPath expressions. All must evaluate to true for this selector to match.	false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

CustomQuota.spec.sources[index].selectors[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

CustomQuota.spec.scopeSelectors[index]

A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

CustomQuota.spec.scopeSelectors[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

CustomQuota.status

CustomQuotaStatus defines the observed state of GlobalResourceQuota.

Name	Type	Description	Required
conditions	[]object	Conditions	true
targets	[]object	Targeting GVK	true
claims	[]object	Objects regarding this policy	false
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false
usage	object	Usage measurements	false

CustomQuota.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

CustomQuota.status.targets[index]

Name	Type	Description	Required
group	string		true
kind	string		true
version	string		true
cel	string	CEL expression evaluated against the source object. The object is available as “object”. Must evaluate to kubernetes.Quantity or list<kubernetes.Quantity>. Mutually exclusive with path and must be empty when op is “count”.	false
op	enum	Operation used to evaluate usage. Enum: add, sub, count Default: add	false
path	string	Path on GVK where usage is evaluated. Must be empty when op is “count”. Required and non-empty for all other operations.	false
scope	string	Scope of the GVK where usage is evaluated.	false
selectors	[]object	Provide more granular selectors for these sources The ScopeSelector and NamespaceSelector are always applied Allowing these selectors to make further selecting on the resulting subset.	false

CustomQuota.status.targets[index].selectors[index]

Name	Type	Description	Required
celExpressions	[]string	Additional CEL expressions evaluated against the selected object. The object is available as “object”. All must evaluate to true for this selector to match. CEL expressions and fieldSelectors may be used together.	false
fieldSelectors	[]string	Additional boolean JSONPath expressions. All must evaluate to true for this selector to match.	false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

CustomQuota.status.targets[index].selectors[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

CustomQuota.status.claims[index]

Name	Type	Description	Required
group	string		true
kind	string		true
name	string	Name of the referent.	true
uid	string	UID of the tracked Tenant to pin point tracking	true
usage	int or string	Resource Quantity for given item	true
version	string		true
namespace	string	Namespace of the referent, when not specified it acts as LocalObjectReference.	false

CustomQuota.status.usage

Usage measurements

Name	Type	Description	Required
available	int or string	Used is the current observed total available of the resource (limit - used).	false
used	int or string	Used is the current observed total usage of the resource.	false

GlobalCustomQuota

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	GlobalCustomQuota	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	ClusterCustomQuotaSpec.	true
status	object	CustomQuotaStatus defines the observed state of GlobalResourceQuota.	false

GlobalCustomQuota.spec

ClusterCustomQuotaSpec.

Name	Type	Description	Required
limit	int or string	Resource Quantity as limit	true
options	object	Additional Options for the CustomQuotaSpecification Default: map[emitMetricPerClaimUsage:false]	true
sources	[]object	Target resource	true
namespaceSelectors	[]object	Select specifc namespaces where this Quota selects items.	false
scopeSelectors	[]object	Select items governed by this quota	false

GlobalCustomQuota.spec.options

Additional Options for the CustomQuotaSpecification

Name	Type	Description	Required
emitMetricPerClaimUsage	boolean	Additionally expose usage metrics for each claim contributing to the quota. This is disabled by default to avoid high cardinality in the metrics, but can be enabled for more granular monitoring and alerting. Default: false	false

GlobalCustomQuota.spec.sources[index]

Name	Type	Description	Required
kind	string	Kind of the referent. Use “*” to match all kinds.	true
apiVersion	string	API version, API group, or API group/version selector of the referent. Empty APIVersion means the core Kubernetes API version “v1”. Use “” to explicitly match all API groups and versions. Examples: - "" means core “v1”. - “v1” means core “v1”. - “apps” means any version in the “apps” API group. - “apps/v1” means the “apps/v1” API group/version. - “apps/” means any version in the “apps” API group.	false
cel	string	CEL expression evaluated against the source object. The object is available as “object”. Must evaluate to kubernetes.Quantity or list<kubernetes.Quantity>. Mutually exclusive with path and must be empty when op is “count”.	false
op	enum	Operation used to evaluate usage. Enum: add, sub, count Default: add	false
path	string	Path on GVK where usage is evaluated. Must be empty when op is “count”. Required and non-empty for all other operations.	false
selectors	[]object	Provide more granular selectors for these sources The ScopeSelector and NamespaceSelector are always applied Allowing these selectors to make further selecting on the resulting subset.	false

GlobalCustomQuota.spec.sources[index].selectors[index]

Name	Type	Description	Required
celExpressions	[]string	Additional CEL expressions evaluated against the selected object. The object is available as “object”. All must evaluate to true for this selector to match. CEL expressions and fieldSelectors may be used together.	false
fieldSelectors	[]string	Additional boolean JSONPath expressions. All must evaluate to true for this selector to match.	false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalCustomQuota.spec.sources[index].selectors[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalCustomQuota.spec.namespaceSelectors[index]

Selector for resources and their labels or selecting origin namespaces

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalCustomQuota.spec.namespaceSelectors[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalCustomQuota.spec.scopeSelectors[index]

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalCustomQuota.spec.scopeSelectors[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalCustomQuota.status

CustomQuotaStatus defines the observed state of GlobalResourceQuota.

Name	Type	Description	Required
conditions	[]object	Conditions	true
targets	[]object	Targeting GVK	true
claims	[]object	Objects regarding this policy	false
namespaces	[]string	Observed Namespaces	false
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false
usage	object	Usage measurements	false

GlobalCustomQuota.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

GlobalCustomQuota.status.targets[index]

Name	Type	Description	Required
group	string		true
kind	string		true
version	string		true
cel	string	CEL expression evaluated against the source object. The object is available as “object”. Must evaluate to kubernetes.Quantity or list<kubernetes.Quantity>. Mutually exclusive with path and must be empty when op is “count”.	false
op	enum	Operation used to evaluate usage. Enum: add, sub, count Default: add	false
path	string	Path on GVK where usage is evaluated. Must be empty when op is “count”. Required and non-empty for all other operations.	false
scope	string	Scope of the GVK where usage is evaluated.	false
selectors	[]object	Provide more granular selectors for these sources The ScopeSelector and NamespaceSelector are always applied Allowing these selectors to make further selecting on the resulting subset.	false

GlobalCustomQuota.status.targets[index].selectors[index]

Name	Type	Description	Required
celExpressions	[]string	Additional CEL expressions evaluated against the selected object. The object is available as “object”. All must evaluate to true for this selector to match. CEL expressions and fieldSelectors may be used together.	false
fieldSelectors	[]string	Additional boolean JSONPath expressions. All must evaluate to true for this selector to match.	false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalCustomQuota.status.targets[index].selectors[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalCustomQuota.status.claims[index]

Name	Type	Description	Required
group	string		true
kind	string		true
name	string	Name of the referent.	true
uid	string	UID of the tracked Tenant to pin point tracking	true
usage	int or string	Resource Quantity for given item	true
version	string		true
namespace	string	Namespace of the referent, when not specified it acts as LocalObjectReference.	false

GlobalCustomQuota.status.usage

Usage measurements

Name	Type	Description	Required
available	int or string	Used is the current observed total available of the resource (limit - used).	false
used	int or string	Used is the current observed total usage of the resource.	false

GlobalResourceQuota

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	GlobalResourceQuota	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	GlobalResourceQuotaSpec defines a native ResourceQuota shared by every namespace matched by any namespace selector.	true
status	object		false

GlobalResourceQuota.spec

GlobalResourceQuotaSpec defines a native ResourceQuota shared by every namespace matched by any namespace selector.

Name	Type	Description	Required
quota	object	Quota is the native Kubernetes ResourceQuota specification enforced across the selected namespaces.	true
namespaceSelectors	[]object	NamespaceSelectors select the namespaces that share this quota. Selectors are ORed; requirements within one selector are ANDed. An empty label selector matches all namespaces.	false

GlobalResourceQuota.spec.quota

Quota is the native Kubernetes ResourceQuota specification enforced across the selected namespaces.

Name	Type	Description	Required
hard	map[string]int or string	hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
scopeSelector	object	scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.	false
scopes	[]string	A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.	false

GlobalResourceQuota.spec.quota.scopeSelector

scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.

Name	Type	Description	Required
matchExpressions	[]object	A list of scope selector requirements by scope of the resources.	false

GlobalResourceQuota.spec.quota.scopeSelector.matchExpressions[index]

A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.

Name	Type	Description	Required
operator	string	Represents a scope’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.	true
scopeName	string	The name of the scope that the selector applies to.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalResourceQuota.spec.namespaceSelectors[index]

Selector for resources and their labels or selecting origin namespaces

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalResourceQuota.spec.namespaceSelectors[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalResourceQuota.status

Name	Type	Description	Required
conditions	[]object	Conditions report reconciliation and admission readiness.	true
total	object	Total contains aggregate quota usage across all selected namespaces.	true
namespaceCount	integer	NamespaceSize is the number of selected namespaces. Default: 0	false
namespaceUsage	map[string]object	NamespaceUsage contains observed quota usage per selected namespace.	false
namespaces	[]string	Namespaces is the ordered set of selected namespace names.	false
observedGeneration	integer	ObservedGeneration is the most recent generation observed by the controller. Format: int64	false

GlobalResourceQuota.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

GlobalResourceQuota.status.total

Total contains aggregate quota usage across all selected namespaces.

Name	Type	Description	Required
available	map[string]int or string	Available is max(Hard-Used, 0).	false
hard	map[string]int or string	Hard is the configured shared limit.	false
used	map[string]int or string	Used is the usage observed across the relevant namespace set.	false

GlobalResourceQuota.status.namespaceUsage[key]

Name	Type	Description	Required
used	map[string]int or string	Used is the usage observed in this namespace.	false

GlobalTenantResource

GlobalTenantResource allows to propagate resource replications to a specific subset of Tenant resources.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	GlobalTenantResource	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	GlobalTenantResourceSpec defines the desired state of GlobalTenantResource.	true
status	object	GlobalTenantResourceStatus defines the observed state of GlobalTenantResource.	false

GlobalTenantResource.spec

GlobalTenantResourceSpec defines the desired state of GlobalTenantResource.

Name	Type	Description	Required
resources	[]object	Defines the rules to select targeting Namespace, along with the objects that must be replicated.	true
resyncPeriod	string	Define the period of time upon a second reconciliation must be invoked. Keep in mind that any change to the manifests will trigger a new reconciliation. Default: 60s	true
settings	object	Deprecated: configure resources[].policy instead. Admission converts Settings into policies for blocks without one. Retained as a compatibility fallback for stored resources that have not passed through admission again. Default: map[]	true
cordoned	boolean	When cordoning a replication it will no longer execute any applies or deletions (paused). This is useful for maintenances Default: false	false
dependsOn	[]object	DependsOn may contain a meta.NamespacedObjectReference slice with references to TenantResource resources that must be ready before this TenantResource can be reconciled.	false
pruningOnDelete	boolean	When the replicated resource manifest is deleted, all the objects replicated so far will be automatically deleted. Disable this to keep replicated resources although the deletion of the replication manifest. Deprecated: use resources[].policy.deletion instead. Retained for blocks without a policy. Default: true	false
scope	enum	Resource Scope, Can either be - Tenant: Create Resources for each tenant in selected Tenants - Namespace: Create Resources for each namespace in selected Tenants Enum: Namespace, Tenant, None Default: Namespace	false
serviceAccount	object	Local ServiceAccount which will perform all the actions defined in the TenantResource You must provide permissions accordingly to that ServiceAccount	false
tenantSelector	object	Defines the Tenant selector used target the tenants on which resources must be propagated.	false

GlobalTenantResource.spec.resources[index]

Name	Type	Description	Required
additionalMetadata	object	Besides the Capsule metadata required by TenantResource controller, defines additional metadata that must be added to the replicated resources.	false
context	object	Provide additional template context, which can be used throughout all the declared items for the replication	false
generators	[]object	Templates for advanced use cases	false
namespaceSelector	object	Defines the Namespace selector to select the Tenant Namespaces on which the resources must be propagated. In case of nil value, all the Tenant Namespaces are targeted.	false
namespacedItems	[]object	List of the resources already existing in other Namespaces that must be replicated.	false
policy	object	Policy controls creation, admission protection, SSA conflict ownership, and deletion for every raw item, namespaced item, and generator in this block. An explicit policy replaces the deprecated spec.settings for this block. When omitted, admission converts legacy settings and pruningOnDelete into a policy.	false
rawItems	[]object	List of raw resources that must be replicated.	false

GlobalTenantResource.spec.resources[index].additionalMetadata

Besides the Capsule metadata required by TenantResource controller, defines additional metadata that must be added to the replicated resources.

Name	Type	Description	Required
annotations	map[string]string		false
labels	map[string]string		false

GlobalTenantResource.spec.resources[index].context

Provide additional template context, which can be used throughout all the declared items for the replication

Name	Type	Description	Required
resources	[]object		false

GlobalTenantResource.spec.resources[index].context.resources[index]

Name	Type	Description	Required
kind	string	Kind of the referent. Use “*” to match all kinds.	true
apiVersion	string	API version, API group, or API group/version selector of the referent. Empty APIVersion means the core Kubernetes API version “v1”. Use “” to explicitly match all API groups and versions. Examples: - "" means core “v1”. - “v1” means core “v1”. - “apps” means any version in the “apps” API group. - “apps/v1” means the “apps/v1” API group/version. - “apps/” means any version in the “apps” API group.	false
index	string	Index to mount the resource in the template context	false
name	string	Name of the values referent. This is useful when you traying to get a specific resource	false
namespace	string	Namespace of the values referent.	false
optional	boolean	Only relevant if name is set. If an item is not optional, there will be an error thrown when it does not exist Default: true	false
selector	object	Selector which allows to get any amount of these resources based on labels	false

GlobalTenantResource.spec.resources[index].context.resources[index].selector

Selector which allows to get any amount of these resources based on labels

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalTenantResource.spec.resources[index].context.resources[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalTenantResource.spec.resources[index].generators[index]

Name	Type	Description	Required
missingKey	enum	Missing Key Option for templating Enum: invalid, zero, error Default: zero	false
template	string	Template contains any amount of yaml which is applied to Kubernetes. This can be a single resource or multiple resources	false

GlobalTenantResource.spec.resources[index].namespaceSelector

Defines the Namespace selector to select the Tenant Namespaces on which the resources must be propagated. In case of nil value, all the Tenant Namespaces are targeted.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalTenantResource.spec.resources[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalTenantResource.spec.resources[index].namespacedItems[index]

Reference

Name	Type	Description	Required
kind	string	Kind of the referent. Use “*” to match all kinds.	true
apiVersion	string	API version, API group, or API group/version selector of the referent. Empty APIVersion means the core Kubernetes API version “v1”. Use “” to explicitly match all API groups and versions. Examples: - "" means core “v1”. - “v1” means core “v1”. - “apps” means any version in the “apps” API group. - “apps/v1” means the “apps/v1” API group/version. - “apps/” means any version in the “apps” API group.	false
name	string	Name of the values referent. This is useful when you traying to get a specific resource	false
namespace	string	Namespace of the values referent.	false
optional	boolean	Only relevant if name is set. If an item is not optional, there will be an error thrown when it does not exist Default: true	false
selector	object	Selector which allows to get any amount of these resources based on labels	false

GlobalTenantResource.spec.resources[index].namespacedItems[index].selector

Selector which allows to get any amount of these resources based on labels

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalTenantResource.spec.resources[index].namespacedItems[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalTenantResource.spec.resources[index].policy

Policy controls creation, admission protection, SSA conflict ownership, and deletion for every raw item, namespaced item, and generator in this block. An explicit policy replaces the deprecated spec.settings for this block. When omitted, admission converts legacy settings and pruningOnDelete into a policy.

Name	Type	Description	Required
condition	string	Condition is an optional CEL boolean expression checked before each target is applied. object is the existing destination object, or null when absent; now is the evaluation timestamp. False skips rendered content without pruning or releasing an already managed object. Policy and protection are still reconciled for managed objects. Evaluation errors prevent writes. Rendering still occurs before evaluation. Omitting Condition preserves the existing apply behavior. Conditions do not affect deletion or orphaning.	false
creation	enum	Creation controls how an existing target is handled. Owner requires the resource to have been created by the applying controller and otherwise returns an error. Merge adopts an existing resource when possible and creates the resource when it does not exist. Enum: Owner, Merge Default: Owner	false
deletion	enum	Deletion controls what happens when the parent stops managing the target. Remove deletes resources created for the parent and relinquishes adopted resources. Orphan keeps the resource and removes Capsule’s lifecycle metadata. Removal is the default. Enum: Remove, Orphan Default: Remove	false
force	boolean	Force allows server-side apply to acquire conflicting field ownership. Default: false	false
protect	boolean	Protect prevents users from changing or deleting the target through admission while it is managed. Protection is enabled by default. Default: true	false

GlobalTenantResource.spec.settings

Deprecated: configure resources[].policy instead. Admission converts Settings into policies for blocks without one. Retained as a compatibility fallback for stored resources that have not passed through admission again.

Name	Type	Description	Required
adopt	boolean	Enabling this allows TenanResources to interact with objects which were not created by a TenantResource. In this case on prune no deletion of the entire object is made. Deprecated: use resources[].policy.creation: Merge instead. Default: false	false
force	boolean	Force indicates that in case of conflicts with server-side apply, the client should acquire ownership of the conflicting field. You may create collisions with this. Deprecated: use resources[].policy.force instead. Default: false	false

GlobalTenantResource.spec.dependsOn[index]

LocalObjectReference contains enough information to locate the referenced Kubernetes resource object.

Name	Type	Description	Required
name	string	Name of the referent.	true

GlobalTenantResource.spec.serviceAccount

Local ServiceAccount which will perform all the actions defined in the TenantResource You must provide permissions accordingly to that ServiceAccount

Name	Type	Description	Required
name	string	Name of the referent.	true
namespace	string	Namespace of the referent.	true

GlobalTenantResource.spec.tenantSelector

Defines the Tenant selector used target the tenants on which resources must be propagated.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

GlobalTenantResource.spec.tenantSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

GlobalTenantResource.status

GlobalTenantResourceStatus defines the observed state of GlobalTenantResource.

Name	Type	Description	Required
size	integer	Number of managed resources.	true
conditions	[]object	Condition of the GlobalTenantResource.	false
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false
processedItems	[]object	List of resources managed by the API object.	false
selectedTenants	[]string	List of Tenants addressed by the GlobalTenantResource.	false
serviceAccount	object	Serviceaccount used for impersonation	false

GlobalTenantResource.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

GlobalTenantResource.status.processedItems[index]

Advanced Status Item for pin pointing items in tenants/namespaces.

Name	Type	Required
group	string	false
kind	string	false
name	string	false
namespace	string	false
origin	string	false
status	object	false
tenant	string	false
version	string	false

GlobalTenantResource.status.processedItems[index].status

Name	Type	Description	Required
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
clusterScoped	boolean	Indicates whether the referenced resource is cluster-scoped.	false
created	boolean	Indicates wether the resource was created or adopted	false
lastApply	string	An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources. Populated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency Format: date-time	false
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	false
policy	object	Policy is a lifecycle snapshot for protection and cleanup. Conditions are evaluated from the source block and omitted here. Failed updates retain the previous snapshot; a first successful content write records its policy even if metadata reconciliation fails, so cleanup respects explicit retention. An absent policy on an already applied item preserves legacy behavior.	false

GlobalTenantResource.status.processedItems[index].status.policy

Policy is a lifecycle snapshot for protection and cleanup. Conditions are evaluated from the source block and omitted here. Failed updates retain the previous snapshot; a first successful content write records its policy even if metadata reconciliation fails, so cleanup respects explicit retention. An absent policy on an already applied item preserves legacy behavior.

Name	Type	Description	Required
creation	enum	Creation controls how an existing target is handled. Owner requires the resource to have been created by the applying controller and otherwise returns an error. Merge adopts an existing resource when possible and creates the resource when it does not exist. Enum: Owner, Merge Default: Owner	false
deletion	enum	Deletion controls what happens when the parent stops managing the target. Remove deletes resources created for the parent and relinquishes adopted resources. Orphan keeps the resource and removes Capsule’s lifecycle metadata. Removal is the default. Enum: Remove, Orphan Default: Remove	false
force	boolean	Force allows server-side apply to acquire conflicting field ownership. Default: false	false
protect	boolean	Protect prevents users from changing or deleting the target through admission while it is managed. Protection is enabled by default. Default: true	false

GlobalTenantResource.status.serviceAccount

Serviceaccount used for impersonation

Name	Type	Description	Required
name	string	Name of the referent.	true
namespace	string	Namespace of the referent.	true

QuantityLedger

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	QuantityLedger	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	QuotaLedgerSpec contains the immutable target reference.	false
status	object	QuantityLedgerStatus contains the mutable coordination state used by admission and quota controllers.	false

QuantityLedger.spec

QuotaLedgerSpec contains the immutable target reference.

Name	Type	Description	Required
targetRef	object	TargetRef points to the quota object that this ledger belongs to.	true

QuantityLedger.spec.targetRef

TargetRef points to the quota object that this ledger belongs to.

Name	Type	Description	Required
kind	string	Kind of the target quota resource, for example “CustomQuota” or “GlobalCustomQuota”.	true
name	string	Name of the target quota resource.	true
apiGroup	string	APIGroup of the target quota resource, for example “capsule.clastix.io”.	false
namespace	string	Namespace of the target quota resource. Must be empty for cluster-scoped targets.	false
uid	string	UID of the target quota resource. Optional, but useful for stale reference detection.	false

QuantityLedger.status

QuantityLedgerStatus contains the mutable coordination state used by admission and quota controllers.

Name	Type	Description	Required
allocated	int or string	Allocated is the admission-owned total that has been accepted by the webhook. It must be updated only through optimistic concurrency on QuantityLedger.	false
conditions	[]object	Conditions for the resource claim	false
pendingDeletes	[]object	Pending delete hints carried over from admission delete handling.	false
reservations	[]object	Active inflight reservations for this quota.	false
reserved	int or string	Reserved is the aggregate sum of all active reservations. Controllers/webhooks should treat this as derived data from Reservations.	false
resourceQuota	object	ResourceQuota contains coordination state for GlobalResourceQuota. It is unset for CustomQuota and GlobalCustomQuota ledgers.	false

QuantityLedger.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

QuantityLedger.status.pendingDeletes[index]

QuantityLedgerPendingDelete tracks objects that are expected to disappear from claims soon, but may still temporarily appear during rebuild due to propagation delay.

Name	Type	Description	Required
createdAt	string	Format: date-time	true
objectRef	object	QuotaLedgerObjectRef identifies the object for which a reservation exists. UID may be empty for CREATE admission before the object is persisted.	true
id	string	ID identifies the admission request that added this hint. It allows a failed multi-quota admission to roll back only its own hint.	false

QuantityLedger.status.pendingDeletes[index].objectRef

QuotaLedgerObjectRef identifies the object for which a reservation exists. UID may be empty for CREATE admission before the object is persisted.

Name	Type	Description	Required
apiVersion	string	APIVersion of the tracked object, for example “v1”.	true
kind	string	Kind of the tracked object, for example “Pod”.	true
apiGroup	string	APIGroup of the tracked object.	false
name	string	Name of the tracked object.	false
namespace	string	Namespace of the tracked object.	false
uid	string	UID of the tracked object.	false

QuantityLedger.status.reservations[index]

QuantityLedgerReservation represents one active inflight reservation. ID should be stable for retries of the same admission request. In practice, admission.Request.UID is a good default.

Name	Type	Description	Required
createdAt	string	Time the reservation was first created. Format: date-time	true
id	string	Unique reservation identifier.	true
objectRef	object	Object that this reservation is intended to create/update.	true
updatedAt	string	Time the reservation was last refreshed or updated. Format: date-time	true
usage	int or string	Amount reserved for this request.	true
delta	int or string	Delta is the additional amount held against the quota while the admitted object is materializing. For creates this is normally equal to Usage. For updates it is max(newUsage-oldUsage, 0), so admission never releases capacity before the API server has persisted the update. A nil value is interpreted as Usage for backwards compatibility with ledgers written before this field was introduced.	false
expiresAt	string	Time after which the reservation may be considered stale. Format: date-time	false

QuantityLedger.status.reservations[index].objectRef

Object that this reservation is intended to create/update.

Name	Type	Description	Required
apiVersion	string	APIVersion of the tracked object, for example “v1”.	true
kind	string	Kind of the tracked object, for example “Pod”.	true
apiGroup	string	APIGroup of the tracked object.	false
name	string	Name of the tracked object.	false
namespace	string	Namespace of the tracked object.	false
uid	string	UID of the tracked object.	false

QuantityLedger.status.resourceQuota

ResourceQuota contains coordination state for GlobalResourceQuota. It is unset for CustomQuota and GlobalCustomQuota ledgers.

Name	Type	Description	Required
allocated	map[string]int or string	Allocated is Used plus all active reservations.	false
initialized	boolean	Initialized is true after every selected namespace has reported ResourceQuota status for this quota.	false
namespaces	[]string	Namespaces is the selected namespace set represented by Used.	false
observedGeneration	integer	ObservedGeneration is the GlobalResourceQuota generation represented by this ledger state. Format: int64	false
reservations	[]object	Reservations contains inflight admission operations.	false
reserved	map[string]int or string	Reserved is derived from Reservations.	false
used	map[string]int or string	Used is the usage observed from ResourceQuota status across all selected namespaces.	false

QuantityLedger.status.resourceQuota.reservations[index]

QuantityLedgerResourceQuotaReservation is an atomic reservation against all resources tracked by a GlobalResourceQuota.

Name	Type	Description	Required
createdAt	string	Format: date-time	true
id	string	Unique reservation identifier.	true
objectRef	object	Object that this reservation is intended to create/update.	true
updatedAt	string	Format: date-time	true
delta	map[string]int or string	Delta is the positive amount held while ResourceQuota status catches up.	false
expiresAt	string	Format: date-time	false
usage	map[string]int or string	Usage is the calculated usage of the admitted object.	false

QuantityLedger.status.resourceQuota.reservations[index].objectRef

Object that this reservation is intended to create/update.

Name	Type	Description	Required
apiVersion	string	APIVersion of the tracked object, for example “v1”.	true
kind	string	Kind of the tracked object, for example “Pod”.	true
apiGroup	string	APIGroup of the tracked object.	false
name	string	Name of the tracked object.	false
namespace	string	Namespace of the tracked object.	false
uid	string	UID of the tracked object.	false

ResourcePoolClaim

ResourcePoolClaim is the Schema for the resourcepoolclaims API.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	ResourcePoolClaim	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object		true
status	object	ResourceQuotaClaimStatus defines the observed state of ResourceQuotaClaim.	false

ResourcePoolClaim.spec

Name	Type	Description	Required
claim	map[string]int or string	Amount which should be claimed for the resourcequota	true
pool	string	If there’s the possability to claim from multiple global Quotas You must be specific about which one you want to claim resources from Once bound to a ResourcePool, this field is immutable	true

ResourcePoolClaim.status

ResourceQuotaClaimStatus defines the observed state of ResourceQuotaClaim.

Name	Type	Description	Required
conditions	[]object	Conditions for the resource claim	true
allocation	object	Tracks the Usage from Claimed from this claim and available resources	false
condition	object	Deprecated: Use Conditions	false
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false
pool	object	Reference to the GlobalQuota being claimed from	false

ResourcePoolClaim.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

ResourcePoolClaim.status.allocation

Tracks the Usage from Claimed from this claim and available resources

Name	Type	Description	Required
available	map[string]int or string	Used to track the usage of the resource in the pool (diff hard - claimed). May be used for further automation	false
hard	map[string]int or string	Hard is the set of enforced hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
used	map[string]int or string	Used is the current observed total usage of the resource in the namespace.	false

ResourcePoolClaim.status.condition

Deprecated: Use Conditions

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

ResourcePoolClaim.status.pool

Reference to the GlobalQuota being claimed from

Name	Type	Description	Required
name	string	Name of the referent.	true
uid	string	UID of the tracked Tenant to pin point tracking	true

ResourcePool

Resourcepools allows you to define a set of resources as known from ResoureQuotas. The Resourcepools are defined at cluster-scope an should be administrated by cluster-administrators. However they create an interface, where cluster-administrators can define from which namespaces resources from a Resourcepool can be claimed. The claiming is done via a namespaced CRD called ResourcePoolClaim. Then it’s up the group of users within these namespaces, to manage the resources they consume per namespace. Each Resourcepool provisions a ResourceQuotainto all the selected namespaces. Then essentially the ResourcePoolClaims, when they can be assigned to the ResourcePool stack resources on top of that ResourceQuota based on the namspace, where the ResourcePoolClaim was made from.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	ResourcePool	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	ResourcePoolSpec.	true
status	object	GlobalResourceQuotaStatus defines the observed state of GlobalResourceQuota.	false

ResourcePool.spec

ResourcePoolSpec.

Name	Type	Description	Required
quota	object	Define the resourcequota served by this resourcepool.	true
config	object	Additional Configuration Default: map[]	false
defaults	map[string]int or string	The Defaults given for each namespace, the default is not counted towards the total allocation When you use claims it’s recommended to provision Defaults as the prevent the scheduling of any resources	false
selectors	[]object	Selector to match the namespaces that should be managed by the GlobalResourceQuota	false

ResourcePool.spec.quota

Define the resourcequota served by this resourcepool.

Name	Type	Description	Required
hard	map[string]int or string	hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
scopeSelector	object	scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.	false
scopes	[]string	A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.	false

ResourcePool.spec.quota.scopeSelector

Name	Type	Description	Required
matchExpressions	[]object	A list of scope selector requirements by scope of the resources.	false

ResourcePool.spec.quota.scopeSelector.matchExpressions[index]

A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.

Name	Type	Description	Required
operator	string	Represents a scope’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.	true
scopeName	string	The name of the scope that the selector applies to.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

ResourcePool.spec.config

Additional Configuration

Name	Type	Description	Required
defaultsZero	boolean	With this option all resources which can be allocated are set to 0 for the resourcequota defaults. (Default false) Default: false	false
deleteBoundResources	boolean	When a resourcepool is deleted, the resourceclaims bound to it are disassociated from the resourcepool but not deleted. By Enabling this option, the resourceclaims will be deleted when the resourcepool is deleted, if they are in bound state. (Default false) Default: false	false
orderedQueue	boolean	Claims are queued whenever they are allocated to a pool. A pool tries to allocate claims in order based on their creation date. But no matter their creation time, if a claim is requesting too much resources it’s put into the queue but if a lower priority claim still has enough space in the available resources, it will be able to claim them. Eventough it’s priority was lower Enabling this option respects to Order. Meaning the Creationtimestamp matters and if a resource is put into the queue, no other claim can claim the same resources with lower priority. (Default false) Default: false	false

ResourcePool.spec.selectors[index]

Selector for resources and their labels or selecting origin namespaces

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

ResourcePool.spec.selectors[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

ResourcePool.status

GlobalResourceQuotaStatus defines the observed state of GlobalResourceQuota.

Name	Type	Description	Required
conditions	[]object	Conditions for the resource claim	true
allocation	object	Tracks the Usage from Claimed against what has been granted from the pool	false
claimCount	integer	Amount of claims Default: 0	false
claims	map[string][]object	Tracks the quotas for the Resource.	false
exhaustions	map[string]object	Exhaustions from claims associated with the pool	false
namespaceCount	integer	How many namespaces are considered Default: 0	false
namespaces	[]string	Namespaces which are considered for claims	false
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false

ResourcePool.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

ResourcePool.status.allocation

Tracks the Usage from Claimed against what has been granted from the pool

Name	Type	Description	Required
available	map[string]int or string	Used to track the usage of the resource in the pool (diff hard - claimed). May be used for further automation	false
hard	map[string]int or string	Hard is the set of enforced hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
used	map[string]int or string	Used is the current observed total usage of the resource in the namespace.	false

ResourcePool.status.claims[key][index]

ResourceQuotaClaimStatus defines the observed state of ResourceQuotaClaim.

Name	Type	Description	Required
name	string	Name of the referent.	true
namespace	string	Namespace of the referent.	true
uid	string	UID of the tracked Tenant to pin point tracking	true
claims	map[string]int or string	Claimed resources	false

ResourcePool.status.exhaustions[key]

Name	Type	Description	Required
available	int or string	Available Resources to be claimed	false
requesting	int or string	Requesting Resources	false

RuleStatus

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	RuleStatus	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	[]object		false
status	object	RuleStatus contains the accumulated rules applying to namespace it’s deployed in.	false

RuleStatus.spec[index]

For future implementation where users might manage RuleStatus CRs themselves

Name	Type	Description	Required
audience	[]object	Audience limits this rule to matching request subjects. An empty audience matches every request.	false
enforce	object	Enforcement for given rule	false
mutate	[]object	Mutate applies ordered typed mutations before enforcement. It shares the rule’s namespace selection and audience. Workload mutations apply on Pod creation.	false
quota	[]object	Quota contains native Kubernetes ResourceQuota specifications shared by all namespaces selected by this rule. Unlike Enforce, quota accounting is independent of the request audience.	false

RuleStatus.spec[index].audience[index]

Name	Type	Description	Required
kind	enum	Enum: User, Group, ServiceAccount, Custom	true
name	string		true

RuleStatus.spec[index].enforce

Enforcement for given rule

Name	Type	Description	Required
action	enum	Declare the action being performed on the enforcement rule: deny: On match, deny admission request allow: On match, allowed admission request audit: On match, audit (post event) of admission request Enum: allow, deny, audit Default: deny	false
conditions	[]object	Conditions gate this entire enforcement rule, including workloads, services, metadata, ingress, network and storage. All conditions must be true; empty means apply. Conditions run during admission; managed metadata in a conditional rule is applied on matching requests only and is not reconciled in the background.	false
ingress	object	Enforcement for Ingress and Gateway API resource hostnames.	false
metadata	[]object	Enforcement for object metadata on namespaced resources.	false
network	object	Network configures network enforcement for namespace resources.	false
services	object	Enforcement for Services.	false
storage	object	Additional access to PersistentVolumes without a tenant ownership label.	false
workloads	object	Enforcement for native workload kinds and Pod properties.	false

RuleStatus.spec[index].enforce.conditions[index]

AdmissionCondition is a Boolean CEL gate for a mutation entry or enforcement rule. It can inspect object and request metadata, but cannot generate mutation values. During PVC volume access evaluation, volume contains the referenced PV; otherwise volume is null. Conditions never modify the PVC or PV.

Name	Type	Description	Required
expression	string	Expression must evaluate to bool. Any false condition skips this block. An evaluation error rejects the request unless another condition is false.	true
name	string	Name identifies a condition in admission errors. Names must be unique within a block.	false

RuleStatus.spec[index].enforce.ingress

Enforcement for Ingress and Gateway API resource hostnames.

Name	Type	Description	Required
hostnames	[]object	Hostnames defines allowed, denied, or audited hostname expressions. A resource targeted by an allow or deny rule must declare non-empty values in all hostname fields. Audit-only rules record missing hostnames without denying them.	false
types	[]enum	Types defines the resource kinds to which hostname enforcement applies. Enum: Ingress, Route, ListenerSet, HTTPRoute, Gateway, TLSRoute, GRPCRoute	false

RuleStatus.spec[index].enforce.ingress.hostnames[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.metadata[index]

MetadataRule defines metadata constraints for namespaced resources.

Name	Type	Description	Required
kinds	[]string	Kinds of the referents. Use “*” to match all kinds.	true
annotations	map[string]object	Annotations defines metadata policies by annotation key.	false
apiGroups	[]string	API groups or API group/version selectors of the referents. Empty or omitted APIGroups means the core Kubernetes API version “v1”. Use “” to match all API groups and versions. Examples: - [] or [""] means core “v1”. - [“v1”] means core “v1”. - [“apps”] means any version in the “apps” API group. - [“apps/v1”] means only “apps/v1”. - [“apps”, “batch/v1”] means any “apps” version and “batch/v1”. - [""] means all API groups and versions.	false
labels	map[string]object	Labels defines metadata policies by label key.	false

RuleStatus.spec[index].enforce.metadata[index].annotations[key]

Name	Type	Description	Required
default	string	Default is applied by admission mutation when the concrete metadata key is absent. It is not reconciled after admission.	false
managed	string	Managed is enforced by admission mutation and reconciled by the RuleStatus controller using server-side apply when the rule configuration changes. Present metadata matching the effective managed value is exempt from metadata value validation, including overlapping deny rules. The last applicable managed value for each concrete key wins. During admission, only rules matching the request’s audience contribute to the effective managed value.	false
required	boolean	Required enforces that the metadata key must be present. This is only meaningful with action=allow. Deny and audit rules do not require missing metadata to exist. Default: false	false
values	[]object	Values defines allowed, denied, or audited values for the metadata key. With action=allow and no values, only Required is enforced. With action=deny or action=audit and no values, any present value matches, including an empty string.	false

RuleStatus.spec[index].enforce.metadata[index].annotations[key].values[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.metadata[index].labels[key]

Name	Type	Description	Required
default	string	Default is applied by admission mutation when the concrete metadata key is absent. It is not reconciled after admission.	false
managed	string	Managed is enforced by admission mutation and reconciled by the RuleStatus controller using server-side apply when the rule configuration changes. Present metadata matching the effective managed value is exempt from metadata value validation, including overlapping deny rules. The last applicable managed value for each concrete key wins. During admission, only rules matching the request’s audience contribute to the effective managed value.	false
required	boolean	Required enforces that the metadata key must be present. This is only meaningful with action=allow. Deny and audit rules do not require missing metadata to exist. Default: false	false
values	[]object	Values defines allowed, denied, or audited values for the metadata key. With action=allow and no values, only Required is enforced. With action=deny or action=audit and no values, any present value matches, including an empty string.	false

RuleStatus.spec[index].enforce.metadata[index].labels[key].values[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.network

Network configures network enforcement for namespace resources.

Name	Type	Description	Required
policies	object	Policies constrains native NetworkPolicies.	false

RuleStatus.spec[index].enforce.network.policies

Policies constrains native NetworkPolicies.

Name	Type	Description	Required
egress	object	Egress constrains destination address grants. Selector-based peers are unaffected. No constraints are applied when omitted.	false
ingress	object	Ingress constrains source address grants. Selector-based peers are unaffected. No constraints are applied when omitted.	false

RuleStatus.spec[index].enforce.network.policies.egress

Egress constrains destination address grants. Selector-based peers are unaffected. No constraints are applied when omitted.

Name	Type	Description	Required
cidrs	[]string	CIDRs matches addresses granted by ipBlock.cidr after subtracting except. Omitted peers on an ingress or egress rule grant both IPv4 and IPv6 address space. An empty NetworkPolicy ingress or egress list has no grants in that direction to validate. Each direction is evaluated independently. An empty CIDRs list imposes no constraint. Ordered allow/deny decisions apply to every granted address: deny rejects overlapping grants, allow requires complete coverage, and audit reports overlaps. A later allow cannot authorize addresses outside its CIDRs.	false

RuleStatus.spec[index].enforce.network.policies.ingress

Ingress constrains source address grants. Selector-based peers are unaffected. No constraints are applied when omitted.

Name	Type	Description	Required
cidrs	[]string	CIDRs matches addresses granted by ipBlock.cidr after subtracting except. Omitted peers on an ingress or egress rule grant both IPv4 and IPv6 address space. An empty NetworkPolicy ingress or egress list has no grants in that direction to validate. Each direction is evaluated independently. An empty CIDRs list imposes no constraint. Ordered allow/deny decisions apply to every granted address: deny rejects overlapping grants, allow requires complete coverage, and audit reports overlaps. A later allow cannot authorize addresses outside its CIDRs.	false

RuleStatus.spec[index].enforce.services

Enforcement for Services.

Name	Type	Description	Required
externalIPs	object	ExternalIPs defines constraints for spec.externalIPs.	false
externalNames	object	ExternalNames defines additional constraints for Services of type ExternalName.	false
loadBalancers	object	LoadBalancers defines additional constraints for Services of type LoadBalancer.	false
nodePorts	object	NodePorts defines additional constraints for nodePort values.	false
types	[]enum	Types defines the Service types matched by this rule. Supported values: - ClusterIP - NodePort - LoadBalancer - ExternalName Enum: ClusterIP, NodePort, LoadBalancer, ExternalName	false

RuleStatus.spec[index].enforce.services.externalIPs

ExternalIPs defines constraints for spec.externalIPs.

Name	Type	Description	Required
cidrs	[]string	CIDRs restricts spec.externalIPs. Individual IP addresses are treated as host CIDRs (/32 for IPv4 and /128 for IPv6). For deny rules, empty means all external IPs are denied. For allow and audit rules, empty means no external IP restriction.	false

RuleStatus.spec[index].enforce.services.externalNames

ExternalNames defines additional constraints for Services of type ExternalName.

Name	Type	Description	Required
hostnames	[]object	Hostnames restricts spec.externalName. Empty means no additional hostname restriction once ExternalName is allowed by types.	false

RuleStatus.spec[index].enforce.services.externalNames.hostnames[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.services.loadBalancers

LoadBalancers defines additional constraints for Services of type LoadBalancer.

Name	Type	Description	Required
cidrs	[]string	CIDRs restricts spec.loadBalancerIP and spec.loadBalancerSourceRanges. Empty means no additional CIDR restriction once LoadBalancer is allowed by types.	false

RuleStatus.spec[index].enforce.services.nodePorts

NodePorts defines additional constraints for nodePort values.

Name	Type	Description	Required
ports	[]object	Ports restricts explicitly requested nodePort values. Empty means no additional port restriction once NodePort is allowed by types.	false

RuleStatus.spec[index].enforce.services.nodePorts.ports[index]

Name	Type	Description	Required
from	integer	Format: int32 Minimum: 1 Maximum: 65535	true
to	integer	Format: int32 Minimum: 1 Maximum: 65535	true

RuleStatus.spec[index].enforce.storage

Additional access to PersistentVolumes without a tenant ownership label.

Name	Type	Description	Required
volumes	[]object	Volumes matches additional PersistentVolumes. Entries are ORed. Conditions on the enclosing enforcement rule must also match. The last matching allow/deny decides the exception; audit never grants access.	false

RuleStatus.spec[index].enforce.storage.volumes[index]

PersistentVolumeMatch matches labels on the referenced PersistentVolume.

Name	Type	Description	Required
selector	object	Selector matches existing PV labels, independently of PVC.spec.selector. An explicit empty selector matches all PVs. Capsule’s tenant ownership label cannot be overridden by a selector or an allow decision.	true
name	string	Name identifies this match in admission and audit messages.	false

RuleStatus.spec[index].enforce.storage.volumes[index].selector

Selector matches existing PV labels, independently of PVC.spec.selector. An explicit empty selector matches all PVs. Capsule’s tenant ownership label cannot be overridden by a selector or an allow decision.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].enforce.storage.volumes[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].enforce.workloads

Enforcement for native workload kinds and Pod properties.

Name	Type	Description	Required
disruptionBudgets	object	DisruptionBudgets constrains PDB coverage of selected Pods and controller templates. Only whole-workload targets apply; omitted targets select Pods.	false
placement	object	Placement matches the Pod scheduler and scheduling constraints.	false
qosClasses	[]string	Define Pod QoS classes matched by this enforcement rule. Supported values are Guaranteed, Burstable and BestEffort.	false
registries	[]object	Define registries which are allowed to be used within this tenant The rules are aggregated, since you can use Regular Expressions the match registry endpoints	false
resources	object	Resources defines mutation and enforcement policies for Pod and container resource requests and limits. The workload targets select where the policies apply. With no targets, resource policies apply to all compatible locations: Pod-level resources, regular containers, and init containers. Resource names unsupported at Pod level still apply to compatible container locations. Mutation is applied when a Pod is created, or when an explicitly targeted controller is created or updated (to its Pod template). Remove and MatchRequest manage explicit values, Default fills an absent value, and Ratio fills an absent limit from its request. An explicit Ratio violation is then handled by the enclosing allow, deny, or audit action.	false
schedulers	[]object	Schedulers defines additional schedulerName matchers for selected Pods and Pod templates. These are combined with placement.schedulers under the same rule action; a match in either field counts as a match. Deprecated: use Placement.Schedulers instead.	false
security	object	Security matches effective Pod and container security profiles.	false
targets	[]string	Targets selects native workloads and, optionally, parts of their Pod specs. With no workload policies, the action matches the selected kinds themselves. With policies, targets scopes those policies; it does not also match the kind. Omitted targets preserve Pod-only defaults. Controller targets are opt-in. Whole-controller targets include every compatible location in the Pod template. Existing pod and pod/* part targets retain their established policy scope.	false

RuleStatus.spec[index].enforce.workloads.disruptionBudgets

DisruptionBudgets constrains PDB coverage of selected Pods and controller templates. Only whole-workload targets apply; omitted targets select Pods.

Name	Type	Description	Required
allowOverlap	boolean	AllowOverlap permits multiple PDBs to select the same Pod or selected controller template. False rejects overlap for allow/deny actions and reports violations for audit. True permits overlap in an allow rule; omitted does not participate in evaluation. Rule ordering is preserved. Checks run on PDB selector changes and selected Pod/template label changes. Conditions and audience apply to the admission request, including PDB writes. Cross-resource admission reads are not an atomic transaction.	false
evictableReplicas	object	EvictableReplicas bounds the configured number of evictable replicas of each selected Deployment, StatefulSet, ReplicaSet or ReplicationController. Uses desired spec.replicas and Kubernetes’ upward percentage rounding; zero replicas are exempt. Checks controller writes, /scale, and PDB writes. Bounds are inclusive nonnegative integers; omitted bounds are unrestricted. Allow requires compliance; deny/audit match violations, like resource constraints. Other targets do not have a desired replica count and skip this property. At least one supported controller target must be explicitly selected.	false
unhealthyPodEvictionPolicies	[]enum	UnhealthyPodEvictionPolicies matches the effective PDB policy. Omission in a PDB means IfHealthyBudget. Allow accepts listed values; deny/audit match listed values, like other enum lists. Empty imposes no restriction. Enum: IfHealthyBudget, AlwaysAllow	false

RuleStatus.spec[index].enforce.workloads.disruptionBudgets.evictableReplicas

EvictableReplicas bounds the configured number of evictable replicas of each selected Deployment, StatefulSet, ReplicaSet or ReplicationController. Uses desired spec.replicas and Kubernetes’ upward percentage rounding; zero replicas are exempt. Checks controller writes, /scale, and PDB writes. Bounds are inclusive nonnegative integers; omitted bounds are unrestricted. Allow requires compliance; deny/audit match violations, like resource constraints. Other targets do not have a desired replica count and skip this property. At least one supported controller target must be explicitly selected.

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.spec[index].enforce.workloads.placement

Placement matches the Pod scheduler and scheduling constraints.

Name	Type	Description	Required
affinity	[]object	Affinity matches each complete required or preferred affinity term.	false
nodeSelector	[]object	NodeSelector matches each node selector entry. Empty matchers match any entry. Placement rules apply to the pod target; allowing an entry does not require it.	false
schedulers	[]object	Schedulers defines schedulerName matchers for selected Pods and Pod templates. The rule is evaluated against pod.spec.schedulerName. Empty schedulerName is ignored and is not normalized to default-scheduler.	false
tolerations	[]object	Tolerations matches each toleration, including injected and wildcard tolerations.	false
topologySpreadConstraints	[]object	TopologySpreadConstraints matches each complete spread constraint.	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index]

WorkloadAffinityMatch matches an entire affinity term. Fields are ANDed; entries in enforce.workloads.placement.affinity are alternatives. {} matches any term.

Name	Type	Description	Required
fieldRequirements	[]object	FieldRequirements constrains node matchFields. When Requirements is set, matchFields must be explicitly permitted here to avoid an unchecked path.	false
labelSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
modes	[]enum	Modes defaults to both scheduling modes; it does not require their presence. Enum: required, preferred	false
namespaceScope	enum	SameNamespace requires no namespaceSelector and only the Pod’s namespace in namespaces (or an omitted namespaces list). Any imposes no restriction. Enum: SameNamespace, Any	false
namespaceSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
namespaces	object	Namespaces constrains each explicitly supplied namespace.	false
requirements	[]object	Requirements constrains node matchExpressions.	false
topologyKey	object	TopologyKey applies to Pod affinity and anti-affinity.	false
types	[]enum	Types defaults to all types. Type-specific constraints only match the types on which they are meaningful. Enum: nodeAffinity, podAffinity, podAntiAffinity	false
weight	object	Weight requires modes: [preferred].	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].fieldRequirements[index]

PlacementRequirementMatch matches one selector requirement. Every supplied value must match Values. Operators that have no values remain valid when explicitly permitted. Node selectors additionally support Gt and Lt.

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].fieldRequirements[index].key

PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].fieldRequirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].labelSelector

PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].labelSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].labelSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].labelSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].namespaceSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].namespaces

Namespaces constrains each explicitly supplied namespace.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].topologyKey

TopologyKey applies to Pod affinity and anti-affinity.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.affinity[index].weight

Weight requires modes: [preferred].

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.spec[index].enforce.workloads.placement.nodeSelector[index]

WorkloadNodeSelectorMatch matches a nodeSelector key/value pair. Omitted fields are unrestricted, so {} matches every entry, including an empty label value.

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.spec[index].enforce.workloads.placement.nodeSelector[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.nodeSelector[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.schedulers[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.tolerations[index]

Name	Type	Description	Required
effects	[]enum	Effects matches the literal effect. An empty effect on a Pod tolerates all effects and does not match an allowlist of individual effects. Enum: NoSchedule, PreferNoSchedule, NoExecute,	false
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Operators matches Equal (including an omitted operator) or Exists. Enum: Equal, Exists	false
tolerationSeconds	object	TolerationDurationMatch treats an absent tolerationSeconds as unlimited.	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.spec[index].enforce.workloads.placement.tolerations[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.tolerations[index].tolerationSeconds

TolerationDurationMatch treats an absent tolerationSeconds as unlimited.

Name	Type	Description	Required
allowUnlimited	boolean	AllowUnlimited defaults to true. Set false to require a finite duration.	false
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.spec[index].enforce.workloads.placement.tolerations[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.topologySpreadConstraints[index]

Name	Type	Description	Required
labelSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
maxSkew	object	PlacementRange defines inclusive bounds. An omitted bound is unrestricted.	false
minDomains	object	MinDomains uses 1 when the Pod omits minDomains.	false
nodeAffinityPolicy	[]enum	NodeAffinityPolicy uses Honor when the Pod omits the field. Enum: Honor, Ignore	false
nodeTaintsPolicy	[]enum	NodeTaintsPolicy uses Ignore when the Pod omits the field. Enum: Honor, Ignore	false
topologyKey	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
whenUnsatisfiable	[]enum	Enum: DoNotSchedule, ScheduleAnyway	false

RuleStatus.spec[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

RuleStatus.spec[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.spec[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.placement.topologySpreadConstraints[index].maxSkew

PlacementRange defines inclusive bounds. An omitted bound is unrestricted.

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.spec[index].enforce.workloads.placement.topologySpreadConstraints[index].minDomains

MinDomains uses 1 when the Pod omits minDomains.

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.spec[index].enforce.workloads.placement.topologySpreadConstraints[index].topologyKey

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.registries[index]

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false
policy	[]string	Allowed PullPolicy for the given registry. Supplying no value allows all policies.	false

RuleStatus.spec[index].enforce.workloads.resources

Resources defines mutation and enforcement policies for Pod and container resource requests and limits. The workload targets select where the policies apply. With no targets, resource policies apply to all compatible locations: Pod-level resources, regular containers, and init containers. Resource names unsupported at Pod level still apply to compatible container locations. Mutation is applied when a Pod is created, or when an explicitly targeted controller is created or updated (to its Pod template). Remove and MatchRequest manage explicit values, Default fills an absent value, and Ratio fills an absent limit from its request. An explicit Ratio violation is then handled by the enclosing allow, deny, or audit action.

Name	Type	Description	Required
limits	map[string]object	Limits defines policies for resource limits.	false
requests	map[string]object	Requests defines policies for resource requests.	false

RuleStatus.spec[index].enforce.workloads.resources.limits[key]

WorkloadResourceLimitPolicy defines how a resource limit is mutated and enforced.

Name	Type	Description	Required
policy	enum	Policy selects how the limit is handled: Preserve leaves it unchanged, Default fills an absent limit, Remove deletes it, MatchRequest manages it to equal the request, and Ratio defaults an absent limit and enforces the maximum multiplier against explicitly supplied limits. Enum: Preserve, Default, Remove, MatchRequest, Ratio	true
value	int or string	Value is the quantity applied by Default or the maximum limit-to-request multiplier applied by Ratio.	false

RuleStatus.spec[index].enforce.workloads.resources.requests[key]

WorkloadResourceRequestPolicy defines how a resource request is mutated.

Name	Type	Description	Required
policy	enum	Policy selects how the request is handled: Preserve leaves it unchanged, Default fills an absent request, and Remove deletes it. Enum: Preserve, Default, Remove	true
value	int or string	Value is the quantity applied by the Default policy.	false

RuleStatus.spec[index].enforce.workloads.schedulers[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.security

Security matches effective Pod and container security profiles.

Name	Type	Description	Required
appArmorProfiles	[]object	AppArmorProfiles matches effective Linux container profiles, including legacy AppArmor annotations before Pod defaults. It uses the same target and missing-profile semantics as SeccompProfiles. Privileged containers are Unconfined. Matching a Localhost name does not verify its installation.	false
seccompProfiles	[]object	SeccompProfiles matches effective Linux container profiles, resolving container overrides before Pod defaults. Privileged containers are Unconfined. Missing profiles do not match any type, so an allow-list rejects them. Omitted targets check regular, init and ephemeral containers; pod explicitly selects only the Pod default. Controller targets check their Pod templates.	false

RuleStatus.spec[index].enforce.workloads.security.appArmorProfiles[index]

WorkloadSecurityProfileMatch matches a profile type and optionally its local name/path. Entries are alternatives; localhostProfiles constrains only Localhost.

Name	Type	Description	Required
types	[]enum	Types selects profile types. Missing profiles never match. Enum: RuntimeDefault, Localhost, Unconfined	true
localhostProfiles	[]object	LocalhostProfiles matches seccomp paths relative to the kubelet seccomp directory, or loaded AppArmor profile names. Omitted permits any Localhost profile. Requires Localhost in types. Expressions use exact, exp and negate.	false

RuleStatus.spec[index].enforce.workloads.security.appArmorProfiles[index].localhostProfiles[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].enforce.workloads.security.seccompProfiles[index]

WorkloadSecurityProfileMatch matches a profile type and optionally its local name/path. Entries are alternatives; localhostProfiles constrains only Localhost.

Name	Type	Description	Required
types	[]enum	Types selects profile types. Missing profiles never match. Enum: RuntimeDefault, Localhost, Unconfined	true
localhostProfiles	[]object	LocalhostProfiles matches seccomp paths relative to the kubelet seccomp directory, or loaded AppArmor profile names. Omitted permits any Localhost profile. Requires Localhost in types. Expressions use exact, exp and negate.	false

RuleStatus.spec[index].enforce.workloads.security.seccompProfiles[index].localhostProfiles[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.spec[index].mutate[index]

NamespaceRuleMutation applies typed mutations independently of enforce.action. Mutated values remain subject to all applicable enforcement rules.

Name	Type	Description	Required
action	enum	Action chooses how explicitly supplied properties are applied. Merge fills an empty schedulerName and absent security profiles, sets hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, and map keys, adds missing imagePullSecrets by name, upserts lists, and conjoins required affinity. Replace replaces each supplied property in full: scheduler, hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, nodeSelector, registries.imagePullSecrets, tolerations, topologySpreadConstraints, affinity, or security profiles. Omitted properties are retained. Supplying affinity replaces all its branches, including omitted ones. Enum: merge, replace Default: merge	false
conditions	[]object	Conditions gate this entire mutation entry and inspect the object after preceding mutations. All conditions must be true; empty means apply.	false
workloads	object	WorkloadMutation contains typed native Pod values. Empty maps/lists are preserved so replace can clear a property; nil means the property is omitted. It applies on Pod creation. Security.ReadOnlyRootFilesystem and Registries.ImagePullPolicy also apply to newly added ephemeral containers on subresource updates. Existing containers are never reconciled. On merge, later entries override hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, matching map keys, tolerations and spread constraints. On merge, required affinity restrictions from applicable entries are ANDed.	false

RuleStatus.spec[index].mutate[index].conditions[index]

Name	Type	Description	Required
expression	string	Expression must evaluate to bool. Any false condition skips this block. An evaluation error rejects the request unless another condition is false.	true
name	string	Name identifies a condition in admission errors. Names must be unique within a block.	false

RuleStatus.spec[index].mutate[index].workloads

WorkloadMutation contains typed native Pod values. Empty maps/lists are preserved so replace can clear a property; nil means the property is omitted. It applies on Pod creation. Security.ReadOnlyRootFilesystem and Registries.ImagePullPolicy also apply to newly added ephemeral containers on subresource updates. Existing containers are never reconciled. On merge, later entries override hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, matching map keys, tolerations and spread constraints. On merge, required affinity restrictions from applicable entries are ANDed.

Name	Type	Description	Required
placement	object	Placement configures the Pod scheduler and scheduling constraints.	false
registries	object	Registries configures Pod image pull secrets and selected containers’ pull policies.	false
security	object	Security configures Pod and container security settings.	false
targets	[]enum	Targets selects compatible Pod locations. Omitted or empty selects all compatible locations. The pod target includes Pod-level properties and all container groups; pod/containers, pod/initcontainers and pod/ephemeralcontainers narrow selection to one group. Controller templates and volumes are not supported. Enum: pod, pod/containers, pod/initcontainers, pod/ephemeralcontainers	false

RuleStatus.spec[index].mutate[index].workloads.placement

Placement configures the Pod scheduler and scheduling constraints.

Name	Type	Description	Required
affinity	object	Affinity on merge conjoins required restrictions and upserts preferred terms. Preferred terms match by their complete term, excluding weight and normalized ordering. A different term is added; a matching term gets the supplied weight. Required node affinity is distributed over existing OR alternatives, with at most 256 resulting alternatives. Empty node selector terms match no nodes. Replace replaces all affinity, including any branches omitted from the rule.	false
nodeSelector	map[string]string	NodeSelector sets configured keys on merge, or replaces the map on replace.	false
scheduler	string	Scheduler sets spec.schedulerName on Pod creation. Merge fills only an empty schedulerName, preserving all non-empty names, including default-scheduler. Kubernetes defaults omitted schedulerName before admission. Use replace with a condition to override default-scheduler while preserving custom schedulers. Replace always overwrites schedulerName when the entry’s conditions match. Omitted or null leaves the Pod value unchanged.	false
tolerations	[]object	Tolerations merges by key, operator (default Equal), value and effect. Changing any identity field adds another toleration, retaining the old one. A matching toleration’s duration is replaced; omitting it makes it unlimited. Replace replaces the entire list, including entries with other identities.	false
topologySpreadConstraints	[]object	TopologySpreadConstraints merges by topologyKey and whenUnsatisfiable. The entire matching constraint is replaced, including its selector and optional fields; other constraints remain. Replace replaces the entire list.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity

Affinity on merge conjoins required restrictions and upserts preferred terms. Preferred terms match by their complete term, excluding weight and normalized ordering. A different term is added; a matching term gets the supplied weight. Required node affinity is distributed over existing OR alternatives, with at most 256 resulting alternatives. Empty node selector terms match no nodes. Replace replaces all affinity, including any branches omitted from the rule.

Name	Type	Description	Required
nodeAffinity	object	Describes node affinity scheduling rules for the pod.	false
podAffinity	object	Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).	false
podAntiAffinity	object	Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.nodeAffinity

Describes node affinity scheduling rules for the pod.

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding “weight” to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	object	If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it’s a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).

Name	Type	Description	Required
preference	object	A node selector term, associated with the corresponding weight.	true
weight	integer	Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100. Format: int32	true

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference

A node selector term, associated with the corresponding weight.

Name	Type	Description	Required
matchExpressions	[]object	A list of node selector requirements by node’s labels.	false
matchFields	[]object	A list of node selector requirements by node’s fields.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference.matchExpressions[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference.matchFields[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution

If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.

Name	Type	Description	Required
nodeSelectorTerms	[]object	Required. A list of node selector terms. The terms are ORed.	true

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index]

A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.

Name	Type	Description	Required
matchExpressions	[]object	A list of node selector requirements by node’s labels.	false
matchFields	[]object	A list of node selector requirements by node’s fields.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index].matchExpressions[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index].matchFields[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity

Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding “weight” to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	[]object	If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)

Name	Type	Description	Required
podAffinityTerm	object	Required. A pod affinity term, associated with the corresponding weight.	true
weight	integer	weight associated with matching the corresponding podAffinityTerm, in the range 1-100. Format: int32	true

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm

Required. A pod affinity term, associated with the corresponding weight.

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector

A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index]

Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity

Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and subtracting “weight” from the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	[]object	If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)

Name	Type	Description	Required
podAffinityTerm	object	Required. A pod affinity term, associated with the corresponding weight.	true
weight	integer	weight associated with matching the corresponding podAffinityTerm, in the range 1-100. Format: int32	true

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm

Required. A pod affinity term, associated with the corresponding weight.

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index]

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.placement.tolerations[index]

The pod this Toleration is attached to tolerates any taint that matches the triple <key,value,effect> using the matching operator .

Name	Type	Description	Required
effect	string	Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.	false
key	string	Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.	false
operator	string	Operator represents a key’s relationship to the value. Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category. Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).	false
tolerationSeconds	integer	TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system. Format: int64	false
value	string	Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.	false

RuleStatus.spec[index].mutate[index].workloads.placement.topologySpreadConstraints[index]

TopologySpreadConstraint specifies how to spread matching pods among the given topology.

Name	Type	Description	Required
maxSkew	integer	MaxSkew describes the degree to which pods may be unevenly distributed. When `whenUnsatisfiable=DoNotSchedule`, it is the maximum permitted difference between the number of matching pods in the target topology and the global minimum. The global minimum is the minimum number of matching pods in an eligible domain or zero if the number of eligible domains is less than MinDomains. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 2/2/1: In this case, the global minimum is 1.	zone1
topologyKey	string	TopologyKey is the key of node labels. Nodes that have a label with this key and identical values are considered to be in the same topology. We consider each <key, value> as a “bucket”, and try to put balanced number of pods into each bucket. We define a domain as a particular instance of a topology. Also, we define an eligible domain as a domain whose nodes meet the requirements of nodeAffinityPolicy and nodeTaintsPolicy. e.g. If TopologyKey is “kubernetes.io/hostname”, each Node is a domain of that topology. And, if TopologyKey is “topology.kubernetes.io/zone”, each zone is a domain of that topology. It’s a required field.	true
whenUnsatisfiable	string	WhenUnsatisfiable indicates how to deal with a pod if it doesn’t satisfy the spread constraint. - DoNotSchedule (default) tells the scheduler not to schedule it. - ScheduleAnyway tells the scheduler to schedule the pod in any location, but giving higher precedence to topologies that would help reduce the skew. A constraint is considered “Unsatisfiable” for an incoming pod if and only if every possible node assignment for that pod would violate “MaxSkew” on some topology. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 3/1/1:	zone1
labelSelector	object	LabelSelector is used to find matching pods. Pods that match this label selector are counted to determine the number of pods in their corresponding topology domain.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select the pods over which spreading will be calculated. The keys are used to lookup values from the incoming pod labels, those key-value labels are ANDed with labelSelector to select the group of existing pods over which spreading will be calculated for the incoming pod. The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. MatchLabelKeys cannot be set when LabelSelector isn’t set. Keys that don’t exist in the incoming pod labels will be ignored. A null or empty list means only match against labelSelector. This is a beta field and requires the MatchLabelKeysInPodTopologySpread feature gate to be enabled (enabled by default).	false
minDomains	integer	MinDomains indicates a minimum number of eligible domains. When the number of eligible domains with matching topology keys is less than minDomains, Pod Topology Spread treats “global minimum” as 0, and then the calculation of Skew is performed. And when the number of eligible domains with matching topology keys equals or greater than minDomains, this value has no effect on scheduling. As a result, when the number of eligible domains is less than minDomains, scheduler won’t schedule more than maxSkew Pods to those domains. If value is nil, the constraint behaves as if MinDomains is equal to 1. Valid values are integers greater than 0. When value is not nil, WhenUnsatisfiable must be DoNotSchedule. For example, in a 3-zone cluster, MaxSkew is set to 2, MinDomains is set to 5 and pods with the same labelSelector spread as 2/2/2:	zone1
nodeAffinityPolicy	string	NodeAffinityPolicy indicates how we will treat Pod’s nodeAffinity/nodeSelector when calculating pod topology spread skew. Options are: - Honor: only nodes matching nodeAffinity/nodeSelector are included in the calculations. - Ignore: nodeAffinity/nodeSelector are ignored. All nodes are included in the calculations. If this value is nil, the behavior is equivalent to the Honor policy.	false
nodeTaintsPolicy	string	NodeTaintsPolicy indicates how we will treat node taints when calculating pod topology spread skew. Options are: - Honor: nodes without taints, along with tainted nodes for which the incoming pod has a toleration, are included. - Ignore: node taints are ignored. All nodes are included. If this value is nil, the behavior is equivalent to the Ignore policy.	false

RuleStatus.spec[index].mutate[index].workloads.placement.topologySpreadConstraints[index].labelSelector

LabelSelector is used to find matching pods. Pods that match this label selector are counted to determine the number of pods in their corresponding topology domain.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.spec[index].mutate[index].workloads.placement.topologySpreadConstraints[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.spec[index].mutate[index].workloads.registries

Registries configures Pod image pull secrets and selected containers’ pull policies.

Name	Type	Description	Required
imagePullPolicy	enum	ImagePullPolicy sets imagePullPolicy on every selected regular or init container at Pod creation, and newly added ephemeral containers on subresource updates. Both merge and replace overwrite explicit and Kubernetes-defaulted values. Omitted or null preserves the existing policy. Applies to all Pod OSes. Enum: Always, IfNotPresent, Never	false
imagePullSecrets	[]object	ImagePullSecrets sets spec.imagePullSecrets on Pod creation. Merge appends missing names and removes existing duplicates, retaining first-occurrence order. Replace sets the complete list; an explicit empty list clears it. Omitted or null preserves it. Requires the pod target or omitted/empty targets. References always use the Pod’s namespace; Capsule does not create, copy, or check the referenced Secrets.	false

RuleStatus.spec[index].mutate[index].workloads.registries.imagePullSecrets[index]

LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.

Name	Type	Description	Required
name	string	Name of the referent. This field is effectively required, but due to backwards compatibility is allowed to be empty. Instances of this type with an empty value here are almost certainly wrong. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names Default:	false

RuleStatus.spec[index].mutate[index].workloads.security

Security configures Pod and container security settings.

Name	Type	Description	Required
appArmorProfile	object	AppArmorProfile supplies the Pod-level securityContext.appArmorProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it. AppArmor and any Localhost profile must be available on eligible nodes.	false
hostUsers	boolean	HostUsers sets spec.hostUsers on both merge and replace. False requests a separate user namespace; true uses the host user namespace. Omitted or null leaves the Pod value unchanged. Requires Kubernetes/runtime support.	false
readOnlyRootFilesystem	boolean	ReadOnlyRootFilesystem sets securityContext.readOnlyRootFilesystem on every selected regular or init container at Pod creation, and newly added ephemeral containers on subresource updates. Both merge and replace overwrite the value. False is an explicit setting; nil preserves it. Windows Pods are skipped.	false
seccompProfile	object	SeccompProfile supplies the Pod-level securityContext.seccompProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it.	false

RuleStatus.spec[index].mutate[index].workloads.security.appArmorProfile

AppArmorProfile supplies the Pod-level securityContext.appArmorProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it. AppArmor and any Localhost profile must be available on eligible nodes.

Name	Type	Description	Required
type	string	type indicates which kind of AppArmor profile will be applied. Valid options are: Localhost - a profile pre-loaded on the node. RuntimeDefault - the container runtime’s default profile. Unconfined - no AppArmor enforcement.	true
localhostProfile	string	localhostProfile indicates a profile loaded on the node that should be used. The profile must be preconfigured on the node to work. Must match the loaded name of the profile. Must be set if and only if type is “Localhost”.	false

RuleStatus.spec[index].mutate[index].workloads.security.seccompProfile

SeccompProfile supplies the Pod-level securityContext.seccompProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it.

Name	Type	Description	Required
type	string	type indicates which kind of seccomp profile will be applied. Valid options are: Localhost - a profile defined in a file on the node should be used. RuntimeDefault - the container runtime default profile should be used. Unconfined - no profile should be applied.	true
localhostProfile	string	localhostProfile indicates a profile defined in a file on the node should be used. The profile must be preconfigured on the node to work. Must be a descending path, relative to the kubelet’s configured seccomp profile location. Must be set if type is “Localhost”. Must NOT be set for any other type.	false

RuleStatus.spec[index].quota[index]

ResourceQuotaRule defines a named ResourceQuota specification generated by a Tenant rule. Name is the durable identity of the generated GlobalResourceQuota and must be unique across all rules of a Tenant.

Name	Type	Description	Required
name	string	Name is the stable identity of this quota within the Tenant. Changing the name replaces the generated GlobalResourceQuota; changing the quota or its namespace selector updates the existing object.	true
hard	map[string]int or string	hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
scopeSelector	object	scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.	false
scopes	[]string	A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.	false

RuleStatus.spec[index].quota[index].scopeSelector

Name	Type	Description	Required
matchExpressions	[]object	A list of scope selector requirements by scope of the resources.	false

RuleStatus.spec[index].quota[index].scopeSelector.matchExpressions[index]

A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.

Name	Type	Description	Required
operator	string	Represents a scope’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.	true
scopeName	string	The name of the scope that the selector applies to.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status

RuleStatus contains the accumulated rules applying to namespace it’s deployed in.

Name	Type	Description	Required
conditions	[]object	Conditions	true
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false
rule	object	Deprecated: use Rules. Rule contains a legacy flattened view and cannot fully represent action-aware rules.	false
rules	[]object	Rules contains the effective namespace rules after tenant rule selection. Order is preserved from the originating Tenant rules.	false

RuleStatus.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

RuleStatus.status.rule

Deprecated: use Rules. Rule contains a legacy flattened view and cannot fully represent action-aware rules.

Name	Type	Description	Required
audience	[]object	Audience limits this rule to matching request subjects. An empty audience matches every request.	false
enforce	object	Enforcement for given rule	false
mutate	[]object	Mutate applies ordered typed mutations before enforcement. It shares the rule’s namespace selection and audience. Workload mutations apply on Pod creation.	false
quota	[]object	Quota contains native Kubernetes ResourceQuota specifications shared by all namespaces selected by this rule. Unlike Enforce, quota accounting is independent of the request audience.	false

RuleStatus.status.rule.audience[index]

Name	Type	Description	Required
kind	enum	Enum: User, Group, ServiceAccount, Custom	true
name	string		true

RuleStatus.status.rule.enforce

Enforcement for given rule

Name	Type	Description	Required
action	enum	Declare the action being performed on the enforcement rule: deny: On match, deny admission request allow: On match, allowed admission request audit: On match, audit (post event) of admission request Enum: allow, deny, audit Default: deny	false
conditions	[]object	Conditions gate this entire enforcement rule, including workloads, services, metadata, ingress, network and storage. All conditions must be true; empty means apply. Conditions run during admission; managed metadata in a conditional rule is applied on matching requests only and is not reconciled in the background.	false
ingress	object	Enforcement for Ingress and Gateway API resource hostnames.	false
metadata	[]object	Enforcement for object metadata on namespaced resources.	false
network	object	Network configures network enforcement for namespace resources.	false
services	object	Enforcement for Services.	false
storage	object	Additional access to PersistentVolumes without a tenant ownership label.	false
workloads	object	Enforcement for native workload kinds and Pod properties.	false

RuleStatus.status.rule.enforce.conditions[index]

Name	Type	Description	Required
expression	string	Expression must evaluate to bool. Any false condition skips this block. An evaluation error rejects the request unless another condition is false.	true
name	string	Name identifies a condition in admission errors. Names must be unique within a block.	false

RuleStatus.status.rule.enforce.ingress

Enforcement for Ingress and Gateway API resource hostnames.

Name	Type	Description	Required
hostnames	[]object	Hostnames defines allowed, denied, or audited hostname expressions. A resource targeted by an allow or deny rule must declare non-empty values in all hostname fields. Audit-only rules record missing hostnames without denying them.	false
types	[]enum	Types defines the resource kinds to which hostname enforcement applies. Enum: Ingress, Route, ListenerSet, HTTPRoute, Gateway, TLSRoute, GRPCRoute	false

RuleStatus.status.rule.enforce.ingress.hostnames[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.metadata[index]

MetadataRule defines metadata constraints for namespaced resources.

Name	Type	Description	Required
kinds	[]string	Kinds of the referents. Use “*” to match all kinds.	true
annotations	map[string]object	Annotations defines metadata policies by annotation key.	false
apiGroups	[]string	API groups or API group/version selectors of the referents. Empty or omitted APIGroups means the core Kubernetes API version “v1”. Use “” to match all API groups and versions. Examples: - [] or [""] means core “v1”. - [“v1”] means core “v1”. - [“apps”] means any version in the “apps” API group. - [“apps/v1”] means only “apps/v1”. - [“apps”, “batch/v1”] means any “apps” version and “batch/v1”. - [""] means all API groups and versions.	false
labels	map[string]object	Labels defines metadata policies by label key.	false

RuleStatus.status.rule.enforce.metadata[index].annotations[key]

Name	Type	Description	Required
default	string	Default is applied by admission mutation when the concrete metadata key is absent. It is not reconciled after admission.	false
managed	string	Managed is enforced by admission mutation and reconciled by the RuleStatus controller using server-side apply when the rule configuration changes. Present metadata matching the effective managed value is exempt from metadata value validation, including overlapping deny rules. The last applicable managed value for each concrete key wins. During admission, only rules matching the request’s audience contribute to the effective managed value.	false
required	boolean	Required enforces that the metadata key must be present. This is only meaningful with action=allow. Deny and audit rules do not require missing metadata to exist. Default: false	false
values	[]object	Values defines allowed, denied, or audited values for the metadata key. With action=allow and no values, only Required is enforced. With action=deny or action=audit and no values, any present value matches, including an empty string.	false

RuleStatus.status.rule.enforce.metadata[index].annotations[key].values[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.metadata[index].labels[key]

Name	Type	Description	Required
default	string	Default is applied by admission mutation when the concrete metadata key is absent. It is not reconciled after admission.	false
managed	string	Managed is enforced by admission mutation and reconciled by the RuleStatus controller using server-side apply when the rule configuration changes. Present metadata matching the effective managed value is exempt from metadata value validation, including overlapping deny rules. The last applicable managed value for each concrete key wins. During admission, only rules matching the request’s audience contribute to the effective managed value.	false
required	boolean	Required enforces that the metadata key must be present. This is only meaningful with action=allow. Deny and audit rules do not require missing metadata to exist. Default: false	false
values	[]object	Values defines allowed, denied, or audited values for the metadata key. With action=allow and no values, only Required is enforced. With action=deny or action=audit and no values, any present value matches, including an empty string.	false

RuleStatus.status.rule.enforce.metadata[index].labels[key].values[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.network

Network configures network enforcement for namespace resources.

Name	Type	Description	Required
policies	object	Policies constrains native NetworkPolicies.	false

RuleStatus.status.rule.enforce.network.policies

Policies constrains native NetworkPolicies.

Name	Type	Description	Required
egress	object	Egress constrains destination address grants. Selector-based peers are unaffected. No constraints are applied when omitted.	false
ingress	object	Ingress constrains source address grants. Selector-based peers are unaffected. No constraints are applied when omitted.	false

RuleStatus.status.rule.enforce.network.policies.egress

Egress constrains destination address grants. Selector-based peers are unaffected. No constraints are applied when omitted.

Name	Type	Description	Required
cidrs	[]string	CIDRs matches addresses granted by ipBlock.cidr after subtracting except. Omitted peers on an ingress or egress rule grant both IPv4 and IPv6 address space. An empty NetworkPolicy ingress or egress list has no grants in that direction to validate. Each direction is evaluated independently. An empty CIDRs list imposes no constraint. Ordered allow/deny decisions apply to every granted address: deny rejects overlapping grants, allow requires complete coverage, and audit reports overlaps. A later allow cannot authorize addresses outside its CIDRs.	false

RuleStatus.status.rule.enforce.network.policies.ingress

Ingress constrains source address grants. Selector-based peers are unaffected. No constraints are applied when omitted.

Name	Type	Description	Required
cidrs	[]string	CIDRs matches addresses granted by ipBlock.cidr after subtracting except. Omitted peers on an ingress or egress rule grant both IPv4 and IPv6 address space. An empty NetworkPolicy ingress or egress list has no grants in that direction to validate. Each direction is evaluated independently. An empty CIDRs list imposes no constraint. Ordered allow/deny decisions apply to every granted address: deny rejects overlapping grants, allow requires complete coverage, and audit reports overlaps. A later allow cannot authorize addresses outside its CIDRs.	false

RuleStatus.status.rule.enforce.services

Enforcement for Services.

Name	Type	Description	Required
externalIPs	object	ExternalIPs defines constraints for spec.externalIPs.	false
externalNames	object	ExternalNames defines additional constraints for Services of type ExternalName.	false
loadBalancers	object	LoadBalancers defines additional constraints for Services of type LoadBalancer.	false
nodePorts	object	NodePorts defines additional constraints for nodePort values.	false
types	[]enum	Types defines the Service types matched by this rule. Supported values: - ClusterIP - NodePort - LoadBalancer - ExternalName Enum: ClusterIP, NodePort, LoadBalancer, ExternalName	false

RuleStatus.status.rule.enforce.services.externalIPs

ExternalIPs defines constraints for spec.externalIPs.

Name	Type	Description	Required
cidrs	[]string	CIDRs restricts spec.externalIPs. Individual IP addresses are treated as host CIDRs (/32 for IPv4 and /128 for IPv6). For deny rules, empty means all external IPs are denied. For allow and audit rules, empty means no external IP restriction.	false

RuleStatus.status.rule.enforce.services.externalNames

ExternalNames defines additional constraints for Services of type ExternalName.

Name	Type	Description	Required
hostnames	[]object	Hostnames restricts spec.externalName. Empty means no additional hostname restriction once ExternalName is allowed by types.	false

RuleStatus.status.rule.enforce.services.externalNames.hostnames[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.services.loadBalancers

LoadBalancers defines additional constraints for Services of type LoadBalancer.

Name	Type	Description	Required
cidrs	[]string	CIDRs restricts spec.loadBalancerIP and spec.loadBalancerSourceRanges. Empty means no additional CIDR restriction once LoadBalancer is allowed by types.	false

RuleStatus.status.rule.enforce.services.nodePorts

NodePorts defines additional constraints for nodePort values.

Name	Type	Description	Required
ports	[]object	Ports restricts explicitly requested nodePort values. Empty means no additional port restriction once NodePort is allowed by types.	false

RuleStatus.status.rule.enforce.services.nodePorts.ports[index]

Name	Type	Description	Required
from	integer	Format: int32 Minimum: 1 Maximum: 65535	true
to	integer	Format: int32 Minimum: 1 Maximum: 65535	true

RuleStatus.status.rule.enforce.storage

Additional access to PersistentVolumes without a tenant ownership label.

Name	Type	Description	Required
volumes	[]object	Volumes matches additional PersistentVolumes. Entries are ORed. Conditions on the enclosing enforcement rule must also match. The last matching allow/deny decides the exception; audit never grants access.	false

RuleStatus.status.rule.enforce.storage.volumes[index]

PersistentVolumeMatch matches labels on the referenced PersistentVolume.

Name	Type	Description	Required
selector	object	Selector matches existing PV labels, independently of PVC.spec.selector. An explicit empty selector matches all PVs. Capsule’s tenant ownership label cannot be overridden by a selector or an allow decision.	true
name	string	Name identifies this match in admission and audit messages.	false

RuleStatus.status.rule.enforce.storage.volumes[index].selector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.enforce.storage.volumes[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.enforce.workloads

Enforcement for native workload kinds and Pod properties.

Name	Type	Description	Required
disruptionBudgets	object	DisruptionBudgets constrains PDB coverage of selected Pods and controller templates. Only whole-workload targets apply; omitted targets select Pods.	false
placement	object	Placement matches the Pod scheduler and scheduling constraints.	false
qosClasses	[]string	Define Pod QoS classes matched by this enforcement rule. Supported values are Guaranteed, Burstable and BestEffort.	false
registries	[]object	Define registries which are allowed to be used within this tenant The rules are aggregated, since you can use Regular Expressions the match registry endpoints	false
resources	object	Resources defines mutation and enforcement policies for Pod and container resource requests and limits. The workload targets select where the policies apply. With no targets, resource policies apply to all compatible locations: Pod-level resources, regular containers, and init containers. Resource names unsupported at Pod level still apply to compatible container locations. Mutation is applied when a Pod is created, or when an explicitly targeted controller is created or updated (to its Pod template). Remove and MatchRequest manage explicit values, Default fills an absent value, and Ratio fills an absent limit from its request. An explicit Ratio violation is then handled by the enclosing allow, deny, or audit action.	false
schedulers	[]object	Schedulers defines additional schedulerName matchers for selected Pods and Pod templates. These are combined with placement.schedulers under the same rule action; a match in either field counts as a match. Deprecated: use Placement.Schedulers instead.	false
security	object	Security matches effective Pod and container security profiles.	false
targets	[]string	Targets selects native workloads and, optionally, parts of their Pod specs. With no workload policies, the action matches the selected kinds themselves. With policies, targets scopes those policies; it does not also match the kind. Omitted targets preserve Pod-only defaults. Controller targets are opt-in. Whole-controller targets include every compatible location in the Pod template. Existing pod and pod/* part targets retain their established policy scope.	false

RuleStatus.status.rule.enforce.workloads.disruptionBudgets

DisruptionBudgets constrains PDB coverage of selected Pods and controller templates. Only whole-workload targets apply; omitted targets select Pods.

Name	Type	Description	Required
allowOverlap	boolean	AllowOverlap permits multiple PDBs to select the same Pod or selected controller template. False rejects overlap for allow/deny actions and reports violations for audit. True permits overlap in an allow rule; omitted does not participate in evaluation. Rule ordering is preserved. Checks run on PDB selector changes and selected Pod/template label changes. Conditions and audience apply to the admission request, including PDB writes. Cross-resource admission reads are not an atomic transaction.	false
evictableReplicas	object	EvictableReplicas bounds the configured number of evictable replicas of each selected Deployment, StatefulSet, ReplicaSet or ReplicationController. Uses desired spec.replicas and Kubernetes’ upward percentage rounding; zero replicas are exempt. Checks controller writes, /scale, and PDB writes. Bounds are inclusive nonnegative integers; omitted bounds are unrestricted. Allow requires compliance; deny/audit match violations, like resource constraints. Other targets do not have a desired replica count and skip this property. At least one supported controller target must be explicitly selected.	false
unhealthyPodEvictionPolicies	[]enum	UnhealthyPodEvictionPolicies matches the effective PDB policy. Omission in a PDB means IfHealthyBudget. Allow accepts listed values; deny/audit match listed values, like other enum lists. Empty imposes no restriction. Enum: IfHealthyBudget, AlwaysAllow	false

RuleStatus.status.rule.enforce.workloads.disruptionBudgets.evictableReplicas

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rule.enforce.workloads.placement

Placement matches the Pod scheduler and scheduling constraints.

Name	Type	Description	Required
affinity	[]object	Affinity matches each complete required or preferred affinity term.	false
nodeSelector	[]object	NodeSelector matches each node selector entry. Empty matchers match any entry. Placement rules apply to the pod target; allowing an entry does not require it.	false
schedulers	[]object	Schedulers defines schedulerName matchers for selected Pods and Pod templates. The rule is evaluated against pod.spec.schedulerName. Empty schedulerName is ignored and is not normalized to default-scheduler.	false
tolerations	[]object	Tolerations matches each toleration, including injected and wildcard tolerations.	false
topologySpreadConstraints	[]object	TopologySpreadConstraints matches each complete spread constraint.	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index]

WorkloadAffinityMatch matches an entire affinity term. Fields are ANDed; entries in enforce.workloads.placement.affinity are alternatives. {} matches any term.

Name	Type	Description	Required
fieldRequirements	[]object	FieldRequirements constrains node matchFields. When Requirements is set, matchFields must be explicitly permitted here to avoid an unchecked path.	false
labelSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
modes	[]enum	Modes defaults to both scheduling modes; it does not require their presence. Enum: required, preferred	false
namespaceScope	enum	SameNamespace requires no namespaceSelector and only the Pod’s namespace in namespaces (or an omitted namespaces list). Any imposes no restriction. Enum: SameNamespace, Any	false
namespaceSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
namespaces	object	Namespaces constrains each explicitly supplied namespace.	false
requirements	[]object	Requirements constrains node matchExpressions.	false
topologyKey	object	TopologyKey applies to Pod affinity and anti-affinity.	false
types	[]enum	Types defaults to all types. Type-specific constraints only match the types on which they are meaningful. Enum: nodeAffinity, podAffinity, podAntiAffinity	false
weight	object	Weight requires modes: [preferred].	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].fieldRequirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].fieldRequirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].fieldRequirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].labelSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].labelSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].labelSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].labelSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].namespaceSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].namespaces

Namespaces constrains each explicitly supplied namespace.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].topologyKey

TopologyKey applies to Pod affinity and anti-affinity.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.affinity[index].weight

Weight requires modes: [preferred].

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rule.enforce.workloads.placement.nodeSelector[index]

WorkloadNodeSelectorMatch matches a nodeSelector key/value pair. Omitted fields are unrestricted, so {} matches every entry, including an empty label value.

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rule.enforce.workloads.placement.nodeSelector[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.nodeSelector[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.schedulers[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.tolerations[index]

Name	Type	Description	Required
effects	[]enum	Effects matches the literal effect. An empty effect on a Pod tolerates all effects and does not match an allowlist of individual effects. Enum: NoSchedule, PreferNoSchedule, NoExecute,	false
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Operators matches Equal (including an omitted operator) or Exists. Enum: Equal, Exists	false
tolerationSeconds	object	TolerationDurationMatch treats an absent tolerationSeconds as unlimited.	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rule.enforce.workloads.placement.tolerations[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.tolerations[index].tolerationSeconds

TolerationDurationMatch treats an absent tolerationSeconds as unlimited.

Name	Type	Description	Required
allowUnlimited	boolean	AllowUnlimited defaults to true. Set false to require a finite duration.	false
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rule.enforce.workloads.placement.tolerations[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.topologySpreadConstraints[index]

Name	Type	Description	Required
labelSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
maxSkew	object	PlacementRange defines inclusive bounds. An omitted bound is unrestricted.	false
minDomains	object	MinDomains uses 1 when the Pod omits minDomains.	false
nodeAffinityPolicy	[]enum	NodeAffinityPolicy uses Honor when the Pod omits the field. Enum: Honor, Ignore	false
nodeTaintsPolicy	[]enum	NodeTaintsPolicy uses Ignore when the Pod omits the field. Enum: Honor, Ignore	false
topologyKey	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
whenUnsatisfiable	[]enum	Enum: DoNotSchedule, ScheduleAnyway	false

RuleStatus.status.rule.enforce.workloads.placement.topologySpreadConstraints[index].labelSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

RuleStatus.status.rule.enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rule.enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.placement.topologySpreadConstraints[index].maxSkew

PlacementRange defines inclusive bounds. An omitted bound is unrestricted.

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rule.enforce.workloads.placement.topologySpreadConstraints[index].minDomains

MinDomains uses 1 when the Pod omits minDomains.

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rule.enforce.workloads.placement.topologySpreadConstraints[index].topologyKey

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.registries[index]

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false
policy	[]string	Allowed PullPolicy for the given registry. Supplying no value allows all policies.	false

RuleStatus.status.rule.enforce.workloads.resources

Name	Type	Description	Required
limits	map[string]object	Limits defines policies for resource limits.	false
requests	map[string]object	Requests defines policies for resource requests.	false

RuleStatus.status.rule.enforce.workloads.resources.limits[key]

WorkloadResourceLimitPolicy defines how a resource limit is mutated and enforced.

Name	Type	Description	Required
policy	enum	Policy selects how the limit is handled: Preserve leaves it unchanged, Default fills an absent limit, Remove deletes it, MatchRequest manages it to equal the request, and Ratio defaults an absent limit and enforces the maximum multiplier against explicitly supplied limits. Enum: Preserve, Default, Remove, MatchRequest, Ratio	true
value	int or string	Value is the quantity applied by Default or the maximum limit-to-request multiplier applied by Ratio.	false

RuleStatus.status.rule.enforce.workloads.resources.requests[key]

WorkloadResourceRequestPolicy defines how a resource request is mutated.

Name	Type	Description	Required
policy	enum	Policy selects how the request is handled: Preserve leaves it unchanged, Default fills an absent request, and Remove deletes it. Enum: Preserve, Default, Remove	true
value	int or string	Value is the quantity applied by the Default policy.	false

RuleStatus.status.rule.enforce.workloads.schedulers[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.security

Security matches effective Pod and container security profiles.

Name	Type	Description	Required
appArmorProfiles	[]object	AppArmorProfiles matches effective Linux container profiles, including legacy AppArmor annotations before Pod defaults. It uses the same target and missing-profile semantics as SeccompProfiles. Privileged containers are Unconfined. Matching a Localhost name does not verify its installation.	false
seccompProfiles	[]object	SeccompProfiles matches effective Linux container profiles, resolving container overrides before Pod defaults. Privileged containers are Unconfined. Missing profiles do not match any type, so an allow-list rejects them. Omitted targets check regular, init and ephemeral containers; pod explicitly selects only the Pod default. Controller targets check their Pod templates.	false

RuleStatus.status.rule.enforce.workloads.security.appArmorProfiles[index]

WorkloadSecurityProfileMatch matches a profile type and optionally its local name/path. Entries are alternatives; localhostProfiles constrains only Localhost.

Name	Type	Description	Required
types	[]enum	Types selects profile types. Missing profiles never match. Enum: RuntimeDefault, Localhost, Unconfined	true
localhostProfiles	[]object	LocalhostProfiles matches seccomp paths relative to the kubelet seccomp directory, or loaded AppArmor profile names. Omitted permits any Localhost profile. Requires Localhost in types. Expressions use exact, exp and negate.	false

RuleStatus.status.rule.enforce.workloads.security.appArmorProfiles[index].localhostProfiles[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.enforce.workloads.security.seccompProfiles[index]

WorkloadSecurityProfileMatch matches a profile type and optionally its local name/path. Entries are alternatives; localhostProfiles constrains only Localhost.

Name	Type	Description	Required
types	[]enum	Types selects profile types. Missing profiles never match. Enum: RuntimeDefault, Localhost, Unconfined	true
localhostProfiles	[]object	LocalhostProfiles matches seccomp paths relative to the kubelet seccomp directory, or loaded AppArmor profile names. Omitted permits any Localhost profile. Requires Localhost in types. Expressions use exact, exp and negate.	false

RuleStatus.status.rule.enforce.workloads.security.seccompProfiles[index].localhostProfiles[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rule.mutate[index]

NamespaceRuleMutation applies typed mutations independently of enforce.action. Mutated values remain subject to all applicable enforcement rules.

Name	Type	Description	Required
action	enum	Action chooses how explicitly supplied properties are applied. Merge fills an empty schedulerName and absent security profiles, sets hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, and map keys, adds missing imagePullSecrets by name, upserts lists, and conjoins required affinity. Replace replaces each supplied property in full: scheduler, hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, nodeSelector, registries.imagePullSecrets, tolerations, topologySpreadConstraints, affinity, or security profiles. Omitted properties are retained. Supplying affinity replaces all its branches, including omitted ones. Enum: merge, replace Default: merge	false
conditions	[]object	Conditions gate this entire mutation entry and inspect the object after preceding mutations. All conditions must be true; empty means apply.	false
workloads	object	WorkloadMutation contains typed native Pod values. Empty maps/lists are preserved so replace can clear a property; nil means the property is omitted. It applies on Pod creation. Security.ReadOnlyRootFilesystem and Registries.ImagePullPolicy also apply to newly added ephemeral containers on subresource updates. Existing containers are never reconciled. On merge, later entries override hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, matching map keys, tolerations and spread constraints. On merge, required affinity restrictions from applicable entries are ANDed.	false

RuleStatus.status.rule.mutate[index].conditions[index]

Name	Type	Description	Required
expression	string	Expression must evaluate to bool. Any false condition skips this block. An evaluation error rejects the request unless another condition is false.	true
name	string	Name identifies a condition in admission errors. Names must be unique within a block.	false

RuleStatus.status.rule.mutate[index].workloads

Name	Type	Description	Required
placement	object	Placement configures the Pod scheduler and scheduling constraints.	false
registries	object	Registries configures Pod image pull secrets and selected containers’ pull policies.	false
security	object	Security configures Pod and container security settings.	false
targets	[]enum	Targets selects compatible Pod locations. Omitted or empty selects all compatible locations. The pod target includes Pod-level properties and all container groups; pod/containers, pod/initcontainers and pod/ephemeralcontainers narrow selection to one group. Controller templates and volumes are not supported. Enum: pod, pod/containers, pod/initcontainers, pod/ephemeralcontainers	false

RuleStatus.status.rule.mutate[index].workloads.placement

Placement configures the Pod scheduler and scheduling constraints.

Name	Type	Description	Required
affinity	object	Affinity on merge conjoins required restrictions and upserts preferred terms. Preferred terms match by their complete term, excluding weight and normalized ordering. A different term is added; a matching term gets the supplied weight. Required node affinity is distributed over existing OR alternatives, with at most 256 resulting alternatives. Empty node selector terms match no nodes. Replace replaces all affinity, including any branches omitted from the rule.	false
nodeSelector	map[string]string	NodeSelector sets configured keys on merge, or replaces the map on replace.	false
scheduler	string	Scheduler sets spec.schedulerName on Pod creation. Merge fills only an empty schedulerName, preserving all non-empty names, including default-scheduler. Kubernetes defaults omitted schedulerName before admission. Use replace with a condition to override default-scheduler while preserving custom schedulers. Replace always overwrites schedulerName when the entry’s conditions match. Omitted or null leaves the Pod value unchanged.	false
tolerations	[]object	Tolerations merges by key, operator (default Equal), value and effect. Changing any identity field adds another toleration, retaining the old one. A matching toleration’s duration is replaced; omitting it makes it unlimited. Replace replaces the entire list, including entries with other identities.	false
topologySpreadConstraints	[]object	TopologySpreadConstraints merges by topologyKey and whenUnsatisfiable. The entire matching constraint is replaced, including its selector and optional fields; other constraints remain. Replace replaces the entire list.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity

Name	Type	Description	Required
nodeAffinity	object	Describes node affinity scheduling rules for the pod.	false
podAffinity	object	Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).	false
podAntiAffinity	object	Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.nodeAffinity

Describes node affinity scheduling rules for the pod.

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding “weight” to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	object	If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it’s a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).

Name	Type	Description	Required
preference	object	A node selector term, associated with the corresponding weight.	true
weight	integer	Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100. Format: int32	true

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference

A node selector term, associated with the corresponding weight.

Name	Type	Description	Required
matchExpressions	[]object	A list of node selector requirements by node’s labels.	false
matchFields	[]object	A list of node selector requirements by node’s fields.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference.matchExpressions[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference.matchFields[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution

Name	Type	Description	Required
nodeSelectorTerms	[]object	Required. A list of node selector terms. The terms are ORed.	true

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index]

A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.

Name	Type	Description	Required
matchExpressions	[]object	A list of node selector requirements by node’s labels.	false
matchFields	[]object	A list of node selector requirements by node’s fields.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index].matchExpressions[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index].matchFields[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity

Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding “weight” to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	[]object	If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)

Name	Type	Description	Required
podAffinityTerm	object	Required. A pod affinity term, associated with the corresponding weight.	true
weight	integer	weight associated with matching the corresponding podAffinityTerm, in the range 1-100. Format: int32	true

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm

Required. A pod affinity term, associated with the corresponding weight.

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index]

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity

Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and subtracting “weight” from the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	[]object	If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)

Name	Type	Description	Required
podAffinityTerm	object	Required. A pod affinity term, associated with the corresponding weight.	true
weight	integer	weight associated with matching the corresponding podAffinityTerm, in the range 1-100. Format: int32	true

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm

Required. A pod affinity term, associated with the corresponding weight.

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index]

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.placement.tolerations[index]

The pod this Toleration is attached to tolerates any taint that matches the triple <key,value,effect> using the matching operator .

Name	Type	Description	Required
effect	string	Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.	false
key	string	Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.	false
operator	string	Operator represents a key’s relationship to the value. Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category. Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).	false
tolerationSeconds	integer	TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system. Format: int64	false
value	string	Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.	false

RuleStatus.status.rule.mutate[index].workloads.placement.topologySpreadConstraints[index]

TopologySpreadConstraint specifies how to spread matching pods among the given topology.

Name	Type	Description	Required
maxSkew	integer	MaxSkew describes the degree to which pods may be unevenly distributed. When `whenUnsatisfiable=DoNotSchedule`, it is the maximum permitted difference between the number of matching pods in the target topology and the global minimum. The global minimum is the minimum number of matching pods in an eligible domain or zero if the number of eligible domains is less than MinDomains. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 2/2/1: In this case, the global minimum is 1.	zone1
topologyKey	string	TopologyKey is the key of node labels. Nodes that have a label with this key and identical values are considered to be in the same topology. We consider each <key, value> as a “bucket”, and try to put balanced number of pods into each bucket. We define a domain as a particular instance of a topology. Also, we define an eligible domain as a domain whose nodes meet the requirements of nodeAffinityPolicy and nodeTaintsPolicy. e.g. If TopologyKey is “kubernetes.io/hostname”, each Node is a domain of that topology. And, if TopologyKey is “topology.kubernetes.io/zone”, each zone is a domain of that topology. It’s a required field.	true
whenUnsatisfiable	string	WhenUnsatisfiable indicates how to deal with a pod if it doesn’t satisfy the spread constraint. - DoNotSchedule (default) tells the scheduler not to schedule it. - ScheduleAnyway tells the scheduler to schedule the pod in any location, but giving higher precedence to topologies that would help reduce the skew. A constraint is considered “Unsatisfiable” for an incoming pod if and only if every possible node assignment for that pod would violate “MaxSkew” on some topology. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 3/1/1:	zone1
labelSelector	object	LabelSelector is used to find matching pods. Pods that match this label selector are counted to determine the number of pods in their corresponding topology domain.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select the pods over which spreading will be calculated. The keys are used to lookup values from the incoming pod labels, those key-value labels are ANDed with labelSelector to select the group of existing pods over which spreading will be calculated for the incoming pod. The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. MatchLabelKeys cannot be set when LabelSelector isn’t set. Keys that don’t exist in the incoming pod labels will be ignored. A null or empty list means only match against labelSelector. This is a beta field and requires the MatchLabelKeysInPodTopologySpread feature gate to be enabled (enabled by default).	false
minDomains	integer	MinDomains indicates a minimum number of eligible domains. When the number of eligible domains with matching topology keys is less than minDomains, Pod Topology Spread treats “global minimum” as 0, and then the calculation of Skew is performed. And when the number of eligible domains with matching topology keys equals or greater than minDomains, this value has no effect on scheduling. As a result, when the number of eligible domains is less than minDomains, scheduler won’t schedule more than maxSkew Pods to those domains. If value is nil, the constraint behaves as if MinDomains is equal to 1. Valid values are integers greater than 0. When value is not nil, WhenUnsatisfiable must be DoNotSchedule. For example, in a 3-zone cluster, MaxSkew is set to 2, MinDomains is set to 5 and pods with the same labelSelector spread as 2/2/2:	zone1
nodeAffinityPolicy	string	NodeAffinityPolicy indicates how we will treat Pod’s nodeAffinity/nodeSelector when calculating pod topology spread skew. Options are: - Honor: only nodes matching nodeAffinity/nodeSelector are included in the calculations. - Ignore: nodeAffinity/nodeSelector are ignored. All nodes are included in the calculations. If this value is nil, the behavior is equivalent to the Honor policy.	false
nodeTaintsPolicy	string	NodeTaintsPolicy indicates how we will treat node taints when calculating pod topology spread skew. Options are: - Honor: nodes without taints, along with tainted nodes for which the incoming pod has a toleration, are included. - Ignore: node taints are ignored. All nodes are included. If this value is nil, the behavior is equivalent to the Ignore policy.	false

RuleStatus.status.rule.mutate[index].workloads.placement.topologySpreadConstraints[index].labelSelector

LabelSelector is used to find matching pods. Pods that match this label selector are counted to determine the number of pods in their corresponding topology domain.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rule.mutate[index].workloads.placement.topologySpreadConstraints[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rule.mutate[index].workloads.registries

Registries configures Pod image pull secrets and selected containers’ pull policies.

Name	Type	Description	Required
imagePullPolicy	enum	ImagePullPolicy sets imagePullPolicy on every selected regular or init container at Pod creation, and newly added ephemeral containers on subresource updates. Both merge and replace overwrite explicit and Kubernetes-defaulted values. Omitted or null preserves the existing policy. Applies to all Pod OSes. Enum: Always, IfNotPresent, Never	false
imagePullSecrets	[]object	ImagePullSecrets sets spec.imagePullSecrets on Pod creation. Merge appends missing names and removes existing duplicates, retaining first-occurrence order. Replace sets the complete list; an explicit empty list clears it. Omitted or null preserves it. Requires the pod target or omitted/empty targets. References always use the Pod’s namespace; Capsule does not create, copy, or check the referenced Secrets.	false

RuleStatus.status.rule.mutate[index].workloads.registries.imagePullSecrets[index]

LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.

Name	Type	Description	Required
name	string	Name of the referent. This field is effectively required, but due to backwards compatibility is allowed to be empty. Instances of this type with an empty value here are almost certainly wrong. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names Default:	false

RuleStatus.status.rule.mutate[index].workloads.security

Security configures Pod and container security settings.

Name	Type	Description	Required
appArmorProfile	object	AppArmorProfile supplies the Pod-level securityContext.appArmorProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it. AppArmor and any Localhost profile must be available on eligible nodes.	false
hostUsers	boolean	HostUsers sets spec.hostUsers on both merge and replace. False requests a separate user namespace; true uses the host user namespace. Omitted or null leaves the Pod value unchanged. Requires Kubernetes/runtime support.	false
readOnlyRootFilesystem	boolean	ReadOnlyRootFilesystem sets securityContext.readOnlyRootFilesystem on every selected regular or init container at Pod creation, and newly added ephemeral containers on subresource updates. Both merge and replace overwrite the value. False is an explicit setting; nil preserves it. Windows Pods are skipped.	false
seccompProfile	object	SeccompProfile supplies the Pod-level securityContext.seccompProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it.	false

RuleStatus.status.rule.mutate[index].workloads.security.appArmorProfile

Name	Type	Description	Required
type	string	type indicates which kind of AppArmor profile will be applied. Valid options are: Localhost - a profile pre-loaded on the node. RuntimeDefault - the container runtime’s default profile. Unconfined - no AppArmor enforcement.	true
localhostProfile	string	localhostProfile indicates a profile loaded on the node that should be used. The profile must be preconfigured on the node to work. Must match the loaded name of the profile. Must be set if and only if type is “Localhost”.	false

RuleStatus.status.rule.mutate[index].workloads.security.seccompProfile

Name	Type	Description	Required
type	string	type indicates which kind of seccomp profile will be applied. Valid options are: Localhost - a profile defined in a file on the node should be used. RuntimeDefault - the container runtime default profile should be used. Unconfined - no profile should be applied.	true
localhostProfile	string	localhostProfile indicates a profile defined in a file on the node should be used. The profile must be preconfigured on the node to work. Must be a descending path, relative to the kubelet’s configured seccomp profile location. Must be set if type is “Localhost”. Must NOT be set for any other type.	false

RuleStatus.status.rule.quota[index]

Name	Type	Description	Required
name	string	Name is the stable identity of this quota within the Tenant. Changing the name replaces the generated GlobalResourceQuota; changing the quota or its namespace selector updates the existing object.	true
hard	map[string]int or string	hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
scopeSelector	object	scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.	false
scopes	[]string	A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.	false

RuleStatus.status.rule.quota[index].scopeSelector

Name	Type	Description	Required
matchExpressions	[]object	A list of scope selector requirements by scope of the resources.	false

RuleStatus.status.rule.quota[index].scopeSelector.matchExpressions[index]

A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.

Name	Type	Description	Required
operator	string	Represents a scope’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.	true
scopeName	string	The name of the scope that the selector applies to.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index]

For future implementation where users might manage RuleStatus CRs themselves

Name	Type	Description	Required
audience	[]object	Audience limits this rule to matching request subjects. An empty audience matches every request.	false
enforce	object	Enforcement for given rule	false
mutate	[]object	Mutate applies ordered typed mutations before enforcement. It shares the rule’s namespace selection and audience. Workload mutations apply on Pod creation.	false
quota	[]object	Quota contains native Kubernetes ResourceQuota specifications shared by all namespaces selected by this rule. Unlike Enforce, quota accounting is independent of the request audience.	false

RuleStatus.status.rules[index].audience[index]

Name	Type	Description	Required
kind	enum	Enum: User, Group, ServiceAccount, Custom	true
name	string		true

RuleStatus.status.rules[index].enforce

Enforcement for given rule

Name	Type	Description	Required
action	enum	Declare the action being performed on the enforcement rule: deny: On match, deny admission request allow: On match, allowed admission request audit: On match, audit (post event) of admission request Enum: allow, deny, audit Default: deny	false
conditions	[]object	Conditions gate this entire enforcement rule, including workloads, services, metadata, ingress, network and storage. All conditions must be true; empty means apply. Conditions run during admission; managed metadata in a conditional rule is applied on matching requests only and is not reconciled in the background.	false
ingress	object	Enforcement for Ingress and Gateway API resource hostnames.	false
metadata	[]object	Enforcement for object metadata on namespaced resources.	false
network	object	Network configures network enforcement for namespace resources.	false
services	object	Enforcement for Services.	false
storage	object	Additional access to PersistentVolumes without a tenant ownership label.	false
workloads	object	Enforcement for native workload kinds and Pod properties.	false

RuleStatus.status.rules[index].enforce.conditions[index]

Name	Type	Description	Required
expression	string	Expression must evaluate to bool. Any false condition skips this block. An evaluation error rejects the request unless another condition is false.	true
name	string	Name identifies a condition in admission errors. Names must be unique within a block.	false

RuleStatus.status.rules[index].enforce.ingress

Enforcement for Ingress and Gateway API resource hostnames.

Name	Type	Description	Required
hostnames	[]object	Hostnames defines allowed, denied, or audited hostname expressions. A resource targeted by an allow or deny rule must declare non-empty values in all hostname fields. Audit-only rules record missing hostnames without denying them.	false
types	[]enum	Types defines the resource kinds to which hostname enforcement applies. Enum: Ingress, Route, ListenerSet, HTTPRoute, Gateway, TLSRoute, GRPCRoute	false

RuleStatus.status.rules[index].enforce.ingress.hostnames[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.metadata[index]

MetadataRule defines metadata constraints for namespaced resources.

Name	Type	Description	Required
kinds	[]string	Kinds of the referents. Use “*” to match all kinds.	true
annotations	map[string]object	Annotations defines metadata policies by annotation key.	false
apiGroups	[]string	API groups or API group/version selectors of the referents. Empty or omitted APIGroups means the core Kubernetes API version “v1”. Use “” to match all API groups and versions. Examples: - [] or [""] means core “v1”. - [“v1”] means core “v1”. - [“apps”] means any version in the “apps” API group. - [“apps/v1”] means only “apps/v1”. - [“apps”, “batch/v1”] means any “apps” version and “batch/v1”. - [""] means all API groups and versions.	false
labels	map[string]object	Labels defines metadata policies by label key.	false

RuleStatus.status.rules[index].enforce.metadata[index].annotations[key]

Name	Type	Description	Required
default	string	Default is applied by admission mutation when the concrete metadata key is absent. It is not reconciled after admission.	false
managed	string	Managed is enforced by admission mutation and reconciled by the RuleStatus controller using server-side apply when the rule configuration changes. Present metadata matching the effective managed value is exempt from metadata value validation, including overlapping deny rules. The last applicable managed value for each concrete key wins. During admission, only rules matching the request’s audience contribute to the effective managed value.	false
required	boolean	Required enforces that the metadata key must be present. This is only meaningful with action=allow. Deny and audit rules do not require missing metadata to exist. Default: false	false
values	[]object	Values defines allowed, denied, or audited values for the metadata key. With action=allow and no values, only Required is enforced. With action=deny or action=audit and no values, any present value matches, including an empty string.	false

RuleStatus.status.rules[index].enforce.metadata[index].annotations[key].values[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.metadata[index].labels[key]

Name	Type	Description	Required
default	string	Default is applied by admission mutation when the concrete metadata key is absent. It is not reconciled after admission.	false
managed	string	Managed is enforced by admission mutation and reconciled by the RuleStatus controller using server-side apply when the rule configuration changes. Present metadata matching the effective managed value is exempt from metadata value validation, including overlapping deny rules. The last applicable managed value for each concrete key wins. During admission, only rules matching the request’s audience contribute to the effective managed value.	false
required	boolean	Required enforces that the metadata key must be present. This is only meaningful with action=allow. Deny and audit rules do not require missing metadata to exist. Default: false	false
values	[]object	Values defines allowed, denied, or audited values for the metadata key. With action=allow and no values, only Required is enforced. With action=deny or action=audit and no values, any present value matches, including an empty string.	false

RuleStatus.status.rules[index].enforce.metadata[index].labels[key].values[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.network

Network configures network enforcement for namespace resources.

Name	Type	Description	Required
policies	object	Policies constrains native NetworkPolicies.	false

RuleStatus.status.rules[index].enforce.network.policies

Policies constrains native NetworkPolicies.

Name	Type	Description	Required
egress	object	Egress constrains destination address grants. Selector-based peers are unaffected. No constraints are applied when omitted.	false
ingress	object	Ingress constrains source address grants. Selector-based peers are unaffected. No constraints are applied when omitted.	false

RuleStatus.status.rules[index].enforce.network.policies.egress

Egress constrains destination address grants. Selector-based peers are unaffected. No constraints are applied when omitted.

Name	Type	Description	Required
cidrs	[]string	CIDRs matches addresses granted by ipBlock.cidr after subtracting except. Omitted peers on an ingress or egress rule grant both IPv4 and IPv6 address space. An empty NetworkPolicy ingress or egress list has no grants in that direction to validate. Each direction is evaluated independently. An empty CIDRs list imposes no constraint. Ordered allow/deny decisions apply to every granted address: deny rejects overlapping grants, allow requires complete coverage, and audit reports overlaps. A later allow cannot authorize addresses outside its CIDRs.	false

RuleStatus.status.rules[index].enforce.network.policies.ingress

Ingress constrains source address grants. Selector-based peers are unaffected. No constraints are applied when omitted.

Name	Type	Description	Required
cidrs	[]string	CIDRs matches addresses granted by ipBlock.cidr after subtracting except. Omitted peers on an ingress or egress rule grant both IPv4 and IPv6 address space. An empty NetworkPolicy ingress or egress list has no grants in that direction to validate. Each direction is evaluated independently. An empty CIDRs list imposes no constraint. Ordered allow/deny decisions apply to every granted address: deny rejects overlapping grants, allow requires complete coverage, and audit reports overlaps. A later allow cannot authorize addresses outside its CIDRs.	false

RuleStatus.status.rules[index].enforce.services

Enforcement for Services.

Name	Type	Description	Required
externalIPs	object	ExternalIPs defines constraints for spec.externalIPs.	false
externalNames	object	ExternalNames defines additional constraints for Services of type ExternalName.	false
loadBalancers	object	LoadBalancers defines additional constraints for Services of type LoadBalancer.	false
nodePorts	object	NodePorts defines additional constraints for nodePort values.	false
types	[]enum	Types defines the Service types matched by this rule. Supported values: - ClusterIP - NodePort - LoadBalancer - ExternalName Enum: ClusterIP, NodePort, LoadBalancer, ExternalName	false

RuleStatus.status.rules[index].enforce.services.externalIPs

ExternalIPs defines constraints for spec.externalIPs.

Name	Type	Description	Required
cidrs	[]string	CIDRs restricts spec.externalIPs. Individual IP addresses are treated as host CIDRs (/32 for IPv4 and /128 for IPv6). For deny rules, empty means all external IPs are denied. For allow and audit rules, empty means no external IP restriction.	false

RuleStatus.status.rules[index].enforce.services.externalNames

ExternalNames defines additional constraints for Services of type ExternalName.

Name	Type	Description	Required
hostnames	[]object	Hostnames restricts spec.externalName. Empty means no additional hostname restriction once ExternalName is allowed by types.	false

RuleStatus.status.rules[index].enforce.services.externalNames.hostnames[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.services.loadBalancers

LoadBalancers defines additional constraints for Services of type LoadBalancer.

Name	Type	Description	Required
cidrs	[]string	CIDRs restricts spec.loadBalancerIP and spec.loadBalancerSourceRanges. Empty means no additional CIDR restriction once LoadBalancer is allowed by types.	false

RuleStatus.status.rules[index].enforce.services.nodePorts

NodePorts defines additional constraints for nodePort values.

Name	Type	Description	Required
ports	[]object	Ports restricts explicitly requested nodePort values. Empty means no additional port restriction once NodePort is allowed by types.	false

RuleStatus.status.rules[index].enforce.services.nodePorts.ports[index]

Name	Type	Description	Required
from	integer	Format: int32 Minimum: 1 Maximum: 65535	true
to	integer	Format: int32 Minimum: 1 Maximum: 65535	true

RuleStatus.status.rules[index].enforce.storage

Additional access to PersistentVolumes without a tenant ownership label.

Name	Type	Description	Required
volumes	[]object	Volumes matches additional PersistentVolumes. Entries are ORed. Conditions on the enclosing enforcement rule must also match. The last matching allow/deny decides the exception; audit never grants access.	false

RuleStatus.status.rules[index].enforce.storage.volumes[index]

PersistentVolumeMatch matches labels on the referenced PersistentVolume.

Name	Type	Description	Required
selector	object	Selector matches existing PV labels, independently of PVC.spec.selector. An explicit empty selector matches all PVs. Capsule’s tenant ownership label cannot be overridden by a selector or an allow decision.	true
name	string	Name identifies this match in admission and audit messages.	false

RuleStatus.status.rules[index].enforce.storage.volumes[index].selector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].enforce.storage.volumes[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].enforce.workloads

Enforcement for native workload kinds and Pod properties.

Name	Type	Description	Required
disruptionBudgets	object	DisruptionBudgets constrains PDB coverage of selected Pods and controller templates. Only whole-workload targets apply; omitted targets select Pods.	false
placement	object	Placement matches the Pod scheduler and scheduling constraints.	false
qosClasses	[]string	Define Pod QoS classes matched by this enforcement rule. Supported values are Guaranteed, Burstable and BestEffort.	false
registries	[]object	Define registries which are allowed to be used within this tenant The rules are aggregated, since you can use Regular Expressions the match registry endpoints	false
resources	object	Resources defines mutation and enforcement policies for Pod and container resource requests and limits. The workload targets select where the policies apply. With no targets, resource policies apply to all compatible locations: Pod-level resources, regular containers, and init containers. Resource names unsupported at Pod level still apply to compatible container locations. Mutation is applied when a Pod is created, or when an explicitly targeted controller is created or updated (to its Pod template). Remove and MatchRequest manage explicit values, Default fills an absent value, and Ratio fills an absent limit from its request. An explicit Ratio violation is then handled by the enclosing allow, deny, or audit action.	false
schedulers	[]object	Schedulers defines additional schedulerName matchers for selected Pods and Pod templates. These are combined with placement.schedulers under the same rule action; a match in either field counts as a match. Deprecated: use Placement.Schedulers instead.	false
security	object	Security matches effective Pod and container security profiles.	false
targets	[]string	Targets selects native workloads and, optionally, parts of their Pod specs. With no workload policies, the action matches the selected kinds themselves. With policies, targets scopes those policies; it does not also match the kind. Omitted targets preserve Pod-only defaults. Controller targets are opt-in. Whole-controller targets include every compatible location in the Pod template. Existing pod and pod/* part targets retain their established policy scope.	false

RuleStatus.status.rules[index].enforce.workloads.disruptionBudgets

DisruptionBudgets constrains PDB coverage of selected Pods and controller templates. Only whole-workload targets apply; omitted targets select Pods.

Name	Type	Description	Required
allowOverlap	boolean	AllowOverlap permits multiple PDBs to select the same Pod or selected controller template. False rejects overlap for allow/deny actions and reports violations for audit. True permits overlap in an allow rule; omitted does not participate in evaluation. Rule ordering is preserved. Checks run on PDB selector changes and selected Pod/template label changes. Conditions and audience apply to the admission request, including PDB writes. Cross-resource admission reads are not an atomic transaction.	false
evictableReplicas	object	EvictableReplicas bounds the configured number of evictable replicas of each selected Deployment, StatefulSet, ReplicaSet or ReplicationController. Uses desired spec.replicas and Kubernetes’ upward percentage rounding; zero replicas are exempt. Checks controller writes, /scale, and PDB writes. Bounds are inclusive nonnegative integers; omitted bounds are unrestricted. Allow requires compliance; deny/audit match violations, like resource constraints. Other targets do not have a desired replica count and skip this property. At least one supported controller target must be explicitly selected.	false
unhealthyPodEvictionPolicies	[]enum	UnhealthyPodEvictionPolicies matches the effective PDB policy. Omission in a PDB means IfHealthyBudget. Allow accepts listed values; deny/audit match listed values, like other enum lists. Empty imposes no restriction. Enum: IfHealthyBudget, AlwaysAllow	false

RuleStatus.status.rules[index].enforce.workloads.disruptionBudgets.evictableReplicas

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rules[index].enforce.workloads.placement

Placement matches the Pod scheduler and scheduling constraints.

Name	Type	Description	Required
affinity	[]object	Affinity matches each complete required or preferred affinity term.	false
nodeSelector	[]object	NodeSelector matches each node selector entry. Empty matchers match any entry. Placement rules apply to the pod target; allowing an entry does not require it.	false
schedulers	[]object	Schedulers defines schedulerName matchers for selected Pods and Pod templates. The rule is evaluated against pod.spec.schedulerName. Empty schedulerName is ignored and is not normalized to default-scheduler.	false
tolerations	[]object	Tolerations matches each toleration, including injected and wildcard tolerations.	false
topologySpreadConstraints	[]object	TopologySpreadConstraints matches each complete spread constraint.	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index]

WorkloadAffinityMatch matches an entire affinity term. Fields are ANDed; entries in enforce.workloads.placement.affinity are alternatives. {} matches any term.

Name	Type	Description	Required
fieldRequirements	[]object	FieldRequirements constrains node matchFields. When Requirements is set, matchFields must be explicitly permitted here to avoid an unchecked path.	false
labelSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
modes	[]enum	Modes defaults to both scheduling modes; it does not require their presence. Enum: required, preferred	false
namespaceScope	enum	SameNamespace requires no namespaceSelector and only the Pod’s namespace in namespaces (or an omitted namespaces list). Any imposes no restriction. Enum: SameNamespace, Any	false
namespaceSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
namespaces	object	Namespaces constrains each explicitly supplied namespace.	false
requirements	[]object	Requirements constrains node matchExpressions.	false
topologyKey	object	TopologyKey applies to Pod affinity and anti-affinity.	false
types	[]enum	Types defaults to all types. Type-specific constraints only match the types on which they are meaningful. Enum: nodeAffinity, podAffinity, podAntiAffinity	false
weight	object	Weight requires modes: [preferred].	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].fieldRequirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].fieldRequirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].fieldRequirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].labelSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].labelSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].labelSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].labelSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].namespaceSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].namespaces

Namespaces constrains each explicitly supplied namespace.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].topologyKey

TopologyKey applies to Pod affinity and anti-affinity.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.affinity[index].weight

Weight requires modes: [preferred].

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rules[index].enforce.workloads.placement.nodeSelector[index]

WorkloadNodeSelectorMatch matches a nodeSelector key/value pair. Omitted fields are unrestricted, so {} matches every entry, including an empty label value.

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rules[index].enforce.workloads.placement.nodeSelector[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.nodeSelector[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.schedulers[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.tolerations[index]

Name	Type	Description	Required
effects	[]enum	Effects matches the literal effect. An empty effect on a Pod tolerates all effects and does not match an allowlist of individual effects. Enum: NoSchedule, PreferNoSchedule, NoExecute,	false
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Operators matches Equal (including an omitted operator) or Exists. Enum: Equal, Exists	false
tolerationSeconds	object	TolerationDurationMatch treats an absent tolerationSeconds as unlimited.	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rules[index].enforce.workloads.placement.tolerations[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.tolerations[index].tolerationSeconds

TolerationDurationMatch treats an absent tolerationSeconds as unlimited.

Name	Type	Description	Required
allowUnlimited	boolean	AllowUnlimited defaults to true. Set false to require a finite duration.	false
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rules[index].enforce.workloads.placement.tolerations[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.topologySpreadConstraints[index]

Name	Type	Description	Required
labelSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
maxSkew	object	PlacementRange defines inclusive bounds. An omitted bound is unrestricted.	false
minDomains	object	MinDomains uses 1 when the Pod omits minDomains.	false
nodeAffinityPolicy	[]enum	NodeAffinityPolicy uses Honor when the Pod omits the field. Enum: Honor, Ignore	false
nodeTaintsPolicy	[]enum	NodeTaintsPolicy uses Ignore when the Pod omits the field. Enum: Honor, Ignore	false
topologyKey	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
whenUnsatisfiable	[]enum	Enum: DoNotSchedule, ScheduleAnyway	false

RuleStatus.status.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

RuleStatus.status.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

RuleStatus.status.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].maxSkew

PlacementRange defines inclusive bounds. An omitted bound is unrestricted.

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].minDomains

MinDomains uses 1 when the Pod omits minDomains.

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

RuleStatus.status.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].topologyKey

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.registries[index]

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false
policy	[]string	Allowed PullPolicy for the given registry. Supplying no value allows all policies.	false

RuleStatus.status.rules[index].enforce.workloads.resources

Name	Type	Description	Required
limits	map[string]object	Limits defines policies for resource limits.	false
requests	map[string]object	Requests defines policies for resource requests.	false

RuleStatus.status.rules[index].enforce.workloads.resources.limits[key]

WorkloadResourceLimitPolicy defines how a resource limit is mutated and enforced.

Name	Type	Description	Required
policy	enum	Policy selects how the limit is handled: Preserve leaves it unchanged, Default fills an absent limit, Remove deletes it, MatchRequest manages it to equal the request, and Ratio defaults an absent limit and enforces the maximum multiplier against explicitly supplied limits. Enum: Preserve, Default, Remove, MatchRequest, Ratio	true
value	int or string	Value is the quantity applied by Default or the maximum limit-to-request multiplier applied by Ratio.	false

RuleStatus.status.rules[index].enforce.workloads.resources.requests[key]

WorkloadResourceRequestPolicy defines how a resource request is mutated.

Name	Type	Description	Required
policy	enum	Policy selects how the request is handled: Preserve leaves it unchanged, Default fills an absent request, and Remove deletes it. Enum: Preserve, Default, Remove	true
value	int or string	Value is the quantity applied by the Default policy.	false

RuleStatus.status.rules[index].enforce.workloads.schedulers[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.security

Security matches effective Pod and container security profiles.

Name	Type	Description	Required
appArmorProfiles	[]object	AppArmorProfiles matches effective Linux container profiles, including legacy AppArmor annotations before Pod defaults. It uses the same target and missing-profile semantics as SeccompProfiles. Privileged containers are Unconfined. Matching a Localhost name does not verify its installation.	false
seccompProfiles	[]object	SeccompProfiles matches effective Linux container profiles, resolving container overrides before Pod defaults. Privileged containers are Unconfined. Missing profiles do not match any type, so an allow-list rejects them. Omitted targets check regular, init and ephemeral containers; pod explicitly selects only the Pod default. Controller targets check their Pod templates.	false

RuleStatus.status.rules[index].enforce.workloads.security.appArmorProfiles[index]

WorkloadSecurityProfileMatch matches a profile type and optionally its local name/path. Entries are alternatives; localhostProfiles constrains only Localhost.

Name	Type	Description	Required
types	[]enum	Types selects profile types. Missing profiles never match. Enum: RuntimeDefault, Localhost, Unconfined	true
localhostProfiles	[]object	LocalhostProfiles matches seccomp paths relative to the kubelet seccomp directory, or loaded AppArmor profile names. Omitted permits any Localhost profile. Requires Localhost in types. Expressions use exact, exp and negate.	false

RuleStatus.status.rules[index].enforce.workloads.security.appArmorProfiles[index].localhostProfiles[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].enforce.workloads.security.seccompProfiles[index]

WorkloadSecurityProfileMatch matches a profile type and optionally its local name/path. Entries are alternatives; localhostProfiles constrains only Localhost.

Name	Type	Description	Required
types	[]enum	Types selects profile types. Missing profiles never match. Enum: RuntimeDefault, Localhost, Unconfined	true
localhostProfiles	[]object	LocalhostProfiles matches seccomp paths relative to the kubelet seccomp directory, or loaded AppArmor profile names. Omitted permits any Localhost profile. Requires Localhost in types. Expressions use exact, exp and negate.	false

RuleStatus.status.rules[index].enforce.workloads.security.seccompProfiles[index].localhostProfiles[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

RuleStatus.status.rules[index].mutate[index]

NamespaceRuleMutation applies typed mutations independently of enforce.action. Mutated values remain subject to all applicable enforcement rules.

Name	Type	Description	Required
action	enum	Action chooses how explicitly supplied properties are applied. Merge fills an empty schedulerName and absent security profiles, sets hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, and map keys, adds missing imagePullSecrets by name, upserts lists, and conjoins required affinity. Replace replaces each supplied property in full: scheduler, hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, nodeSelector, registries.imagePullSecrets, tolerations, topologySpreadConstraints, affinity, or security profiles. Omitted properties are retained. Supplying affinity replaces all its branches, including omitted ones. Enum: merge, replace Default: merge	false
conditions	[]object	Conditions gate this entire mutation entry and inspect the object after preceding mutations. All conditions must be true; empty means apply.	false
workloads	object	WorkloadMutation contains typed native Pod values. Empty maps/lists are preserved so replace can clear a property; nil means the property is omitted. It applies on Pod creation. Security.ReadOnlyRootFilesystem and Registries.ImagePullPolicy also apply to newly added ephemeral containers on subresource updates. Existing containers are never reconciled. On merge, later entries override hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, matching map keys, tolerations and spread constraints. On merge, required affinity restrictions from applicable entries are ANDed.	false

RuleStatus.status.rules[index].mutate[index].conditions[index]

Name	Type	Description	Required
expression	string	Expression must evaluate to bool. Any false condition skips this block. An evaluation error rejects the request unless another condition is false.	true
name	string	Name identifies a condition in admission errors. Names must be unique within a block.	false

RuleStatus.status.rules[index].mutate[index].workloads

Name	Type	Description	Required
placement	object	Placement configures the Pod scheduler and scheduling constraints.	false
registries	object	Registries configures Pod image pull secrets and selected containers’ pull policies.	false
security	object	Security configures Pod and container security settings.	false
targets	[]enum	Targets selects compatible Pod locations. Omitted or empty selects all compatible locations. The pod target includes Pod-level properties and all container groups; pod/containers, pod/initcontainers and pod/ephemeralcontainers narrow selection to one group. Controller templates and volumes are not supported. Enum: pod, pod/containers, pod/initcontainers, pod/ephemeralcontainers	false

RuleStatus.status.rules[index].mutate[index].workloads.placement

Placement configures the Pod scheduler and scheduling constraints.

Name	Type	Description	Required
affinity	object	Affinity on merge conjoins required restrictions and upserts preferred terms. Preferred terms match by their complete term, excluding weight and normalized ordering. A different term is added; a matching term gets the supplied weight. Required node affinity is distributed over existing OR alternatives, with at most 256 resulting alternatives. Empty node selector terms match no nodes. Replace replaces all affinity, including any branches omitted from the rule.	false
nodeSelector	map[string]string	NodeSelector sets configured keys on merge, or replaces the map on replace.	false
scheduler	string	Scheduler sets spec.schedulerName on Pod creation. Merge fills only an empty schedulerName, preserving all non-empty names, including default-scheduler. Kubernetes defaults omitted schedulerName before admission. Use replace with a condition to override default-scheduler while preserving custom schedulers. Replace always overwrites schedulerName when the entry’s conditions match. Omitted or null leaves the Pod value unchanged.	false
tolerations	[]object	Tolerations merges by key, operator (default Equal), value and effect. Changing any identity field adds another toleration, retaining the old one. A matching toleration’s duration is replaced; omitting it makes it unlimited. Replace replaces the entire list, including entries with other identities.	false
topologySpreadConstraints	[]object	TopologySpreadConstraints merges by topologyKey and whenUnsatisfiable. The entire matching constraint is replaced, including its selector and optional fields; other constraints remain. Replace replaces the entire list.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity

Name	Type	Description	Required
nodeAffinity	object	Describes node affinity scheduling rules for the pod.	false
podAffinity	object	Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).	false
podAntiAffinity	object	Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity

Describes node affinity scheduling rules for the pod.

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding “weight” to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	object	If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it’s a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).

Name	Type	Description	Required
preference	object	A node selector term, associated with the corresponding weight.	true
weight	integer	Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100. Format: int32	true

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference

A node selector term, associated with the corresponding weight.

Name	Type	Description	Required
matchExpressions	[]object	A list of node selector requirements by node’s labels.	false
matchFields	[]object	A list of node selector requirements by node’s fields.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference.matchExpressions[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference.matchFields[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution

Name	Type	Description	Required
nodeSelectorTerms	[]object	Required. A list of node selector terms. The terms are ORed.	true

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index]

A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.

Name	Type	Description	Required
matchExpressions	[]object	A list of node selector requirements by node’s labels.	false
matchFields	[]object	A list of node selector requirements by node’s fields.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index].matchExpressions[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index].matchFields[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity

Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding “weight” to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	[]object	If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)

Name	Type	Description	Required
podAffinityTerm	object	Required. A pod affinity term, associated with the corresponding weight.	true
weight	integer	weight associated with matching the corresponding podAffinityTerm, in the range 1-100. Format: int32	true

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm

Required. A pod affinity term, associated with the corresponding weight.

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index]

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity

Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and subtracting “weight” from the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	[]object	If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)

Name	Type	Description	Required
podAffinityTerm	object	Required. A pod affinity term, associated with the corresponding weight.	true
weight	integer	weight associated with matching the corresponding podAffinityTerm, in the range 1-100. Format: int32	true

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm

Required. A pod affinity term, associated with the corresponding weight.

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index]

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.tolerations[index]

The pod this Toleration is attached to tolerates any taint that matches the triple <key,value,effect> using the matching operator .

Name	Type	Description	Required
effect	string	Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.	false
key	string	Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.	false
operator	string	Operator represents a key’s relationship to the value. Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category. Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).	false
tolerationSeconds	integer	TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system. Format: int64	false
value	string	Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.topologySpreadConstraints[index]

TopologySpreadConstraint specifies how to spread matching pods among the given topology.

Name	Type	Description	Required
maxSkew	integer	MaxSkew describes the degree to which pods may be unevenly distributed. When `whenUnsatisfiable=DoNotSchedule`, it is the maximum permitted difference between the number of matching pods in the target topology and the global minimum. The global minimum is the minimum number of matching pods in an eligible domain or zero if the number of eligible domains is less than MinDomains. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 2/2/1: In this case, the global minimum is 1.	zone1
topologyKey	string	TopologyKey is the key of node labels. Nodes that have a label with this key and identical values are considered to be in the same topology. We consider each <key, value> as a “bucket”, and try to put balanced number of pods into each bucket. We define a domain as a particular instance of a topology. Also, we define an eligible domain as a domain whose nodes meet the requirements of nodeAffinityPolicy and nodeTaintsPolicy. e.g. If TopologyKey is “kubernetes.io/hostname”, each Node is a domain of that topology. And, if TopologyKey is “topology.kubernetes.io/zone”, each zone is a domain of that topology. It’s a required field.	true
whenUnsatisfiable	string	WhenUnsatisfiable indicates how to deal with a pod if it doesn’t satisfy the spread constraint. - DoNotSchedule (default) tells the scheduler not to schedule it. - ScheduleAnyway tells the scheduler to schedule the pod in any location, but giving higher precedence to topologies that would help reduce the skew. A constraint is considered “Unsatisfiable” for an incoming pod if and only if every possible node assignment for that pod would violate “MaxSkew” on some topology. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 3/1/1:	zone1
labelSelector	object	LabelSelector is used to find matching pods. Pods that match this label selector are counted to determine the number of pods in their corresponding topology domain.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select the pods over which spreading will be calculated. The keys are used to lookup values from the incoming pod labels, those key-value labels are ANDed with labelSelector to select the group of existing pods over which spreading will be calculated for the incoming pod. The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. MatchLabelKeys cannot be set when LabelSelector isn’t set. Keys that don’t exist in the incoming pod labels will be ignored. A null or empty list means only match against labelSelector. This is a beta field and requires the MatchLabelKeysInPodTopologySpread feature gate to be enabled (enabled by default).	false
minDomains	integer	MinDomains indicates a minimum number of eligible domains. When the number of eligible domains with matching topology keys is less than minDomains, Pod Topology Spread treats “global minimum” as 0, and then the calculation of Skew is performed. And when the number of eligible domains with matching topology keys equals or greater than minDomains, this value has no effect on scheduling. As a result, when the number of eligible domains is less than minDomains, scheduler won’t schedule more than maxSkew Pods to those domains. If value is nil, the constraint behaves as if MinDomains is equal to 1. Valid values are integers greater than 0. When value is not nil, WhenUnsatisfiable must be DoNotSchedule. For example, in a 3-zone cluster, MaxSkew is set to 2, MinDomains is set to 5 and pods with the same labelSelector spread as 2/2/2:	zone1
nodeAffinityPolicy	string	NodeAffinityPolicy indicates how we will treat Pod’s nodeAffinity/nodeSelector when calculating pod topology spread skew. Options are: - Honor: only nodes matching nodeAffinity/nodeSelector are included in the calculations. - Ignore: nodeAffinity/nodeSelector are ignored. All nodes are included in the calculations. If this value is nil, the behavior is equivalent to the Honor policy.	false
nodeTaintsPolicy	string	NodeTaintsPolicy indicates how we will treat node taints when calculating pod topology spread skew. Options are: - Honor: nodes without taints, along with tainted nodes for which the incoming pod has a toleration, are included. - Ignore: node taints are ignored. All nodes are included. If this value is nil, the behavior is equivalent to the Ignore policy.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.topologySpreadConstraints[index].labelSelector

LabelSelector is used to find matching pods. Pods that match this label selector are counted to determine the number of pods in their corresponding topology domain.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

RuleStatus.status.rules[index].mutate[index].workloads.placement.topologySpreadConstraints[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

RuleStatus.status.rules[index].mutate[index].workloads.registries

Registries configures Pod image pull secrets and selected containers’ pull policies.

Name	Type	Description	Required
imagePullPolicy	enum	ImagePullPolicy sets imagePullPolicy on every selected regular or init container at Pod creation, and newly added ephemeral containers on subresource updates. Both merge and replace overwrite explicit and Kubernetes-defaulted values. Omitted or null preserves the existing policy. Applies to all Pod OSes. Enum: Always, IfNotPresent, Never	false
imagePullSecrets	[]object	ImagePullSecrets sets spec.imagePullSecrets on Pod creation. Merge appends missing names and removes existing duplicates, retaining first-occurrence order. Replace sets the complete list; an explicit empty list clears it. Omitted or null preserves it. Requires the pod target or omitted/empty targets. References always use the Pod’s namespace; Capsule does not create, copy, or check the referenced Secrets.	false

RuleStatus.status.rules[index].mutate[index].workloads.registries.imagePullSecrets[index]

LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.

Name	Type	Description	Required
name	string	Name of the referent. This field is effectively required, but due to backwards compatibility is allowed to be empty. Instances of this type with an empty value here are almost certainly wrong. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names Default:	false

RuleStatus.status.rules[index].mutate[index].workloads.security

Security configures Pod and container security settings.

Name	Type	Description	Required
appArmorProfile	object	AppArmorProfile supplies the Pod-level securityContext.appArmorProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it. AppArmor and any Localhost profile must be available on eligible nodes.	false
hostUsers	boolean	HostUsers sets spec.hostUsers on both merge and replace. False requests a separate user namespace; true uses the host user namespace. Omitted or null leaves the Pod value unchanged. Requires Kubernetes/runtime support.	false
readOnlyRootFilesystem	boolean	ReadOnlyRootFilesystem sets securityContext.readOnlyRootFilesystem on every selected regular or init container at Pod creation, and newly added ephemeral containers on subresource updates. Both merge and replace overwrite the value. False is an explicit setting; nil preserves it. Windows Pods are skipped.	false
seccompProfile	object	SeccompProfile supplies the Pod-level securityContext.seccompProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it.	false

RuleStatus.status.rules[index].mutate[index].workloads.security.appArmorProfile

Name	Type	Description	Required
type	string	type indicates which kind of AppArmor profile will be applied. Valid options are: Localhost - a profile pre-loaded on the node. RuntimeDefault - the container runtime’s default profile. Unconfined - no AppArmor enforcement.	true
localhostProfile	string	localhostProfile indicates a profile loaded on the node that should be used. The profile must be preconfigured on the node to work. Must match the loaded name of the profile. Must be set if and only if type is “Localhost”.	false

RuleStatus.status.rules[index].mutate[index].workloads.security.seccompProfile

Name	Type	Description	Required
type	string	type indicates which kind of seccomp profile will be applied. Valid options are: Localhost - a profile defined in a file on the node should be used. RuntimeDefault - the container runtime default profile should be used. Unconfined - no profile should be applied.	true
localhostProfile	string	localhostProfile indicates a profile defined in a file on the node should be used. The profile must be preconfigured on the node to work. Must be a descending path, relative to the kubelet’s configured seccomp profile location. Must be set if type is “Localhost”. Must NOT be set for any other type.	false

RuleStatus.status.rules[index].quota[index]

Name	Type	Description	Required
name	string	Name is the stable identity of this quota within the Tenant. Changing the name replaces the generated GlobalResourceQuota; changing the quota or its namespace selector updates the existing object.	true
hard	map[string]int or string	hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
scopeSelector	object	scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.	false
scopes	[]string	A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.	false

RuleStatus.status.rules[index].quota[index].scopeSelector

Name	Type	Description	Required
matchExpressions	[]object	A list of scope selector requirements by scope of the resources.	false

RuleStatus.status.rules[index].quota[index].scopeSelector.matchExpressions[index]

A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.

Name	Type	Description	Required
operator	string	Represents a scope’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.	true
scopeName	string	The name of the scope that the selector applies to.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

TenantOwner

TenantOwner is the Schema for the tenantowners API.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	TenantOwner	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	spec defines the desired state of TenantOwner.	true
status	object	status defines the observed state of TenantOwner.	false

TenantOwner.spec

spec defines the desired state of TenantOwner.

Name	Type	Description	Required
kind	enum	Kind of entity. Possible values are “User”, “Group”, and “ServiceAccount” Enum: User, Group, ServiceAccount	true
name	string	Name of the entity.	true
aggregate	boolean	Adds the given subject as capsule user. When enabled this subject does not have to be mentioned in the CapsuleConfiguration as Capsule User. In almost all scenarios Tenant Owners must be Capsule Users. Default: true	false
clusterRoles	[]string	Defines additional cluster-roles for the specific Owner. Default: [admin capsule-namespace-deleter]	false

TenantOwner.status

status defines the observed state of TenantOwner.

Name	Type	Description	Required
conditions	[]object	Conditions contains the reconciliation conditions for this TenantOwner.	false
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false
tenants	[]string	Tenants lists the names of all Tenants that this TenantOwner is currently matched to via the Tenant’s spec.permissions.matchOwners selectors.	false

TenantOwner.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

TenantResource

TenantResource allows a Tenant Owner, if enabled with proper RBAC, to propagate resources in its Namespace. The object must be deployed in a Tenant Namespace, and cannot reference object living in non-Tenant namespaces. For such cases, the GlobalTenantResource must be used.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	TenantResource	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	TenantResourceSpec defines the desired state of TenantResource.	true
status	object	TenantResourceStatus defines the observed state of TenantResource.	false

TenantResource.spec

TenantResourceSpec defines the desired state of TenantResource.

Name	Type	Description	Required
resources	[]object	Defines the rules to select targeting Namespace, along with the objects that must be replicated.	true
resyncPeriod	string	Define the period of time upon a second reconciliation must be invoked. Keep in mind that any change to the manifests will trigger a new reconciliation. Default: 60s	true
settings	object	Deprecated: configure resources[].policy instead. Admission converts Settings into policies for blocks without one. Retained as a compatibility fallback for stored resources that have not passed through admission again. Default: map[]	true
cordoned	boolean	When cordoning a replication it will no longer execute any applies or deletions (paused). This is useful for maintenances Default: false	false
dependsOn	[]object	DependsOn may contain a meta.NamespacedObjectReference slice with references to TenantResource resources that must be ready before this TenantResource can be reconciled.	false
pruningOnDelete	boolean	When the replicated resource manifest is deleted, all the objects replicated so far will be automatically deleted. Disable this to keep replicated resources although the deletion of the replication manifest. Deprecated: use resources[].policy.deletion instead. Retained for blocks without a policy. Default: true	false
serviceAccount	object	Local ServiceAccount which will perform all the actions defined in the TenantResource You must provide permissions accordingly to that ServiceAccount	false

TenantResource.spec.resources[index]

Name	Type	Description	Required
additionalMetadata	object	Besides the Capsule metadata required by TenantResource controller, defines additional metadata that must be added to the replicated resources.	false
context	object	Provide additional template context, which can be used throughout all the declared items for the replication	false
generators	[]object	Templates for advanced use cases	false
namespaceSelector	object	Defines the Namespace selector to select the Tenant Namespaces on which the resources must be propagated. In case of nil value, all the Tenant Namespaces are targeted.	false
namespacedItems	[]object	List of the resources already existing in other Namespaces that must be replicated.	false
policy	object	Policy controls creation, admission protection, SSA conflict ownership, and deletion for every raw item, namespaced item, and generator in this block. An explicit policy replaces the deprecated spec.settings for this block. When omitted, admission converts legacy settings and pruningOnDelete into a policy.	false
rawItems	[]object	List of raw resources that must be replicated.	false

TenantResource.spec.resources[index].additionalMetadata

Besides the Capsule metadata required by TenantResource controller, defines additional metadata that must be added to the replicated resources.

Name	Type	Description	Required
annotations	map[string]string		false
labels	map[string]string		false

TenantResource.spec.resources[index].context

Provide additional template context, which can be used throughout all the declared items for the replication

Name	Type	Description	Required
resources	[]object		false

TenantResource.spec.resources[index].context.resources[index]

Name	Type	Description	Required
kind	string	Kind of the referent. Use “*” to match all kinds.	true
apiVersion	string	API version, API group, or API group/version selector of the referent. Empty APIVersion means the core Kubernetes API version “v1”. Use “” to explicitly match all API groups and versions. Examples: - "" means core “v1”. - “v1” means core “v1”. - “apps” means any version in the “apps” API group. - “apps/v1” means the “apps/v1” API group/version. - “apps/” means any version in the “apps” API group.	false
index	string	Index to mount the resource in the template context	false
name	string	Name of the values referent. This is useful when you traying to get a specific resource	false
namespace	string	Namespace of the values referent.	false
optional	boolean	Only relevant if name is set. If an item is not optional, there will be an error thrown when it does not exist Default: true	false
selector	object	Selector which allows to get any amount of these resources based on labels	false

TenantResource.spec.resources[index].context.resources[index].selector

Selector which allows to get any amount of these resources based on labels

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

TenantResource.spec.resources[index].context.resources[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

TenantResource.spec.resources[index].generators[index]

Name	Type	Description	Required
missingKey	enum	Missing Key Option for templating Enum: invalid, zero, error Default: zero	false
template	string	Template contains any amount of yaml which is applied to Kubernetes. This can be a single resource or multiple resources	false

TenantResource.spec.resources[index].namespaceSelector

Defines the Namespace selector to select the Tenant Namespaces on which the resources must be propagated. In case of nil value, all the Tenant Namespaces are targeted.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

TenantResource.spec.resources[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

TenantResource.spec.resources[index].namespacedItems[index]

Reference

Name	Type	Description	Required
kind	string	Kind of the referent. Use “*” to match all kinds.	true
apiVersion	string	API version, API group, or API group/version selector of the referent. Empty APIVersion means the core Kubernetes API version “v1”. Use “” to explicitly match all API groups and versions. Examples: - "" means core “v1”. - “v1” means core “v1”. - “apps” means any version in the “apps” API group. - “apps/v1” means the “apps/v1” API group/version. - “apps/” means any version in the “apps” API group.	false
name	string	Name of the values referent. This is useful when you traying to get a specific resource	false
namespace	string	Namespace of the values referent.	false
optional	boolean	Only relevant if name is set. If an item is not optional, there will be an error thrown when it does not exist Default: true	false
selector	object	Selector which allows to get any amount of these resources based on labels	false

TenantResource.spec.resources[index].namespacedItems[index].selector

Selector which allows to get any amount of these resources based on labels

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

TenantResource.spec.resources[index].namespacedItems[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

TenantResource.spec.resources[index].policy

Name	Type	Description	Required
condition	string	Condition is an optional CEL boolean expression checked before each target is applied. object is the existing destination object, or null when absent; now is the evaluation timestamp. False skips rendered content without pruning or releasing an already managed object. Policy and protection are still reconciled for managed objects. Evaluation errors prevent writes. Rendering still occurs before evaluation. Omitting Condition preserves the existing apply behavior. Conditions do not affect deletion or orphaning.	false
creation	enum	Creation controls how an existing target is handled. Owner requires the resource to have been created by the applying controller and otherwise returns an error. Merge adopts an existing resource when possible and creates the resource when it does not exist. Enum: Owner, Merge Default: Owner	false
deletion	enum	Deletion controls what happens when the parent stops managing the target. Remove deletes resources created for the parent and relinquishes adopted resources. Orphan keeps the resource and removes Capsule’s lifecycle metadata. Removal is the default. Enum: Remove, Orphan Default: Remove	false
force	boolean	Force allows server-side apply to acquire conflicting field ownership. Default: false	false
protect	boolean	Protect prevents users from changing or deleting the target through admission while it is managed. Protection is enabled by default. Default: true	false

TenantResource.spec.settings

Name	Type	Description	Required
adopt	boolean	Enabling this allows TenanResources to interact with objects which were not created by a TenantResource. In this case on prune no deletion of the entire object is made. Deprecated: use resources[].policy.creation: Merge instead. Default: false	false
force	boolean	Force indicates that in case of conflicts with server-side apply, the client should acquire ownership of the conflicting field. You may create collisions with this. Deprecated: use resources[].policy.force instead. Default: false	false

TenantResource.spec.dependsOn[index]

LocalObjectReference contains enough information to locate the referenced Kubernetes resource object.

Name	Type	Description	Required
name	string	Name of the referent.	true

TenantResource.spec.serviceAccount

Local ServiceAccount which will perform all the actions defined in the TenantResource You must provide permissions accordingly to that ServiceAccount

Name	Type	Description	Required
name	string	Name of the referent.	true

TenantResource.status

TenantResourceStatus defines the observed state of TenantResource.

Name	Type	Description	Required
size	integer	Number of managed resources.	true
conditions	[]object	Condition of the GlobalTenantResource.	false
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false
processedItems	[]object	List of resources managed by the API object.	false
serviceAccount	object	Serviceaccount used for impersonation	false

TenantResource.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

TenantResource.status.processedItems[index]

Advanced Status Item for pin pointing items in tenants/namespaces.

Name	Type	Required
group	string	false
kind	string	false
name	string	false
namespace	string	false
origin	string	false
status	object	false
tenant	string	false
version	string	false

TenantResource.status.processedItems[index].status

Name	Type	Description	Required
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
clusterScoped	boolean	Indicates whether the referenced resource is cluster-scoped.	false
created	boolean	Indicates wether the resource was created or adopted	false
lastApply	string	An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources. Populated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency Format: date-time	false
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	false
policy	object	Policy is a lifecycle snapshot for protection and cleanup. Conditions are evaluated from the source block and omitted here. Failed updates retain the previous snapshot; a first successful content write records its policy even if metadata reconciliation fails, so cleanup respects explicit retention. An absent policy on an already applied item preserves legacy behavior.	false

TenantResource.status.processedItems[index].status.policy

Name	Type	Description	Required
creation	enum	Creation controls how an existing target is handled. Owner requires the resource to have been created by the applying controller and otherwise returns an error. Merge adopts an existing resource when possible and creates the resource when it does not exist. Enum: Owner, Merge Default: Owner	false
deletion	enum	Deletion controls what happens when the parent stops managing the target. Remove deletes resources created for the parent and relinquishes adopted resources. Orphan keeps the resource and removes Capsule’s lifecycle metadata. Removal is the default. Enum: Remove, Orphan Default: Remove	false
force	boolean	Force allows server-side apply to acquire conflicting field ownership. Default: false	false
protect	boolean	Protect prevents users from changing or deleting the target through admission while it is managed. Protection is enabled by default. Default: true	false

TenantResource.status.serviceAccount

Serviceaccount used for impersonation

Name	Type	Description	Required
name	string	Name of the referent.	true
namespace	string	Namespace of the referent.	true

Tenant

Tenant is the Schema for the tenants API.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta2	true
kind	string	Tenant	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	TenantSpec defines the desired state of Tenant.	false
status	object	Returns the observed state of the Tenant.	false

Tenant.spec

TenantSpec defines the desired state of Tenant.

Name	Type	Description	Required
additionalRoleBindings	[]object	Specifies additional RoleBindings assigned to the Tenant. Capsule will ensure that all namespaces in the Tenant always contain the RoleBinding for the given ClusterRole. Optional.	false
containerRegistries	object	Deprecated: Use Enforcement.Registries instead Specifies the trusted Image Registries assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed trusted registries. Optional.	false
cordoned	boolean	Toggling the Tenant resources cordoning, when enable resources cannot be deleted. Default: false	false
data	JSON	Specify additional data relating to the tenant. Mainly useable in templating and more accessible than labels/annotations.	false
deviceClasses	object	Specifies options for the DeviceClass resources.	false
forceTenantPrefix	boolean	Use this if you want to disable/enable the Tenant name prefix to specific Tenants, overriding global forceTenantPrefix in CapsuleConfiguration. When set to ’true’, it enforces Namespaces created for this Tenant to be named with the Tenant name prefix, separated by a dash (i.e. for Tenant ‘foo’, namespace names must be prefixed with ‘foo-’), this is useful to avoid Namespace name collision. When set to ‘false’, it allows Namespaces created for this Tenant to be named anything. Overrides CapsuleConfiguration global forceTenantPrefix for the Tenant only. If unset, Tenant uses CapsuleConfiguration’s forceTenantPrefix Optional	false
gatewayOptions	object	Specifies options for the GatewayClass resources.	false
imagePullPolicies	[]enum	Deprecated: Use Enforcement.Registries instead Specify the allowed values for the imagePullPolicies option in Pod resources. Capsule assures that all Pod resources created in the Tenant can use only one of the allowed policy. Optional. Enum: Always, Never, IfNotPresent	false
ingressOptions	object	Specifies options for the Ingress resources, such as allowed hostnames and IngressClass. Optional.	false
limitRanges	object	Deprecated: Use Tenant Replications instead (https://projectcapsule.dev/docs/replications/) Specifies the resource min/max usage restrictions to the Tenant. The assigned values are inherited by any namespace created in the Tenant. Optional.	false
namespaceOptions	object	Specifies options for the Namespaces, such as additional metadata or maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional.	false
networkPolicies	object	Deprecated: Use Tenant Replications instead (https://projectcapsule.dev/docs/replications/) Specifies the NetworkPolicies assigned to the Tenant. The assigned NetworkPolicies are inherited by any namespace created in the Tenant. Optional.	false
nodeSelector	map[string]string	Specifies the label to control the placement of pods on a given pool of worker nodes. All namespaces created within the Tenant will have the node selector annotation. This annotation tells the Kubernetes scheduler to place pods on the nodes having the selector label. Optional.	false
owners	[]object	Specifies the owners of the Tenant. Optional	false
permissions	object	Specify Permissions for the Tenant.	false
podOptions	object	Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/) Specifies options for the Pods deployed in the Tenant namespaces, such as additional metadata.	false
preventDeletion	boolean	Prevent accidental deletion of the Tenant. When enabled, the deletion request will be declined. Default: false	false
priorityClasses	object	Specifies the allowed priorityClasses assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed PriorityClasses. A default value can be specified, and all the Pod resources created will inherit the declared class. Optional.	false
resourceQuotas	object	Deprecated: Use Rules Quota (https://projectcapsule.dev/docs/tenants/rules/#quotas) Specifies a list of ResourceQuota resources assigned to the Tenant. The assigned values are inherited by any namespace created in the Tenant. The Capsule operator aggregates ResourceQuota at Tenant level, so that the hard quota is never crossed for the given Tenant. This permits the Tenant owner to consume resources in the Tenant regardless of the namespace. Optional.	false
rules	[]object	Specify enforcement specifications for the scope of the Tenant. We are moving all configuration enforcement. per namespace into a rule construct. It’s currently not final. Read More: https://projectcapsule.dev/docs/tenants/rules/	false
runtimeClasses	object	Specifies the allowed RuntimeClasses assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed RuntimeClasses. Optional.	false
serviceOptions	object	Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/) Specifies options for the Service, such as additional metadata or block of certain type of Services. Optional.	false
storageClasses	object	Specifies the allowed StorageClasses assigned to the Tenant. Capsule assures that all PersistentVolumeClaim resources created in the Tenant can use only one of the allowed StorageClasses. A default value can be specified, and all the PersistentVolumeClaim resources created will inherit the declared class. Optional.	false

Tenant.spec.additionalRoleBindings[index]

Name	Type	Description	Required
clusterRoleName	string		true
subjects	[]object		true
annotations	map[string]string	Additional Annotations for the synchronized rolebindings	false
labels	map[string]string	Additional Labels for the synchronized rolebindings	false

Tenant.spec.additionalRoleBindings[index].subjects[index]

Subject contains a reference to the object or user identities a role binding applies to. This can either hold a direct API object reference, or a value for non-objects such as user and group names.

Name	Type	Description	Required
kind	string	Kind of object being referenced. Values defined by this API group are “User”, “Group”, and “ServiceAccount”. If the Authorizer does not recognized the kind value, the Authorizer should report an error.	true
name	string	Name of the object being referenced.	true
apiGroup	string	APIGroup holds the API group of the referenced subject. Defaults to "" for ServiceAccount subjects. Defaults to “rbac.authorization.k8s.io” for User and Group subjects.	false
namespace	string	Namespace of the referenced object. If the object kind is non-namespace, such as “User” or “Group”, and this value is not empty the Authorizer should report an error.	false

Tenant.spec.containerRegistries

Deprecated: Use Enforcement.Registries instead

Specifies the trusted Image Registries assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed trusted registries. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false

Tenant.spec.deviceClasses

Specifies options for the DeviceClass resources.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.deviceClasses.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.gatewayOptions

Specifies options for the GatewayClass resources.

Name	Type	Description	Required
allowedClasses	object		false

Tenant.spec.gatewayOptions.allowedClasses

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false
default	string		false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.gatewayOptions.allowedClasses.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.ingressOptions

Specifies options for the Ingress resources, such as allowed hostnames and IngressClass. Optional.

Name	Type	Description	Required
allowWildcardHostnames	boolean	Toggles the ability for Ingress resources created in a Tenant to have a hostname wildcard.	false
allowedClasses	object	Specifies the allowed IngressClasses assigned to the Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed IngressClasses. A default value can be specified, and all the Ingress resources created will inherit the declared class. Optional.	false
allowedHostnames	object	Specifies the allowed hostnames in Ingresses for the given Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed hostnames. Optional.	false
hostnameCollisionScope	enum	Defines the scope of hostname collision check performed when Tenant Owners create Ingress with allowed hostnames. - Cluster: disallow the creation of an Ingress if the pair hostname and path is already used across the Namespaces managed by Capsule. - Tenant: disallow the creation of an Ingress if the pair hostname and path is already used across the Namespaces of the Tenant. - Namespace: disallow the creation of an Ingress if the pair hostname and path is already used in the Ingress Namespace. Optional. Enum: Cluster, Tenant, Namespace, Disabled Default: Disabled	false

Tenant.spec.ingressOptions.allowedClasses

Specifies the allowed IngressClasses assigned to the Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed IngressClasses. A default value can be specified, and all the Ingress resources created will inherit the declared class. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false
default	string		false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.ingressOptions.allowedClasses.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.ingressOptions.allowedHostnames

Specifies the allowed hostnames in Ingresses for the given Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed hostnames. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false

Tenant.spec.limitRanges

Deprecated: Use Tenant Replications instead (https://projectcapsule.dev/docs/replications/)

Specifies the resource min/max usage restrictions to the Tenant. The assigned values are inherited by any namespace created in the Tenant. Optional.

Name	Type	Description	Required
items	[]object		false

Tenant.spec.limitRanges.items[index]

LimitRangeSpec defines a min/max usage limit for resources that match on kind.

Name	Type	Description	Required
limits	[]object	Limits is the list of LimitRangeItem objects that are enforced.	true

Tenant.spec.limitRanges.items[index].limits[index]

LimitRangeItem defines a min/max usage limit for any resource that matches on kind.

Name	Type	Description	Required
type	string	Type of resource that this limit applies to.	true
default	map[string]int or string	Default resource requirement limit value by resource name if resource limit is omitted.	false
defaultRequest	map[string]int or string	DefaultRequest is the default resource requirement request value by resource name if resource request is omitted.	false
max	map[string]int or string	Max usage constraints on this kind by resource name.	false
maxLimitRequestRatio	map[string]int or string	MaxLimitRequestRatio if specified, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value; this represents the max burst for the named resource.	false
min	map[string]int or string	Min usage constraints on this kind by resource name.	false

Tenant.spec.namespaceOptions

Specifies options for the Namespaces, such as additional metadata or maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional.

Name	Type	Description	Required
additionalMetadata	object	Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/#namespace) Specifies additional labels and annotations the Capsule operator places on any Namespace resource in the Tenant. Optional.	false
additionalMetadataList	[]object	Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/#namespace) Specifies additional labels and annotations the Capsule operator places on any Namespace resource in the Tenant via a list. Optional.	false
forbiddenAnnotations	object	Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/#namespace) Define the annotations that a Tenant Owner cannot set for their Namespace resources.	false
forbiddenLabels	object	Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/#namespace) Define the labels that a Tenant Owner cannot set for their Namespace resources.	false
managedMetadataOnly	boolean	If enabled only metadata from additionalMetadata is reconciled to the namespaces. Default: false	false
quota	integer	Specifies the maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional. Format: int32 Minimum: 1	false
requiredMetadata	object	Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/#namespace) Required Metadata for namespace within this tenant	false

Tenant.spec.namespaceOptions.additionalMetadata

Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/#namespace)

Specifies additional labels and annotations the Capsule operator places on any Namespace resource in the Tenant. Optional.

Name	Type	Description	Required
annotations	map[string]string		false
labels	map[string]string		false

Tenant.spec.namespaceOptions.additionalMetadataList[index]

Name	Type	Description	Required
annotations	map[string]string		false
labels	map[string]string		false
namespaceSelector	object	A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.	false

Tenant.spec.namespaceOptions.additionalMetadataList[index].namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.namespaceOptions.additionalMetadataList[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.namespaceOptions.forbiddenAnnotations

Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/#namespace)

Define the annotations that a Tenant Owner cannot set for their Namespace resources.

Name	Type	Description	Required
denied	[]string		false
deniedRegex	string		false

Tenant.spec.namespaceOptions.forbiddenLabels

Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/#namespace)

Define the labels that a Tenant Owner cannot set for their Namespace resources.

Name	Type	Description	Required
denied	[]string		false
deniedRegex	string		false

Tenant.spec.namespaceOptions.requiredMetadata

Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/#namespace)

Required Metadata for namespace within this tenant

Name	Type	Description	Required
annotations	map[string]string	Annotations that must be defined for each namespace	false
labels	map[string]string	Labels that must be defined for each namespace	false

Tenant.spec.networkPolicies

Deprecated: Use Tenant Replications instead (https://projectcapsule.dev/docs/replications/)

Specifies the NetworkPolicies assigned to the Tenant. The assigned NetworkPolicies are inherited by any namespace created in the Tenant. Optional.

Name	Type	Description	Required
items	[]object		false

Tenant.spec.networkPolicies.items[index]

NetworkPolicySpec provides the specification of a NetworkPolicy

Name	Type	Description	Required
egress	[]object	egress is a list of egress rules to be applied to the selected pods. Outgoing traffic is allowed if there are no NetworkPolicies selecting the pod (and cluster policy otherwise allows the traffic), OR if the traffic matches at least one egress rule across all of the NetworkPolicy objects whose podSelector matches the pod. If this field is empty then this NetworkPolicy limits all outgoing traffic (and serves solely to ensure that the pods it selects are isolated by default). This field is beta-level in 1.8	false
ingress	[]object	ingress is a list of ingress rules to be applied to the selected pods. Traffic is allowed to a pod if there are no NetworkPolicies selecting the pod (and cluster policy otherwise allows the traffic), OR if the traffic source is the pod’s local node, OR if the traffic matches at least one ingress rule across all of the NetworkPolicy objects whose podSelector matches the pod. If this field is empty then this NetworkPolicy does not allow any traffic (and serves solely to ensure that the pods it selects are isolated by default)	false
podSelector	object	podSelector selects the pods to which this NetworkPolicy object applies. The array of rules is applied to any pods selected by this field. An empty selector matches all pods in the policy’s namespace. Multiple network policies can select the same set of pods. In this case, the ingress rules for each are combined additively. This field is optional. If it is not specified, it defaults to an empty selector.	false
policyTypes	[]string	policyTypes is a list of rule types that the NetworkPolicy relates to. Valid options are [“Ingress”], [“Egress”], or [“Ingress”, “Egress”]. If this field is not specified, it will default based on the existence of ingress or egress rules; policies that contain an egress section are assumed to affect egress, and all policies (whether or not they contain an ingress section) are assumed to affect ingress. If you want to write an egress-only policy, you must explicitly specify policyTypes [ “Egress” ]. Likewise, if you want to write a policy that specifies that no egress is allowed, you must specify a policyTypes value that include “Egress” (since such a policy would not include an egress section and would otherwise default to just [ “Ingress” ]). This field is beta-level in 1.8	false

Tenant.spec.networkPolicies.items[index].egress[index]

NetworkPolicyEgressRule describes a particular set of traffic that is allowed out of pods matched by a NetworkPolicySpec’s podSelector. The traffic must match both ports and to. This type is beta-level in 1.8

Name	Type	Description	Required
ports	[]object	ports is a list of destination ports for outgoing traffic. Each item in this list is combined using a logical OR. If this field is empty or missing, this rule matches all ports (traffic not restricted by port). If this field is present and contains at least one item, then this rule allows traffic only if the traffic matches at least one port in the list.	false
to	[]object	to is a list of destinations for outgoing traffic of pods selected for this rule. Items in this list are combined using a logical OR operation. If this field is empty or missing, this rule matches all destinations (traffic not restricted by destination). If this field is present and contains at least one item, this rule allows traffic only if the traffic matches at least one item in the to list.	false

Tenant.spec.networkPolicies.items[index].egress[index].ports[index]

NetworkPolicyPort describes a port to allow traffic on

Name	Type	Description	Required
endPort	integer	endPort indicates that the range of ports from port to endPort if set, inclusive, should be allowed by the policy. This field cannot be defined if the port field is not defined or if the port field is defined as a named (string) port. The endPort must be equal or greater than port. Format: int32	false
port	int or string	port represents the port on the given protocol. This can either be a numerical or named port on a pod. If this field is not provided, this matches all port names and numbers. If present, only traffic on the specified protocol AND port will be matched.	false
protocol	string	protocol represents the protocol (TCP, UDP, or SCTP) which traffic must match. If not specified, this field defaults to TCP.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index]

NetworkPolicyPeer describes a peer to allow traffic to/from. Only certain combinations of fields are allowed

Name	Type	Description	Required
ipBlock	object	ipBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.	false
namespaceSelector	object	namespaceSelector selects namespaces using cluster-scoped labels. This field follows standard label selector semantics; if present but empty, it selects all namespaces. If podSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the namespaces selected by namespaceSelector. Otherwise it selects all pods in the namespaces selected by namespaceSelector.	false
podSelector	object	podSelector is a label selector which selects pods. This field follows standard label selector semantics; if present but empty, it selects all pods. If namespaceSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the Namespaces selected by NamespaceSelector. Otherwise it selects the pods matching podSelector in the policy’s own namespace.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].ipBlock

ipBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.

Name	Type	Description	Required
cidr	string	cidr is a string representing the IPBlock Valid examples are “192.168.1.0/24” or “2001:db8::/64”	true
except	[]string	except is a slice of CIDRs that should not be included within an IPBlock Valid examples are “192.168.1.0/24” or “2001:db8::/64” Except values will be rejected if they are outside the cidr range	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].namespaceSelector

namespaceSelector selects namespaces using cluster-scoped labels. This field follows standard label selector semantics; if present but empty, it selects all namespaces.

If podSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the namespaces selected by namespaceSelector. Otherwise it selects all pods in the namespaces selected by namespaceSelector.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].podSelector

podSelector is a label selector which selects pods. This field follows standard label selector semantics; if present but empty, it selects all pods.

If namespaceSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the Namespaces selected by NamespaceSelector. Otherwise it selects the pods matching podSelector in the policy’s own namespace.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].podSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.networkPolicies.items[index].ingress[index]

NetworkPolicyIngressRule describes a particular set of traffic that is allowed to the pods matched by a NetworkPolicySpec’s podSelector. The traffic must match both ports and from.

Name	Type	Description	Required
from	[]object	from is a list of sources which should be able to access the pods selected for this rule. Items in this list are combined using a logical OR operation. If this field is empty or missing, this rule matches all sources (traffic not restricted by source). If this field is present and contains at least one item, this rule allows traffic only if the traffic matches at least one item in the from list.	false
ports	[]object	ports is a list of ports which should be made accessible on the pods selected for this rule. Each item in this list is combined using a logical OR. If this field is empty or missing, this rule matches all ports (traffic not restricted by port). If this field is present and contains at least one item, then this rule allows traffic only if the traffic matches at least one port in the list.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index]

NetworkPolicyPeer describes a peer to allow traffic to/from. Only certain combinations of fields are allowed

Name	Type	Description	Required
ipBlock	object	ipBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.	false
namespaceSelector	object	namespaceSelector selects namespaces using cluster-scoped labels. This field follows standard label selector semantics; if present but empty, it selects all namespaces. If podSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the namespaces selected by namespaceSelector. Otherwise it selects all pods in the namespaces selected by namespaceSelector.	false
podSelector	object	podSelector is a label selector which selects pods. This field follows standard label selector semantics; if present but empty, it selects all pods. If namespaceSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the Namespaces selected by NamespaceSelector. Otherwise it selects the pods matching podSelector in the policy’s own namespace.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].ipBlock

ipBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.

Name	Type	Description	Required
cidr	string	cidr is a string representing the IPBlock Valid examples are “192.168.1.0/24” or “2001:db8::/64”	true
except	[]string	except is a slice of CIDRs that should not be included within an IPBlock Valid examples are “192.168.1.0/24” or “2001:db8::/64” Except values will be rejected if they are outside the cidr range	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].namespaceSelector

namespaceSelector selects namespaces using cluster-scoped labels. This field follows standard label selector semantics; if present but empty, it selects all namespaces.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].podSelector

podSelector is a label selector which selects pods. This field follows standard label selector semantics; if present but empty, it selects all pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].podSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.networkPolicies.items[index].ingress[index].ports[index]

NetworkPolicyPort describes a port to allow traffic on

Name	Type	Description	Required
endPort	integer	endPort indicates that the range of ports from port to endPort if set, inclusive, should be allowed by the policy. This field cannot be defined if the port field is not defined or if the port field is defined as a named (string) port. The endPort must be equal or greater than port. Format: int32	false
port	int or string	port represents the port on the given protocol. This can either be a numerical or named port on a pod. If this field is not provided, this matches all port names and numbers. If present, only traffic on the specified protocol AND port will be matched.	false
protocol	string	protocol represents the protocol (TCP, UDP, or SCTP) which traffic must match. If not specified, this field defaults to TCP.	false

Tenant.spec.networkPolicies.items[index].podSelector

podSelector selects the pods to which this NetworkPolicy object applies. The array of rules is applied to any pods selected by this field. An empty selector matches all pods in the policy’s namespace. Multiple network policies can select the same set of pods. In this case, the ingress rules for each are combined additively. This field is optional. If it is not specified, it defaults to an empty selector.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].podSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.owners[index]

Name	Type	Description	Required
kind	enum	Kind of entity. Possible values are “User”, “Group”, and “ServiceAccount” Enum: User, Group, ServiceAccount	true
name	string	Name of the entity.	true
annotations	map[string]string	Additional Annotations for the synchronized rolebindings	false
clusterRoles	[]string	Defines additional cluster-roles for the specific Owner. Default: [admin capsule-namespace-deleter]	false
labels	map[string]string	Additional Labels for the synchronized rolebindings	false
proxySettings	[]object	Proxy settings for tenant owner.	false

Tenant.spec.owners[index].proxySettings[index]

Name	Type	Description	Required
kind	enum	Enum: Nodes, StorageClasses, IngressClasses, PriorityClasses, RuntimeClasses, PersistentVolumes	true
operations	[]enum	Enum: List, Update, Delete	true

Tenant.spec.permissions

Specify Permissions for the Tenant.

Name	Type	Description	Required
allowOwnerPromotion	boolean	ClusterRoles granted to the promoted ServiceAccounts across the Tenant Default: true	false
matchOwners	[]object	Matches TenantOwner objects which are promoted to owners of this tenant The elements are OR operations and independent. You can see the resulting Tenant Owners in the Status.Owners specification of the Tenant.	false

Tenant.spec.permissions.matchOwners[index]

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.permissions.matchOwners[index].matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.podOptions

Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/)

Specifies options for the Pods deployed in the Tenant namespaces, such as additional metadata.

Name	Type	Description	Required
additionalMetadata	object	Specifies additional labels and annotations the Capsule operator places on any Pod resource in the Tenant. Optional.	false

Tenant.spec.podOptions.additionalMetadata

Specifies additional labels and annotations the Capsule operator places on any Pod resource in the Tenant. Optional.

Name	Type	Description	Required
annotations	map[string]string		false
labels	map[string]string		false

Tenant.spec.priorityClasses

Specifies the allowed priorityClasses assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed PriorityClasses. A default value can be specified, and all the Pod resources created will inherit the declared class. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false
default	string		false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.priorityClasses.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.resourceQuotas

Deprecated: Use Rules Quota (https://projectcapsule.dev/docs/tenants/rules/#quotas)

Specifies a list of ResourceQuota resources assigned to the Tenant. The assigned values are inherited by any namespace created in the Tenant. The Capsule operator aggregates ResourceQuota at Tenant level, so that the hard quota is never crossed for the given Tenant. This permits the Tenant owner to consume resources in the Tenant regardless of the namespace. Optional.

Name	Type	Description	Required
items	[]object		false
scope	enum	Define if the Resource Budget should compute resource across all Namespaces in the Tenant or individually per cluster. Default is Tenant Enum: Tenant, Namespace Default: Tenant	false

Tenant.spec.resourceQuotas.items[index]

ResourceQuotaSpec defines the desired hard limits to enforce for Quota.

Name	Type	Description	Required
hard	map[string]int or string	hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
scopeSelector	object	scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.	false
scopes	[]string	A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.	false

Tenant.spec.resourceQuotas.items[index].scopeSelector

Name	Type	Description	Required
matchExpressions	[]object	A list of scope selector requirements by scope of the resources.	false

Tenant.spec.resourceQuotas.items[index].scopeSelector.matchExpressions[index]

A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.

Name	Type	Description	Required
operator	string	Represents a scope’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.	true
scopeName	string	The name of the scope that the selector applies to.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index]

Rules Distributed via Tenants

Name	Type	Description	Required
audience	[]object	Audience limits this rule to matching request subjects. An empty audience matches every request.	false
enforce	object	Enforcement for given rule	false
mutate	[]object	Mutate applies ordered typed mutations before enforcement. It shares the rule’s namespace selection and audience. Workload mutations apply on Pod creation.	false
namespaceSelector	object	Select namespaces which are going to be targeted with this rule	false
permissions	object	Permissions for given rule	false
quota	[]object	Quota contains native Kubernetes ResourceQuota specifications shared by all namespaces selected by this rule. Unlike Enforce, quota accounting is independent of the request audience.	false

Tenant.spec.rules[index].audience[index]

Name	Type	Description	Required
kind	enum	Enum: User, Group, ServiceAccount, Custom	true
name	string		true

Tenant.spec.rules[index].enforce

Enforcement for given rule

Name	Type	Description	Required
action	enum	Declare the action being performed on the enforcement rule: deny: On match, deny admission request allow: On match, allowed admission request audit: On match, audit (post event) of admission request Enum: allow, deny, audit Default: deny	false
conditions	[]object	Conditions gate this entire enforcement rule, including workloads, services, metadata, ingress, network and storage. All conditions must be true; empty means apply. Conditions run during admission; managed metadata in a conditional rule is applied on matching requests only and is not reconciled in the background.	false
ingress	object	Enforcement for Ingress and Gateway API resource hostnames.	false
metadata	[]object	Enforcement for object metadata on namespaced resources.	false
network	object	Network configures network enforcement for namespace resources.	false
services	object	Enforcement for Services.	false
storage	object	Additional access to PersistentVolumes without a tenant ownership label.	false
workloads	object	Enforcement for native workload kinds and Pod properties.	false

Tenant.spec.rules[index].enforce.conditions[index]

Name	Type	Description	Required
expression	string	Expression must evaluate to bool. Any false condition skips this block. An evaluation error rejects the request unless another condition is false.	true
name	string	Name identifies a condition in admission errors. Names must be unique within a block.	false

Tenant.spec.rules[index].enforce.ingress

Enforcement for Ingress and Gateway API resource hostnames.

Name	Type	Description	Required
hostnames	[]object	Hostnames defines allowed, denied, or audited hostname expressions. A resource targeted by an allow or deny rule must declare non-empty values in all hostname fields. Audit-only rules record missing hostnames without denying them.	false
types	[]enum	Types defines the resource kinds to which hostname enforcement applies. Enum: Ingress, Route, ListenerSet, HTTPRoute, Gateway, TLSRoute, GRPCRoute	false

Tenant.spec.rules[index].enforce.ingress.hostnames[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.metadata[index]

MetadataRule defines metadata constraints for namespaced resources.

Name	Type	Description	Required
kinds	[]string	Kinds of the referents. Use “*” to match all kinds.	true
annotations	map[string]object	Annotations defines metadata policies by annotation key.	false
apiGroups	[]string	API groups or API group/version selectors of the referents. Empty or omitted APIGroups means the core Kubernetes API version “v1”. Use “” to match all API groups and versions. Examples: - [] or [""] means core “v1”. - [“v1”] means core “v1”. - [“apps”] means any version in the “apps” API group. - [“apps/v1”] means only “apps/v1”. - [“apps”, “batch/v1”] means any “apps” version and “batch/v1”. - [""] means all API groups and versions.	false
labels	map[string]object	Labels defines metadata policies by label key.	false

Tenant.spec.rules[index].enforce.metadata[index].annotations[key]

Name	Type	Description	Required
default	string	Default is applied by admission mutation when the concrete metadata key is absent. It is not reconciled after admission.	false
managed	string	Managed is enforced by admission mutation and reconciled by the RuleStatus controller using server-side apply when the rule configuration changes. Present metadata matching the effective managed value is exempt from metadata value validation, including overlapping deny rules. The last applicable managed value for each concrete key wins. During admission, only rules matching the request’s audience contribute to the effective managed value.	false
required	boolean	Required enforces that the metadata key must be present. This is only meaningful with action=allow. Deny and audit rules do not require missing metadata to exist. Default: false	false
values	[]object	Values defines allowed, denied, or audited values for the metadata key. With action=allow and no values, only Required is enforced. With action=deny or action=audit and no values, any present value matches, including an empty string.	false

Tenant.spec.rules[index].enforce.metadata[index].annotations[key].values[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.metadata[index].labels[key]

Name	Type	Description	Required
default	string	Default is applied by admission mutation when the concrete metadata key is absent. It is not reconciled after admission.	false
managed	string	Managed is enforced by admission mutation and reconciled by the RuleStatus controller using server-side apply when the rule configuration changes. Present metadata matching the effective managed value is exempt from metadata value validation, including overlapping deny rules. The last applicable managed value for each concrete key wins. During admission, only rules matching the request’s audience contribute to the effective managed value.	false
required	boolean	Required enforces that the metadata key must be present. This is only meaningful with action=allow. Deny and audit rules do not require missing metadata to exist. Default: false	false
values	[]object	Values defines allowed, denied, or audited values for the metadata key. With action=allow and no values, only Required is enforced. With action=deny or action=audit and no values, any present value matches, including an empty string.	false

Tenant.spec.rules[index].enforce.metadata[index].labels[key].values[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.network

Network configures network enforcement for namespace resources.

Name	Type	Description	Required
policies	object	Policies constrains native NetworkPolicies.	false

Tenant.spec.rules[index].enforce.network.policies

Policies constrains native NetworkPolicies.

Name	Type	Description	Required
egress	object	Egress constrains destination address grants. Selector-based peers are unaffected. No constraints are applied when omitted.	false
ingress	object	Ingress constrains source address grants. Selector-based peers are unaffected. No constraints are applied when omitted.	false

Tenant.spec.rules[index].enforce.network.policies.egress

Egress constrains destination address grants. Selector-based peers are unaffected. No constraints are applied when omitted.

Name	Type	Description	Required
cidrs	[]string	CIDRs matches addresses granted by ipBlock.cidr after subtracting except. Omitted peers on an ingress or egress rule grant both IPv4 and IPv6 address space. An empty NetworkPolicy ingress or egress list has no grants in that direction to validate. Each direction is evaluated independently. An empty CIDRs list imposes no constraint. Ordered allow/deny decisions apply to every granted address: deny rejects overlapping grants, allow requires complete coverage, and audit reports overlaps. A later allow cannot authorize addresses outside its CIDRs.	false

Tenant.spec.rules[index].enforce.network.policies.ingress

Ingress constrains source address grants. Selector-based peers are unaffected. No constraints are applied when omitted.

Name	Type	Description	Required
cidrs	[]string	CIDRs matches addresses granted by ipBlock.cidr after subtracting except. Omitted peers on an ingress or egress rule grant both IPv4 and IPv6 address space. An empty NetworkPolicy ingress or egress list has no grants in that direction to validate. Each direction is evaluated independently. An empty CIDRs list imposes no constraint. Ordered allow/deny decisions apply to every granted address: deny rejects overlapping grants, allow requires complete coverage, and audit reports overlaps. A later allow cannot authorize addresses outside its CIDRs.	false

Tenant.spec.rules[index].enforce.services

Enforcement for Services.

Name	Type	Description	Required
externalIPs	object	ExternalIPs defines constraints for spec.externalIPs.	false
externalNames	object	ExternalNames defines additional constraints for Services of type ExternalName.	false
loadBalancers	object	LoadBalancers defines additional constraints for Services of type LoadBalancer.	false
nodePorts	object	NodePorts defines additional constraints for nodePort values.	false
types	[]enum	Types defines the Service types matched by this rule. Supported values: - ClusterIP - NodePort - LoadBalancer - ExternalName Enum: ClusterIP, NodePort, LoadBalancer, ExternalName	false

Tenant.spec.rules[index].enforce.services.externalIPs

ExternalIPs defines constraints for spec.externalIPs.

Name	Type	Description	Required
cidrs	[]string	CIDRs restricts spec.externalIPs. Individual IP addresses are treated as host CIDRs (/32 for IPv4 and /128 for IPv6). For deny rules, empty means all external IPs are denied. For allow and audit rules, empty means no external IP restriction.	false

Tenant.spec.rules[index].enforce.services.externalNames

ExternalNames defines additional constraints for Services of type ExternalName.

Name	Type	Description	Required
hostnames	[]object	Hostnames restricts spec.externalName. Empty means no additional hostname restriction once ExternalName is allowed by types.	false

Tenant.spec.rules[index].enforce.services.externalNames.hostnames[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.services.loadBalancers

LoadBalancers defines additional constraints for Services of type LoadBalancer.

Name	Type	Description	Required
cidrs	[]string	CIDRs restricts spec.loadBalancerIP and spec.loadBalancerSourceRanges. Empty means no additional CIDR restriction once LoadBalancer is allowed by types.	false

Tenant.spec.rules[index].enforce.services.nodePorts

NodePorts defines additional constraints for nodePort values.

Name	Type	Description	Required
ports	[]object	Ports restricts explicitly requested nodePort values. Empty means no additional port restriction once NodePort is allowed by types.	false

Tenant.spec.rules[index].enforce.services.nodePorts.ports[index]

Name	Type	Description	Required
from	integer	Format: int32 Minimum: 1 Maximum: 65535	true
to	integer	Format: int32 Minimum: 1 Maximum: 65535	true

Tenant.spec.rules[index].enforce.storage

Additional access to PersistentVolumes without a tenant ownership label.

Name	Type	Description	Required
volumes	[]object	Volumes matches additional PersistentVolumes. Entries are ORed. Conditions on the enclosing enforcement rule must also match. The last matching allow/deny decides the exception; audit never grants access.	false

Tenant.spec.rules[index].enforce.storage.volumes[index]

PersistentVolumeMatch matches labels on the referenced PersistentVolume.

Name	Type	Description	Required
selector	object	Selector matches existing PV labels, independently of PVC.spec.selector. An explicit empty selector matches all PVs. Capsule’s tenant ownership label cannot be overridden by a selector or an allow decision.	true
name	string	Name identifies this match in admission and audit messages.	false

Tenant.spec.rules[index].enforce.storage.volumes[index].selector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].enforce.storage.volumes[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].enforce.workloads

Enforcement for native workload kinds and Pod properties.

Name	Type	Description	Required
disruptionBudgets	object	DisruptionBudgets constrains PDB coverage of selected Pods and controller templates. Only whole-workload targets apply; omitted targets select Pods.	false
placement	object	Placement matches the Pod scheduler and scheduling constraints.	false
qosClasses	[]string	Define Pod QoS classes matched by this enforcement rule. Supported values are Guaranteed, Burstable and BestEffort.	false
registries	[]object	Define registries which are allowed to be used within this tenant The rules are aggregated, since you can use Regular Expressions the match registry endpoints	false
resources	object	Resources defines mutation and enforcement policies for Pod and container resource requests and limits. The workload targets select where the policies apply. With no targets, resource policies apply to all compatible locations: Pod-level resources, regular containers, and init containers. Resource names unsupported at Pod level still apply to compatible container locations. Mutation is applied when a Pod is created, or when an explicitly targeted controller is created or updated (to its Pod template). Remove and MatchRequest manage explicit values, Default fills an absent value, and Ratio fills an absent limit from its request. An explicit Ratio violation is then handled by the enclosing allow, deny, or audit action.	false
schedulers	[]object	Schedulers defines additional schedulerName matchers for selected Pods and Pod templates. These are combined with placement.schedulers under the same rule action; a match in either field counts as a match. Deprecated: use Placement.Schedulers instead.	false
security	object	Security matches effective Pod and container security profiles.	false
targets	[]string	Targets selects native workloads and, optionally, parts of their Pod specs. With no workload policies, the action matches the selected kinds themselves. With policies, targets scopes those policies; it does not also match the kind. Omitted targets preserve Pod-only defaults. Controller targets are opt-in. Whole-controller targets include every compatible location in the Pod template. Existing pod and pod/* part targets retain their established policy scope.	false

Tenant.spec.rules[index].enforce.workloads.disruptionBudgets

DisruptionBudgets constrains PDB coverage of selected Pods and controller templates. Only whole-workload targets apply; omitted targets select Pods.

Name	Type	Description	Required
allowOverlap	boolean	AllowOverlap permits multiple PDBs to select the same Pod or selected controller template. False rejects overlap for allow/deny actions and reports violations for audit. True permits overlap in an allow rule; omitted does not participate in evaluation. Rule ordering is preserved. Checks run on PDB selector changes and selected Pod/template label changes. Conditions and audience apply to the admission request, including PDB writes. Cross-resource admission reads are not an atomic transaction.	false
evictableReplicas	object	EvictableReplicas bounds the configured number of evictable replicas of each selected Deployment, StatefulSet, ReplicaSet or ReplicationController. Uses desired spec.replicas and Kubernetes’ upward percentage rounding; zero replicas are exempt. Checks controller writes, /scale, and PDB writes. Bounds are inclusive nonnegative integers; omitted bounds are unrestricted. Allow requires compliance; deny/audit match violations, like resource constraints. Other targets do not have a desired replica count and skip this property. At least one supported controller target must be explicitly selected.	false
unhealthyPodEvictionPolicies	[]enum	UnhealthyPodEvictionPolicies matches the effective PDB policy. Omission in a PDB means IfHealthyBudget. Allow accepts listed values; deny/audit match listed values, like other enum lists. Empty imposes no restriction. Enum: IfHealthyBudget, AlwaysAllow	false

Tenant.spec.rules[index].enforce.workloads.disruptionBudgets.evictableReplicas

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

Tenant.spec.rules[index].enforce.workloads.placement

Placement matches the Pod scheduler and scheduling constraints.

Name	Type	Description	Required
affinity	[]object	Affinity matches each complete required or preferred affinity term.	false
nodeSelector	[]object	NodeSelector matches each node selector entry. Empty matchers match any entry. Placement rules apply to the pod target; allowing an entry does not require it.	false
schedulers	[]object	Schedulers defines schedulerName matchers for selected Pods and Pod templates. The rule is evaluated against pod.spec.schedulerName. Empty schedulerName is ignored and is not normalized to default-scheduler.	false
tolerations	[]object	Tolerations matches each toleration, including injected and wildcard tolerations.	false
topologySpreadConstraints	[]object	TopologySpreadConstraints matches each complete spread constraint.	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index]

WorkloadAffinityMatch matches an entire affinity term. Fields are ANDed; entries in enforce.workloads.placement.affinity are alternatives. {} matches any term.

Name	Type	Description	Required
fieldRequirements	[]object	FieldRequirements constrains node matchFields. When Requirements is set, matchFields must be explicitly permitted here to avoid an unchecked path.	false
labelSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
modes	[]enum	Modes defaults to both scheduling modes; it does not require their presence. Enum: required, preferred	false
namespaceScope	enum	SameNamespace requires no namespaceSelector and only the Pod’s namespace in namespaces (or an omitted namespaces list). Any imposes no restriction. Enum: SameNamespace, Any	false
namespaceSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
namespaces	object	Namespaces constrains each explicitly supplied namespace.	false
requirements	[]object	Requirements constrains node matchExpressions.	false
topologyKey	object	TopologyKey applies to Pod affinity and anti-affinity.	false
types	[]enum	Types defaults to all types. Type-specific constraints only match the types on which they are meaningful. Enum: nodeAffinity, podAffinity, podAntiAffinity	false
weight	object	Weight requires modes: [preferred].	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].fieldRequirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].fieldRequirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].fieldRequirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].labelSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].labelSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].labelSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].labelSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].namespaceSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].namespaceSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].namespaces

Namespaces constrains each explicitly supplied namespace.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].topologyKey

TopologyKey applies to Pod affinity and anti-affinity.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.affinity[index].weight

Weight requires modes: [preferred].

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

Tenant.spec.rules[index].enforce.workloads.placement.nodeSelector[index]

WorkloadNodeSelectorMatch matches a nodeSelector key/value pair. Omitted fields are unrestricted, so {} matches every entry, including an empty label value.

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

Tenant.spec.rules[index].enforce.workloads.placement.nodeSelector[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.nodeSelector[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.schedulers[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.tolerations[index]

Name	Type	Description	Required
effects	[]enum	Effects matches the literal effect. An empty effect on a Pod tolerates all effects and does not match an allowlist of individual effects. Enum: NoSchedule, PreferNoSchedule, NoExecute,	false
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Operators matches Equal (including an omitted operator) or Exists. Enum: Equal, Exists	false
tolerationSeconds	object	TolerationDurationMatch treats an absent tolerationSeconds as unlimited.	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

Tenant.spec.rules[index].enforce.workloads.placement.tolerations[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.tolerations[index].tolerationSeconds

TolerationDurationMatch treats an absent tolerationSeconds as unlimited.

Name	Type	Description	Required
allowUnlimited	boolean	AllowUnlimited defaults to true. Set false to require a finite duration.	false
max	integer	Format: int64	false
min	integer	Format: int64	false

Tenant.spec.rules[index].enforce.workloads.placement.tolerations[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.topologySpreadConstraints[index]

Name	Type	Description	Required
labelSelector	object	PlacementLabelSelectorMatch checks every effective selector requirement. matchLabels is normalized to In, and dynamic matchLabelKeys/mismatchLabelKeys are checked as In/NotIn using the incoming Pod’s label values. Missing dynamic labels are ignored, matching Kubernetes semantics.	false
maxSkew	object	PlacementRange defines inclusive bounds. An omitted bound is unrestricted.	false
minDomains	object	MinDomains uses 1 when the Pod omits minDomains.	false
nodeAffinityPolicy	[]enum	NodeAffinityPolicy uses Honor when the Pod omits the field. Enum: Honor, Ignore	false
nodeTaintsPolicy	[]enum	NodeTaintsPolicy uses Ignore when the Pod omits the field. Enum: Honor, Ignore	false
topologyKey	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
whenUnsatisfiable	[]enum	Enum: DoNotSchedule, ScheduleAnyway	false

Tenant.spec.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector

Name	Type	Description	Required
required	boolean	Required requires at least one effective selector requirement.	false
requirements	[]object	Requirements is an allowlist within this matcher. If empty, requirements are unrestricted. Each actual requirement must match one complete entry.	false

Tenant.spec.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index]

Name	Type	Description	Required
key	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false
operators	[]enum	Enum: In, NotIn, Exists, DoesNotExist, Gt, Lt	false
values	object	PlacementExpressionMatch uses the shared exact/exp/negate matching semantics. The rule admission validator requires exact or exp. Keeping this check out of CEL avoids multiplying its estimated cost across the nested placement lists and the three representations of namespace rules in RuleStatus.	false

Tenant.spec.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index].key

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].labelSelector.requirements[index].values

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].maxSkew

PlacementRange defines inclusive bounds. An omitted bound is unrestricted.

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

Tenant.spec.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].minDomains

MinDomains uses 1 when the Pod omits minDomains.

Name	Type	Description	Required
max	integer	Format: int64	false
min	integer	Format: int64	false

Tenant.spec.rules[index].enforce.workloads.placement.topologySpreadConstraints[index].topologyKey

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.registries[index]

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false
policy	[]string	Allowed PullPolicy for the given registry. Supplying no value allows all policies.	false

Tenant.spec.rules[index].enforce.workloads.resources

Name	Type	Description	Required
limits	map[string]object	Limits defines policies for resource limits.	false
requests	map[string]object	Requests defines policies for resource requests.	false

Tenant.spec.rules[index].enforce.workloads.resources.limits[key]

WorkloadResourceLimitPolicy defines how a resource limit is mutated and enforced.

Name	Type	Description	Required
policy	enum	Policy selects how the limit is handled: Preserve leaves it unchanged, Default fills an absent limit, Remove deletes it, MatchRequest manages it to equal the request, and Ratio defaults an absent limit and enforces the maximum multiplier against explicitly supplied limits. Enum: Preserve, Default, Remove, MatchRequest, Ratio	true
value	int or string	Value is the quantity applied by Default or the maximum limit-to-request multiplier applied by Ratio.	false

Tenant.spec.rules[index].enforce.workloads.resources.requests[key]

WorkloadResourceRequestPolicy defines how a resource request is mutated.

Name	Type	Description	Required
policy	enum	Policy selects how the request is handled: Preserve leaves it unchanged, Default fills an absent request, and Remove deletes it. Enum: Preserve, Default, Remove	true
value	int or string	Value is the quantity applied by the Default policy.	false

Tenant.spec.rules[index].enforce.workloads.schedulers[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.security

Security matches effective Pod and container security profiles.

Name	Type	Description	Required
appArmorProfiles	[]object	AppArmorProfiles matches effective Linux container profiles, including legacy AppArmor annotations before Pod defaults. It uses the same target and missing-profile semantics as SeccompProfiles. Privileged containers are Unconfined. Matching a Localhost name does not verify its installation.	false
seccompProfiles	[]object	SeccompProfiles matches effective Linux container profiles, resolving container overrides before Pod defaults. Privileged containers are Unconfined. Missing profiles do not match any type, so an allow-list rejects them. Omitted targets check regular, init and ephemeral containers; pod explicitly selects only the Pod default. Controller targets check their Pod templates.	false

Tenant.spec.rules[index].enforce.workloads.security.appArmorProfiles[index]

WorkloadSecurityProfileMatch matches a profile type and optionally its local name/path. Entries are alternatives; localhostProfiles constrains only Localhost.

Name	Type	Description	Required
types	[]enum	Types selects profile types. Missing profiles never match. Enum: RuntimeDefault, Localhost, Unconfined	true
localhostProfiles	[]object	LocalhostProfiles matches seccomp paths relative to the kubelet seccomp directory, or loaded AppArmor profile names. Omitted permits any Localhost profile. Requires Localhost in types. Expressions use exact, exp and negate.	false

Tenant.spec.rules[index].enforce.workloads.security.appArmorProfiles[index].localhostProfiles[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].enforce.workloads.security.seccompProfiles[index]

WorkloadSecurityProfileMatch matches a profile type and optionally its local name/path. Entries are alternatives; localhostProfiles constrains only Localhost.

Name	Type	Description	Required
types	[]enum	Types selects profile types. Missing profiles never match. Enum: RuntimeDefault, Localhost, Unconfined	true
localhostProfiles	[]object	LocalhostProfiles matches seccomp paths relative to the kubelet seccomp directory, or loaded AppArmor profile names. Omitted permits any Localhost profile. Requires Localhost in types. Expressions use exact, exp and negate.	false

Tenant.spec.rules[index].enforce.workloads.security.seccompProfiles[index].localhostProfiles[index]

At least one of Exact or Exp must be set. Both may be set together.

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false

Tenant.spec.rules[index].mutate[index]

NamespaceRuleMutation applies typed mutations independently of enforce.action. Mutated values remain subject to all applicable enforcement rules.

Name	Type	Description	Required
action	enum	Action chooses how explicitly supplied properties are applied. Merge fills an empty schedulerName and absent security profiles, sets hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, and map keys, adds missing imagePullSecrets by name, upserts lists, and conjoins required affinity. Replace replaces each supplied property in full: scheduler, hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, nodeSelector, registries.imagePullSecrets, tolerations, topologySpreadConstraints, affinity, or security profiles. Omitted properties are retained. Supplying affinity replaces all its branches, including omitted ones. Enum: merge, replace Default: merge	false
conditions	[]object	Conditions gate this entire mutation entry and inspect the object after preceding mutations. All conditions must be true; empty means apply.	false
workloads	object	WorkloadMutation contains typed native Pod values. Empty maps/lists are preserved so replace can clear a property; nil means the property is omitted. It applies on Pod creation. Security.ReadOnlyRootFilesystem and Registries.ImagePullPolicy also apply to newly added ephemeral containers on subresource updates. Existing containers are never reconciled. On merge, later entries override hostUsers, readOnlyRootFilesystem, registries.imagePullPolicy, matching map keys, tolerations and spread constraints. On merge, required affinity restrictions from applicable entries are ANDed.	false

Tenant.spec.rules[index].mutate[index].conditions[index]

Name	Type	Description	Required
expression	string	Expression must evaluate to bool. Any false condition skips this block. An evaluation error rejects the request unless another condition is false.	true
name	string	Name identifies a condition in admission errors. Names must be unique within a block.	false

Tenant.spec.rules[index].mutate[index].workloads

Name	Type	Description	Required
placement	object	Placement configures the Pod scheduler and scheduling constraints.	false
registries	object	Registries configures Pod image pull secrets and selected containers’ pull policies.	false
security	object	Security configures Pod and container security settings.	false
targets	[]enum	Targets selects compatible Pod locations. Omitted or empty selects all compatible locations. The pod target includes Pod-level properties and all container groups; pod/containers, pod/initcontainers and pod/ephemeralcontainers narrow selection to one group. Controller templates and volumes are not supported. Enum: pod, pod/containers, pod/initcontainers, pod/ephemeralcontainers	false

Tenant.spec.rules[index].mutate[index].workloads.placement

Placement configures the Pod scheduler and scheduling constraints.

Name	Type	Description	Required
affinity	object	Affinity on merge conjoins required restrictions and upserts preferred terms. Preferred terms match by their complete term, excluding weight and normalized ordering. A different term is added; a matching term gets the supplied weight. Required node affinity is distributed over existing OR alternatives, with at most 256 resulting alternatives. Empty node selector terms match no nodes. Replace replaces all affinity, including any branches omitted from the rule.	false
nodeSelector	map[string]string	NodeSelector sets configured keys on merge, or replaces the map on replace.	false
scheduler	string	Scheduler sets spec.schedulerName on Pod creation. Merge fills only an empty schedulerName, preserving all non-empty names, including default-scheduler. Kubernetes defaults omitted schedulerName before admission. Use replace with a condition to override default-scheduler while preserving custom schedulers. Replace always overwrites schedulerName when the entry’s conditions match. Omitted or null leaves the Pod value unchanged.	false
tolerations	[]object	Tolerations merges by key, operator (default Equal), value and effect. Changing any identity field adds another toleration, retaining the old one. A matching toleration’s duration is replaced; omitting it makes it unlimited. Replace replaces the entire list, including entries with other identities.	false
topologySpreadConstraints	[]object	TopologySpreadConstraints merges by topologyKey and whenUnsatisfiable. The entire matching constraint is replaced, including its selector and optional fields; other constraints remain. Replace replaces the entire list.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity

Name	Type	Description	Required
nodeAffinity	object	Describes node affinity scheduling rules for the pod.	false
podAffinity	object	Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).	false
podAntiAffinity	object	Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity

Describes node affinity scheduling rules for the pod.

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding “weight” to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	object	If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it’s a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).

Name	Type	Description	Required
preference	object	A node selector term, associated with the corresponding weight.	true
weight	integer	Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100. Format: int32	true

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference

A node selector term, associated with the corresponding weight.

Name	Type	Description	Required
matchExpressions	[]object	A list of node selector requirements by node’s labels.	false
matchFields	[]object	A list of node selector requirements by node’s fields.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference.matchExpressions[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].preference.matchFields[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution

Name	Type	Description	Required
nodeSelectorTerms	[]object	Required. A list of node selector terms. The terms are ORed.	true

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index]

A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.

Name	Type	Description	Required
matchExpressions	[]object	A list of node selector requirements by node’s labels.	false
matchFields	[]object	A list of node selector requirements by node’s fields.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index].matchExpressions[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[index].matchFields[index]

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	The label key that the selector applies to.	true
operator	string	Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity

Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding “weight” to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	[]object	If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)

Name	Type	Description	Required
podAffinityTerm	object	Required. A pod affinity term, associated with the corresponding weight.	true
weight	integer	weight associated with matching the corresponding podAffinityTerm, in the range 1-100. Format: int32	true

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm

Required. A pod affinity term, associated with the corresponding weight.

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index]

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity

Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).

Name	Type	Description	Required
preferredDuringSchedulingIgnoredDuringExecution	[]object	The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and subtracting “weight” from the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.	false
requiredDuringSchedulingIgnoredDuringExecution	[]object	If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index]

The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)

Name	Type	Description	Required
podAffinityTerm	object	Required. A pod affinity term, associated with the corresponding weight.	true
weight	integer	weight associated with matching the corresponding podAffinityTerm, in the range 1-100. Format: int32	true

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm

Required. A pod affinity term, associated with the corresponding weight.

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[index].podAffinityTerm.namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index]

Name	Type	Description	Required
topologyKey	string	This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.	true
labelSelector	object	A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key in (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both matchLabelKeys and labelSelector. Also, matchLabelKeys cannot be set when labelSelector isn’t set.	false
mismatchLabelKeys	[]string	MismatchLabelKeys is a set of pod label keys to select which pods will be taken into consideration. The keys are used to lookup values from the incoming pod labels, those key-value labels are merged with `labelSelector` as `key notin (value)` to select the group of existing pods which pods will be taken into consideration for the incoming pod’s pod (anti) affinity. Keys that don’t exist in the incoming pod labels will be ignored. The default value is empty. The same key is forbidden to exist in both mismatchLabelKeys and labelSelector. Also, mismatchLabelKeys cannot be set when labelSelector isn’t set.	false
namespaceSelector	object	A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means “this pod’s namespace”. An empty selector ({}) matches all namespaces.	false
namespaces	[]string	namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means “this pod’s namespace”.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector

A label query over a set of resources, in this case pods. If it’s null, this PodAffinityTerm matches with no Pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.tolerations[index]

The pod this Toleration is attached to tolerates any taint that matches the triple <key,value,effect> using the matching operator .

Name	Type	Description	Required
effect	string	Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.	false
key	string	Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.	false
operator	string	Operator represents a key’s relationship to the value. Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category. Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).	false
tolerationSeconds	integer	TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system. Format: int64	false
value	string	Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.topologySpreadConstraints[index]

TopologySpreadConstraint specifies how to spread matching pods among the given topology.

Name	Type	Description	Required
maxSkew	integer	MaxSkew describes the degree to which pods may be unevenly distributed. When `whenUnsatisfiable=DoNotSchedule`, it is the maximum permitted difference between the number of matching pods in the target topology and the global minimum. The global minimum is the minimum number of matching pods in an eligible domain or zero if the number of eligible domains is less than MinDomains. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 2/2/1: In this case, the global minimum is 1.	zone1
topologyKey	string	TopologyKey is the key of node labels. Nodes that have a label with this key and identical values are considered to be in the same topology. We consider each <key, value> as a “bucket”, and try to put balanced number of pods into each bucket. We define a domain as a particular instance of a topology. Also, we define an eligible domain as a domain whose nodes meet the requirements of nodeAffinityPolicy and nodeTaintsPolicy. e.g. If TopologyKey is “kubernetes.io/hostname”, each Node is a domain of that topology. And, if TopologyKey is “topology.kubernetes.io/zone”, each zone is a domain of that topology. It’s a required field.	true
whenUnsatisfiable	string	WhenUnsatisfiable indicates how to deal with a pod if it doesn’t satisfy the spread constraint. - DoNotSchedule (default) tells the scheduler not to schedule it. - ScheduleAnyway tells the scheduler to schedule the pod in any location, but giving higher precedence to topologies that would help reduce the skew. A constraint is considered “Unsatisfiable” for an incoming pod if and only if every possible node assignment for that pod would violate “MaxSkew” on some topology. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 3/1/1:	zone1
labelSelector	object	LabelSelector is used to find matching pods. Pods that match this label selector are counted to determine the number of pods in their corresponding topology domain.	false
matchLabelKeys	[]string	MatchLabelKeys is a set of pod label keys to select the pods over which spreading will be calculated. The keys are used to lookup values from the incoming pod labels, those key-value labels are ANDed with labelSelector to select the group of existing pods over which spreading will be calculated for the incoming pod. The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. MatchLabelKeys cannot be set when LabelSelector isn’t set. Keys that don’t exist in the incoming pod labels will be ignored. A null or empty list means only match against labelSelector. This is a beta field and requires the MatchLabelKeysInPodTopologySpread feature gate to be enabled (enabled by default).	false
minDomains	integer	MinDomains indicates a minimum number of eligible domains. When the number of eligible domains with matching topology keys is less than minDomains, Pod Topology Spread treats “global minimum” as 0, and then the calculation of Skew is performed. And when the number of eligible domains with matching topology keys equals or greater than minDomains, this value has no effect on scheduling. As a result, when the number of eligible domains is less than minDomains, scheduler won’t schedule more than maxSkew Pods to those domains. If value is nil, the constraint behaves as if MinDomains is equal to 1. Valid values are integers greater than 0. When value is not nil, WhenUnsatisfiable must be DoNotSchedule. For example, in a 3-zone cluster, MaxSkew is set to 2, MinDomains is set to 5 and pods with the same labelSelector spread as 2/2/2:	zone1
nodeAffinityPolicy	string	NodeAffinityPolicy indicates how we will treat Pod’s nodeAffinity/nodeSelector when calculating pod topology spread skew. Options are: - Honor: only nodes matching nodeAffinity/nodeSelector are included in the calculations. - Ignore: nodeAffinity/nodeSelector are ignored. All nodes are included in the calculations. If this value is nil, the behavior is equivalent to the Honor policy.	false
nodeTaintsPolicy	string	NodeTaintsPolicy indicates how we will treat node taints when calculating pod topology spread skew. Options are: - Honor: nodes without taints, along with tainted nodes for which the incoming pod has a toleration, are included. - Ignore: node taints are ignored. All nodes are included. If this value is nil, the behavior is equivalent to the Ignore policy.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.topologySpreadConstraints[index].labelSelector

LabelSelector is used to find matching pods. Pods that match this label selector are counted to determine the number of pods in their corresponding topology domain.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].mutate[index].workloads.placement.topologySpreadConstraints[index].labelSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].mutate[index].workloads.registries

Registries configures Pod image pull secrets and selected containers’ pull policies.

Name	Type	Description	Required
imagePullPolicy	enum	ImagePullPolicy sets imagePullPolicy on every selected regular or init container at Pod creation, and newly added ephemeral containers on subresource updates. Both merge and replace overwrite explicit and Kubernetes-defaulted values. Omitted or null preserves the existing policy. Applies to all Pod OSes. Enum: Always, IfNotPresent, Never	false
imagePullSecrets	[]object	ImagePullSecrets sets spec.imagePullSecrets on Pod creation. Merge appends missing names and removes existing duplicates, retaining first-occurrence order. Replace sets the complete list; an explicit empty list clears it. Omitted or null preserves it. Requires the pod target or omitted/empty targets. References always use the Pod’s namespace; Capsule does not create, copy, or check the referenced Secrets.	false

Tenant.spec.rules[index].mutate[index].workloads.registries.imagePullSecrets[index]

LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.

Name	Type	Description	Required
name	string	Name of the referent. This field is effectively required, but due to backwards compatibility is allowed to be empty. Instances of this type with an empty value here are almost certainly wrong. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names Default:	false

Tenant.spec.rules[index].mutate[index].workloads.security

Security configures Pod and container security settings.

Name	Type	Description	Required
appArmorProfile	object	AppArmorProfile supplies the Pod-level securityContext.appArmorProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it. AppArmor and any Localhost profile must be available on eligible nodes.	false
hostUsers	boolean	HostUsers sets spec.hostUsers on both merge and replace. False requests a separate user namespace; true uses the host user namespace. Omitted or null leaves the Pod value unchanged. Requires Kubernetes/runtime support.	false
readOnlyRootFilesystem	boolean	ReadOnlyRootFilesystem sets securityContext.readOnlyRootFilesystem on every selected regular or init container at Pod creation, and newly added ephemeral containers on subresource updates. Both merge and replace overwrite the value. False is an explicit setting; nil preserves it. Windows Pods are skipped.	false
seccompProfile	object	SeccompProfile supplies the Pod-level securityContext.seccompProfile on Linux Pod creation. Merge fills an absent profile; replace replaces the complete profile. Explicit container profiles are preserved. Nil omits it.	false

Tenant.spec.rules[index].mutate[index].workloads.security.appArmorProfile

Name	Type	Description	Required
type	string	type indicates which kind of AppArmor profile will be applied. Valid options are: Localhost - a profile pre-loaded on the node. RuntimeDefault - the container runtime’s default profile. Unconfined - no AppArmor enforcement.	true
localhostProfile	string	localhostProfile indicates a profile loaded on the node that should be used. The profile must be preconfigured on the node to work. Must match the loaded name of the profile. Must be set if and only if type is “Localhost”.	false

Tenant.spec.rules[index].mutate[index].workloads.security.seccompProfile

Name	Type	Description	Required
type	string	type indicates which kind of seccomp profile will be applied. Valid options are: Localhost - a profile defined in a file on the node should be used. RuntimeDefault - the container runtime default profile should be used. Unconfined - no profile should be applied.	true
localhostProfile	string	localhostProfile indicates a profile defined in a file on the node should be used. The profile must be preconfigured on the node to work. Must be a descending path, relative to the kubelet’s configured seccomp profile location. Must be set if type is “Localhost”. Must NOT be set for any other type.	false

Tenant.spec.rules[index].namespaceSelector

Select namespaces which are going to be targeted with this rule

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].permissions

Permissions for given rule

Name	Type	Description	Required
bindings	[]object	Bindings defines additional RoleBindings for namespaces selected by this rule.	false
promotions	[]object	Define Promotion Rules which distributed additional ClusterRoles across the Tenant for promoted ServiceAccounts.	false

Tenant.spec.rules[index].permissions.bindings[index]

Name	Type	Description	Required
clusterRoleName	string		true
subjects	[]object		true
annotations	map[string]string	Additional Annotations for the synchronized rolebindings	false
labels	map[string]string	Additional Labels for the synchronized rolebindings	false

Tenant.spec.rules[index].permissions.bindings[index].subjects[index]

Subject contains a reference to the object or user identities a role binding applies to. This can either hold a direct API object reference, or a value for non-objects such as user and group names.

Name	Type	Description	Required
kind	string	Kind of object being referenced. Values defined by this API group are “User”, “Group”, and “ServiceAccount”. If the Authorizer does not recognized the kind value, the Authorizer should report an error.	true
name	string	Name of the object being referenced.	true
apiGroup	string	APIGroup holds the API group of the referenced subject. Defaults to "" for ServiceAccount subjects. Defaults to “rbac.authorization.k8s.io” for User and Group subjects.	false
namespace	string	Namespace of the referenced object. If the object kind is non-namespace, such as “User” or “Group”, and this value is not empty the Authorizer should report an error.	false

Tenant.spec.rules[index].permissions.promotions[index]

Name	Type	Description	Required
clusterRoles	[]string	ClusterRoles granted to the promoted ServiceAccounts across the Tenant kubebuilder:validation:Minimum=1	false
selector	object	Match ServiceAccounts which are promoted which are granted these additional ClusterRoles across the Tenant	false

Tenant.spec.rules[index].permissions.promotions[index].selector

Match ServiceAccounts which are promoted which are granted these additional ClusterRoles across the Tenant

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.rules[index].permissions.promotions[index].selector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.rules[index].quota[index]

Name	Type	Description	Required
name	string	Name is the stable identity of this quota within the Tenant. Changing the name replaces the generated GlobalResourceQuota; changing the quota or its namespace selector updates the existing object.	true
hard	map[string]int or string	hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
scopeSelector	object	scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.	false
scopes	[]string	A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.	false

Tenant.spec.rules[index].quota[index].scopeSelector

Name	Type	Description	Required
matchExpressions	[]object	A list of scope selector requirements by scope of the resources.	false

Tenant.spec.rules[index].quota[index].scopeSelector.matchExpressions[index]

A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.

Name	Type	Description	Required
operator	string	Represents a scope’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.	true
scopeName	string	The name of the scope that the selector applies to.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.runtimeClasses

Specifies the allowed RuntimeClasses assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed RuntimeClasses. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false
default	string		false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.runtimeClasses.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.serviceOptions

Deprecated: Use Rules Metadata instead (https://projectcapsule.dev/docs/rules/enforcement/metadata/)

Specifies options for the Service, such as additional metadata or block of certain type of Services. Optional.

Name	Type	Description	Required
additionalMetadata	object	Specifies additional labels and annotations the Capsule operator places on any Service resource in the Tenant. Optional.	false
allowedServices	object	Block or deny certain type of Services. Optional.	false
externalIPs	object	Specifies the external IPs that can be used in Services with type ClusterIP. An empty list means no IPs are allowed. Optional.	false
forbiddenAnnotations	object	Define the annotations that a Tenant Owner cannot set for their Service resources.	false
forbiddenLabels	object	Define the labels that a Tenant Owner cannot set for their Service resources.	false

Tenant.spec.serviceOptions.additionalMetadata

Specifies additional labels and annotations the Capsule operator places on any Service resource in the Tenant. Optional.

Name	Type	Description	Required
annotations	map[string]string		false
labels	map[string]string		false

Tenant.spec.serviceOptions.allowedServices

Block or deny certain type of Services. Optional.

Name	Type	Description	Required
externalName	boolean	Specifies if ExternalName service type resources are allowed for the Tenant. Default is true. Optional. Default: true	false
loadBalancer	boolean	Specifies if LoadBalancer service type resources are allowed for the Tenant. Default is true. Optional. Default: true	false
nodePort	boolean	Specifies if NodePort service type resources are allowed for the Tenant. Default is true. Optional. Default: true	false

Tenant.spec.serviceOptions.externalIPs

Specifies the external IPs that can be used in Services with type ClusterIP. An empty list means no IPs are allowed. Optional.

Name	Type	Description	Required
allowed	[]string		true

Tenant.spec.serviceOptions.forbiddenAnnotations

Define the annotations that a Tenant Owner cannot set for their Service resources.

Name	Type	Description	Required
denied	[]string		false
deniedRegex	string		false

Tenant.spec.serviceOptions.forbiddenLabels

Define the labels that a Tenant Owner cannot set for their Service resources.

Name	Type	Description	Required
denied	[]string		false
deniedRegex	string		false

Tenant.spec.storageClasses

Specifies the allowed StorageClasses assigned to the Tenant. Capsule assures that all PersistentVolumeClaim resources created in the Tenant can use only one of the allowed StorageClasses. A default value can be specified, and all the PersistentVolumeClaim resources created will inherit the declared class. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false
default	string		false
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.storageClasses.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.status

Returns the observed state of the Tenant.

Name	Type	Description	Required
conditions	[]object	Tenant Condition	true
size	integer	How many namespaces are assigned to the Tenant.	true
state	enum	The operational state of the Tenant. Possible values are “Active”, “Cordoned” or “Terminating”. Enum: Cordoned, Active, Terminating Default: Active	true
classes	object	Available Class Types within Tenant	false
namespaces	[]string	List of namespaces assigned to the Tenant. (Deprecated)	false
observedGeneration	integer	ObservedGeneration is the most recent generation the controller has observed. Format: int64	false
owners	[]object	Collected owners for this tenant	false
promotions	[]object	Promoted ServiceAccounts across the Tenant	false
spaces	[]object	Tracks state for the namespaces associated with this tenant	false

Tenant.status.conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

Tenant.status.classes

Available Class Types within Tenant

Name	Type	Description	Required
device	[]string	Available DeviceClasses	false
gateway	[]string	Available GatewayClasses	false
priority	[]string	Available PriorityClasses	false
runtime	[]string	Available StorageClasses	false
storage	[]string	Available Storageclasses (Only collected if any matching condition is specified)	false

Tenant.status.owners[index]

Name	Type	Description	Required
kind	enum	Kind of entity. Possible values are “User”, “Group”, and “ServiceAccount” Enum: User, Group, ServiceAccount	true
name	string	Name of the entity.	true
clusterRoles	[]string	Defines additional cluster-roles for the specific Owner. Default: [admin capsule-namespace-deleter]	false

Tenant.status.promotions[index]

Name	Type	Description	Required
kind	enum	Kind of entity. Possible values are “User”, “Group”, and “ServiceAccount” Enum: User, Group, ServiceAccount	true
name	string	Name of the entity.	true
clusterRoles	[]string	Defines additional cluster-roles for the specific Owner. Default: [admin capsule-namespace-deleter]	false
targets	[]string	Defines additional cluster-roles for the specific Owner.	false

Tenant.status.spaces[index]

Name	Type	Description	Required
conditions	[]object	Conditions	true
name	string	Namespace Name	true
enforce	object	Managed Metadata	false
metadata	object	Managed Metadata	false
uid	string	Namespace UID	false

Tenant.status.spaces[index].conditions[index]

Condition contains details for one aspect of the current state of this API Resource.

Name	Type	Description	Required
lastTransitionTime	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. Format: date-time	true
message	string	message is a human readable message indicating details about the transition. This may be an empty string.	true
reason	string	reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.	true
status	enum	status of the condition, one of True, False, Unknown. Enum: True, False, Unknown	true
type	string	type of condition in CamelCase or in foo.example.com/CamelCase.	true
observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. Format: int64 Minimum: 0	false

Tenant.status.spaces[index].enforce

Managed Metadata

Name	Type	Description	Required
registry	[]object	Registries which are allowed within this namespace	false

Tenant.status.spaces[index].enforce.registry[index]

Name	Type	Description	Required
exact	[]string	Exact matches one of the provided values exactly.	false
exp	string	Exp matches regular expression.	false
negate	boolean	Negate regular Expression Default: false	false
policy	[]string	Allowed PullPolicy for the given registry. Supplying no value allows all policies.	false

Tenant.status.spaces[index].metadata

Managed Metadata

Name	Type	Description	Required
annotations	map[string]string	Managed Annotations	false
labels	map[string]string	Managed Labels	false

capsule.clastix.io/v1beta1

Resource Types:

Tenant

Tenant

Tenant is the Schema for the tenants API.

Name	Type	Description	Required
apiVersion	string	capsule.clastix.io/v1beta1	true
kind	string	Tenant	true
metadata	object	Refer to the Kubernetes API documentation for the fields of the `metadata` field.	true
spec	object	TenantSpec defines the desired state of Tenant.	true
status	object	Returns the observed state of the Tenant.	false

Tenant.spec

TenantSpec defines the desired state of Tenant.

Name	Type	Description	Required
owners	[]object	Specifies the owners of the Tenant. Mandatory.	true
additionalRoleBindings	[]object	Specifies additional RoleBindings assigned to the Tenant. Capsule will ensure that all namespaces in the Tenant always contain the RoleBinding for the given ClusterRole. Optional.	false
containerRegistries	object	Specifies the trusted Image Registries assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed trusted registries. Optional.	false
imagePullPolicies	[]enum	Specify the allowed values for the imagePullPolicies option in Pod resources. Capsule assures that all Pod resources created in the Tenant can use only one of the allowed policy. Optional. Enum: Always, Never, IfNotPresent	false
ingressOptions	object	Specifies options for the Ingress resources, such as allowed hostnames and IngressClass. Optional.	false
limitRanges	object	Specifies the resource min/max usage restrictions to the Tenant. The assigned values are inherited by any namespace created in the Tenant. Optional.	false
namespaceOptions	object	Specifies options for the Namespaces, such as additional metadata or maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional.	false
networkPolicies	object	Specifies the NetworkPolicies assigned to the Tenant. The assigned NetworkPolicies are inherited by any namespace created in the Tenant. Optional.	false
nodeSelector	map[string]string	Specifies the label to control the placement of pods on a given pool of worker nodes. All namespaces created within the Tenant will have the node selector annotation. This annotation tells the Kubernetes scheduler to place pods on the nodes having the selector label. Optional.	false
priorityClasses	object	Specifies the allowed priorityClasses assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed PriorityClasses. Optional.	false
resourceQuotas	object	Specifies a list of ResourceQuota resources assigned to the Tenant. The assigned values are inherited by any namespace created in the Tenant. The Capsule operator aggregates ResourceQuota at Tenant level, so that the hard quota is never crossed for the given Tenant. This permits the Tenant owner to consume resources in the Tenant regardless of the namespace. Optional.	false
serviceOptions	object	Specifies options for the Service, such as additional metadata or block of certain type of Services. Optional.	false
storageClasses	object	Specifies the allowed StorageClasses assigned to the Tenant. Capsule assures that all PersistentVolumeClaim resources created in the Tenant can use only one of the allowed StorageClasses. Optional.	false

Tenant.spec.owners[index]

Name	Type	Description	Required
kind	enum	Kind of tenant owner. Possible values are “User”, “Group”, and “ServiceAccount” Enum: User, Group, ServiceAccount	true
name	string	Name of tenant owner.	true
proxySettings	[]object	Proxy settings for tenant owner.	false

Tenant.spec.owners[index].proxySettings[index]

Name	Type	Description	Required
kind	enum	Enum: Nodes, StorageClasses, IngressClasses, PriorityClasses	true
operations	[]enum	Enum: List, Update, Delete	true

Tenant.spec.additionalRoleBindings[index]

Name	Type	Description	Required
clusterRoleName	string		true
subjects	[]object		true
annotations	map[string]string	Additional Annotations for the synchronized rolebindings	false
labels	map[string]string	Additional Labels for the synchronized rolebindings	false

Tenant.spec.additionalRoleBindings[index].subjects[index]

Subject contains a reference to the object or user identities a role binding applies to. This can either hold a direct API object reference, or a value for non-objects such as user and group names.

Name	Type	Description	Required
kind	string	Kind of object being referenced. Values defined by this API group are “User”, “Group”, and “ServiceAccount”. If the Authorizer does not recognized the kind value, the Authorizer should report an error.	true
name	string	Name of the object being referenced.	true
apiGroup	string	APIGroup holds the API group of the referenced subject. Defaults to "" for ServiceAccount subjects. Defaults to “rbac.authorization.k8s.io” for User and Group subjects.	false
namespace	string	Namespace of the referenced object. If the object kind is non-namespace, such as “User” or “Group”, and this value is not empty the Authorizer should report an error.	false

Tenant.spec.containerRegistries

Specifies the trusted Image Registries assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed trusted registries. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false

Tenant.spec.ingressOptions

Specifies options for the Ingress resources, such as allowed hostnames and IngressClass. Optional.

Name	Type	Description	Required
allowedClasses	object	Specifies the allowed IngressClasses assigned to the Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed IngressClasses. Optional.	false
allowedHostnames	object	Specifies the allowed hostnames in Ingresses for the given Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed hostnames. Optional.	false
hostnameCollisionScope	enum	Defines the scope of hostname collision check performed when Tenant Owners create Ingress with allowed hostnames. - Cluster: disallow the creation of an Ingress if the pair hostname and path is already used across the Namespaces managed by Capsule. - Tenant: disallow the creation of an Ingress if the pair hostname and path is already used across the Namespaces of the Tenant. - Namespace: disallow the creation of an Ingress if the pair hostname and path is already used in the Ingress Namespace. Optional. Enum: Cluster, Tenant, Namespace, Disabled Default: Disabled	false

Tenant.spec.ingressOptions.allowedClasses

Specifies the allowed IngressClasses assigned to the Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed IngressClasses. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false

Tenant.spec.ingressOptions.allowedHostnames

Specifies the allowed hostnames in Ingresses for the given Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed hostnames. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false

Tenant.spec.limitRanges

Specifies the resource min/max usage restrictions to the Tenant. The assigned values are inherited by any namespace created in the Tenant. Optional.

Name	Type	Description	Required
items	[]object		false

Tenant.spec.limitRanges.items[index]

LimitRangeSpec defines a min/max usage limit for resources that match on kind.

Name	Type	Description	Required
limits	[]object	Limits is the list of LimitRangeItem objects that are enforced.	true

Tenant.spec.limitRanges.items[index].limits[index]

LimitRangeItem defines a min/max usage limit for any resource that matches on kind.

Name	Type	Description	Required
type	string	Type of resource that this limit applies to.	true
default	map[string]int or string	Default resource requirement limit value by resource name if resource limit is omitted.	false
defaultRequest	map[string]int or string	DefaultRequest is the default resource requirement request value by resource name if resource request is omitted.	false
max	map[string]int or string	Max usage constraints on this kind by resource name.	false
maxLimitRequestRatio	map[string]int or string	MaxLimitRequestRatio if specified, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value; this represents the max burst for the named resource.	false
min	map[string]int or string	Min usage constraints on this kind by resource name.	false

Tenant.spec.namespaceOptions

Name	Type	Description	Required
additionalMetadata	object	Specifies additional labels and annotations the Capsule operator places on any Namespace resource in the Tenant. Optional.	false
quota	integer	Specifies the maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional. Format: int32 Minimum: 1	false

Tenant.spec.namespaceOptions.additionalMetadata

Specifies additional labels and annotations the Capsule operator places on any Namespace resource in the Tenant. Optional.

Name	Type	Description	Required
annotations	map[string]string		false
labels	map[string]string		false

Tenant.spec.networkPolicies

Specifies the NetworkPolicies assigned to the Tenant. The assigned NetworkPolicies are inherited by any namespace created in the Tenant. Optional.

Name	Type	Description	Required
items	[]object		false

Tenant.spec.networkPolicies.items[index]

NetworkPolicySpec provides the specification of a NetworkPolicy

Name	Type	Description	Required
egress	[]object	egress is a list of egress rules to be applied to the selected pods. Outgoing traffic is allowed if there are no NetworkPolicies selecting the pod (and cluster policy otherwise allows the traffic), OR if the traffic matches at least one egress rule across all of the NetworkPolicy objects whose podSelector matches the pod. If this field is empty then this NetworkPolicy limits all outgoing traffic (and serves solely to ensure that the pods it selects are isolated by default). This field is beta-level in 1.8	false
ingress	[]object	ingress is a list of ingress rules to be applied to the selected pods. Traffic is allowed to a pod if there are no NetworkPolicies selecting the pod (and cluster policy otherwise allows the traffic), OR if the traffic source is the pod’s local node, OR if the traffic matches at least one ingress rule across all of the NetworkPolicy objects whose podSelector matches the pod. If this field is empty then this NetworkPolicy does not allow any traffic (and serves solely to ensure that the pods it selects are isolated by default)	false
podSelector	object	podSelector selects the pods to which this NetworkPolicy object applies. The array of rules is applied to any pods selected by this field. An empty selector matches all pods in the policy’s namespace. Multiple network policies can select the same set of pods. In this case, the ingress rules for each are combined additively. This field is optional. If it is not specified, it defaults to an empty selector.	false
policyTypes	[]string	policyTypes is a list of rule types that the NetworkPolicy relates to. Valid options are [“Ingress”], [“Egress”], or [“Ingress”, “Egress”]. If this field is not specified, it will default based on the existence of ingress or egress rules; policies that contain an egress section are assumed to affect egress, and all policies (whether or not they contain an ingress section) are assumed to affect ingress. If you want to write an egress-only policy, you must explicitly specify policyTypes [ “Egress” ]. Likewise, if you want to write a policy that specifies that no egress is allowed, you must specify a policyTypes value that include “Egress” (since such a policy would not include an egress section and would otherwise default to just [ “Ingress” ]). This field is beta-level in 1.8	false

Tenant.spec.networkPolicies.items[index].egress[index]

Name	Type	Description	Required
ports	[]object	ports is a list of destination ports for outgoing traffic. Each item in this list is combined using a logical OR. If this field is empty or missing, this rule matches all ports (traffic not restricted by port). If this field is present and contains at least one item, then this rule allows traffic only if the traffic matches at least one port in the list.	false
to	[]object	to is a list of destinations for outgoing traffic of pods selected for this rule. Items in this list are combined using a logical OR operation. If this field is empty or missing, this rule matches all destinations (traffic not restricted by destination). If this field is present and contains at least one item, this rule allows traffic only if the traffic matches at least one item in the to list.	false

Tenant.spec.networkPolicies.items[index].egress[index].ports[index]

NetworkPolicyPort describes a port to allow traffic on

Name	Type	Description	Required
endPort	integer	endPort indicates that the range of ports from port to endPort if set, inclusive, should be allowed by the policy. This field cannot be defined if the port field is not defined or if the port field is defined as a named (string) port. The endPort must be equal or greater than port. Format: int32	false
port	int or string	port represents the port on the given protocol. This can either be a numerical or named port on a pod. If this field is not provided, this matches all port names and numbers. If present, only traffic on the specified protocol AND port will be matched.	false
protocol	string	protocol represents the protocol (TCP, UDP, or SCTP) which traffic must match. If not specified, this field defaults to TCP.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index]

NetworkPolicyPeer describes a peer to allow traffic to/from. Only certain combinations of fields are allowed

Name	Type	Description	Required
ipBlock	object	ipBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.	false
namespaceSelector	object	namespaceSelector selects namespaces using cluster-scoped labels. This field follows standard label selector semantics; if present but empty, it selects all namespaces. If podSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the namespaces selected by namespaceSelector. Otherwise it selects all pods in the namespaces selected by namespaceSelector.	false
podSelector	object	podSelector is a label selector which selects pods. This field follows standard label selector semantics; if present but empty, it selects all pods. If namespaceSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the Namespaces selected by NamespaceSelector. Otherwise it selects the pods matching podSelector in the policy’s own namespace.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].ipBlock

ipBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.

Name	Type	Description	Required
cidr	string	cidr is a string representing the IPBlock Valid examples are “192.168.1.0/24” or “2001:db8::/64”	true
except	[]string	except is a slice of CIDRs that should not be included within an IPBlock Valid examples are “192.168.1.0/24” or “2001:db8::/64” Except values will be rejected if they are outside the cidr range	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].namespaceSelector

namespaceSelector selects namespaces using cluster-scoped labels. This field follows standard label selector semantics; if present but empty, it selects all namespaces.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].podSelector

podSelector is a label selector which selects pods. This field follows standard label selector semantics; if present but empty, it selects all pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].egress[index].to[index].podSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.networkPolicies.items[index].ingress[index]

NetworkPolicyIngressRule describes a particular set of traffic that is allowed to the pods matched by a NetworkPolicySpec’s podSelector. The traffic must match both ports and from.

Name	Type	Description	Required
from	[]object	from is a list of sources which should be able to access the pods selected for this rule. Items in this list are combined using a logical OR operation. If this field is empty or missing, this rule matches all sources (traffic not restricted by source). If this field is present and contains at least one item, this rule allows traffic only if the traffic matches at least one item in the from list.	false
ports	[]object	ports is a list of ports which should be made accessible on the pods selected for this rule. Each item in this list is combined using a logical OR. If this field is empty or missing, this rule matches all ports (traffic not restricted by port). If this field is present and contains at least one item, then this rule allows traffic only if the traffic matches at least one port in the list.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index]

NetworkPolicyPeer describes a peer to allow traffic to/from. Only certain combinations of fields are allowed

Name	Type	Description	Required
ipBlock	object	ipBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.	false
namespaceSelector	object	namespaceSelector selects namespaces using cluster-scoped labels. This field follows standard label selector semantics; if present but empty, it selects all namespaces. If podSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the namespaces selected by namespaceSelector. Otherwise it selects all pods in the namespaces selected by namespaceSelector.	false
podSelector	object	podSelector is a label selector which selects pods. This field follows standard label selector semantics; if present but empty, it selects all pods. If namespaceSelector is also set, then the NetworkPolicyPeer as a whole selects the pods matching podSelector in the Namespaces selected by NamespaceSelector. Otherwise it selects the pods matching podSelector in the policy’s own namespace.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].ipBlock

ipBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.

Name	Type	Description	Required
cidr	string	cidr is a string representing the IPBlock Valid examples are “192.168.1.0/24” or “2001:db8::/64”	true
except	[]string	except is a slice of CIDRs that should not be included within an IPBlock Valid examples are “192.168.1.0/24” or “2001:db8::/64” Except values will be rejected if they are outside the cidr range	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].namespaceSelector

namespaceSelector selects namespaces using cluster-scoped labels. This field follows standard label selector semantics; if present but empty, it selects all namespaces.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].namespaceSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].podSelector

podSelector is a label selector which selects pods. This field follows standard label selector semantics; if present but empty, it selects all pods.

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].ingress[index].from[index].podSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.networkPolicies.items[index].ingress[index].ports[index]

NetworkPolicyPort describes a port to allow traffic on

Name	Type	Description	Required
endPort	integer	endPort indicates that the range of ports from port to endPort if set, inclusive, should be allowed by the policy. This field cannot be defined if the port field is not defined or if the port field is defined as a named (string) port. The endPort must be equal or greater than port. Format: int32	false
port	int or string	port represents the port on the given protocol. This can either be a numerical or named port on a pod. If this field is not provided, this matches all port names and numbers. If present, only traffic on the specified protocol AND port will be matched.	false
protocol	string	protocol represents the protocol (TCP, UDP, or SCTP) which traffic must match. If not specified, this field defaults to TCP.	false

Tenant.spec.networkPolicies.items[index].podSelector

Name	Type	Description	Required
matchExpressions	[]object	matchExpressions is a list of label selector requirements. The requirements are ANDed.	false
matchLabels	map[string]string	matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.	false

Tenant.spec.networkPolicies.items[index].podSelector.matchExpressions[index]

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

Name	Type	Description	Required
key	string	key is the label key that the selector applies to.	true
operator	string	operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.	true
values	[]string	values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.priorityClasses

Specifies the allowed priorityClasses assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed PriorityClasses. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false

Tenant.spec.resourceQuotas

Name	Type	Description	Required
items	[]object		false
scope	enum	Define if the Resource Budget should compute resource across all Namespaces in the Tenant or individually per cluster. Default is Tenant Enum: Tenant, Namespace Default: Tenant	false

Tenant.spec.resourceQuotas.items[index]

ResourceQuotaSpec defines the desired hard limits to enforce for Quota.

Name	Type	Description	Required
hard	map[string]int or string	hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/	false
scopeSelector	object	scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.	false
scopes	[]string	A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.	false

Tenant.spec.resourceQuotas.items[index].scopeSelector

Name	Type	Description	Required
matchExpressions	[]object	A list of scope selector requirements by scope of the resources.	false

Tenant.spec.resourceQuotas.items[index].scopeSelector.matchExpressions[index]

A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.

Name	Type	Description	Required
operator	string	Represents a scope’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.	true
scopeName	string	The name of the scope that the selector applies to.	true
values	[]string	An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.	false

Tenant.spec.serviceOptions

Specifies options for the Service, such as additional metadata or block of certain type of Services. Optional.

Name	Type	Description	Required
additionalMetadata	object	Specifies additional labels and annotations the Capsule operator places on any Service resource in the Tenant. Optional.	false
allowedServices	object	Block or deny certain type of Services. Optional.	false
externalIPs	object	Specifies the external IPs that can be used in Services with type ClusterIP. An empty list means no IPs are allowed. Optional.	false
forbiddenAnnotations	object	Define the annotations that a Tenant Owner cannot set for their Service resources.	false
forbiddenLabels	object	Define the labels that a Tenant Owner cannot set for their Service resources.	false

Tenant.spec.serviceOptions.additionalMetadata

Specifies additional labels and annotations the Capsule operator places on any Service resource in the Tenant. Optional.

Name	Type	Description	Required
annotations	map[string]string		false
labels	map[string]string		false

Tenant.spec.serviceOptions.allowedServices

Block or deny certain type of Services. Optional.

Name	Type	Description	Required
externalName	boolean	Specifies if ExternalName service type resources are allowed for the Tenant. Default is true. Optional. Default: true	false
loadBalancer	boolean	Specifies if LoadBalancer service type resources are allowed for the Tenant. Default is true. Optional. Default: true	false
nodePort	boolean	Specifies if NodePort service type resources are allowed for the Tenant. Default is true. Optional. Default: true	false

Tenant.spec.serviceOptions.externalIPs

Specifies the external IPs that can be used in Services with type ClusterIP. An empty list means no IPs are allowed. Optional.

Name	Type	Description	Required
allowed	[]string		true

Tenant.spec.serviceOptions.forbiddenAnnotations

Define the annotations that a Tenant Owner cannot set for their Service resources.

Name	Type	Description	Required
denied	[]string		false
deniedRegex	string		false

Tenant.spec.serviceOptions.forbiddenLabels

Define the labels that a Tenant Owner cannot set for their Service resources.

Name	Type	Description	Required
denied	[]string		false
deniedRegex	string		false

Tenant.spec.storageClasses

Specifies the allowed StorageClasses assigned to the Tenant. Capsule assures that all PersistentVolumeClaim resources created in the Tenant can use only one of the allowed StorageClasses. Optional.

Name	Type	Description	Required
allowed	[]string	Match exact elements which are allowed as class names within this tenant	false
allowedRegex	string	Deprecated: will be removed in a future release Match elements by regex.	false

Tenant.status

Returns the observed state of the Tenant.

Name	Type	Description	Required
size	integer	How many namespaces are assigned to the Tenant.	true
state	enum	The operational state of the Tenant. Possible values are “Active”, “Cordoned”. Enum: Cordoned, Active Default: Active	true
namespaces	[]string	List of namespaces assigned to the Tenant.	false

Documentation

Where to start

1 - What's New ✨

Events 📅

Swiss Dev Confederation 🇨🇭

KubeCon + CloudNativeCon North America 🇺🇸

KCD Sofia 2026 🇧🇬

Swiss Cloud Native Day 🇨🇭

Security 🔒

Breaking Changes ⚠️

Features ✨

Documentation 📚

Ecosystem 🌐

Roadmap 🗺️

2 - Overview

The Problem

How Capsule Works

Who Does What

Key Features

Capsule Controller

Get Started

3 - Quickstart 🚀

Installation

Create Your First Tenant

Tenant Owners

As a Tenant Owner

Create a namespace

Enforce label constraints

Namespace quota

List with the Proxy

Going Further

Next Steps

3.1 - Going Further

Optional

Pod Security Standards

See it in action

Service Restrictions

Permission Bindings

Resource Distribution with GlobalTenantResource

LimitRanges

Networkpolicies

Quota Management

Full Tenant

Next Steps

4 - Operating

4.1 - Setup

4.1.1 - Installation

Requirements

Installation

Production

Scalability

QPS/Burst

Workers

Cache Synchronisation

Leader Election Timeout

API Priority and Fairness (APF)

Strict RBAC

Attention

Chart versions v0.13.11 and earlier

Admission Policies

Certificate Management

Webhooks

System-critical Components

Nodes

Namespaces

Protected

GitOps

ArgoCD

Information

FluxCD

Security

Signature

Provenance

Software Bill of Materials (SBOM)

Compatibility

4.1.2 - Configuration

CapsuleConfiguration

administrators

users

ignoreUserWithGroups

Swiss Dev Confederation

KubeCon + CloudNativeCon North America

KCD Sofia 2026

Swiss Cloud Native Day

`administrators`

`users`

`ignoreUserWithGroups`

`enableTLSReconciler`

`forceTenantPrefix`

`nodeMetadata`

`overrides`

`protectedNamespaceRegex`

`allowServiceAccountPromotion`

`cacheInvalidation`

`rbac`

`impersonation`

`admission`

Keycloak new URLs without `/auth` makes Rancher crash