Kubernetes

Kubernetes is DIT's container orchestration platform. All production workloads run on Kubernetes, providing automated deployment, scaling, and management of containerized applications.

DIT clusters run Kubernetes v1.33+.

DIT Kubernetes Stack

Component	Tool	Purpose
Container Runtime	containerd	Container execution
CNI	Cilium	Networking and network policies
Gateway / Ingress	Traefik	HTTP routing and load balancing
Certificate Manager	cert-manager	TLS certificate automation
GitOps	FluxCD	Continuous delivery

Traefik Gateway

DIT uses Traefik as the ingress controller and API gateway. All external HTTP traffic is routed through Traefik, which provides:

• Automatic TLS termination via cert-manager
• HTTP/2 and gRPC support
• Request routing and load balancing
• Middleware for authentication, rate limiting, etc.

Cilium CNI

DIT uses Cilium as the Container Network Interface (CNI) provider. Cilium provides:

• eBPF-based networking for high performance
• Network policy enforcement via CiliumNetworkPolicy
• Service mesh capabilities
• Network observability

Network Policy Required

All applications must define network policy resources. You can use either CiliumNetworkPolicy or standard Kubernetes NetworkPolicy resources. See the Network Policies section below.

Security Requirements

Pod Security Standards (Restricted)

Restricted Pod Security Required

All DIT namespaces enforce the restricted Pod Security Standard. Pods that do not conform to restricted security requirements will be rejected by the admission controller.

The restricted Pod Security Standard is applied at the cluster level by default. However, you can override this policy for specific namespaces by setting the appropriate labels:

apiVersion: v1
kind: Namespace
metadata:
  name: my-app-production
  labels:
    pod-security.kubernetes.io/enforce: baseline    # Set or override enforcement level

Adjust the label values if you need to relax or further restrict policies for a given namespace.

The restricted policy requires:

• Containers must run as non-root
• Containers must not allow privilege escalation
• Containers must drop all capabilities
• Containers must use a read-only root filesystem (recommended)
• Seccomp profile must be set to RuntimeDefault or Localhost

Security Context

All pods must include security contexts that conform to the restricted policy:

spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 1000
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities:
          drop:
            - ALL

Network Policies

Network Policy Required

All applications must define network policy resources for both ingress and egress traffic. You can use either CiliumNetworkPolicy (Cilium-specific) or standard Kubernetes NetworkPolicy resources.

Network policies are typically defined in the HelmRelease values (see FluxCD documentation).

CiliumNetworkPolicy Example

DIT uses Cilium as the CNI provider, which supports advanced CiliumNetworkPolicy features:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: my-app
  namespace: my-app-production
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/name: my-app

  ingress:
    - fromEndpoints:
        - matchLabels:
            app.kubernetes.io/instance: traefik-traefik
            io.kubernetes.pod.namespace: traefik
      toPorts:
        - ports:
            - port: "8080"

  egress:
    # DNS resolution (required for all apps)
    - toEndpoints:
        - matchLabels:
            k8s-app: kube-dns
            io.kubernetes.pod.namespace: kube-system
      toPorts:
        - ports:
            - port: "53"
          rules:
            dns:
              - matchPattern: "*"

    # External HTTPS (e.g., example.com)
    - toFQDNs:
        - matchName: example.com
      toPorts:
        - ports:
            - port: "443"

    # Internal service communication
    - toEndpoints:
        - matchLabels:
            app.kubernetes.io/name: redis
      toPorts:
        - ports:
            - port: "6379"

Key CiliumNetworkPolicy features:

• Label-based selection – Use matchLabels with io.kubernetes.pod.namespace for cross-namespace rules
• FQDN egress – Use toFQDNs for external services
• DNS rules – DNS egress is required for most applications
• Layer 7 policies – Cilium supports HTTP-aware policies

Choosing Between CiliumNetworkPolicy and NetworkPolicy

• Use CiliumNetworkPolicy when you need advanced features like FQDN-based egress rules, Layer 7 policies, or DNS matching
• Use standard NetworkPolicy for simpler use cases or when portability across different CNI providers is desired

Resource Management

Resource Requests and Limits

All containers must specify resource requests and limits:

resources:
  requests:
    memory: "256Mi"
    cpu: "100m"
  limits:
    memory: "512Mi"
    cpu: "500m"

Resource Quotas

Namespaces have resource quotas enforced:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: my-app-production
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi
    pods: "20"

Limit Ranges

Default limits are applied to containers without explicit specifications:

apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: my-app-production
spec:
  limits:
    - default:
        memory: "256Mi"
        cpu: "200m"
      defaultRequest:
        memory: "128Mi"
        cpu: "100m"
      type: Container

Health Checks

All applications must expose the following probe endpoints:

Endpoint	Probe	Purpose
/health	Liveness	Is the application running? (restart if fails)
/status	Readiness & Startup	Is the application ready to serve traffic?

Separate Endpoints Required

Liveness and readiness probes must use different endpoints.

Liveness Probes

Detect when a container needs to be restarted. Use the /health endpoint:

livenessProbe:
  httpGet:
    path: /health
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 3
  failureThreshold: 3

Readiness Probes

Determine when a container is ready to accept traffic. Use the /status endpoint:

readinessProbe:
  httpGet:
    path: /status
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5
  timeoutSeconds: 3
  failureThreshold: 3

Startup Probes

For applications with slow startup times, use the /status or a custom endpoint:

startupProbe:
  httpGet:
    path: /status
    port: 8080
  failureThreshold: 30
  periodSeconds: 10

Checklist

Before deploying to production, verify:

• Deployment uses specific image tags (no latest)
• Resource requests and limits are defined
• Security context conforms to restricted Pod Security standards
• Liveness and readiness probes are defined (/health and /status)
• Network policies (CiliumNetworkPolicy or NetworkPolicy) are defined for ingress and egress
• Secrets are managed via Infisical + Sealed Secrets

---

By following these Kubernetes standards, DIT ensures secure, scalable, and maintainable container orchestration across all environments.