Capítulo 38 — Kubernetes e Orquestração de Containers

┌─────────────────────────────────────────────────────────┐
│                  Control Plane                            │
│                                                          │
│  kube-apiserver                                          │
│  ├─ API REST exposta em HTTPS                           │
│  ├─ Valida e processa requests                          │
│  └─ Persiste estado em etcd                              │
│                                                          │
│  etcd                                                    │
│  ├─ Banco chave-valor distribuído                        │
│  ├─ Source of truth do cluster                          │
│  └─ Réplicas para HA (3 ou 5 nós)                       │
│                                                          │
│  kube-scheduler                                          │
│  ├─ Decide em qual nó pods executam                     │
│  ├─ Considera recursos, afinidade, taints               │
│  └─ Recalcula continuamente                             │
│                                                          │
│  kube-controller-manager                                 │
│  ├─ ReplicaSet controller                               │
│  ├─ Deployment controller                              │
│  ├─ Service controller                                  │
│  ├─ Node controller                                     │
│  ├─ Namespace controller                                │
│  └─ ... (mais de 30 controllers)                        │
│                                                          │
│  cloud-controller-manager (opcional)                    │
│  ├─ Integração com cloud provider                       │
│  └─ Provisiona LoadBalancer, PersistentDisk             │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│                  Worker Node                              │
│                                                          │
│  kubelet                                                 │
│  ├─ Agente que reporta ao control plane                 │
│  ├─ Inicia pods via container runtime                   │
│  └─ Health checks (liveness, readiness)                 │
│                                                          │
│  kube-proxy                                              │
│  ├─ Mantém regras iptables/IPVS                        │
│  └─ Habilita comunicação entre pods e Services          │
│                                                          │
│  container runtime (containerd ou CRI-O)                │
│  ├─ Executa containers                                  │
│  └─ Puxa imagens de registries                          │
│                                                          │
│  Pods                                                    │
│  ├─ Menor unidade implantável                            │
│  ├─ 1+ containers que compartilham network/storage      │
│  └─ Scheduling atômico                                  │
└─────────────────────────────────────────────────────────┘

Zone A         Zone B         Zone C
kube-apiserver  kube-apiserver  kube-apiserver
etcd (a)        etcd (b)        etcd (c)
scheduler       scheduler       scheduler

Workers distribuídos:
node-pool-app-1 (a)   node-pool-app-1 (b)   node-pool-app-1 (c)
node-pool-data-1 (a)  node-pool-data-1 (b)  node-pool-data-1 (c)

apiVersion: v1
kind: Pod
metadata:
  name: api-solicitacoes-7d4b
  namespace: tenant-org-a
  labels:
    app: api-solicitacoes
    tier: backend
    tenant: org-a
spec:
  containers:
  - name: api
    image: registry.example.com/api-solicitacoes:1.2.3
    ports:
    - containerPort: 8080
      name: http
    env:
    - name: DB_HOST
      valueFrom:
        secretKeyRef:
          name: db-credentials
          key: host
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: db-credentials
          key: password
    resources:
      requests:
        cpu: 250m
        memory: 512Mi
      limits:
        cpu: 1000m
        memory: 2Gi
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
      initialDelaySeconds: 30
      periodSeconds: 10
    readinessProbe:
      httpGet:
        path: /ready
        port: 8080
      initialDelaySeconds: 5
      periodSeconds: 5
    securityContext:
      runAsNonRoot: true
      runAsUser: 1000
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop: [ALL]
    volumeMounts:
    - name: tmp
      mountPath: /tmp
  volumes:
  - name: tmp
    emptyDir: {}

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-solicitacoes
  namespace: tenant-org-a
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-solicitacoes
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: api-solicitacoes
        tier: backend
        tenant: org-a
    spec:
      containers:
      - name: api
        image: registry.example.com/api-solicitacoes:1.2.3
        # ... (como no pod spec)

Estado inicial: 3 réplicas com versão v1
Estado desejado: 3 réplicas com versão v2

Kubernetes:
1. Cria 1 réplica v2 (maxSurge: 1)
   Estado: 3 v1 + 1 v2
2. Termina 1 réplica v1 (maxUnavailable: 0)
   Estado: 2 v1 + 1 v2 + 1 v2 = 2 v1 + 2 v2
3. Cria mais 1 réplica v2
   Estado: 2 v1 + 2 v2 + 1 v2 = 2 v1 + 3 v2
4. Termina mais 1 réplica v1
   Estado: 1 v1 + 3 v2
... continua até 0 v1 + 3 v2

Liveness probe falha
  │
  └─ Kubernetes mata o pod
      │
      └─ ReplicaSet cria novo pod

Readiness probe falha
  │
  └─ Pod removido de Service endpoints
      │
      └─ Tráfego não chega ao pod

startupProbe:
  httpGet:
    path: /healthz
    port: 8080
  failureThreshold: 30
  periodSeconds: 10

Service com IP interno do cluster
  │
  └─ Roteamento para pods via selector
       │
       └─ kube-proxy atualiza iptables/IPVS

Service exposto em porta fixa em cada node
  │
  └─ ClusterIP + porta em cada node (30000-32767)
       │
       └─ Acessível via <node-ip>:<node-port>

Service provisiona Load Balancer externo
  │
  └─ Cloud provider cria LB
       │
       └─ Tráfego roteado para nodes/pods

Service exposto via Ingress Controller
  │
  └─ Roteamento baseado em host/path
       │
       └─ TLS termination, rate limiting, etc.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: portal-cidadao
  namespace: tenant-org-a
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/rate-limit: "100"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - portal.org-a.example.com
    secretName: portal-tls
  rules:
  - host: portal.org-a.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: frontend-portal
            port:
              number: 80
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: api-gateway
            port:
              number: 8080

Service A → Service B
     │
     └─ Sidecar proxy intercepta
          │
          ├─ mTLS automático
          ├─ Tracing distribuído
          ├─ Retry policies
          └─ Circuit breaking

Namespaces:
- tenant-org-a
- tenant-org-b
- tenant-org-c
- platform-system (recursos compartilhados)
- monitoring (Prometheus, Grafana)
- logging (Elasticsearch, Kibana)
- ingress-nginx
- cert-manager

apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-org-a-quota
  namespace: tenant-org-a
spec:
  hard:
    requests.cpu: "20"
    requests.memory: 40Gi
    limits.cpu: "40"
    limits.memory: 80Gi
    persistentvolumeclaims: "10"
    pods: "100"
    services: "50"
    secrets: "100"
    configmaps: "100"
    loadbalancers: "5"  # limite de LBs externos

apiVersion: v1
kind: LimitRange
metadata:
  name: tenant-org-a-limits
  namespace: tenant-org-a
spec:
  limits:
  - type: Container
    default:
      cpu: 500m
      memory: 1Gi
    defaultRequest:
      cpu: 100m
      memory: 256Mi
    max:
      cpu: 2
      memory: 4Gi
    min:
      cpu: 50m
      memory: 64Mi

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: tenant-org-a-isolation
  namespace: tenant-org-a
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          tenant: org-a
    - namespaceSelector:
        matchLabels:
          name: ingress-nginx
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          tenant: org-a
  - to:
    - namespaceSelector:
        matchLabels:
          name: kube-system
    ports:
    - protocol: UDP
      port: 53

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: tenant-org-a
data:
  LOG_LEVEL: "INFO"
  SERVICE_NAME: "api-solicitacoes"
  CACHE_TTL_SECONDS: "300"
  FEATURE_FLAG_NEW_UI: "true"

spec:
  containers:
  - name: api
    envFrom:
    - configMapRef:
        name: app-config

apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
  namespace: tenant-org-a
type: Opaque
stringData:
  username: app_user
  password: super-secret-password-123

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: tenant-org-a
spec:
  refreshInterval: 5m
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: db-credentials
  data:
  - secretKey: username
    remoteRef:
      key: tenant-org-a/db
      property: username
  - secretKey: password
    remoteRef:
      key: tenant-org-a/db
      property: password

chart: api-solicitacoes
├── Chart.yaml          (metadata da chart)
├── values.yaml         (configuração padrão)
├── templates/
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── ingress.yaml
│   ├── configmap.yaml
│   ├── hpa.yaml
│   └── _helpers.tpl
├── values-dev.yaml     (override para dev)
├── values-staging.yaml (override para staging)
└── values-prod.yaml    (override para prod)

# Chart.yaml
apiVersion: v2
name: api-solicitacoes
description: API de solicitações
type: application
version: 1.2.3
appVersion: "1.2.3"

# values.yaml
replicaCount: 3
image:
  repository: registry.example.com/api-solicitacoes
  tag: "1.2.3"
  pullPolicy: IfNotPresent
service:
  type: ClusterIP
  port: 80
ingress:
  enabled: true
  className: nginx
  hosts:
    - host: api.org-a.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: api-tls
      hosts:
        - api.org-a.example.com
resources:
  requests:
    cpu: 250m
    memory: 512Mi
  limits:
    cpu: 1000m
    memory: 2Gi
autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 30
  targetCPUUtilizationPercentage: 70

# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "api-solicitacoes.fullname" . }}
  namespace: {{ .Values.namespace }}
  labels:
    {{- include "api-solicitacoes.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      {{- include "api-solicitacoes.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      labels:
        {{- include "api-solicitacoes.selectorLabels" . | nindent 8 }}
    spec:
      containers:
      - name: {{ .Chart.Name }}
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
        imagePullPolicy: {{ .Values.image.pullPolicy }}
        resources:
          {{- toYaml .Values.resources | nindent 10 }}

helm install api-solicitacoes ./chart \
  --namespace tenant-org-a \
  --values values-prod.yaml \
  --set image.tag=1.2.3

helm history api-solicitacoes -n tenant-org-a
REVISION  UPDATED                   STATUS     CHART             DESCRIPTION
1         2026-07-01 10:30:00       deployed   api-solicitacoes-1.2.3  Install complete
2         2026-07-08 14:15:00       deployed   api-solicitacoes-1.2.4  Upgrade complete
3         2026-07-15 09:00:00       superseded api-solicitacoes-1.3.0  Upgrade complete

helm rollback api-solicitacoes 2 -n tenant-org-a

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-solicitacoes-hpa
  namespace: tenant-org-a
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-solicitacoes
  minReplicas: 3
  maxReplicas: 30
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "100"
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 30
      policies:
      - type: Percent
        value: 100
        periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 10
        periodSeconds: 60

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: api-solicitacoes-vpa
  namespace: tenant-org-a
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-solicitacoes
  updatePolicy:
    updateMode: Auto
  resourcePolicy:
    containerPolicies:
    - containerName: api
      minAllowed:
        cpu: 100m
        memory: 256Mi
      maxAllowed:
        cpu: 2
        memory: 4Gi

apiVersion: autoscaling.k8s.io/v1
kind: ClusterAutoscaler
metadata:
  name: cluster-autoscaler
  namespace: kube-system
spec:
  scaleDown:
    enabled: true
    delayAfterAdd: 10m
    delayAfterDelete: 10m
    unneededTime: 5m
  minNodes: 6
  maxNodes: 50
  nodeGroups:
  - name: node-pool-app
    minSize: 3
    maxSize: 20
    instanceType: m5.large
  - name: node-pool-data
    minSize: 3
    maxSize: 10
    instanceType: r5.xlarge

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: workflow-processor
  namespace: tenant-org-a
spec:
  scaleTargetRef:
    name: workflow-processor
  minReplicaCount: 2
  maxReplicaCount: 50
  triggers:
  - type: rabbitmq
    metadata:
      host: rabbitmq.platform.svc.cluster.local
      protocol: amqp
      queueName: workflow.pending
      mode: QueueLength
      value: "10"

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: elasticsearch-data
  namespace: tenant-org-a
spec:
  serviceName: elasticsearch-data
  replicas: 3
  selector:
    matchLabels:
      app: elasticsearch-data
  template:
    metadata:
      labels:
        app: elasticsearch-data
    spec:
      containers:
      - name: elasticsearch
        image: docker.elastic.co/elasticsearch/elasticsearch:8.10.0
        volumeMounts:
        - name: data
          mountPath: /usr/share/elasticsearch/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ReadWriteOnce]
      storageClassName: gp3
      resources:
        requests:
          storage: 100Gi

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-data
  namespace: tenant-org-a
spec:
  accessModes:
  - ReadWriteOnce
  storageClassName: gp3
  resources:
    requests:
      storage: 100Gi

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3-replicated
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  iops: "3000"
  throughput: "125"
  encrypted: "true"
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Retain
allowVolumeExpansion: true

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: developer
  namespace: tenant-org-a
rules:
- apiGroups: [""]
  resources: ["pods", "services", "configmaps"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "update", "patch"]

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: developer-binding
  namespace: tenant-org-a
subjects:
- kind: User
  name: alice@example.com
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: developer
  apiGroup: rbac.authorization.k8s.io

apiVersion: v1
kind: Namespace
metadata:
  name: tenant-org-a
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
spec:
  validations:
  - expression: "object.spec.containers.all(c, c.image.startsWith('registry.example.com/'))"

spec:
  imagePullSecrets:
  - name: registry-credentials
  containers:
  - name: api
    image: registry.example.com/api-solicitacoes:1.2.3

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: tenant-org-a
spec:
  mtls:
    mode: STRICT

Git repository (source of truth)
    │
    └─ Argo CD monitora mudanças
         │
         ├─ Detecta divergência
         │
         └─ Sincroniza cluster para estado declarado
              │
              └─ Auto-sync ou manual approval

platform-gitops/
├── apps/
│   ├── api-solicitacoes/
│   │   ├── base/
│   │   │   ├── kustomization.yaml
│   │   │   ├── deployment.yaml
│   │   │   └── service.yaml
│   │   └── overlays/
│   │       ├── dev/
│   │       ├── staging/
│   │       └── prod/
│   ├── frontend-portal/
│   └── ...
└── infrastructure/
    ├── namespaces/
    ├── network-policies/
    ├── rbac/
    └── secrets-external/

# apps/api-solicitacoes/base/kustomization.yaml
resources:
- deployment.yaml
- service.yaml
- ingress.yaml

# apps/api-solicitacoes/overlays/prod/kustomization.yaml
bases:
- ../../base
replicas:
- name: api-solicitacoes
  count: 10
images:
- name: api-solicitacoes
  newTag: 1.2.3

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: api-solicitacoes-org-a
  namespace: argocd
spec:
  project: platform
  source:
    repoURL: https://git.example.com/platform-gitops.git
    targetRevision: main
    path: apps/api-solicitacoes/overlays/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: tenant-org-a
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=false

# Listar pods
kubectl get pods -n tenant-org-a

# Logs de um pod
kubectl logs api-solicitacoes-7d4b -n tenant-org-a --tail=100

# Logs de múltiplos pods (Deployment)
kubectl logs -l app=api-solicitacoes -n tenant-org-a --tail=100

# Executar shell em pod
kubectl exec -it api-solicitacoes-7d4b -n tenant-org-a -- /bin/sh

# Descrever pod (events, status)
kubectl describe pod api-solicitacoes-7d4b -n tenant-org-a

# Eventos recentes
kubectl get events -n tenant-org-a --sort-by='.lastTimestamp'

# Uso de recursos
kubectl top pods -n tenant-org-a
kubectl top nodes

# Estado de deployments
kubectl rollout status deployment/api-solicitacoes -n tenant-org-a

# Histórico de rollout
kubectl rollout history deployment/api-solicitacoes -n tenant-org-a

# Rollback
kubectl rollout undo deployment/api-solicitacoes -n tenant-org-a

# Port forward para debug
kubectl port-forward api-solicitacoes-7d4b 8080:8080 -n tenant-org-a

# 1. Verificar logs
kubectl logs api-solicitacoes-7d4b -n tenant-org-a

# 2. Verificar events (OOM, ImagePullBackOff, etc.)
kubectl describe pod api-solicitacoes-7d4b -n tenant-org-a

# 3. Cenários comuns:
# - ImagePullBackOff: registry inacessível ou tag incorreta
# - OOMKilled: aumentar memory limit
# - CrashLoopBackOff: bug na aplicação (ver stack trace em logs)

# 1. Verificar pods do deployment
kubectl get pods -l app=api-solicitacoes -n tenant-org-a

# 2. Verificar endpoints do Service
kubectl get endpoints api-solicitacoes -n tenant-org-a

# 3. Cenários comuns:
# - Selector incorreto: pods não match
# - Porta incorreta no Service vs Pod
# - Readiness probe falhando

# 1. Verificar uso de recursos
kubectl top pods -n tenant-org-a
kubectl top nodes

# 2. Verificar métricas de aplicação
# Via Prometheus/Grafana

# 3. Verificar throttling de CPU
# dmesg ou container logs podem mostrar throttling

1.17 → 1.18 → 1.19 → 1.20 → 1.21

1. Ler release notes (breaking changes)
2. Atualizar cluster de desenvolvimento
3. Validar suite de testes
4. Atualizar cluster de staging
5. Smoke tests em staging (1 semana)
6. Atualizar cluster de produção
   a. Upgrade control plane (1 nó por vez)
   b. Aguardar estabilização
   c. Upgrade worker nodes (drain + replace)
7. Validar workloads em produção

Node em produção
  │
  ├─ kubectl drain (remove pods)
  │   └─ Pods são realocados para outros nodes
  │
  ├─ Terminate node
  │
  ├─ Novo node provisionado com nova versão
  │
  └─ kubectl uncordon (permite novos pods)

Operator (controller customizado)
  │
  ├─ Observa recursos customizados (CRDs)
  │
  ├─ Reconcilia estado desejado
  │
  └─ Aplica ações (create, update, delete)

apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
  name: tenant-org-a-db
  namespace: tenant-org-a
spec:
  teamId: "platform"
  volume:
    size: 100Gi
    storageClass: gp3
  numberOfInstances: 3
  users:
    app_user:
    - superuser
    - createdb
  databases:
    tenant_org_a: app_user
  postgresql:
    version: "15"
  resources:
    requests:
      cpu: 500m
      memory: 2Gi
    limits:
      cpu: 2
      memory: 8Gi