Log in to the cluster with SSO

Kubernetes has no user database — there's no User object and no command to create one. The API server only knows how to verify an identity that your Identity Provider (IdP) already vouches for.

Authentication flow: you (kubectl + kubelogin) log in → the IdP (Keycloak/Azure AD/Okta) returns an ID token → you send that token straight to the cluster's API server → the API server verifies the signature against the IdP's JWKS → Kubernetes RBAC decides what you can do.

note

The token goes directly from you to the API server — the IdP isn't in the request path itself. If the IdP goes down temporarily, a token you already cached still works until it expires, and the cluster keeps running normally.

warning

Logging in successfully doesn't mean you have permission to do anything. A user who authenticates but has no RoleBinding gets 403 Forbidden on a command like kubectl get pods. This is expected behavior, not a misconfiguration.

Three tiers, not one

Getting a GPU workload running for your team involves 3 separate layers — mixing them up is the most common source of confusion:

Tier	Answers	Configured via
Authentication	Who are you?	OIDC (Issuer URL, Client ID) — set once by the Cluster Admin
Authorization	What can you do?	Kubernetes RBAC (RoleBinding / ClusterRoleBinding) — this guide
Quota	How much can you use?	`ResourceQuota` / `LimitRange` (CPU, RAM, and `requests.nvidia.com/gpu`) — this guide

A user can pass authentication and still be blocked by either of the other two: authorized for a namespace but with no quota left gets a scheduling error, not a permissions error.

Before you start

Item	Requirement
Cluster	An admin has already enabled OIDC authentication — see Configure OIDC authentication for the cluster.
Tools	`kubectl` and `kubelogin` installed on your machine.
Network	Your machine can reach both the cluster's API server and the IdP.
HTTPS	The IdP has a valid certificate — if it uses an internal CA, import that CA on your machine.
Account	A username and password on the IdP, already added to the appropriate group.

You also need 4 pieces of information from your cluster administrator — you can't build a kubeconfig without them:

Information	Used for
API server URL	The `server` field in the kubeconfig.
Cluster CA certificate	The `certificate-authority-data` field.
Issuer URL (IdP)	The same Issuer URL the admin entered when configuring OIDC on the Portal.
Client ID	The same Client ID the admin entered when configuring OIDC on the Portal.

Install kubelogin

Install krew (the kubectl plugin manager)

Follow the official krew install guide at krew.sigs.k8s.io.

Add krew to your PATH

echo 'export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
# Using zsh: replace ~/.bashrc with ~/.zshrc

# Confirm
kubectl krew version

warning

PATH changes in ~/.bashrc only apply to an interactive shell. A command run over ssh host 'kubectl ...' or from a script doesn't load that file. In those cases, add the path explicitly: export PATH="$HOME/.krew/bin:$PATH"; kubectl get nodes.

kubectl krew install oidc-login

# Upgrade later
kubectl krew upgrade oidc-login

# See installed version
kubectl krew list

Verify the installation

# 1. Does the binary run?
kubectl oidc-login --version
# kubelogin version v1.36.4

# 2. Does kubectl recognize the plugin?
kubectl plugin list
# /root/.krew/bin/kubectl-oidc_login

# 3. Can you reach the IdP? (replace with your real issuer URL)
curl -sI https://sso.example.com/realms/kubernetes/.well-known/openid-configuration | head -1
# HTTP/2 200

Install error	Cause	Fix
`unknown command "krew"`	Step 1 wasn't run, or Step 2 was skipped	Add `~/.krew/bin` to `PATH`, then `source ~/.bashrc`
`unknown command "oidc-login"`	Plugin not installed, or `~/.krew/bin` missing from `PATH`	Run `kubectl krew list` to check, then revisit Step 2
Works manually, fails over SSH	`~/.bashrc` isn't loaded in a non-interactive shell	Prepend `export PATH="$HOME/.krew/bin:$PATH"` to the command
kubelogin version is very old	Installed a while ago, never updated	`kubectl krew upgrade oidc-login`

Create a kubeconfig for the user

An OIDC kubeconfig contains no password or token — it only describes how to call kubelogin to get one. That means a single file works for the whole team: each person logs in with their own account and gets exactly their own permissions.

Full sample file

apiVersion: v1
kind: Config
current-context: my-gpu-cluster-sso

clusters:
  - name: my-gpu-cluster
    cluster:
      server: https://api.my-gpu-cluster.example.com
      certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0t...

contexts:
  - name: my-gpu-cluster-sso
    context:
      cluster: my-gpu-cluster
      user: sso-user
      namespace: ml-training

users:
  - name: sso-user
    user:
      exec:
        apiVersion: client.authentication.k8s.io/v1beta1
        command: kubectl
        args:
          - oidc-login
          - get-token
          - --oidc-issuer-url=https://sso.example.com/realms/kubernetes
          - --oidc-client-id=kubernetes
          - --oidc-extra-scope=email
          - --oidc-extra-scope=profile
          # Machine with no browser: add the 2 lines below
          # - --grant-type=device-code
          # - --skip-open-browser
        interactiveMode: IfAvailable
        installHint: |
          kubelogin isn't installed. Install it with:
            kubectl krew install oidc-login

tip

--oidc-issuer-url and --oidc-client-id are the same Issuer URL and Client ID values your cluster administrator configured on the Portal (see Configure OIDC authentication for the cluster) — ask your administrator for these two exact values.

File structure — three blocks, joined by name

clusters   ──┐   which cluster       (address + CA)
             ├── contexts ── links a cluster to a user, plus a default namespace
users      ──┘   how to log in

current-context selects which context is active. A single file can hold multiple clusters and contexts — switch between them with kubectl config use-context.

The clusters block:

Field	Meaning
`server`	The API server's HTTPS address. Ask your administrator.
`certificate-authority-data`	The cluster's CA, base64-encoded. Used to verify the API server's certificate — not a secret.
`certificate-authority`	Alternative to the field above, pointing to a `.crt` file path. Use only one of the two.
`insecure-skip-tls-verify`	Avoid this. Skips certificate verification — only for a disposable lab environment.

The contexts block:

Field	Meaning
`cluster`	Cluster name — must match an entry in `clusters`.
`user`	User name — must match an entry in `users`.
`namespace`	Default namespace when you don't pass `-n`.

The exec block, field by field

This block decides how you log in. kubectl runs the command with args, then reads the token from its output.

exec fields

Field	Value	Meaning
`apiVersion`	`client.authentication.k8s.io/v1beta1`	Exec credential protocol version
`command`	`kubectl`	The command that gets called (since kubelogin is a plugin)
`args`	a list	Arguments passed to the command
`interactiveMode`	`IfAvailable`	When it's allowed to take over the terminal
`installHint`	text	Message shown when kubelogin isn't installed

Required arguments

Argument	Meaning
`oidc-login get-token`	The plugin name and subcommand — must come first, in this order
`--oidc-issuer-url`	IdP address — must match the cluster's configuration character for character
`--oidc-client-id`	The application identifier registered on the IdP — also the `aud` value the API server checks

Commonly used optional arguments

Argument	When you need it
`--oidc-client-secret`	Only when the client is confidential
`--oidc-extra-scope`	Request an extra scope, usually `email` and `profile`
`--grant-type`	`auto` (default), `authcode`, `device-code`, `password`
`--skip-open-browser`	Machine with no browser
`--certificate-authority`	CA for the IdP (different from the cluster's CA)
`--force-refresh`	Ignore the cache and get a fresh token — useful right after a group change
`-v=1` … `-v=5`	Log level; use `-v=5` to troubleshoot

warning

Don't add --oidc-extra-scope=groups. Group membership comes from a protocol mapper your IdP administrator configures, not from a scope. Requesting a groups scope the IdP doesn't define fails with invalid_scope.

Token caching

The first run saves the token to ~/.kube/cache/oidc-login. Later runs read from the cache and auto-refresh, only prompting you to log in again once the refresh token expires.

# Clear the cache when you need to force a re-login (required after a group change)
rm -rf ~/.kube/cache/oidc-login          # Linux/macOS

Remove-Item -Recurse -Force "$env:USERPROFILE\.kube\cache\oidc-login"   # Windows

Confirm you're logged in

kubectl auth whoami

ATTRIBUTE   VALUE
Username    oidc:nguyenvana
Groups      [oidc:ml-team system:authenticated]

Seeing the correct username and group means authentication is working. If you then get 403, that's an authorization issue — see Assign Kubernetes permissions.

Machines without a browser

A jump host, server, or container can't open a browser. Using the default kubeconfig there hangs and eventually fails with context deadline exceeded.

Device code (recommended): add 2 lines to args:
```
- --grant-type=device-code
- --skip-open-browser
```
The terminal prints a link with a code already filled in — open it on your phone or another laptop to log in, then return to the terminal and wait a few seconds. The code is valid for 10 minutes.
SSH port forwarding: keep the default kubeconfig, connect with ssh -L 8000:localhost:8000 -L 18000:localhost:18000 user@jumphost, then run kubectl normally on the jump host.
Authcode keyboard: --grant-type=authcode-keyboard — open the URL, log in, then paste the returned code into the terminal.
Password grant: --grant-type=password --username=... — doesn't work if the account has MFA enabled, and is deprecated. Use only for a quick test.

note

Don't use OIDC login for CI/CD pipelines — use a ServiceAccount token instead (see use case 7 below).

Assign Kubernetes permissions

Mapping rule

The cluster administrator configures a prefix (the Username Prefix / Groups Prefix entered under Advanced when configuring OIDC on the Portal) that's added to every identity coming from the IdP:

On the IdP	Claim in the token	Name used in RBAC
User `nguyenvana`	`preferred_username`	`oidc:nguyenvana`
Group `ml-team`	`groups[]`	`oidc:ml-team`

Not sure what the prefix is? Run kubectl auth whoami — the value it prints is exactly the name to use in RBAC.

tip

Always assign permissions by Group, not by User. Binding to kind: User means editing RBAC on every cluster each time someone joins or leaves the team. Binding to kind: Group means you only add or remove members on the IdP. The one exception is a break-glass account.

Four built-in ClusterRoles

ClusterRole	Grants	Typical use
`cluster-admin`	Full access, every namespace	Infrastructure administrators
`admin`	Full access within 1 namespace, including creating RoleBindings	MLOps team leads
`edit`	Edit resources, but not RBAC itself	Data scientists and ML engineers running training/inference jobs
`view`	Read-only, excludes Secrets	Reviewers, observers

A ClusterRoleBinding grants cluster-wide access; a RoleBinding scopes access to 1 namespace (and can still reference a ClusterRole).

Common use cases

Cluster-wide administrator

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: sso-cluster-admins
subjects:
  - kind: Group
    name: "oidc:gpu-admins"
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

Cluster-wide read-only access

Same as use case 1, but with roleRef.name: view. The view ClusterRole intentionally excludes reading Secrets.

A team owns its own namespace

Team lead gets admin, data scientists/ML engineers get edit:

# ML team lead
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ml-training-leads
  namespace: ml-training
subjects:
  - kind: Group
    name: "oidc:ml-team-lead"
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: admin
  apiGroup: rbac.authorization.k8s.io
---
# Data scientists / ML engineers
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ml-training-users
  namespace: ml-training
subjects:
  - kind: Group
    name: "oidc:ml-team"
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: edit
  apiGroup: rbac.authorization.k8s.io

Allow listing namespaces

Namespace is a cluster-scoped resource, so a RoleBinding alone isn't enough:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: namespace-lister
rules:
  - apiGroups: [""]
    resources: ["namespaces"]
    verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: all-users-list-namespaces
subjects:
  - kind: Group
    name: "system:authenticated"
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: namespace-lister
  apiGroup: rbac.authorization.k8s.io

warning

This exposes every namespace's name to anyone who's logged in.

Multiple teams sharing a cluster, with GPU quota

RBAC alone doesn't limit how much a team can use — pair it with a ResourceQuota that covers GPU:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: ml-team-quota
  namespace: ml-training
spec:
  hard:
    requests.cpu: "40"
    requests.memory: 160Gi
    requests.nvidia.com/gpu: "8"
    persistentvolumeclaims: "20"
    count/deployments.apps: "50"

warning

A GPU quota is mandatory when multiple teams share a cluster — without it, one team can claim every GPU with a single Deployment.

Narrow permission on one resource type

For example, an on-call team that can view logs and restart training Pods, but can't edit Deployments or read Secrets:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: oncall-responder
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/log", "events"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["delete"]        # restart = delete the pod, the ReplicaSet recreates it
  - apiGroups: ["apps"]
    resources: ["deployments", "statefulsets"]
    verbs: ["get", "list"]

CI/CD with a ServiceAccount, not SSO

For a training/deployment pipeline that submits jobs on a schedule:

kubectl -n ml-training create serviceaccount ci-deployer
kubectl -n ml-training create rolebinding ci-deployer \
  --clusterrole=edit --serviceaccount=ml-training:ci-deployer
kubectl -n ml-training create token ci-deployer --duration=24h

Since Kubernetes 1.24, a ServiceAccount no longer auto-generates a Secret with a permanent token — tokens created with create token are time-limited and safer.

Revoking access when someone changes role

Situation	Action	Takes effect
Moves teams	Change their group on the IdP	Next token refresh
Leaves the company	Disable their IdP account	At most one token lifetime
Needs to be blocked immediately	Disable the account and kill their session	Immediately, for new sessions

danger

A token that's already been issued can't be revoked — this is an inherent limit of bearer tokens. The risk window equals the token's lifetime. For sensitive systems, set that lifetime to 5–10 minutes.

Troubleshooting

401 vs. 403 — always check this first

Code	Meaning	Broken layer	Who fixes it
401	Unauthorized	Authentication — the token wasn't accepted	Cluster admin / IdP
403	Forbidden	Authorization — the token is valid, but lacks permission	Cluster admin (RBAC)

warning

Don't touch RBAC while you're getting a 401. A 401 means the API server never recognized your identity in the first place — adding RoleBindings won't help.

Common errors

Symptom	Cause	Fix
`401` even though login succeeded	Issuer URL in the kubeconfig doesn't match the cluster's configuration, or OIDC isn't enabled on the cluster	Compare the issuer URL character by character; contact your cluster admin
`403`, and `whoami` shows the right name	No RoleBinding yet, or the group name in RBAC doesn't match	Send the `kubectl auth whoami` output to your cluster admin
`Groups` is empty in `whoami`	The IdP hasn't configured a mapper that pushes groups into the token	Contact your IdP administrator
`context deadline exceeded`	kubelogin is waiting for a browser in a non-interactive session	Use `--grant-type=device-code`
`invalid_scope`	Requesting a scope the IdP doesn't define	Remove `--oidc-extra-scope=groups`
`whoami` shows an unexpected identity	The `KUBECONFIG` variable wasn't switched	Set `$env:KUBECONFIG`, or pass `--kubeconfig`
Permissions didn't change after a group change	The old token is still cached	Clear `~/.kube/cache/oidc-login`
`invalid_grant`	Wrong password, or the account is disabled	Check the account on the IdP
`x509: certificate signed by unknown authority`	The IdP uses an internal CA your machine doesn't trust	Add `--certificate-authority=/path/to/ca.crt`
`address already in use`	Port 8000 is already taken	Add `--listen-address=127.0.0.1:8100` and register the new redirect URI
`Missing parameter: code_challenge_method`	The IdP requires PKCE on the device endpoint	Ask your admin to relax the PKCE requirement, or use SSH port forwarding instead
`could not open the browser: exit status 3`	Not an error — the machine has no browser	Look for the "Please visit..." line and open that link manually

Three diagnostic commands, in order

# 1. Who does the API server think I am?
kubectl auth whoami

# 2. Can I do this specific thing?
kubectl auth can-i create deployments -n ml-training

# 3. See everything I can do in one namespace
kubectl auth can-i --list -n ml-training

kubectl oidc-login get-token -v=5 \
  --oidc-issuer-url=https://sso.example.com/realms/kubernetes \
  --oidc-client-id=kubernetes

Getting a token back means the IdP side is fine and the problem is on the cluster side. Failing to get a token means the problem is with the IdP or the network path to it.

Security checklist

Never share a kubeconfig that contains a token. An OIDC kubeconfig is safe to distribute to the whole team because it holds no secret — but never distribute one that already contains a token: or client-key-data: field.
Don't use insecure-skip-tls-verify outside a disposable lab environment.
Enable MFA for any account with cluster-admin permissions.
Clear the token cache when you step away from a shared machine: rm -rf ~/.kube/cache/oidc-login.
Set a default namespace in the context to avoid accidentally operating in the wrong namespace.
Start from view. Request more access only when you have a specific need — don't ask for cluster-admin for convenience.
Assign permissions by group, not by user. The one exception is a break-glass account.
Always set a ResourceQuota for requests.nvidia.com/gpu whenever multiple teams share a cluster — without it, RBAC alone can't stop one team from claiming every GPU.
Keep a break-glass admin kubeconfig. If the IdP goes down, you still need a way into the cluster to fix it — store it somewhere secure.
Review regularly. Every quarter, reconcile IdP group membership against your actual team roster.

This guide assumes a cluster with OIDC authentication already enabled through the Portal (see Configure OIDC authentication for the cluster), with kubelogin installed through krew on Linux. Configuring the Identity Provider itself and enabling OIDC on the cluster are your cluster administrator's responsibility — contact them for the 4 pieces of information listed under Before you start.

Three tiers, not one​

Before you start​

Install kubelogin​

Install krew (the kubectl plugin manager)​

Add krew to your PATH​

Install the oidc-login plugin​

Verify the installation​

Create a kubeconfig for the user​

Full sample file​

File structure — three blocks, joined by name​

The exec block, field by field​

exec fields​

Required arguments​

Commonly used optional arguments​

Token caching​

Confirm you're logged in​

Machines without a browser​

Assign Kubernetes permissions​

Mapping rule​

Four built-in ClusterRoles​

Common use cases​

Cluster-wide administrator​

Cluster-wide read-only access​

A team owns its own namespace​

Allow listing namespaces​

Multiple teams sharing a cluster, with GPU quota​

Narrow permission on one resource type​

CI/CD with a ServiceAccount, not SSO​

Revoking access when someone changes role​

Troubleshooting​

401 vs. 403 — always check this first​

Common errors​

Three diagnostic commands, in order​

Trace a login in detail​

Security checklist​

Three tiers, not one

Before you start

Install kubelogin

Install krew (the kubectl plugin manager)

Add krew to your PATH

Install the oidc-login plugin

Verify the installation

Create a kubeconfig for the user

Full sample file

File structure — three blocks, joined by name

The exec block, field by field

exec fields

Required arguments

Commonly used optional arguments

Token caching

Confirm you're logged in

Machines without a browser

Assign Kubernetes permissions

Mapping rule

Four built-in ClusterRoles

Common use cases

Cluster-wide administrator

Cluster-wide read-only access

A team owns its own namespace

Allow listing namespaces

Multiple teams sharing a cluster, with GPU quota

Narrow permission on one resource type

CI/CD with a ServiceAccount, not SSO

Revoking access when someone changes role

Troubleshooting

401 vs. 403 — always check this first

Common errors

Three diagnostic commands, in order

Trace a login in detail

Security checklist