in a project, we have to manage a Kubernetes on premise - not in cloud.
we need to give people in my team access to kubernetes cluster, without adminitrator role.
for example: the developer member can deploy app into dev namespace only.
How do resolve it?
UserAccount, ClusterRole, Role, these objects are solutions.
in this post, i take a note how to make sure a user has the right permissions towards the right resources using the Role-Based Access Control model.
its definition will be skipped

brief steps:

• Create a x509 certificate file
• Request signing process to k8s cluster
• approve certificate and get certificated file
• create role, rolebinding properly: view pods only
• setting local context
• verify: the created user can not operate anything except for getting list of pods

there are two concepts but Kubernetes does not have objects which represent normal user accounts. Normal users cannot be added to a cluster through an API call.

1
2

User account is for humans. 
Service account is for processes, which run in pods.

Creating individual certificates for cluster users

• Create private key

(i have no knowledge about these keys, they are ambiguous).

1
2
3
4
5
6
7
8
9

the most easiest to understand article.
https://qiita.com/miyuki_samitani/items/e038e2ca7d8b6443657d

申込者が秘密鍵の作成
秘密鍵 + ディスティングイッシュネーム(申込者の情報) = CSRファイルを生成
認証局の申請の際にCSRをつけて申し込み
認証局はCSRのディスティングイッシュネームと申込者の情報が一致するか確認する
認証局は証明書に署名を行い、証明書を発行する
認証局が申込者に証明書、秘密鍵、中間証明書を送る

1
2
3
4
5

openssl genrsa -out monkey192.pem 2048

ll
total 8
-rw-r--r--  1 monkey192  staff  1679 Sep 28 11:18 monkey192.pem

• Create CSR file (Certificate Sign Requesting)

  1 2 3 4

  openssl req -new -key monkey192.pem -out monkey192.csr -subj "/CN=dev" -rw-r--r-- 1 monkey192 staff 883 Sep 28 11:21 monkey192.csr -rw-r--r-- 1 monkey192 staff 1679 Sep 28 11:18 monkey192.pem

• convert csr file to base64 text

  1	$ cat knqyf263.csr | base64 | tr -d '\n'

• Request signing process to k8s cluster

  1 2 3 4 5 6 7 8 9 10 11 12

  cat <<EOF | kubectl apply -f - apiVersion: certificates.k8s.io/v1 kind: CertificateSigningRequest metadata: name: monkey192-request spec: request: <the above csr's base64 text> signerName: kubernetes.io/kube-apiserver-client expirationSeconds: 86400 # one day usages: - client auth EOF

  1	certificatesigningrequest.certificates.k8s.io/monkey192-request created

• Approve certificate signing reques

  1 2 3

  kubectl get csr NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION monkey192-request 54s kubernetes.io/kube-apiserver-client kubernetes-admin 24h Pending

  1 2	kubectl certificate approve monkey192-request certificatesigningrequest.certificates.k8s.io/monkey192-request approved

• Get the certificate from the CSR:
  The certificate value is in Base64-encoded format under status.certificate.
  Export the issued certificate from the CertificateSigningRequest.

  1	$ kubectl get csr monkey192-request -o jsonpath='{.status.certificate}' | base64 -d > monkey192.crt

• setting your local kubectl context

  1 2 3 4 5 6

  kubectl config set-cluster kubernetes --insecure-skip-tls-verify=true --server=<your cluster endpoint> kubectl config set-credentials monkey192 --client-certificate=monkey192.crt --client-key=monkey192.pem --embed-certs=true kubectl config set-context developer --cluster=kubernetes --user=monkey192 Cluster "kubernetes" set. User "monkey192" set. Context "developer" created.

• swicth to created context

  1 2	kubectl config use-context developer Switched to context "developer".

• try get pods

  1 2	kubectl get pods Error from server (Forbidden): pods is forbidden: User "dev" cannot list resource "pods" in API group "" in the namespace "******"

• create ClusterRole & ClusterRoleBinding
  for demonstration, i just create a role which allow view pods only

  1 2 3 4 5 6 7 8 9

  kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: clusterrole-view-only rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "watch", "list"]

• assign ClusterRole to dev user

  1 2 3 4 5 6 7 8 9 10 11 12

  kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: clusterrolebinding-monkey192 subjects: - kind: User name: dev apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: clusterrole-view-only apiGroup: rbac.authorization.k8s.io

• try to get pods list

  1 2 3

  kubectl get pods NAME READY STATUS RESTARTS AGE elasticsearch-565dfb8c5f-gj4d9 1/1 Running 1 18h

  everthing is going on!

• try to get list services

  1 2	kubectl get services Error from server (Forbidden): services is forbidden: User "dev" cannot list resource "services" in API group "" in the namespace "******"

  the dev user can not get list of services, because the associated role is not allowed

when using k8s, there are many things need to discover for improvement security.
All in all, create users for Kubernetes clusters without giving everyone the admin certificate is the best security practice.
and this is a note for me!

What is Basic Authentication?

Basic authentication is a simple authentication schema build into the HTTP protocol.
The client sends HTTP request with the Authorization header that contains a base-64 encoded string from username:password values

For example: Authorization: Basic YXBpdXNlcjpDSUNhcGkwMQ==

I have assinged this task so want to take a note for enabling basic authen in nginx

How to enable basic authen in nginx

Refer nginx site in here

• first, install nginx http://nginx.org/en/download.html

• Generate a Password File
  you can use the htpasswd or apache2-utils (Debian, Ubuntu) or httpd-tools (RHEL/CentOS/Oracle Linux).

  1 2 3 4 5 6 7 8

  htpasswd -c ./.htpasswd admin New password: Re-type new password: Adding password for user admin --- cat .htpasswd admin:$apr1$MnWfNTQr$AiRKKJd.pfSgIN.fMcanJ/ <== Encrypted password

• Configuring nginx
  you can limit access to the whole website with basic authentication by specify the auth_basic directive at server

  1 2 3 4 5 6

  server { ... auth_basic "Basic Auth"; auth_basic_user_file /usr/local/etc/nginx/.htpasswd; ... }

  test config:

  1 2	nginx -t nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok

  reload nginx:

  1 2	nginx -s quit nginx

• Test your setting
  
  

Refer

• WWW-Authenticate
• Basic Auth

Docker notes (Q&A)

1. Dockerfile best practices

2. How to create a user when using base alpine image?
   Alpine uses the command adduser and addgroup for creating users and groups (rather than useradd and usergroup).

1
2
3
4
5

FROM alpine:latest
# Create a group and user
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
# Tell docker that all future commands should run as the appuser user
USER appuser

Contents

• Volumes in Kubernetes
• What differences between volume and persistenceVolume

Refer: Types of Kubernetes Volumes

Volumes in Kubernetes

Containers are ephemeral by design, it means that anything which is stored in container will be lost when the container is stopped.
The might cause proplems with containers that need to persist their data
(eg: database, filestorage …)

In k8s, a volume can be thought as a directory which is accesscible to the containers in a pod/

ssh cmd cheatsheet

• Delete all cached

  1	ssh-add -D

• Add a key

  1	ssh-add ~/.ssh/<private key name>

• Test ssh connection

  1 2	$ ssh -T git@github.com # Attempts to ssh to GitHub

• Create ssh-key

  1	ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

ssh config file ~/.ssh/config

1
2
3
4
5

Host github.com.main # <random for yourself>
  HostName github.com
  User git  # optional
  Port 22   # optional
  IdentityFile ~/.ssh/id_rsa_main  # Your key path

Welcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub.

Quick Start

Create a new post

1

$ hexo new "My New Post"

More info: Writing

Run server

1

$ hexo server

More info: Server

Generate static files

1

$ hexo generate

More info: Generating

Deploy to remote sites

1

$ hexo deploy

More info: Deployment

Theme

i am using tranquilpeak

docker

Tl;dr

• Sensitive Dataはどう管理すればいいのか、良い方法を知りたい
  • Usernames and passwords
  • TLS certificates and keys
  • SSH keys
  • Other important data such as the name of a database or internal server
  • Generic strings or binary content (up to 500 kb in size)
• Docker secretを使って極秘データを一元に管理できる、利用のコンテナだけに安全で渡せる
• Docker secrets are only available to swarm services, not to standalone containers

Docker containerを立ち上げる時に秘密な情報などをコンテナにどう渡せばよいかについて調べましたのでメモです。
Docker Secretがあります。

• Docker Secretを利用しないDocker compose:

  1 2 3 4 5 6 7 8 9 10

  mongo: image: mongodb ports: - 27017:27017 environment: - MONGO_INITDB_DATABASE=mydb - MONGO_INITDB_ROOT_USERNAME=rootuser - MONGO_INITDB_ROOT_PASSWORD=rootpassword volumes: - mongo-db:/data/db

• Docker Secretを利用するDocker compose:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14

  services: mongo: image: mongodb ports: - 27017:27017 environment: - MONGO_INITDB_DATABASE=mydb - MONGO_INITDB_ROOT_USERNAME_FILE=rootuser - MONGO_INITDB_ROOT_PASSWORD_FILE=/run/secrets/db_root_password volumes: - mongo-db:/data/db secrets: db_root_password: file: db_password.txt

  secretsで db_root_passwordの秘密データを定義されます。
  サービス開始するとき、Dockerは/run/secrets/<key-name>にマウントします。

The basic of heml

kubectk apply / create with --dry-run option, you can check whether manifest file is correctly or not without without persisting the resource into your cluster.
--dry-run has three options: none, server, client. I will try to run with these options.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        # Correct: - containerPort: 80
        - containerPort: -1
          name: nginx
        resources:
          limits:
            memory: "128Mi"
            cpu: "500m"

• dry-run: none (default)

  1 2	sh-3.2$ kubectl apply -f nginx-deployment.yaml --dry-run=none The Deployment "nginx" is invalid: spec.template.spec.containers[0].ports[0].containerPort: Invalid value: -1: must be between 1 and 65535, inclusive

• dry-run: client

  1 2	sh-3.2$ kubectl apply -f nginx-deployment.yaml --dry-run=client deployment.apps/nginx created (dry run)

  If client strategy, only print the object that would be sent, without sending it.
  So you don’t know the syntax of manifest file is correct or not.

• dry-run: server

  1 2	sh-3.2$ kubectl apply -f nginx-deployment.yaml --dry-run=server The Deployment "nginx" is invalid: spec.template.spec.containers[0].ports[0].containerPort: Invalid value: -1: must be between 1 and 65535, inclusive

  If server strategy, submit server-side request without persisting the resource.
  The syntax of manifest file will be checked.