blog.witalis.net

gRPCurl - curl for gRPC

 Sun 26 July 2020 by admin

In previous blog post How kubernetes is interacting with docker ? I was writing about dockershim with golang gRPC example code to get list of docker images. This example was trivial but writing code to get it wasn't trival, is there any generic tool to make such requests ? In HTTP world there is a popular tool called curl in gRPC world there is an equivalent of it called grpcurl. It helps to make RPC methods on gRPC server from the cli. So how can we achieve the same results like in my previous blog post ? Installation is pretty straight forward, just download binary. Then download data structure and services from dockershim:

$ wget https://raw.githubusercontent.com/kubernetes/kubernetes/master/staging/src/k8s.io/cri-api/pkg/apis/runtime/v1alpha2/api.proto

download necessary tools to compile proto file into binary form accepted by grpcurl, instruction how to do that is on Compiling your protocol buffers. Then download requirements for this particular proto file gogo/protobuf. Now we can compile proto file into binary format:

$ protoc -I=$GOPATH/src --proto_path=. --descriptor_set_out=api.protoset --include_imports api.proto

using compiled version proto file we can list services

$ grpcurl --protoset api.protoset list
runtime.v1alpha2.ImageService
runtime.v1alpha2.RuntimeService
$ grpcurl --protoset api.protoset list runtime.v1alpha2.ImageService
runtime.v1alpha2.ImageService.ImageFsInfo
runtime.v1alpha2.ImageService.ImageStatus
runtime.v1alpha2.ImageService.ListImages
runtime.v1alpha2.ImageService.PullImage
runtime.v1alpha2.ImageService.RemoveImage
$ grpcurl --protoset api.protoset describe runtime.v1alpha2.ImageService.ListImages
runtime.v1alpha2.ImageService.ListImages is a method:
rpc ListImages ( .runtime.v1alpha2.ListImagesRequest ) returns ( .runtime.v1alpha2.ListImagesResponse );

here we have runtime.v1alpha2.ImageService.ListImages, let's make request:

$ grpcurl -plaintext --protoset api.protoset -unix /var/run/dockershim.sock runtime.v1alpha2.ImageService.ListImages
...
    {
      "id": "sha256:da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e",
      "repoTags": [
        "gcr.io/google-containers/pause:3.1",
        "gcr.io/google_containers/pause-amd64:3.1",
        "k8s.gcr.io/pause:3.1"
      ],
      "repoDigests": [
        "gcr.io/google-containers/pause@sha256:f78411e19d84a252e53bff71a4407a5686c46983a2c2eeed83929b888179acea",
        "gcr.io/google_containers/pause-amd64@sha256:59eec8837a4d942cc19a52b8c09ea75121acc38114a2c68b98983ce9356b8610",
        "k8s.gcr.io/pause@sha256:f78411e19d84a252e53bff71a4407a5686c46983a2c2eeed83929b888179acea"
      ],
      "size": "742472"
    }
...

We can skip compiling proto file, just use plaintext proto file or ask gRPC server directly via reflection. But how can we pass some data to request, grpcurl assist you to get template JSON parameters:

$ grpcurl -msg-template --protoset api.protoset describe  runtime.v1alpha2.ImageStatusRequest
runtime.v1alpha2.ImageStatusRequest is a message:
message ImageStatusRequest {
  .runtime.v1alpha2.ImageSpec image = 1;
  bool verbose = 2;
}

Message template:
{
  "image": {
    "image": "",
    "annotations": {
      "": ""
    }
  },
  "verbose": false
}

now we can compose request:

$ grpcurl -d '{"image": { "image": "gcr.io/google-containers/pause:3.1", "annotations": {  "": "" } }, "verbose": false }' -plaintext --protoset api.protoset -unix /var/run/dockershim.sock runtime.v1alpha2.ImageService.ImageStatus
{
  "image": {
    "id": "sha256:da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e",
    "repoTags": [
      "gcr.io/google-containers/pause:3.1",
      "gcr.io/google_containers/pause-amd64:3.1",
      "k8s.gcr.io/pause:3.1"
    ],
    "repoDigests": [
      "gcr.io/google-containers/pause@sha256:f78411e19d84a252e53bff71a4407a5686c46983a2c2eeed83929b888179acea",
      "gcr.io/google_containers/pause-amd64@sha256:59eec8837a4d942cc19a52b8c09ea75121acc38114a2c68b98983ce9356b8610",
      "k8s.gcr.io/pause@sha256:f78411e19d84a252e53bff71a4407a5686c46983a2c2eeed83929b888179acea"
    ],
    "size": "742472"
  }
}

CNI - the Container Network Interface
Sat 04 July 2020 by admin

CNI is simple interface based on environment variables and JSON config. Both environment variables and JSON config are processed by CNI plugin started by container management system like Kubernetes. CNI plugin is responsible for connecting container namespace into selected network, second responsibility for CNI is to provide IP address it ...

read more

Docker and Kubernetes at network layer
Sun 21 June 2020 by admin

I wrote some posts about how kubernetes is interacting with docker at CRI level, but what about networking ? Docker has developed it's own solution called CNM to maintain network, to interact with it just use commands under docker network. How it looks like in k8s multi node environment:

# docker ...
read more

dropwatch - discover where network packets are dropped
Sat 30 May 2020 by admin

Let's imagine situation where you experience network problem with dropping packets and you've no idea where the problem is located. So first of all prepare environment:

# iptables -A OUTPUT -p icmp -j DROP
# ping -c 3 -W 1 8.8.8.8
PING 8.8.8.8 (8 ...
read more

mkubectx - single command across all your selected kubernetes contexts
Sun 10 May 2020 by admin

More on mkubectx

read more

Pause - most popular container in k8s environment
Sat 02 May 2020 by admin

Probably the most popular container in kubernetes environment. Container image is really small:

$ docker images | grep -i pause
k8s.gcr.io/pause                          3.2                 80d28bedfe5d        2 months ago        683kB

Codebase is also small pause. According to source code it is responsible for doing pretty... nothing, except of dealing ...

read more

How to change default k8s container runtime to containerd ?
Sun 26 April 2020 by admin

Default kubelet container runtime configuration use docker as CRI. Containerd has another implementation of CRI, it should fullfill the same requirements as docker so ie. it's responsible to maintain container images. Containerd extends its functionality by using plugins, one of them is cri. To interact with containerd we can ...

read more

How kubernetes is interacting with docker ?
Sun 19 April 2020 by admin

In a few words Container Runtime Interface is the answer to this questions. But we are going a little bit deeper. First of all what is CRI ? CRI is one of the most mature interface in kubernetes, it's a bridge between kubelet and container runtime. k8-diagram. Creating such interfaces in ...

read more

Kubectl - writing your own plugin
Sat 28 March 2020 by admin

Kubectl is an entrypoint for maintaing k8s clusters, you can find a lot useful switches to extract data. For example to get all containers images, just use:

$ kubectl get pods --all-namespaces -o go-template --template="{{range .items}}{{range .spec.containers}}{{printf \"%s\n\" .image}}{{end}}{{end}}"

Lots of switches ...

read more

Growpart
Sun 17 June 2018 by admin

Easy alternative to fdisk and partprobe when resizing partition to its maximum. Extensively used in cloud environments (i.e. cloud-init).

https://www.systutorials.com/docs/linux/man/1-growpart/

read more