Integrating Hubot and Kubernetes

Recently my team has been integrating Kubernetes with Hubot.

The objective being, Lets provide net new customers a way to write one line and create an entire project from scratch. Keeping in mind everything in our project is in containers. You might be asking, Why? Kubectl is great. This project was primarily to introduce Developers to Kubernetes without requiring a ton of knowledge around containers. We wanted to get them playing with it, launching new projects with containers with less complexity.

 

Real quick: Let me explain what is happening in the slide above. A Technical Project Manager or Program manager wants to create a new project. They run hubot create myfirstproject email@someemail.com this calls master hubot which then creates:

Git repository

Prepopulated data into the git repo

Separate namespace Kubernetes

A hubot to manage their own project/namespace

Jenkins build server which connects back to the new repo

Kubernetes service account

A new hipchat channel

Email invite to repository and chat channel

 

Repos and Hipchat are external to Kubernetes used via API calls. Having such a large organization (100s of development teams) we needed to deploy a generic Jenkins per project. So every project/development team gets their own Jenkins server. All jobs and specialized Jenkins configuration resides in the repository, not initially in Jenkins. All Jenkins jobs can be queried through Chat and run through Chat. Jenkins can be updated in a global fashion for features required across the organization.

We have divided our Hubot scripts based on function.

kubecommands.coffee – Contains any master level Hubot functionality we don’t allow across all Hubots such as creating a new project.

kubeclient.coffee – For communications directly with Kubernetes API. You will notice we are allowing for basic auth. This is changed already as we tokenize everything with OpenID Connect now.

repoclient.coffee – For requests to our repo APIs

chatclient.coffee – For requests to our Chat client, in this case Hipchat

kubeproject.coffee – Hubot commands specific to hubots deployed into projects (limited set of functionality from the Master)

Example: Deploying a New Jenkins deploy

Thats it, thanks for taking a look.