Wed Jun 10 02:48:27 2020
<5547a085> Yeah I agree, its why use consul/nomad instead
<5547a085> I love all the hashicorp tools: consul, nomad, vault, packer, terraform
<5547a085> People are finally starting to realize what a PITA k8s would be like in production after a year, two years in production
<5547a085> A smaller team, even with lots of scaling, can manage equivalent infrastruction without k8s, using hashi consul/nomad. It has been tested up to 10,000 nodes
<5547a085> HPC clusters (hadoop, PBSPro …) usually test in at 50,000 nodes. I think k8s can do up 30,000 nodes?
<5547a085> Docker-swarm has been tested up to 5000 nodes … Mesosphere can do 100k nodes iirc.
<5547a085> Mesos is one of the first, and is what the NSA chose for AWS Gov Cloud. It is 7 node redundant … k8s is only slightly easier to admin than mesos IMO. The best k8s management tools came out last year. Namely, k3s and microk8s (canonical). Only with k3s and microk8s did I find the experience easier. I also tried kube, minikube …
<5547a085> Nomad is the best tool ive ever used for container services hands down
<5547a085> I have scripted my own scheduler before to cloud gpu mine up to a few hundred instances instead of using docker-swarm/k8s lol (using chef and pure bash!)