Easy self-serve Ray Clusters on Kubernetes, no YAML.

So Dev, everyone. My name is Red. I'm an AI infrastructure in MLOps Dev. Just CTO play a quick show of hands, who design not have an idea Ayrat Kubernetes is or Forge what it does just by show of hands? Actually, it's cool. Okay. And what about Ray? Who does not know what is Ray? Ray. Cool. Awesome. So Kubernetes is Trust a platform to run workloads. It's very general purpose. It handles compute resources for you and so on and so forth. Ray is somewhat Singh, but it's really geared towards data workloads and AI workloads. So they're somewhat parallel. And you can also run Ray on top of Kubernetes. They they mix well together and that's called Kubernetes. So the the TLDR here is Kubernetes Six a way for you tool run your AI and ML workloads on Kubernetes. And you just focus on your Python code Mini theory, and all the Infrastructure is handled for you. Code we'll see that in reality, it's a little bit Forge nuanced. So this AI what I just mentioned earlier. There's KubeRay that sits on top of Kubernetes. And then at the lowest level, you have your Community resources, which is your GPT and CPUs and memory, which is what you care about for your workloads. But as a practitioner, if you really wanna use that, you need to know a little bit about Kubernetes. HINTS Kubernetes is very powerful, it's also very complex. And if you want to create resources for you to run your AI workloads on Ray with Kubernetes, you need to know things like Rue Montréal, which is a command line interface that allows you to interact with Kubernetes. It's it's it's Six you're not familiar with this, it's quite complex. LLM quite familiar with it, and I use similar alternatives AI JL because I don't know all the commands by heart. You also need to keep around a lot of YAML files. YAML manifests to define your clusters, your workloads, and so on. And so we design to figure out Web do I keep these file? What if these files is are are not the source of truth anymore for what's really happening in the cluster? So there's a lot of complexity and this is what this image is really trying to illustrate. And so Hat AI, built is a very simple tool. It's a 2026 K8s. TUI e stands for Gemini User Interface. It's a UI in your Gemini. And it's really gonna simplify everything for you so that you Code uses, Kubernetes and Ray together. And all you care about is just your workloads. And Sonja don't need to worry about YAML LLM anymore. It's just like your terminal user interface and you define, it's very opinionated. So it keeps it simple for the User. And you just define your cluster and you use them detect, tool run your workloads. So, it's the command is Krayne 2026 v. This is open source. You can also use it with UBS if you're a Rue user. And I'll show you a quick demo, of what that looks like. So this is the this is the 2026. Is it large enough? PHI think it is. So this is the 2 v. You can see that I have 1 demo cluster that I created 4 hours ago. It's Red. It has 1 worker. But before I show you what it Hat it does, I'll show you the flow to create a cluster. So you just give it a Date. Let's say Demo 32. You can use the mouse as well, which is nice. So if you're not familiar with the Ray Date set up AWS that you have the head master Code if you want, and then you have a bunch of worker nodes. So you can just leave the default here. And then worker groups, let's Ray, for example, I have the group of workers for my CPU workloads, and then I have a group of workers for my GPU workloads. And then I say this 1 is online 1 GPU per SPEAKER. Then it needs to be 7 API, and then here 1, 27. So you get the idea. You can also configure the tasks scaling. So auto scaling means that when your workloads run, they're gonna provision these resources. But once you have no workloads running, everything is gonna scale down. So Singh don't pay for idle resources. You also get for free Jupyter notebook, Versus Code server and SSH that are tool to your local machine. Only you can review it and you can create. So we're not gonna create AWS we already did create 1 before. So I can show you what the Ray dashboard looks like. If you're familiar with Ray, this this you probably know this. You can also have JupyterLab. So this Jupyter is running City in the cloud for you, and it's all tunneled for you. You You don't need to worry about all of Hat. You can just write your code. This is really what I was going for when I built this. Likewise, you can also have the same thing with Versus Code. Let's say, for example, you wanna do code development not on your local teaching, but on a remote machine, Maybe because it has better bandwidth or because it has GPU available or whatnot. And this is it. You have your cluster and it's created and you can run it. And whenever you're done, you can come and you can delete it. Now lead look at a few code examples. Sonja show you 2 quick code examples. This is just a quick help world. So I'm gonna activate my virtual environment. I'll do Python. AI. There's a CLI. It's called Ray submit. I cluster demo Sonja then I just do a UV run AI name dot AI. And this is going to create the JL. Remotely, it's gonna print hello build, and then it's gonna I can go and inspect it. I can see here PHI just submitted City. AI actually just finished Singh it said, right. So this is a help build, but in Dhall, it could be a training workloads, it could be an inference server, it could be many Singh many things. And also Forge the sake of this demo, I set up an infra server for an LLM. I'll show you the code 1st. It's quite simple. So Ray has a hat.com Community called Ray serve. It makes it Easy for you CTO, in 25 lines of code tool deploy an LLM, and then you can use it, for any other reasons if you wanna use it to analyze data Forge your experiments for r and d. And the key thing in here is that nowhere do you need anybody's help. You can just create your clusters Self, and you can deploy your models yourself, and you can use it, in in a closed loop. And this is the the really spirit in which I built Ray, which is to make, Ray Kubernetes and all those tools self serve for you uses that you can just focus on your code. And so this is currently deployed GPT. It's it's current submit JL like Forge. Better, I also have a demo client to Rue Hat this really works Event this is really a demo. So AI Demo client dot AI. What what this demo is doing is essentially teaching, that LLM that is deployed and then tool the proxy back to my own local machine, and I'm sending requests. So I'll Ray, hi. Ray. How AI you? JL, this AWS we GPT to know each other. This is AI 3.5. And I can say, what is the most populous city in Quebec? It's Montreal. So, this is about it really. The the gist of Ray is really not to replace Ray or Coupe or anything like that. It's really there to just supplement cubray to make it Easy for you to use such that you don't need to learn the Code control CLI. You don't need to keep Rue manifest AI. And you just have a terminal user interface and some simple commands so that you can just focus on your code. So, call to action for this JL. This is something that I just built on the Date because I I don't like YAML. So if you don't like YAML tool and you're curious to try AI out on your existing Kubernetes clusters free have Ray installed, then please do so and then reach out. I'd love to get some feedback. Better also, if you're just curious and you wanna build this and add features and improve, I'm I'm very happy to local collaborate and talk. And lastly, this is the QR Code, and this is the GitHub repo for the project. So please feel free to take a look if you're curious and, yeah, right on time. Do we have a question? Do we have a question?