Putting some notes here before I head out:
- My first realization is that we need the applications, and many of them will be in containers. In any previous (pre LLM) performance or usability study, we would build the containers and craft the commands ourselves. We have hundreds of these containers. The thought of going through them again and redoing that work (or rebuilding) is not something I want to do. Instead, I think we can ask an agent to take a container URI, discover the application, how it was built, and then prepare a manifest of "this is what is is in the container and here are types of environments that it will work optimally in." This step would run on a user's local machine, etc., before any job submission to not place the burden on the workload manager.
- As an alternative, we can always fall back to having the build done FOR the environment. But arguably, we've done these builds many times before and should use what we have. Arguably we do not want to wait.
- Now we have a container and a manifest that describes the software and environment needs. We give that to a local dispatch agent, and this agent is able to query the fleetq command to list cluster providers. This is the next local agent that is going to prepare what we submit - an actual jobspec with requires sections for subsystems.
- The submission goes in, and then the entire process described in this repository is orchestrated. In an ideal case, we dispatch to a cluster and get back a result.
One thing that occurs to me, is that we currently do not have agentic logic within the workload manager. The design I described above places the agents with the users, and it feels easier because I don't have to give the workload manager or any component a live token to use an LLM. I was initially thinking of something similar for the "does not fit anywhere" case in the scheduler, but I'm wondering if instead of having the reconciler there, we do a quick return to the submission agent and say "None of them were satisfied, here is the metadata" and then the operation can still happen locally. With this design, we are using agents, but the agent / models / tokens are securely on the level of the user. And each user can use whatever models / agents they have access to, and prefer. Heck, they don't even need to use an agent in that they can just submit a jobspec, as is.
I think I like this approach better because it makes fleetq a more solid infrastructure to not have a requirement of agentic components or credentials.
Putting some notes here before I head out:
One thing that occurs to me, is that we currently do not have agentic logic within the workload manager. The design I described above places the agents with the users, and it feels easier because I don't have to give the workload manager or any component a live token to use an LLM. I was initially thinking of something similar for the "does not fit anywhere" case in the scheduler, but I'm wondering if instead of having the reconciler there, we do a quick return to the submission agent and say "None of them were satisfied, here is the metadata" and then the operation can still happen locally. With this design, we are using agents, but the agent / models / tokens are securely on the level of the user. And each user can use whatever models / agents they have access to, and prefer. Heck, they don't even need to use an agent in that they can just submit a jobspec, as is.
I think I like this approach better because it makes fleetq a more solid infrastructure to not have a requirement of agentic components or credentials.