How do you get the agent state of all of the logged in users?

A Finesse user can only get the agent states for the agents on the team(s) they supervise. So unless all of the agents are in the team they supervise, there isn't a way to get all of the agent states of all logged in users.

If all the agents are indeed in a single team, the supervisor can subscribe to receive Team notifications which would consist of agent state notifications for the agents on the team. Also, they can use the Team-Get Team API to get a list of User objects for each agent in the team. Each User object will contain the agent's state.