API Reference

The reference below is generated directly from the package docstrings.

High-level API

class spindecoupler.RLSide(port, verbose=False)[source]

Bases: object

Communication endpoint used by the reinforcement-learning process.

An RLSide instance owns the server side of the TCP connection and coordinates one high-level RL loop with one external agent loop. The typical call sequence in an environment wrapper is:

  1. Construct RLSide before training starts.

  2. Call resetGetObs() from reset().

  3. Call stepSendActGetObs() from each step().

  4. Call stepExpFinished() once the experiment is over.

The class does not define rewards or episode termination on its own. It only transports actions, observations, and timing metadata between processes.

Parameters:

  • port (int)

  • verbose (bool)

__init__(port, verbose=False)[source]

Create the RL-side communication endpoint and wait for the agent.

Parameters:

  • port (int) – TCP port used by the RL process to listen for the agent connection. Valid ports are restricted by the underlying transport layer to the range 20000 to 49151.

  • verbose (bool) – If True, print lifecycle messages while waiting for the agent and while closing the connection.

Raises:

RuntimeError – If no agent connects before the transport timeout or if the underlying server endpoint cannot be started.

resetGetObs(timeout=10.0)[source]

Request the first observation after an episode reset.

This method is usually called from the RL environment reset() method. It blocks until the agent acknowledges the reset and sends back the first observation.

Parameters:

timeout (float) – Timeout in seconds for each communication operation involved in the request. Values less than or equal to 0.0 disable the timeout at the transport layer.

Returns:

A pair (obs, ato) where obs is the first observation dictionary after the reset and ato is the agent time of observation, that is, the timestamp measured by the agent-side clock when that observation was sampled.

Return type:

Tuple[dict, float]

Raises:

RuntimeError – If the reset command cannot be sent or if the reply is not received successfully.

stepSendActGetObs(action, timeout=10.0)[source]

Send a new action and wait for the corresponding observation.

This method is usually called from the RL environment step() method. It first transfers the new action to the agent and then blocks until the agent reports two pieces of information:

  1. LAT: the actual execution duration of the previous action.

  2. The observation collected after applying the new action, optionally

    accompanied by a reward computed on the agent side.

Parameters:

  • action – Action object to send to the agent. It must be serializable by Python pickle because the transport exchanges pickled payloads.

  • timeout (float) – Timeout in seconds for each communication operation involved in the request. Values less than or equal to 0.0 disable the timeout at the transport layer.

Returns:

A tuple (lat, obs, rew, ato). lat is the duration of the previous action as measured by the agent clock, obs is the observation dictionary after executing the new action, rew is the reward reported by the agent for the current action, and ato is the agent timestamp when the observation was captured.

If your RL code also records a local wall-clock timestamp such as t_wall = time.time(), keep in mind that t_wall belongs to the RL process clock, while lat and ato come from the agent-side clock domain.

Return type:

Tuple[float, dict, float, float]

Raises:

RuntimeError – If sending the action fails or if either response message cannot be received successfully.

stepExpFinished(timeout=10.0)[source]

Notify the agent that the experiment has finished.

Call this method once, after the final RL step, when no more actions will be sent. The method sends a finish indicator to the agent loop so it can perform its own shutdown procedure.

Parameters:

timeout (float) – Reserved for API symmetry with the other high-level methods. The current implementation sends the finish indicator immediately and does not consume the timeout value.

Raises:

RuntimeError – Not raised directly by this wrapper, but downstream socket errors may still surface during shutdown in user-managed teardown code.

class spindecoupler.AgentSide(ipbaselinespart, portbaselinespart, verbose=False)[source]

Bases: object

Communication endpoint used by the external agent process.

An AgentSide instance owns the client side of the TCP connection and lets an agent loop poll for new RL commands without blocking when no command is pending. The agent remains responsible for applying actions, resetting its workspace, sampling observations, and optionally computing rewards.

Parameters:

  • ipbaselinespart (str)

  • portbaselinespart (int)

  • verbose (bool)

class WhatToDo(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Command kinds that the RL process can send to the agent loop.

REC_ACTION_SEND_OBS

Receive a new action, report the duration of the previous action, and later send the observation that results from the new action.

RESET_SEND_OBS

Reset the agent-side episode state and send the first observation after that reset.

FINISH

Stop the experiment and terminate the control loop gracefully.

__init__(ipbaselinespart, portbaselinespart, verbose=False)[source]

Create the agent-side communication endpoint and connect to the RL side.

Parameters:

  • ipbaselinespart (str) – IPv4 address of the RL process that owns the listening server socket. A common pattern is to pass BaseCommPoint.get_ip() when both processes run on the same host.

  • portbaselinespart (int) – TCP port exposed by the RL process.

  • verbose (bool) – If True, print lifecycle messages while connecting and while closing the connection.

Raises:

RuntimeError – If the client socket cannot connect to the RL process.

readWhatToDo(timeout=10.0)[source]

Poll the RL side for the next command.

This method is intended to be called from every agent-loop iteration. The initial poll is non-blocking: if no data are pending in the socket, the method returns None immediately. Once pending data are detected, reading the command itself may block up to timeout seconds.

Parameters:

timeout (float) – Maximum number of seconds allowed for reading a pending command once the socket indicates that data are available. Values less than or equal to 0.0 disable the timeout at the transport layer.

Returns:

None if no command is pending. Otherwise, a tuple whose first element is a WhatToDo value and whose second element is the payload:

  • (WhatToDo.REC_ACTION_SEND_OBS, action) for a normal step.

  • (WhatToDo.RESET_SEND_OBS, None) for a reset request.

  • (WhatToDo.FINISH, None) for shutdown.

Return type:

Optional[Tuple[AgentSide.WhatToDo, Any]]

Raises:

  • RuntimeError – If reading the pending command fails.

  • ValueError – If the RL side sends an unknown command indicator.

stepSendLastActDur(lat)[source]

Send LAT, the duration of the previous action, back to the RL side.

This method should be called immediately after receiving WhatToDo.REC_ACTION_SEND_OBS and before the new action starts running for its full control interval.

Parameters:

lat (float) – Actual duration, in seconds, of the action that was being executed before the newly received action replaced it. This value belongs to the agent clock domain.

Raises:

RuntimeError – If the timing payload cannot be sent.

stepSendObs(obs, agenttime=0.0, rew=0.0)[source]

Send the observation obtained after executing a step action.

This method completes the response cycle that starts with WhatToDo.REC_ACTION_SEND_OBS. It should be called after the agent has applied the action long enough to produce the next observation.

Parameters:

  • obs – Observation dictionary produced by the agent or simulator.

  • agenttime (float) – Agent-side timestamp for when obs was sampled. This is the value that the RL side receives as ATO.

  • rew (float) – Reward associated with the current action, when reward computation is delegated to the agent side. If the RL process owns reward computation, leave this value at its default.

Raises:

RuntimeError – If the observation payload cannot be sent.

resetSendObs(obs, agenttime=0.0)[source]

Send the first observation collected after a reset.

This method should be called after receiving WhatToDo.RESET_SEND_OBS and completing the agent-side reset logic.

Parameters:

  • obs – Observation dictionary collected immediately after the reset.

  • agenttime – Agent-side timestamp for when the reset observation was sampled. This is the value that the RL side receives as ATO.

Raises:

RuntimeError – If the reset observation cannot be sent.

Transport primitives

class rl_spin_decoupler.socketcomms.comms.BaseCommPoint(kind, datachunkmaxsize=4096, port=49054, ipv4='127.0.0.1')[source]

Bases: object

Communication point.

Parameters:

  • kind (Kind)

  • datachunkmaxsize (int)

  • port (int)

  • ipv4 (str)

class Kind(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Kinds of points

__init__(kind, datachunkmaxsize=4096, port=49054, ipv4='127.0.0.1')[source]

Constructor. The point is set at the given port and machine IPv4.

Parameters:

  • kind (Kind)

  • datachunkmaxsize (int)

  • port (int)

  • ipv4 (str)

setDebug(st=True)[source]

Enable or disable debug messages.

Parameters:

st (bool)

sendData(data)[source]

Send that data properly to the other side. Return non-empty string with any error in the connection.

Parameters:

data (Dict)

Return type:

str

readData(timeout=2.0)[source]

Read the data (blocking if timeout > 0.0) from the other side. Return non-empty string if any error occurs in the connection.

Parameters:

timeout (float)

Return type:

Tuple[str, Dict]

checkDataToRead()[source]

Check whether the socket has data to read and return True in that case. This is a non-blocking test.

class rl_spin_decoupler.socketcomms.comms.ClientCommPoint(ip, po)[source]

Bases: BaseCommPoint

Parameters:

  • ip (str)

  • po (int)

__init__(ip, po)[source]

Constructor. Client to connect to that ip:port.

Parameters:

  • ip (str)

  • po (int)

begin()[source]

Start the work for the client.

Return type:

str

end()[source]

Ends the communications for the current work.

Return type:

str

class rl_spin_decoupler.socketcomms.comms.ServerCommPoint(po)[source]

Bases: BaseCommPoint

Parameters:

po (int)

__init__(po)[source]

Constructor. Server listening at that port.

Parameters:

po (int)

begin(timeoutaccept)[source]

Start the work for the server. TIMEOUTACCEPT in seconds.

Parameters:

timeoutaccept (float)

Return type:

str

end()[source]

Ends the communications for the current work.

Return type:

str