Start · End to end

Train it, forecast with it, reuse it¶

A reservoir turns recent observations into a useful memory. The readout learns how to turn that memory into a prediction. Usually training needs one algebraic solve, rather than an epoch loop.

1. Keep training and test data separate¶

Install with pip install resdag. Start with a floating Torch tensor shaped (time, signals) or (batch, time, signals). Convert NumPy data explicitly with torch.as_tensor(array).

import torch
import resdag as rd

t = torch.arange(1400, dtype=torch.float64) * 0.05
series = torch.stack((t.sin(), t.cos()), dim=-1)
training, truth = series[:1200], series[1200:]
options = dict(reservoir_size=100, washout=100, seed=0, readout="svd", dtype=torch.float64)
model = rd.ESN(**options).fit(training)

fit infers the signal count, learns scaling from the supervised training inputs only, and fits the next-sample readout. Washout is the initial period used to synchronize the reservoir before solving. Test samples never enter that solve or the scaling statistics.

For a TSDynamics simulation, install resdag[dynamics] and pass its trajectory directly. The trajectory guide explains timestamps, signal names, resampling and trajectory forecasts.

2. Forecast and measure error¶

prediction = model.forecast(horizon=len(truth))
assert prediction.shape == truth.shape
rmse = (prediction - truth).square().mean().sqrt()
assert torch.isfinite(rmse)

The first prediction estimates the sample immediately after the observations. A horizon of 200 returns 200 future samples, in the original units. Supply history=new_observations to synchronize on a different observed sequence. Otherwise the model uses its saved training history.

Chaotic forecasts eventually diverge. Compare several starting points and seeds on validation data, then evaluate the final choice on an untouched test period. Automatic tuning handles the validation split, candidate comparison and final refit. Matched speed–accuracy comparisons remain in roadmap phase five.

3. Save and restore a Torch state checkpoint¶

torch.save(model.state_dict(), "trained-state.pt")
restored = rd.ESN(**options)
restored.load_state_dict(torch.load("trained-state.pt", weights_only=True, map_location="cpu"))
restored_prediction = restored.forecast(horizon=len(truth))
torch.testing.assert_close(restored_prediction, prediction, rtol=0, atol=0)

This includes fitted weights, scaling, observed history and model metadata. Use the same preset settings when constructing the destination. Its signal count is restored automatically. Custom Model state checkpoints require a matching graph. A checkpoint does not resume an in-progress delayed rollout.

Complete architecture loading without reconstruction and save("model.svg") are tracked in phase four. The current public checkpoint path uses standard PyTorch serialization.

4. Put it inside a larger PyTorch model¶

from torch import nn

pipeline = nn.Sequential(
    nn.Linear(2, 2, dtype=torch.float64),
    restored,
    nn.Linear(2, 1, dtype=torch.float64),
)
restored.reset_state()
batch = training[-20:].unsqueeze(0).detach().requires_grad_()
loss = pipeline(batch).square().mean()
loss.backward()
assert batch.grad is not None and torch.isfinite(batch.grad).all()

Frozen reservoir weights still pass gradients to the preceding encoder. Use ordinary Torch losses and optimizers. Fit or build(input_size=...) before attaching an optimizer, so it sees every parameter.

Calls carry reservoir state. Reset before unrelated windows; train() and eval() do not reset it. For explicit state across chunks:

first, state = restored.forward_stateless(training[:20])
second, state = restored.forward_stateless(training[20:40], states=state)
assert first.shape == second.shape == (20, 2)

These calls leave stored reservoir state unchanged. The returned states can retain gradients across chunks. Detach them yourself when intentionally truncating backpropagation.

5. Customize only what you need¶

Use readout and readout_kwargs to select or configure a solver. Supply custom initializers directly or register them by name. Use model.graph for raw, already-preprocessed graph execution.

For branches, drivers, multiple outputs or a different recurrent cell, build rd.Model(inputs={...}, outputs={...}). It uses the same fit, forecast, forward, state and checkpoint commands. Start with the common model guide, then the public component recipes. Use custom preprocessing to change the units of individual inputs or outputs while retaining automatic inverse conversion. Named trajectories align several timestamped inputs and targets for a custom graph.

6. Choose device and precision explicitly¶

Move both the model and tensor data with .to(device=..., dtype=...). Fitted statistics move with the model. An untouched lazy ESN adopts its first training tensor's dtype/device; explicit constructor placement or .to(...) fixes them.

On MPS, use float32. The default automatic readout uses supported native solves when its numerical checks pass, and reports/warns about any CPU float64 fallback. Set readout_kwargs={"cpu_fallback": False} to require native fitting, or explicitly choose a solver. See readout solvers.

Use compile=True only for supported fast execution, and measure first-time compilation separately. For custom operations that need the whole sequence, choose history execution. It trades extra computation and memory for complete-prefix semantics.