Hamiltonian Neural Networks for a Nonlinear Oscillator Chain

Question: does building Hamiltonian structure into a neural network improve long-horizon dynamics learned from noisy observations?

A Hamiltonian Neural Network (HNN) learns a scalar (H(q,p)) and obtains the dynamics from Hamilton's equations, dq/dt = ∂H/∂p and dp/dt = -∂H/∂q. This project compares that structured model with direct-dynamics baselines, then explores how the choice of MAE or MSE changes training behaviour.

Method

System. A chain of five masses with nonlinear FPUT-beta springs, V(d) = 1/2 k d^2 + 1/4 beta d^4.

Training data. Clean trajectories are simulated, Gaussian measurement noise is added to (q, p), and derivative targets are estimated with central finite differences. The networks therefore train on noisy state-to-derivative pairs rather than analytic forces.

Models.

Model	Predicts	Architecture	Params
ConvHNN	scalar H; dynamics via autodiff	Conv1d	1,313
ConvBaseline	(dq/dt, dp/dt) directly	closely matched Conv1d backbone	1,346
BaselineMLP	(dq/dt, dp/dt) directly	dense, global	19,210

The convolutional baseline is the main control because it keeps the local convolutional design and parameter count close to the HNN while removing the Hamiltonian formulation.

Evaluation. Models are rolled out for 30 s from 15 unseen initial conditions, compared with a 12 s training horizon. The test set includes larger displacements, two displaced masses, and non-zero initial momenta. Results are reported with medians and interquartile ranges (IQRs).

---

1. HNN versus direct-dynamics baselines

The representative rollout shows three complementary views: position over time, true physical energy, and a phase portrait. The HNN remains close to the true orbit for this case, while the direct-dynamics baselines show larger long-term energy drift.

Median over 15 unseen initial conditions (IQR in brackets):

Model	Energy drift %	Trajectory error	Diverged
HNN	6.5 [3.5, 5011]	0.275 [0.24, 0.62]	2/15
Conv baseline	89.7 [75, 2244]	0.312 [0.20, 1.30]	0/15
MLP baseline	1506 [185, 10000]	0.367 [0.28, 8.74]	0/15

The HNN has substantially lower median energy drift than the matched convolutional control. Its trajectory-error advantage is smaller, and the HNN diverges on 2 of 15 test initial conditions. This comparison is descriptive across the test set; no inferential test is reported for this experiment.

The train/validation split is made by trajectory. In this run, the final gaps are HNN +0.007, convolutional baseline +0.019, and MLP +0.054. This suggests the HNN generalises well on the held-out trajectories, but the experiment does not isolate a separate regularisation effect.

---

2. What do the finite-difference target errors look like?

The MAE experiment was initially motivated by the idea that finite differencing might create heavy-tailed derivative errors. Instead of assuming this, the error was measured directly:

target error = noisy finite-difference derivative - true derivative at the clean state

	Skew	Excess kurtosis	Beyond 3σ	Observed σ / predicted σ
dq/dt error	+0.005	+0.006	0.29%	1.004
dp/dt error	-0.004	+0.015	0.27%	1.018
Gaussian reference	0	0	0.27%	1.000

The target errors closely match the Gaussian distribution expected from propagated measurement noise. A central difference of independent Gaussian measurement errors has standard deviation σ√2/(2Δt) = 0.283, close to the observed values. This indicates that propagated measurement noise is the dominant contribution at the chosen timestep.

The analysis uses Q-Q plots and interpretable tail diagnostics rather than treating a normality p-value as the main evidence.

---

3. MAE versus MSE depends on training budget

Mean held-out energy drift over four paired seeds:

Epochs	5	10	20	40	70	120
MSE	64.5%	22.8%	12.0%	4.5%	2.8%	2.0%
MAE	12.0%	7.0%	4.6%	2.5%	1.9%	2.7%

MAE reaches low energy drift faster in these runs, while MSE has lower energy drift at the longest budget tested. The crossover shows why comparing the losses at only one epoch count can be misleading.

For the fixed-budget paired experiments:

Budget	MSE	MAE	Test	p
40 epochs, n=20	5.78%	2.72%	paired t-test	9.8×10⁻¹⁰
120 epochs, n=8	1.88%	2.53%	paired t-test	0.0155

Statistical evaluation

Each MAE run is paired with an MSE run using the same seed, data, initial weights, and batch order. The analysis therefore uses the paired difference

d = MAE metric - MSE metric

for each seed. A negative difference means MAE performed better for that metric.

For each outcome, the hypotheses are:

• H₀ (null hypothesis): there is no systematic difference between MAE and MSE. For the paired t-test this means the mean paired difference is zero, μ_d = 0.
• H₁ (alternative hypothesis): there is a systematic difference between MAE and MSE, so the paired difference is not centred at zero.

The significance level is set before testing. I use α = 0.05, meaning I accept at most a 5% false-positive rate for a single planned test. At 40 epochs I test two outcomes, energy drift and trajectory error. Testing several outcomes increases the chance of finding at least one false positive, so I use a Bonferroni correction: divide the overall α = 0.05 by the number of tests. With two tests, the threshold becomes 0.05 / 2 = 0.025 for each test.

Before choosing the inferential test, I check whether the paired differences are approximately normal using Shapiro-Wilk at α = 0.05. If normality is not rejected, I use a paired t-test. If normality is rejected, I use the Wilcoxon signed-rank test, which does not require normally distributed differences.

Outcome	Shapiro-Wilk p-value	Inferential test	Test p-value	Decision
Energy drift	0.630	paired t-test	9.80 × 10⁻¹⁰	p < 0.025: reject H₀
Trajectory error	0.022	Wilcoxon signed-rank	2.10 × 10⁻⁴	p < 0.025: reject H₀

For energy drift, p = 0.630 > 0.05 in the Shapiro-Wilk test, so there is insufficient evidence against normality and the paired t-test is reasonable. For trajectory error, p = 0.022 < 0.05, so normality is rejected and the Wilcoxon signed-rank test is used instead. A non-significant normality test is not proof of normality; it only means the data do not show strong evidence against it.

At 40 epochs, both inferential p-values are below the Bonferroni-corrected threshold of 0.025, so both tests reject H₀. MAE gives lower energy drift in 20/20 seeds and lower trajectory error in most seeds.

For energy drift, the mean paired difference is -3.06 percentage points with a 95% confidence interval of [-3.63, -2.48]. The interval does not cross zero, which is consistent with the hypothesis-test result. I also report Cohen's d_z, an effect-size measure for paired data:

d_z = mean paired difference / standard deviation of paired differences.

Here, d_z = -2.48. The negative sign shows that MAE has lower drift, while the magnitude shows that the effect is large relative to the run-to-run variation. Unlike the p-value, Cohen's d_z describes the size of the effect rather than only whether there is evidence that an effect exists.

Main conclusion

The statistical evidence shows that MAE is clearly better at the 40-epoch training budget, and this result remains significant after correcting for the two outcome tests. However, this is not a universal advantage. On the same eight seeds trained for 120 epochs, the direction reverses: MSE gives lower energy drift in 7/8 seeds (p = 0.0155 < 0.05). Therefore, the main conclusion is that loss choice interacts with training budget: MAE reaches useful dynamics faster, while MSE gives lower energy drift at the longest budget tested.

---

Limitations

• One measurement-noise level (σ = 0.02) is tested.
• Optimizer settings are fixed (Adam, learning rate 1e-3, batch size 256).
• One model width/depth and one physical system are studied.
• The baseline comparison uses 15 test initial conditions and includes two HNN divergences.
• The paired loss study uses n=20 at 40 epochs, n=8 at 120 epochs, and n=4 for the checkpoint curve.
• Energy drift and trajectory error are related metrics, not independent pieces of evidence.

Reproducing

pip install -r requirements.txt

cd src
python generate_data.py
python train.py
python plot_dynamics.py
python compare.py
python learning_curves.py
python plot_target_noise.py

cd ../experiment
python run_seeds.py 0 19
HNN_EPOCHS=120 python run_seeds.py 0 7
python analyze.py
python epoch_curve.py 0 3
python plot_epoch_curve.py
python analyze_budget.py

The paired sweeps save results incrementally. Raw per-seed experiment results are included as JSON so the statistical analysis can be reproduced without retraining.

Structure

src/         physics, models, training, and baseline comparison
experiment/  paired MAE-versus-MSE experiments and statistics
plots/       generated figures

Reference

Greydanus, S., Dzamba, M., & Yosinski, J. (2019). Hamiltonian Neural Networks. NeurIPS.