Demo: Fitting a sine curve
[2]:
%load_ext autoreload
%autoreload 2
import warnings
warnings.filterwarnings("ignore", category=UserWarning, module="multiprocessing.resource_tracker")
import numpy as np
import matplotlib.pyplot as plt
import nbi
[3]:
print(nbi.__version__)
0.4.1
Let us consider a very simple sine curve model, where \(t\) is time, \(\omega\) is angular frequency, \(A\) is amplitude, and \(\phi_0\) is initial phase: \(f(t) = A\cdot\sin(\omega\cdot t+\phi_0\))
Assume parameters from the following uniform distribution: \(\phi_0\in[0,2\pi]\), \(A\in[-4,4]\), \(\omega\in[2\pi,12\pi]\)
Note that the amplitude is allowed to be negative. Thus expect bi-modal posteriors in the amplitude.
Also assume Gaussian measurement noise fixed to be \(\sigma_0\): \(x_{\rm obs}\sim N(f(t), \sigma=\sigma_0)\)
[4]:
t = np.linspace(0,1,50)
def sine(param):
phi0, A, omega = param
return np.sin(omega * t + phi0) * A
[5]:
# nbi requires prior to be defined with scipy functions
# alternatively, you may supply pre-generated parameters with numpy arrays
from scipy.stats import uniform, truncnorm
prior = {
'phi0': uniform(loc=0, scale=np.pi*2),
'A': uniform(loc=-4, scale=8),
'omega': uniform(loc=2*np.pi, scale=10*np.pi)
}
labels = list(prior.keys())
priors = [prior[k] for k in labels]
[20]:
# a randomly drawn sine curve for illustration
plt.figure(figsize=(5,3))
np.random.seed(0)
# draw random parameter from prior
y_true = [var.rvs(1)[0] for var in priors]
# add fixed gaussian noise of 1
x_err = 0.2
x_obs = sine(y_true) + np.random.normal(size=50) * x_err
plt.errorbar(t, x_obs, yerr=x_err, fmt='.')
plt.show()
Use case 1: Amortized Neural Posterior Estimation
if we want to derive the posterior for a large number of different x_obs, then train nbi for 1 round on a large number of samples generated from the prior.
[6]:
# the noise function can include any component of the forward model that is cheap
# therefore it takes in both data (x) and parameter (y) as input
# Here let us only consider iid Gaussian noise
def noise(x, y):
rand = np.random.normal(0, 1, size=x.shape[0])
# let's say x_err is drawn from [0.05, 0.5]
x_err = np.random.uniform() * 0.45 + 0.05
x_noise = x + rand * x_err
return x_noise, y
[7]:
# hyperparameters for the normalizing flow
flow = {
'n_dims': 3, # dimension of parameter space
'flow_hidden': 64,
'num_blocks': 5,
'n_mog':2 # Number of Mixture of Gaussian as base density
}
# the NBI package provides the "ResNet-GRU" network as the default
# featurizer network for sequential data
featurizer = {
'type': 'resnet-gru',
'norm': 'weight_norm',
'dim_in': 1, # Number of channels for input data
'dim_out': 64, # Output feature vector dimension
'dim_conv_max': 256, # Maximum hidden dimension for CNN
'depth': 3 # Number of 1D ResNet layers
}
# initialize NBI engine
engine = nbi.NBI(
flow=flow,
featurizer=featurizer,
simulator=sine,
priors=priors,
labels=labels,
device='cpu',
n_jobs=10
)
[8]:
engine.fit(
n_sims=12800,
n_rounds=1,
n_epochs=100,
batch_size=256,
lr=0.001,
early_stop_patience=10,
noise=noise # this can also be an array if fixed noise
)
Auto learning rate to min_lr = 2e-06
Generating simulations
1280it [00:00, 1295.21it/s]
---------------------- Round: 0 ----------------------
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
Epoch 0: Train, Loglike in nats: -3.087308: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3634.09it/s]
0%| | 0/1280 [00:00<?, ?it/s][W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
- Val, Loglike in nats: -2.783548: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 2288.41it/s]
Epoch 1: Train, Loglike in nats: -2.195478: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 3940.62it/s]
- Val, Loglike in nats: -1.986147: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8448.17it/s]
Epoch 2: Train, Loglike in nats: -1.785573: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 3893.39it/s]
- Val, Loglike in nats: -1.548288: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8228.18it/s]
Epoch 3: Train, Loglike in nats: -1.659087: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 3866.82it/s]
- Val, Loglike in nats: -1.213384: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8434.26it/s]
Epoch 4: Train, Loglike in nats: -1.229340: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 4038.97it/s]
- Val, Loglike in nats: -1.082149: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8466.85it/s]
Epoch 5: Train, Loglike in nats: -1.287976: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 4088.34it/s]
- Val, Loglike in nats: -1.155724: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 7677.09it/s]
Epoch 6: Train, Loglike in nats: -1.251058: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 4014.02it/s]
- Val, Loglike in nats: -1.406450: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8235.33it/s]
Epoch 7: Train, Loglike in nats: -1.044494: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 4074.09it/s]
- Val, Loglike in nats: -1.487436: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8270.99it/s]
Epoch 8: Train, Loglike in nats: -1.037187: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 4106.95it/s]
- Val, Loglike in nats: -0.881672: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8476.04it/s]
Epoch 9: Train, Loglike in nats: -0.765628: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 4117.42it/s]
- Val, Loglike in nats: -0.857265: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8333.08it/s]
Epoch 10: Train, Loglike in nats: -0.737189: 100%|███████████████████████████████████████████| 11520/11520 [00:02<00:00, 4156.00it/s]
- Val, Loglike in nats: -0.749852: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8405.06it/s]
Epoch 11: Train, Loglike in nats: -0.513982: 100%|███████████████████████████████████████████| 11520/11520 [00:02<00:00, 3970.09it/s]
- Val, Loglike in nats: -0.141688: 100%|███████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8617.25it/s]
Epoch 12: Train, Loglike in nats: 0.108010: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 4092.70it/s]
- Val, Loglike in nats: 0.427636: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8694.70it/s]
Epoch 13: Train, Loglike in nats: 0.599686: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 4170.80it/s]
- Val, Loglike in nats: 1.201011: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8464.31it/s]
Epoch 14: Train, Loglike in nats: 1.498033: 100%|████████████████████████████████████████████| 11520/11520 [00:02<00:00, 4077.56it/s]
- Val, Loglike in nats: 1.896008: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 8746.33it/s]
Epoch 15: Train, Loglike in nats: 1.952197: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3165.01it/s]
- Val, Loglike in nats: 2.429433: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 7121.24it/s]
Epoch 16: Train, Loglike in nats: 2.435560: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3052.03it/s]
- Val, Loglike in nats: 2.450541: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 5794.24it/s]
Epoch 17: Train, Loglike in nats: 2.596513: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3103.48it/s]
- Val, Loglike in nats: 2.660452: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6951.79it/s]
Epoch 18: Train, Loglike in nats: 2.729082: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3104.49it/s]
- Val, Loglike in nats: 2.906046: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6317.56it/s]
Epoch 19: Train, Loglike in nats: 2.871341: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2941.84it/s]
- Val, Loglike in nats: 3.545237: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6327.55it/s]
Epoch 20: Train, Loglike in nats: 2.690105: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3023.93it/s]
- Val, Loglike in nats: 3.426148: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6433.41it/s]
Epoch 21: Train, Loglike in nats: 3.225217: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2977.62it/s]
- Val, Loglike in nats: 3.522245: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6857.05it/s]
Epoch 22: Train, Loglike in nats: 3.363186: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3024.68it/s]
- Val, Loglike in nats: 3.636042: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6594.23it/s]
Epoch 23: Train, Loglike in nats: 3.253713: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3035.76it/s]
- Val, Loglike in nats: 3.460052: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6601.82it/s]
Epoch 24: Train, Loglike in nats: 3.592663: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3089.29it/s]
- Val, Loglike in nats: 3.004652: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6075.72it/s]
Epoch 25: Train, Loglike in nats: 3.449104: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3165.37it/s]
- Val, Loglike in nats: 3.508722: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6031.76it/s]
Epoch 26: Train, Loglike in nats: 3.858269: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3022.67it/s]
- Val, Loglike in nats: 3.392975: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6968.07it/s]
Epoch 27: Train, Loglike in nats: 3.841936: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2996.60it/s]
- Val, Loglike in nats: 3.917620: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6462.49it/s]
Epoch 28: Train, Loglike in nats: 3.773704: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3015.69it/s]
- Val, Loglike in nats: 3.832344: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6261.19it/s]
Epoch 29: Train, Loglike in nats: 3.909618: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2997.46it/s]
- Val, Loglike in nats: 4.148997: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6352.95it/s]
Epoch 30: Train, Loglike in nats: 3.713703: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3029.73it/s]
- Val, Loglike in nats: 3.732015: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6607.44it/s]
Epoch 31: Train, Loglike in nats: 4.163549: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3058.43it/s]
- Val, Loglike in nats: 4.046113: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6512.51it/s]
Epoch 32: Train, Loglike in nats: 4.232159: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3003.54it/s]
- Val, Loglike in nats: 4.211665: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6580.31it/s]
Epoch 33: Train, Loglike in nats: 4.158120: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2942.14it/s]
- Val, Loglike in nats: 4.362492: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6316.84it/s]
Epoch 34: Train, Loglike in nats: 3.911486: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2981.66it/s]
- Val, Loglike in nats: 3.712253: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6468.79it/s]
Epoch 35: Train, Loglike in nats: 4.089507: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2955.99it/s]
- Val, Loglike in nats: 4.302726: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 5717.66it/s]
Epoch 36: Train, Loglike in nats: 4.047397: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3045.86it/s]
- Val, Loglike in nats: 4.131258: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6403.72it/s]
Epoch 37: Train, Loglike in nats: 4.455680: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2968.54it/s]
- Val, Loglike in nats: 4.495085: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6497.45it/s]
Epoch 38: Train, Loglike in nats: 4.279306: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2960.62it/s]
- Val, Loglike in nats: 4.022075: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6360.10it/s]
Epoch 39: Train, Loglike in nats: 4.548883: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2969.54it/s]
- Val, Loglike in nats: 4.347622: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6613.45it/s]
Epoch 40: Train, Loglike in nats: 4.352882: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2951.98it/s]
- Val, Loglike in nats: 4.432039: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6409.87it/s]
Epoch 41: Train, Loglike in nats: 4.413246: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2916.99it/s]
- Val, Loglike in nats: 4.642027: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 5949.55it/s]
Epoch 42: Train, Loglike in nats: 4.691599: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2911.21it/s]
- Val, Loglike in nats: 4.623140: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6465.79it/s]
Epoch 43: Train, Loglike in nats: 4.702567: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3002.44it/s]
- Val, Loglike in nats: 4.862590: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6499.07it/s]
Epoch 44: Train, Loglike in nats: 4.736180: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2939.37it/s]
- Val, Loglike in nats: 4.853620: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6505.33it/s]
Epoch 45: Train, Loglike in nats: 4.570702: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2947.85it/s]
- Val, Loglike in nats: 4.457351: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6513.23it/s]
Epoch 46: Train, Loglike in nats: 4.611096: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2932.06it/s]
- Val, Loglike in nats: 4.809636: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6271.83it/s]
Epoch 47: Train, Loglike in nats: 4.697153: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2935.45it/s]
- Val, Loglike in nats: 4.574687: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 5679.76it/s]
Epoch 48: Train, Loglike in nats: 4.819075: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2953.13it/s]
- Val, Loglike in nats: 4.888621: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6434.79it/s]
Epoch 49: Train, Loglike in nats: 4.931657: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2939.01it/s]
- Val, Loglike in nats: 4.854235: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6333.44it/s]
Epoch 50: Train, Loglike in nats: 4.888239: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2987.17it/s]
- Val, Loglike in nats: 4.701188: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6948.71it/s]
Epoch 51: Train, Loglike in nats: 4.738068: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3158.98it/s]
- Val, Loglike in nats: 4.830042: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6608.50it/s]
Epoch 52: Train, Loglike in nats: 5.017061: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3168.02it/s]
- Val, Loglike in nats: 5.044671: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6677.10it/s]
Epoch 53: Train, Loglike in nats: 4.907747: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3144.24it/s]
- Val, Loglike in nats: 4.964620: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6839.00it/s]
Epoch 54: Train, Loglike in nats: 4.927024: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3166.54it/s]
- Val, Loglike in nats: 5.092577: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6872.96it/s]
Epoch 55: Train, Loglike in nats: 4.949957: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3153.30it/s]
- Val, Loglike in nats: 4.617899: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6826.27it/s]
Epoch 56: Train, Loglike in nats: 5.028600: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3131.85it/s]
- Val, Loglike in nats: 5.043853: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6796.29it/s]
Epoch 57: Train, Loglike in nats: 5.057802: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2983.03it/s]
- Val, Loglike in nats: 5.047560: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 5833.54it/s]
Epoch 58: Train, Loglike in nats: 5.106674: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2971.49it/s]
- Val, Loglike in nats: 5.138825: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6357.34it/s]
Epoch 59: Train, Loglike in nats: 5.147522: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2972.90it/s]
- Val, Loglike in nats: 5.136655: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6400.77it/s]
Epoch 60: Train, Loglike in nats: 5.234972: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2953.36it/s]
- Val, Loglike in nats: 5.243810: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6729.48it/s]
Epoch 61: Train, Loglike in nats: 5.085754: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2972.94it/s]
- Val, Loglike in nats: 5.081370: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6567.10it/s]
Epoch 62: Train, Loglike in nats: 5.219260: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2942.85it/s]
- Val, Loglike in nats: 5.242363: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6300.20it/s]
Epoch 63: Train, Loglike in nats: 5.205916: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2996.35it/s]
- Val, Loglike in nats: 5.120478: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 5911.11it/s]
Epoch 64: Train, Loglike in nats: 5.261810: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2963.23it/s]
- Val, Loglike in nats: 5.233375: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6364.54it/s]
Epoch 65: Train, Loglike in nats: 5.319747: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3064.95it/s]
- Val, Loglike in nats: 5.332101: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6881.05it/s]
Epoch 66: Train, Loglike in nats: 5.307462: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3062.98it/s]
- Val, Loglike in nats: 5.097158: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6501.66it/s]
Epoch 67: Train, Loglike in nats: 5.306693: 100%|████████████████████████████████████████████| 11520/11520 [00:04<00:00, 2596.30it/s]
- Val, Loglike in nats: 5.250185: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6473.35it/s]
Epoch 68: Train, Loglike in nats: 5.376663: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3111.31it/s]
- Val, Loglike in nats: 5.316840: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6842.11it/s]
Epoch 69: Train, Loglike in nats: 5.409171: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3143.25it/s]
- Val, Loglike in nats: 5.336621: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6788.93it/s]
Epoch 70: Train, Loglike in nats: 5.420004: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3139.81it/s]
- Val, Loglike in nats: 5.383515: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6846.34it/s]
Epoch 71: Train, Loglike in nats: 5.426814: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3124.54it/s]
- Val, Loglike in nats: 5.399803: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6616.76it/s]
Epoch 72: Train, Loglike in nats: 5.493544: 100%|████████████████████████████████████████████| 11520/11520 [00:04<00:00, 2771.87it/s]
- Val, Loglike in nats: 5.380247: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6702.79it/s]
Epoch 73: Train, Loglike in nats: 5.475517: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3104.07it/s]
- Val, Loglike in nats: 5.342830: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6763.97it/s]
Epoch 74: Train, Loglike in nats: 5.466399: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3121.96it/s]
- Val, Loglike in nats: 5.476913: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6952.75it/s]
Epoch 75: Train, Loglike in nats: 5.473536: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3149.65it/s]
- Val, Loglike in nats: 5.489121: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6958.49it/s]
Epoch 76: Train, Loglike in nats: 5.513473: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3172.74it/s]
- Val, Loglike in nats: 5.538151: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6762.72it/s]
Epoch 77: Train, Loglike in nats: 5.527736: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3136.83it/s]
- Val, Loglike in nats: 5.436869: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6315.91it/s]
Epoch 78: Train, Loglike in nats: 5.508047: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3279.98it/s]
- Val, Loglike in nats: 5.505529: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6933.99it/s]
Epoch 79: Train, Loglike in nats: 5.573108: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3258.94it/s]
- Val, Loglike in nats: 5.480532: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6505.26it/s]
Epoch 80: Train, Loglike in nats: 5.558843: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3241.29it/s]
- Val, Loglike in nats: 5.473536: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 7182.17it/s]
Epoch 81: Train, Loglike in nats: 5.570381: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3235.77it/s]
- Val, Loglike in nats: 5.403001: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 7323.13it/s]
Epoch 82: Train, Loglike in nats: 5.581730: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3091.31it/s]
- Val, Loglike in nats: 5.518591: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6990.10it/s]
Epoch 83: Train, Loglike in nats: 5.625877: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3170.10it/s]
- Val, Loglike in nats: 5.527087: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6794.85it/s]
Epoch 84: Train, Loglike in nats: 5.601232: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3070.73it/s]
- Val, Loglike in nats: 5.556339: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6661.78it/s]
Epoch 85: Train, Loglike in nats: 5.597750: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3149.29it/s]
- Val, Loglike in nats: 5.633901: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6359.36it/s]
Epoch 86: Train, Loglike in nats: 5.614872: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3240.15it/s]
- Val, Loglike in nats: 5.605390: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6861.36it/s]
Epoch 87: Train, Loglike in nats: 5.655118: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3261.65it/s]
- Val, Loglike in nats: 5.628323: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6985.37it/s]
Epoch 88: Train, Loglike in nats: 5.645738: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3195.31it/s]
- Val, Loglike in nats: 5.573058: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6876.69it/s]
Epoch 89: Train, Loglike in nats: 5.613527: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3269.11it/s]
- Val, Loglike in nats: 5.620242: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 7071.71it/s]
Epoch 90: Train, Loglike in nats: 5.665639: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3224.72it/s]
- Val, Loglike in nats: 5.663519: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 7015.11it/s]
Epoch 91: Train, Loglike in nats: 5.677661: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3139.04it/s]
- Val, Loglike in nats: 5.645561: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6525.44it/s]
Epoch 92: Train, Loglike in nats: 5.662165: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 2994.38it/s]
- Val, Loglike in nats: 5.571723: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6690.46it/s]
Epoch 93: Train, Loglike in nats: 5.648493: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3024.20it/s]
- Val, Loglike in nats: 5.611712: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6512.18it/s]
Epoch 94: Train, Loglike in nats: 5.658863: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3069.27it/s]
- Val, Loglike in nats: 5.682595: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6848.77it/s]
Epoch 95: Train, Loglike in nats: 5.652702: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3026.69it/s]
- Val, Loglike in nats: 5.643842: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 5849.29it/s]
Epoch 96: Train, Loglike in nats: 5.626644: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3033.37it/s]
- Val, Loglike in nats: 5.683327: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6745.47it/s]
Epoch 97: Train, Loglike in nats: 5.683721: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3038.00it/s]
- Val, Loglike in nats: 5.572078: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6459.91it/s]
Epoch 98: Train, Loglike in nats: 5.653526: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3005.69it/s]
- Val, Loglike in nats: 5.568140: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6608.50it/s]
Epoch 99: Train, Loglike in nats: 5.684864: 100%|████████████████████████████████████████████| 11520/11520 [00:03<00:00, 3016.51it/s]
- Val, Loglike in nats: 5.582545: 100%|████████████████████████████████████████████████████████| 1280/1280 [00:00<00:00, 6446.94it/s]
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
Now we can apply the trained model to derive the model posterior for any sine curve. Nbi uses importance sampling to deliver asymptotically exact results if you specify x_err in engine.predict()
[10]:
plt.figure(figsize=(5,3))
np.random.seed(4)
# draw random parameter from prior
y_true = [var.rvs(1)[0] for var in priors]
x_err = 0.1
x_obs = sine(y_true) + np.random.normal(size=50) * x_err
plt.errorbar(t, x_obs, yerr=x_err, fmt='.')
plt.show()
y_pred, weights = engine.predict(x_obs, x_err=np.array([0.1]*50), y_true=y_true, n_samples=12800, corner=True, corner_reweight=True)
plt.show()
Samples outside prior N = 1926
surrogate posterior
Generating simulations
1088it [00:00, 2183.55it/s]
WARNING:root:Too few points to create valid contours
Effective Sample Size = 804.0
Sampling efficiency = 6.3%
Use case 2: Sequential Neural Posterior Estimation (SNPE)
Let’s say we only want to fit the specific sine curve above. In this case, we can train the model over multiple rounds to reduce the number of training samples required. The basic idea is to generate more training samples near the final posterior (as opposed to the broad prior).
[7]:
# Same test data as above
plt.figure(figsize=(5,3))
np.random.seed(4)
# draw random parameter from prior
y_true = [var.rvs(1)[0] for var in priors]
x_err = 0.1
x_obs = sine(y_true) + np.random.normal(size=50) * x_err
plt.errorbar(t, x_obs, yerr=x_err, fmt='.')
plt.show()
[8]:
# hyperparameters remain the same as ANPE
flow = {
'n_dims': 3, # dimension of parameter space
'flow_hidden': 64,
'num_blocks': 5,
'n_mog':4
}
# the NBI package provides the "ResNet-GRU" network as the default
# featurizer network for sequential data
featurizer = {
'type': 'resnet-gru',
'norm': 'weight_norm',
'dim_in': 1,
'dim_out': 64,
'dim_conv_max': 256,
'depth': 3
}
# initialize NBI engine
engine = nbi.NBI(
flow=flow,
featurizer=featurizer,
simulator=sine,
priors=priors,
labels=labels,
path='test2',
device='cpu',
n_jobs=10,
)
[9]:
engine.fit(
x_obs=x_obs,
y_true=y_true,
n_sims=5120, # N=5120 per round
n_rounds=5, # Max 5 rounds. (also see early_stop_train)
n_epochs=50,
batch_size=256,
lr=0.001,
early_stop_train=True, # If sampling efficiency is reduced for this round, stop and revert to previous round
early_stop_patience=10, # Within a round, wait this many epochs before stopping training
noise=np.array([0.1]*50) # assume homogeneous noise; used for importance sampling
)
Auto learning rate to min_lr = 9.999999999999999e-06
Generating simulations
512it [00:00, 1845.76it/s]
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/Keming/anaconda3/envs/py311-arm/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Effective sample size for current/all rounds 0.0/0.0
---------------------- Round: 0 ----------------------
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
Epoch 0: Train, Loglike in nats: -3.178809: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3910.64it/s]
0%| | 0/512 [00:00<?, ?it/s][W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
- Val, Loglike in nats: -2.351445: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 1722.88it/s]
Epoch 1: Train, Loglike in nats: -2.520033: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 4110.11it/s]
- Val, Loglike in nats: -2.315982: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 7416.88it/s]
Epoch 2: Train, Loglike in nats: -2.182982: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3830.64it/s]
- Val, Loglike in nats: -1.903988: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6205.61it/s]
Epoch 3: Train, Loglike in nats: -2.294062: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3591.07it/s]
- Val, Loglike in nats: -1.788503: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 4483.68it/s]
Epoch 4: Train, Loglike in nats: -1.470261: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3334.05it/s]
- Val, Loglike in nats: -1.852802: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 4998.54it/s]
Epoch 5: Train, Loglike in nats: -0.958327: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3558.68it/s]
- Val, Loglike in nats: -0.496098: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6950.21it/s]
Epoch 6: Train, Loglike in nats: -0.592597: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3535.98it/s]
- Val, Loglike in nats: -0.011142: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6655.07it/s]
Epoch 7: Train, Loglike in nats: -0.874985: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3544.96it/s]
- Val, Loglike in nats: -0.874805: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 7336.73it/s]
Epoch 8: Train, Loglike in nats: -0.196669: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3621.08it/s]
- Val, Loglike in nats: 0.357404: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6824.56it/s]
Epoch 9: Train, Loglike in nats: 0.340745: 100%|███████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3493.03it/s]
- Val, Loglike in nats: 0.761815: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6765.33it/s]
Epoch 10: Train, Loglike in nats: -0.063772: 100%|█████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3505.04it/s]
- Val, Loglike in nats: 0.478781: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6865.84it/s]
Epoch 11: Train, Loglike in nats: 0.804854: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3447.36it/s]
- Val, Loglike in nats: 0.972865: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 5571.47it/s]
Epoch 12: Train, Loglike in nats: 0.632102: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3296.29it/s]
- Val, Loglike in nats: -0.165591: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 4841.03it/s]
Epoch 13: Train, Loglike in nats: 0.286750: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3400.08it/s]
- Val, Loglike in nats: 0.878790: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6424.40it/s]
Epoch 14: Train, Loglike in nats: 1.153664: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3460.32it/s]
- Val, Loglike in nats: 0.779362: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6712.56it/s]
Epoch 15: Train, Loglike in nats: 1.310814: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3540.12it/s]
- Val, Loglike in nats: 1.518301: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6703.14it/s]
Epoch 16: Train, Loglike in nats: 1.593314: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3545.71it/s]
- Val, Loglike in nats: 1.603043: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6745.08it/s]
Epoch 17: Train, Loglike in nats: 1.583892: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3520.92it/s]
- Val, Loglike in nats: 1.194725: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6845.27it/s]
Epoch 18: Train, Loglike in nats: 0.606773: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3569.30it/s]
- Val, Loglike in nats: 0.058834: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 5075.58it/s]
Epoch 19: Train, Loglike in nats: 1.089224: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3484.23it/s]
- Val, Loglike in nats: 1.484441: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 5110.29it/s]
Epoch 20: Train, Loglike in nats: 1.714440: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3560.90it/s]
- Val, Loglike in nats: 2.014822: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6942.93it/s]
Epoch 21: Train, Loglike in nats: -0.162407: 100%|█████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3545.26it/s]
- Val, Loglike in nats: -0.072775: 100%|█████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6860.97it/s]
Epoch 22: Train, Loglike in nats: 0.927962: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3551.51it/s]
- Val, Loglike in nats: 1.200364: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 7081.09it/s]
Epoch 23: Train, Loglike in nats: 1.423299: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3567.01it/s]
- Val, Loglike in nats: 1.681204: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6906.51it/s]
Epoch 24: Train, Loglike in nats: 1.991262: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3550.14it/s]
- Val, Loglike in nats: 2.161254: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6694.99it/s]
Epoch 25: Train, Loglike in nats: 2.269061: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3535.69it/s]
- Val, Loglike in nats: 2.183996: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6920.51it/s]
Epoch 26: Train, Loglike in nats: 2.006082: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3573.11it/s]
- Val, Loglike in nats: 2.212944: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 5822.75it/s]
Epoch 27: Train, Loglike in nats: 1.255605: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3484.09it/s]
- Val, Loglike in nats: 2.214038: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 4964.09it/s]
Epoch 28: Train, Loglike in nats: 2.390229: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3561.22it/s]
- Val, Loglike in nats: 2.605304: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6940.78it/s]
Epoch 29: Train, Loglike in nats: 2.499697: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3573.92it/s]
- Val, Loglike in nats: 2.688471: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6664.15it/s]
Epoch 30: Train, Loglike in nats: 2.151875: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3498.21it/s]
- Val, Loglike in nats: 2.298323: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 7000.67it/s]
Epoch 31: Train, Loglike in nats: 2.570245: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3544.75it/s]
- Val, Loglike in nats: 2.756696: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6556.88it/s]
Epoch 32: Train, Loglike in nats: 2.841699: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3549.69it/s]
- Val, Loglike in nats: 2.889441: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6901.76it/s]
Epoch 33: Train, Loglike in nats: 2.867762: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3555.61it/s]
- Val, Loglike in nats: 2.913196: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 4952.79it/s]
Epoch 34: Train, Loglike in nats: 2.844210: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3336.53it/s]
- Val, Loglike in nats: 2.838925: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 5164.00it/s]
Epoch 35: Train, Loglike in nats: 2.925656: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3525.41it/s]
- Val, Loglike in nats: 2.798489: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6783.51it/s]
Epoch 36: Train, Loglike in nats: 2.956311: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3464.58it/s]
- Val, Loglike in nats: 2.919692: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 7027.38it/s]
Epoch 37: Train, Loglike in nats: 3.027276: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3443.03it/s]
- Val, Loglike in nats: 3.017796: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6724.19it/s]
Epoch 38: Train, Loglike in nats: 3.061480: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3523.83it/s]
- Val, Loglike in nats: 3.014896: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6975.95it/s]
Epoch 39: Train, Loglike in nats: 3.091869: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3524.47it/s]
- Val, Loglike in nats: 3.045526: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6818.21it/s]
Epoch 40: Train, Loglike in nats: 3.118678: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3576.96it/s]
- Val, Loglike in nats: 3.096528: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 7550.85it/s]
Epoch 41: Train, Loglike in nats: 3.144019: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 4028.99it/s]
- Val, Loglike in nats: 3.085847: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 5373.36it/s]
Epoch 42: Train, Loglike in nats: 3.138113: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3429.30it/s]
- Val, Loglike in nats: 3.105297: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 4269.05it/s]
Epoch 43: Train, Loglike in nats: 3.174801: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3366.40it/s]
- Val, Loglike in nats: 3.092056: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6451.79it/s]
Epoch 44: Train, Loglike in nats: 3.184265: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3443.95it/s]
- Val, Loglike in nats: 3.138204: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6568.23it/s]
Epoch 45: Train, Loglike in nats: 3.201228: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3376.36it/s]
- Val, Loglike in nats: 3.139873: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6749.97it/s]
Epoch 46: Train, Loglike in nats: 3.187072: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3267.66it/s]
- Val, Loglike in nats: 3.147358: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 6702.88it/s]
Epoch 47: Train, Loglike in nats: 3.206146: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3300.83it/s]
- Val, Loglike in nats: 3.153655: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 5634.87it/s]
Epoch 48: Train, Loglike in nats: 3.213627: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3275.07it/s]
- Val, Loglike in nats: 3.150207: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 4414.59it/s]
Epoch 49: Train, Loglike in nats: 3.218961: 100%|██████████████████████████████████████████████| 4608/4608 [00:01<00:00, 3199.88it/s]
- Val, Loglike in nats: 3.151944: 100%|██████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 4444.60it/s]
Samples outside prior N = 284
Generating simulations
484it [00:00, 1808.32it/s]
WARNING:root:Too few points to create valid contours
WARNING:root:Too few points to create valid contours
Effective sample size for current/all rounds 16.5/16.5
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
---------------------- Round: 1 ----------------------
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
Epoch 0: Train, Loglike in nats: -227.562848: 100%|████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3386.75it/s]
0%| | 0/484 [00:00<?, ?it/s][W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
- Val, Loglike in nats: -4.274474: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 874.38it/s]
Epoch 1: Train, Loglike in nats: -3.720076: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3427.09it/s]
- Val, Loglike in nats: -2.874036: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 5802.19it/s]
Epoch 2: Train, Loglike in nats: -2.695066: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3535.94it/s]
- Val, Loglike in nats: -2.457481: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 5712.21it/s]
Epoch 3: Train, Loglike in nats: -2.217798: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3541.68it/s]
- Val, Loglike in nats: -2.079115: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 6376.74it/s]
Epoch 4: Train, Loglike in nats: -1.919745: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3603.08it/s]
- Val, Loglike in nats: -2.476732: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 6041.86it/s]
Epoch 5: Train, Loglike in nats: -1.852250: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3381.01it/s]
- Val, Loglike in nats: -1.623387: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 3331.77it/s]
Epoch 6: Train, Loglike in nats: -1.275186: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3552.37it/s]
- Val, Loglike in nats: -1.175469: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 3926.30it/s]
Epoch 7: Train, Loglike in nats: -0.990903: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3356.96it/s]
- Val, Loglike in nats: -1.291866: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 6110.98it/s]
Epoch 8: Train, Loglike in nats: -0.707595: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3496.74it/s]
- Val, Loglike in nats: -0.399531: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 6029.62it/s]
Epoch 9: Train, Loglike in nats: -0.477556: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3528.73it/s]
- Val, Loglike in nats: -0.406078: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 5832.13it/s]
Epoch 10: Train, Loglike in nats: -0.167944: 100%|█████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3426.82it/s]
- Val, Loglike in nats: -0.331744: 53%|██████████████████████████████▏ | 256/484 [00:00<00:00, 5891.26it/s]
Epoch 11: Train, Loglike in nats: -0.060062: 100%|█████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3423.77it/s]
- Val, Loglike in nats: 0.097881: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5561.58it/s]
Epoch 12: Train, Loglike in nats: 0.166471: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3442.05it/s]
- Val, Loglike in nats: 0.368781: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3942.52it/s]
Epoch 13: Train, Loglike in nats: 0.241121: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3322.75it/s]
- Val, Loglike in nats: 0.267101: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3449.44it/s]
Epoch 14: Train, Loglike in nats: 0.410803: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3414.56it/s]
- Val, Loglike in nats: 0.558508: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 6167.21it/s]
Epoch 15: Train, Loglike in nats: 0.650889: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3398.49it/s]
- Val, Loglike in nats: 0.662175: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5176.08it/s]
Epoch 16: Train, Loglike in nats: 0.781257: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3039.61it/s]
- Val, Loglike in nats: 0.870337: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5571.54it/s]
Epoch 17: Train, Loglike in nats: 0.684174: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3136.42it/s]
- Val, Loglike in nats: 0.874422: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5640.02it/s]
Epoch 18: Train, Loglike in nats: 0.993196: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3302.39it/s]
- Val, Loglike in nats: 1.182998: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5381.87it/s]
Epoch 19: Train, Loglike in nats: 0.204421: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3378.28it/s]
- Val, Loglike in nats: 0.385963: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5720.03it/s]
Epoch 20: Train, Loglike in nats: 0.688926: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3433.96it/s]
- Val, Loglike in nats: 1.169094: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3740.06it/s]
Epoch 21: Train, Loglike in nats: 1.138873: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3491.87it/s]
- Val, Loglike in nats: 1.270771: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3727.87it/s]
Epoch 22: Train, Loglike in nats: 1.389667: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3562.08it/s]
- Val, Loglike in nats: 1.369481: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5895.47it/s]
Epoch 23: Train, Loglike in nats: 1.494021: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3560.08it/s]
- Val, Loglike in nats: 1.474659: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5432.02it/s]
Epoch 24: Train, Loglike in nats: 1.482213: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3511.09it/s]
- Val, Loglike in nats: 1.397461: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5517.35it/s]
Epoch 25: Train, Loglike in nats: 1.513695: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3405.82it/s]
- Val, Loglike in nats: 1.513018: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5113.15it/s]
Epoch 26: Train, Loglike in nats: 1.630407: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3317.33it/s]
- Val, Loglike in nats: 1.641587: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5939.79it/s]
Epoch 27: Train, Loglike in nats: 1.612592: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3135.02it/s]
- Val, Loglike in nats: 1.528238: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3827.06it/s]
Epoch 28: Train, Loglike in nats: 1.715234: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2812.73it/s]
- Val, Loglike in nats: 1.735319: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3838.89it/s]
Epoch 29: Train, Loglike in nats: 1.732614: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3065.55it/s]
- Val, Loglike in nats: 1.688967: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 4083.77it/s]
Epoch 30: Train, Loglike in nats: 1.689221: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3050.41it/s]
- Val, Loglike in nats: 1.653095: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5363.72it/s]
Epoch 31: Train, Loglike in nats: 1.747054: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3030.68it/s]
- Val, Loglike in nats: 1.770553: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 4455.05it/s]
Epoch 32: Train, Loglike in nats: 1.824224: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2893.79it/s]
- Val, Loglike in nats: 1.833232: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5501.01it/s]
Epoch 33: Train, Loglike in nats: 1.831120: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3100.24it/s]
- Val, Loglike in nats: 1.871883: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 4962.78it/s]
Epoch 34: Train, Loglike in nats: 1.861293: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3075.52it/s]
- Val, Loglike in nats: 1.778226: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 2632.23it/s]
Epoch 35: Train, Loglike in nats: 1.844931: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2676.94it/s]
- Val, Loglike in nats: 1.879002: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3594.74it/s]
Epoch 36: Train, Loglike in nats: 1.926386: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2517.99it/s]
- Val, Loglike in nats: 1.945747: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 4572.23it/s]
Epoch 37: Train, Loglike in nats: 1.909738: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2568.40it/s]
- Val, Loglike in nats: 1.800832: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 4630.69it/s]
Epoch 38: Train, Loglike in nats: 1.907400: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2493.58it/s]
- Val, Loglike in nats: 1.936525: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3786.31it/s]
Epoch 39: Train, Loglike in nats: 1.940091: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3080.35it/s]
- Val, Loglike in nats: 1.914376: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5662.72it/s]
Epoch 40: Train, Loglike in nats: 1.952725: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2490.20it/s]
- Val, Loglike in nats: 1.952243: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 4649.82it/s]
Epoch 41: Train, Loglike in nats: 1.950826: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2379.77it/s]
- Val, Loglike in nats: 1.953348: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3610.81it/s]
Epoch 42: Train, Loglike in nats: 1.958557: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2935.75it/s]
- Val, Loglike in nats: 1.955115: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3640.14it/s]
Epoch 43: Train, Loglike in nats: 1.987062: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2674.26it/s]
- Val, Loglike in nats: 1.971472: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5527.49it/s]
Epoch 44: Train, Loglike in nats: 1.966530: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 3096.91it/s]
- Val, Loglike in nats: 1.984969: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5267.16it/s]
Epoch 45: Train, Loglike in nats: 2.000991: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2820.67it/s]
- Val, Loglike in nats: 1.990955: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5161.30it/s]
Epoch 46: Train, Loglike in nats: 2.002603: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2774.32it/s]
- Val, Loglike in nats: 1.973319: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 4997.75it/s]
Epoch 47: Train, Loglike in nats: 1.986691: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2699.48it/s]
- Val, Loglike in nats: 1.995456: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 4538.93it/s]
Epoch 48: Train, Loglike in nats: 2.016861: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2871.00it/s]
- Val, Loglike in nats: 1.974635: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 3654.83it/s]
Epoch 49: Train, Loglike in nats: 1.993311: 100%|██████████████████████████████████████████████| 4352/4352 [00:01<00:00, 2514.63it/s]
- Val, Loglike in nats: 1.945055: 53%|██████████████████████████████▋ | 256/484 [00:00<00:00, 5519.47it/s]
Samples outside prior N = 300
Generating simulations
482it [00:00, 1319.40it/s]
WARNING:root:Too few points to create valid contours
Effective sample size for current/all rounds 714.3/730.8
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
---------------------- Round: 2 ----------------------
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
Epoch 0: Train, Loglike in nats: -3.126428: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3003.45it/s]
0%| | 0/482 [00:00<?, ?it/s][W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
- Val, Loglike in nats: -0.685096: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 711.50it/s]
Epoch 1: Train, Loglike in nats: -0.178857: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 2886.20it/s]
- Val, Loglike in nats: -0.157790: 53%|██████████████████████████████▎ | 256/482 [00:00<00:00, 4960.55it/s]
Epoch 2: Train, Loglike in nats: 0.432874: 94%|████████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 2874.44it/s]
- Val, Loglike in nats: 0.043271: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 2938.01it/s]
Epoch 3: Train, Loglike in nats: -0.388803: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 2845.52it/s]
- Val, Loglike in nats: 0.172351: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 1948.15it/s]
Epoch 4: Train, Loglike in nats: 0.686426: 94%|████████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3372.91it/s]
- Val, Loglike in nats: 1.082386: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 3562.48it/s]
Epoch 5: Train, Loglike in nats: 1.185858: 94%|████████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3034.14it/s]
- Val, Loglike in nats: 1.408023: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5669.47it/s]
Epoch 6: Train, Loglike in nats: 1.378064: 94%|████████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3074.16it/s]
- Val, Loglike in nats: 1.325811: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5160.33it/s]
Epoch 7: Train, Loglike in nats: 1.530694: 94%|████████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3084.98it/s]
- Val, Loglike in nats: 1.171978: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5018.21it/s]
Epoch 8: Train, Loglike in nats: 1.036830: 94%|████████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3053.08it/s]
- Val, Loglike in nats: 1.429630: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 4336.32it/s]
Epoch 9: Train, Loglike in nats: 1.659337: 94%|████████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3056.33it/s]
- Val, Loglike in nats: 1.614858: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5303.27it/s]
Epoch 10: Train, Loglike in nats: 1.746444: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 2933.91it/s]
- Val, Loglike in nats: 1.595381: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 2691.16it/s]
Epoch 11: Train, Loglike in nats: 0.572571: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 2724.84it/s]
- Val, Loglike in nats: 1.361996: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 3545.84it/s]
Epoch 12: Train, Loglike in nats: 0.887030: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 2850.63it/s]
- Val, Loglike in nats: 1.388081: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5012.73it/s]
Epoch 13: Train, Loglike in nats: 1.688827: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 2922.96it/s]
- Val, Loglike in nats: 1.775293: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5241.81it/s]
Epoch 14: Train, Loglike in nats: 1.816738: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 2983.91it/s]
- Val, Loglike in nats: 1.840165: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5495.88it/s]
Epoch 15: Train, Loglike in nats: 1.900583: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 2889.90it/s]
- Val, Loglike in nats: 1.804649: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5725.80it/s]
Epoch 16: Train, Loglike in nats: 1.952579: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3325.40it/s]
- Val, Loglike in nats: 1.936498: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5918.38it/s]
Epoch 17: Train, Loglike in nats: 1.906857: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3251.45it/s]
- Val, Loglike in nats: 1.938241: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 2470.21it/s]
Epoch 18: Train, Loglike in nats: 1.753342: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3257.02it/s]
- Val, Loglike in nats: 1.649206: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 3627.08it/s]
Epoch 19: Train, Loglike in nats: 1.837964: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3253.59it/s]
- Val, Loglike in nats: 1.858500: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5310.45it/s]
Epoch 20: Train, Loglike in nats: 1.939658: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3560.71it/s]
- Val, Loglike in nats: 1.976185: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5814.07it/s]
Epoch 21: Train, Loglike in nats: 2.016462: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3342.72it/s]
- Val, Loglike in nats: 2.007832: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5240.32it/s]
Epoch 22: Train, Loglike in nats: 1.914216: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3219.02it/s]
- Val, Loglike in nats: 2.056663: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5825.42it/s]
Epoch 23: Train, Loglike in nats: 2.072038: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3356.52it/s]
- Val, Loglike in nats: 2.152986: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 4711.87it/s]
Epoch 24: Train, Loglike in nats: 2.017096: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3412.17it/s]
- Val, Loglike in nats: 1.923773: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 3969.72it/s]
Epoch 25: Train, Loglike in nats: 2.055204: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3360.24it/s]
- Val, Loglike in nats: 1.974297: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 4590.86it/s]
Epoch 26: Train, Loglike in nats: 2.164390: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3342.69it/s]
- Val, Loglike in nats: 2.136933: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 3786.35it/s]
Epoch 27: Train, Loglike in nats: 2.169471: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3485.63it/s]
- Val, Loglike in nats: 2.145892: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5721.86it/s]
Epoch 28: Train, Loglike in nats: 2.186101: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3387.19it/s]
- Val, Loglike in nats: 2.207410: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5933.06it/s]
Epoch 29: Train, Loglike in nats: 2.215851: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3456.79it/s]
- Val, Loglike in nats: 2.093461: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5729.10it/s]
Epoch 30: Train, Loglike in nats: 2.244286: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3374.70it/s]
- Val, Loglike in nats: 2.141585: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5493.44it/s]
Epoch 31: Train, Loglike in nats: 2.216283: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3393.48it/s]
- Val, Loglike in nats: 2.210757: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5894.53it/s]
Epoch 32: Train, Loglike in nats: 2.191318: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3324.01it/s]
- Val, Loglike in nats: 2.152621: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 4177.61it/s]
Epoch 33: Train, Loglike in nats: 2.229294: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3384.96it/s]
- Val, Loglike in nats: 2.161822: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5817.78it/s]
Epoch 34: Train, Loglike in nats: 2.240509: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3315.54it/s]
- Val, Loglike in nats: 2.222312: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5799.84it/s]
Epoch 35: Train, Loglike in nats: 2.253880: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3580.67it/s]
- Val, Loglike in nats: 2.181592: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5990.29it/s]
Epoch 36: Train, Loglike in nats: 2.247014: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3438.16it/s]
- Val, Loglike in nats: 2.294550: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5848.49it/s]
Epoch 37: Train, Loglike in nats: 2.255800: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3481.30it/s]
- Val, Loglike in nats: 2.129786: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 2024.43it/s]
Epoch 38: Train, Loglike in nats: 2.278950: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3273.33it/s]
- Val, Loglike in nats: 2.304597: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 4291.55it/s]
Epoch 39: Train, Loglike in nats: 2.282287: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3360.32it/s]
- Val, Loglike in nats: 2.153277: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5809.60it/s]
Epoch 40: Train, Loglike in nats: 2.301640: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3481.48it/s]
- Val, Loglike in nats: 2.199708: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 4550.14it/s]
Epoch 41: Train, Loglike in nats: 2.293583: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3242.12it/s]
- Val, Loglike in nats: 2.175554: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 2959.26it/s]
Epoch 42: Train, Loglike in nats: 2.289117: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3441.10it/s]
- Val, Loglike in nats: 2.279039: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5801.03it/s]
Epoch 43: Train, Loglike in nats: 2.308299: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3356.59it/s]
- Val, Loglike in nats: 2.259628: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5835.81it/s]
Epoch 44: Train, Loglike in nats: 2.291275: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3459.57it/s]
- Val, Loglike in nats: 2.342046: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5997.55it/s]
Epoch 45: Train, Loglike in nats: 2.312276: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3415.52it/s]
- Val, Loglike in nats: 2.238629: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5764.22it/s]
Epoch 46: Train, Loglike in nats: 2.304259: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3325.27it/s]
- Val, Loglike in nats: 2.222430: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5747.25it/s]
Epoch 47: Train, Loglike in nats: 2.330191: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3460.13it/s]
- Val, Loglike in nats: 2.274152: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5940.64it/s]
Epoch 48: Train, Loglike in nats: 2.327115: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3470.50it/s]
- Val, Loglike in nats: 2.246285: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 5902.99it/s]
Epoch 49: Train, Loglike in nats: 2.314153: 94%|███████████████████████████████████████████▍ | 4096/4338 [00:01<00:00, 3433.11it/s]
- Val, Loglike in nats: 2.221608: 53%|██████████████████████████████▊ | 256/482 [00:00<00:00, 4168.09it/s]
Samples outside prior N = 88
Generating simulations
504it [00:00, 2715.34it/s]
WARNING:root:Too few points to create valid contours
Effective sample size for current/all rounds 1193.1/1923.9
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
---------------------- Round: 3 ----------------------
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
Epoch 0: Train, Loglike in nats: -8.199595: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3563.73it/s]
- Val, Loglike in nats: -0.491954: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 873.86it/s]
Epoch 1: Train, Loglike in nats: -0.478752: 11%|█████▎ | 512/4528 [00:00<00:02, 1772.26it/s][W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
[W ParallelNative.cpp:230] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
Epoch 1: Train, Loglike in nats: -0.049924: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3376.58it/s]
- Val, Loglike in nats: 0.445271: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6056.42it/s]
Epoch 2: Train, Loglike in nats: 0.647779: 96%|█████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3675.66it/s]
- Val, Loglike in nats: 0.973648: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6478.71it/s]
Epoch 3: Train, Loglike in nats: 0.938467: 96%|█████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3880.90it/s]
- Val, Loglike in nats: 1.037072: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6470.04it/s]
Epoch 4: Train, Loglike in nats: 0.594499: 96%|█████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3930.60it/s]
- Val, Loglike in nats: 0.695823: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6636.72it/s]
Epoch 5: Train, Loglike in nats: 0.920950: 96%|█████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3886.24it/s]
- Val, Loglike in nats: 1.280553: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6520.92it/s]
Epoch 6: Train, Loglike in nats: 1.404969: 96%|█████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3894.43it/s]
- Val, Loglike in nats: 1.621338: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 3790.74it/s]
Epoch 7: Train, Loglike in nats: 1.625396: 96%|█████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3914.95it/s]
- Val, Loglike in nats: 1.830127: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 3904.81it/s]
Epoch 8: Train, Loglike in nats: 1.534734: 96%|█████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3943.25it/s]
- Val, Loglike in nats: 1.533903: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6655.60it/s]
Epoch 9: Train, Loglike in nats: 1.622238: 96%|█████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3891.44it/s]
- Val, Loglike in nats: 1.548044: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6844.22it/s]
Epoch 10: Train, Loglike in nats: 1.803787: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3895.58it/s]
- Val, Loglike in nats: 2.010064: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6380.08it/s]
Epoch 11: Train, Loglike in nats: 1.896367: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3889.06it/s]
- Val, Loglike in nats: 1.954187: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6988.05it/s]
Epoch 12: Train, Loglike in nats: 1.850980: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3978.07it/s]
- Val, Loglike in nats: 2.120446: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6814.90it/s]
Epoch 13: Train, Loglike in nats: 1.978908: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3917.64it/s]
- Val, Loglike in nats: 1.962632: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6607.48it/s]
Epoch 14: Train, Loglike in nats: 2.001491: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3885.48it/s]
- Val, Loglike in nats: 2.111832: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6503.07it/s]
Epoch 15: Train, Loglike in nats: 1.963914: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3947.99it/s]
- Val, Loglike in nats: 2.063843: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 3803.25it/s]
Epoch 16: Train, Loglike in nats: 2.063298: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3970.89it/s]
- Val, Loglike in nats: 2.135251: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 4538.77it/s]
Epoch 17: Train, Loglike in nats: 1.940297: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3876.66it/s]
- Val, Loglike in nats: 1.993701: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 3907.33it/s]
Epoch 18: Train, Loglike in nats: 2.032573: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3904.40it/s]
- Val, Loglike in nats: 2.239735: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6429.71it/s]
Epoch 19: Train, Loglike in nats: 2.058820: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3927.06it/s]
- Val, Loglike in nats: 2.268500: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6491.36it/s]
Epoch 20: Train, Loglike in nats: 2.146838: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3942.28it/s]
- Val, Loglike in nats: 2.208792: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6555.08it/s]
Epoch 21: Train, Loglike in nats: 2.198901: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 4011.60it/s]
- Val, Loglike in nats: 2.239750: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6593.81it/s]
Epoch 22: Train, Loglike in nats: 2.158624: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3920.23it/s]
- Val, Loglike in nats: 2.276242: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6286.99it/s]
Epoch 23: Train, Loglike in nats: 2.207117: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3905.97it/s]
- Val, Loglike in nats: 2.407642: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6566.67it/s]
Epoch 24: Train, Loglike in nats: 2.225756: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3944.85it/s]
- Val, Loglike in nats: 2.271960: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6540.11it/s]
Epoch 25: Train, Loglike in nats: 2.256948: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3947.43it/s]
- Val, Loglike in nats: 2.284487: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 3517.11it/s]
Epoch 26: Train, Loglike in nats: 2.220429: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3938.45it/s]
- Val, Loglike in nats: 2.156157: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 3667.04it/s]
Epoch 27: Train, Loglike in nats: 2.168234: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3905.58it/s]
- Val, Loglike in nats: 2.229807: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6317.91it/s]
Epoch 28: Train, Loglike in nats: 2.218117: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3900.67it/s]
- Val, Loglike in nats: 2.368361: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6422.32it/s]
Epoch 29: Train, Loglike in nats: 2.273656: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3828.70it/s]
- Val, Loglike in nats: 2.327561: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6375.65it/s]
Epoch 30: Train, Loglike in nats: 2.256873: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3849.91it/s]
- Val, Loglike in nats: 2.393199: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6460.93it/s]
Epoch 31: Train, Loglike in nats: 2.284647: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3870.17it/s]
- Val, Loglike in nats: 2.443718: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6143.53it/s]
Epoch 32: Train, Loglike in nats: 2.288847: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3920.92it/s]
- Val, Loglike in nats: 2.417039: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6712.82it/s]
Epoch 33: Train, Loglike in nats: 2.278485: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3881.21it/s]
- Val, Loglike in nats: 2.372964: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6585.72it/s]
Epoch 34: Train, Loglike in nats: 2.289699: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3923.30it/s]
- Val, Loglike in nats: 2.350125: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 4651.15it/s]
Epoch 35: Train, Loglike in nats: 2.287399: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3832.88it/s]
- Val, Loglike in nats: 2.281283: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 4067.88it/s]
Epoch 36: Train, Loglike in nats: 2.296871: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3817.32it/s]
- Val, Loglike in nats: 2.459236: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6278.24it/s]
Epoch 37: Train, Loglike in nats: 2.299186: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3735.22it/s]
- Val, Loglike in nats: 2.295779: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6116.97it/s]
Epoch 38: Train, Loglike in nats: 2.336831: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3737.17it/s]
- Val, Loglike in nats: 2.331238: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 5838.09it/s]
Epoch 39: Train, Loglike in nats: 2.334529: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3715.29it/s]
- Val, Loglike in nats: 2.340693: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6161.16it/s]
Epoch 40: Train, Loglike in nats: 2.320522: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3809.15it/s]
- Val, Loglike in nats: 2.346435: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6130.27it/s]
Epoch 41: Train, Loglike in nats: 2.302252: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3752.48it/s]
- Val, Loglike in nats: 2.403435: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6166.32it/s]
Epoch 42: Train, Loglike in nats: 2.323188: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3799.64it/s]
- Val, Loglike in nats: 2.421960: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6435.56it/s]
Epoch 43: Train, Loglike in nats: 2.340944: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3827.87it/s]
- Val, Loglike in nats: 2.392607: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 4691.12it/s]
Epoch 44: Train, Loglike in nats: 2.354024: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3681.29it/s]
- Val, Loglike in nats: 2.460996: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 3954.34it/s]
Epoch 45: Train, Loglike in nats: 2.341909: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3757.12it/s]
- Val, Loglike in nats: 2.385164: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6119.97it/s]
Epoch 46: Train, Loglike in nats: 2.340359: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3825.30it/s]
- Val, Loglike in nats: 2.383424: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6276.55it/s]
Epoch 47: Train, Loglike in nats: 2.349329: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3866.40it/s]
- Val, Loglike in nats: 2.418644: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6614.81it/s]
Epoch 48: Train, Loglike in nats: 2.356575: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3309.05it/s]
- Val, Loglike in nats: 2.362294: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6019.31it/s]
Epoch 49: Train, Loglike in nats: 2.340444: 96%|████████████████████████████████████████████▏ | 4352/4528 [00:01<00:00, 3404.72it/s]
- Val, Loglike in nats: 2.356997: 51%|█████████████████████████████▍ | 256/504 [00:00<00:00, 6057.00it/s]
Samples outside prior N = 32
Generating simulations
509it [00:00, 2701.75it/s]
WARNING:root:Too few points to create valid contours
Effective sample size for current/all rounds 1059.6/2983.6
Early stop: Surrogate posterior did not improve for this round
Loaded state from round 2
[10]:
plt.rcParams['figure.dpi']=100
plt.figure(figsize=(5,5))
plt.subplot(211)
plt.plot(np.array(engine.neff)/5120*100)
plt.ylabel('Sampling Efficiency [%]')
plt.subplot(212)
plt.plot([np.sum(engine.neff[:i+1]) for i in range(4)])
plt.ylabel('Effective Sample Size')
plt.xlabel('# Round')
plt.tight_layout()
As we see, after the 3rd round of training the sampling efficiency is as high as ~25%, meaning the NDE posterior is nearly the true posterior. As expected, the 4th round of training degraded the result due to the use of NLL loss. Therefore, we revert the network to the 3rd round.
Note that every training round contributed to the final effective sample. The third round contributed 1059 effective posterior samples.
To generate more effective posterior samples, let us draw 10000 more samples from the 3nd round NDE. The first corner plot is the (unweighted) NDE surrogate posterior. The second one shows the importance reweighted posterior, which is asymptotically exact
[11]:
y_pred, weights = engine.predict(x_obs, x_err=np.array([0.1]*50),y_true=y_true, n_samples=10000, corner=True, corner_reweight=True)
WARNING:root:Too few points to create valid contours
Samples outside prior N = 177
surrogate posterior
Generating simulations
983it [00:00, 3189.64it/s]
WARNING:root:Too few points to create valid contours
Effective Sample Size = 1680.1
Sampling efficiency = 16.8%
[ ]: