lampe.inference.cnre¶
Contrastive neural ratio estimation (CNRE) components.
The principle of contrastive neural ratio estimation (CNRE) is to predict whether a set \(\Theta = \{\theta^1, \dots, \theta^K\}\) contains or not the parameters that originated an observation \(x\). The elements of \(\Theta\) are drawn independently from the prior \(p(\theta)\) and the element \(\theta^k\) that originates the observation \(x \sim p(x | \theta^k)\) is chosen uniformly within \(\Theta\), such that
where \(r(\theta, x)\) is the likelihood-to-evidence (LTE) ratio. The task is to discriminate between pairs \((\Theta, x)\) for which \(\Theta\) either does or does not contain the nominal parameters of \(x\), similar to the original NRE optimization problem. For this task, the decision function modeling the Bayes optimal classifier is
where \(\gamma \in \mathbb{R}^+\) are the odds of \(\Theta\) containing to not containing the nominal parameters. Consequently, a classifier \(d_\phi(\Theta, x)\) can be equivalently replaced and trained as a composition of ratios \(r_\phi(\theta^k, x)\).
Note
The quantity \(d_\phi(\Theta, x)\) corresponds to \(q_\phi(y \neq 0 | \Theta, x)\) or \(1 - q_\phi(y = 0 | \Theta, x)\) in the notations of Miller et al. (2022).
References
Classes¶
Descriptions¶
- class lampe.inference.cnre.CNRELoss(estimator, cardinality=2, gamma=1.0)¶
Creates a module that calculates the contrastive cross-entropy loss for a NRE network.
Given a batch of \(N \geq 2K\) pairs \((\theta_i, x_i)\), the module returns
\[l = \frac{1}{N} \sum_{i = 1}^N \frac{\gamma}{\gamma + 1} \ell(d_\phi(\Theta_i, x_i)) + \frac{1}{\gamma + 1} \ell(1 - d_\phi(\Theta_{i+K}, x_i))\]where \(\ell(p) = -\log p\) is the negative log-likelihood and \(\Theta_i = \{\theta_i, \dots, \theta_{i+K-1}\}\).
References
Contrastive Neural Ratio Estimation (Miller et al., 2022)- Parameters:
- class lampe.inference.cnre.BCNRELoss(estimator, cardinality=2, gamma=1.0, lmbda=100.0)¶
Creates a module that calculates the balanced contrastive cross-entropy loss for a NRE network.
Given a batch of \(N \geq 2K\) pairs \((\theta_i, x_i)\), the module returns
\[\begin{split}l & = \frac{1}{N} \sum_{i = 1}^N \frac{\gamma}{\gamma + 1} \ell(d_\phi(\Theta_i, x_i)) + \frac{1}{\gamma + 1} \ell(1 - d_\phi(\Theta_{i+K}, x_i)) \\ & + \lambda \left(1 - \frac{1}{N} \sum_{i = 1}^N d_\phi(\Theta_i, x_i) + d_\phi(\Theta_{i+K}, x_i) \right)^2\end{split}\]where \(\ell(p) = -\log p\) is the negative log-likelihood and \(\Theta_i = \{\theta_i, \dots, \theta_{i+K-1}\}\).
References
Balancing Simulation-based Inference for Conservative Posteriors (Delaunoy et al., 2023)- Parameters: