Skip to content

Commit

Permalink
Fix relative urls
Browse files Browse the repository at this point in the history
  • Loading branch information
@BlackHC
BlackHC committed Mar 31, 2024
1 parent 2ea36a5 commit a5e7d69
Showing 1 changed file with 8 additions and 8 deletions.
16 changes: 8 additions & 8 deletions _posts/2024-05-07-clml.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -556,10 +556,10 @@ The following visualization summarizes the relationship between the conditional

<figure markdown="1" class="l-page rounded z-depth-1">
<picture>
<source class="responsive-img-srcset" media="(max-width: 480px)" srcset="/2024/assets/img/2024-05-07-clml/area_under_curve_1.00-480.webp">
<source class="responsive-img-srcset" media="(max-width: 800px)" srcset="/2024/assets/img/2024-05-07-clml/area_under_curve_1.00-800.webp">
<source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="/2024/assets/img/2024-05-07-clml/area_under_curve_1.00-1400.webp">
<img class="img-fluid" src="/2024/assets/img/2024-05-07-clml/area_under_curve_1.00.png" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();">
<source class="responsive-img-srcset" media="(max-width: 480px)" srcset="{{"/assets/img/2024-05-07-clml/area_under_curve_1.00-480.webp" | relative_url}}">
<source class="responsive-img-srcset" media="(max-width: 800px)" srcset="{{"/assets/img/2024-05-07-clml/area_under_curve_1.00-800.webp" | relative_url}}">
<source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="{{"/assets/img/2024-05-07-clml/area_under_curve_1.00-1400.webp" | relative_url}}">
<img class="img-fluid" src="{{"/assets/img/2024-05-07-clml/area_under_curve_1.00.png" | relative_url}}" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();">
</picture>
<figcaption class="caption" markdown="1">
*The relationship between conditional and joint marginal cross-entropies and information.*
Expand Down Expand Up @@ -737,7 +737,7 @@ A **prior-data conflict** occurs when the prior distribution hinders the model f

In the low-data regime, assuming all models converge to the same validation loss given infinite data, we prefer the model that converges the fastest, i.e., with the least amount of training data. A model with a prior well-aligned with the data distribution learns efficiently and generalizes better with limited data.

<figure markdown="1" class="l-page rounded z-depth-1"> <picture> <source class="responsive-img-srcset" media="(max-width: 480px)" srcset="/2024/assets/img/2024-05-07-clml/prior_conflict_and_model_misspecification_0.67-480.webp"> <source class="responsive-img-srcset" media="(max-width: 800px)" srcset="/2024/assets/img/2024-05-07-clml/prior_conflict_and_model_misspecification_0.67-800.webp"> <source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="/2024/assets/img/2024-05-07-clml/prior_conflict_and_model_misspecification_0.67-1400.webp"> <img class="img-fluid" src="/2024/assets/img/2024-05-07-clml/prior_conflict_and_model_misspecification_0.67.svg" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"> </picture>
<figure markdown="1" class="l-page rounded z-depth-1"> <picture> <source class="responsive-img-srcset" media="(max-width: 480px)" srcset="{{"/assets/img/2024-05-07-clml/prior_conflict_and_model_misspecification_0.67-480.webp" | relative_url}}"> <source class="responsive-img-srcset" media="(max-width: 800px)" srcset="{{"/assets/img/2024-05-07-clml/prior_conflict_and_model_misspecification_0.67-800.webp" | relative_url}}"> <source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="{{"/assets/img/2024-05-07-clml/prior_conflict_and_model_misspecification_0.67-1400.webp" | relative_url}}"> <img class="img-fluid" src="{{"/assets/img/2024-05-07-clml/prior_conflict_and_model_misspecification_0.67.svg" | relative_url}}" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"> </picture>
<figcaption class="caption" markdown="1">
*Conditional marginal cross-entropy vs. dataset size under different modeling scenarios.*
**Left: Model misspecification** - Three model hypotheses ($$\h_1$$, $$\h_2$$, $$\h_3$$) converge to different losses due to the model class not containing the true data-generating process. The minimum achievable loss represents the misspecification error.
Expand All @@ -754,7 +754,7 @@ Finally, what happens when there are both model misspecification and a prior-dat

Let's visualize this: the curves will intersect at some point, and the model with the best achievable loss in the infinite data limit might not be the best choice in the low-data regime, depending on how much data we can train on. The optimal model choice may also change based on the amount of available data.

<figure markdown="1" class="l-body rounded z-depth-1"> <picture> <source class="responsive-img-srcset" media="(max-width: 480px)" srcset="/2024/assets/img/2024-05-07-clml/anticorrelated_prior_conflict_and_model_misspecification_1.30-480.webp"> <source class="responsive-img-srcset" media="(max-width: 800px)" srcset="/2024/assets/img/2024-05-07-clml/anticorrelated_prior_conflict_and_model_misspecification_1.30-800.webp"> <source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="/2024/assets/img/2024-05-07-clml/anticorrelated_prior_conflict_and_model_misspecification_1.30-1400.webp"> <img class="img-fluid" src="/2024/assets/img/2024-05-07-clml/anticorrelated_prior_conflict_and_model_misspecification_1.30.svg" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"> </picture>
<figure markdown="1" class="l-body rounded z-depth-1"> <picture> <source class="responsive-img-srcset" media="(max-width: 480px)" srcset="{{"/assets/img/2024-05-07-clml/anticorrelated_prior_conflict_and_model_misspecification_1.30-480.webp" | relative_url}}"> <source class="responsive-img-srcset" media="(max-width: 800px)" srcset="{{"/assets/img/2024-05-07-clml/anticorrelated_prior_conflict_and_model_misspecification_1.30-800.webp" | relative_url}}"> <source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="{{"/assets/img/2024-05-07-clml/anticorrelated_prior_conflict_and_model_misspecification_1.30-1400.webp" | relative_url}}"> <img class="img-fluid" src="{{"/assets/img/2024-05-07-clml/anticorrelated_prior_conflict_and_model_misspecification_1.30.svg" | relative_url}}" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"> </picture>
<figcaption class="caption" markdown="1">
*The conditional marginal cross-entropy is plotted for three different model hypotheses ($$\h_0$$, $$\h_1$$, $$\h_2$$) as a function of dataset size. The models exhibit both prior-data conflict and model misspecification.*
In the small data regime, $$\h_2$$ has the lowest loss due to its prior aligning well with the data distribution, allowing for faster initial learning. However, as more data becomes available, the models' asymptotic performance quickly plateaus. First, $$\h_1$$ takes over, and then finally $$\h_0$$, which converges to the lowest achievable loss in the infinite data limit, indicating it suffers the least from model misspecification. In contrast, $$\h_1$$ and $$\h_2$$ converge to higher loss values due to greater misspecification.
Expand Down Expand Up @@ -928,7 +928,7 @@ include figure.html path="assets/img/2024-05-07-clml/binary_regression_informati
class="l-screen-inset img-fluid rounded z-depth-1"
{% endcomment %}

<figure markdown="1" class="l-page rounded z-depth-1"> <picture> <source class="responsive-img-srcset" media="(max-width: 480px)" srcset="/2024/assets/img/2024-05-07-clml/binary_regression_information_metrics-480.webp"> <source class="responsive-img-srcset" media="(max-width: 800px)" srcset="/2024/assets/img/2024-05-07-clml/binary_regression_information_metrics-800.webp"> <source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="/2024/assets/img/2024-05-07-clml/binary_regression_information_metrics-1400.webp"> <img class="img-fluid" src="/2024/assets/img/2024-05-07-clml/binary_regression_information_metrics.png" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"> </picture>
<figure markdown="1" class="l-page rounded z-depth-1"> <picture> <source class="responsive-img-srcset" media="(max-width: 480px)" srcset="{{"/assets/img/2024-05-07-clml/binary_regression_information_metrics-480.webp" | relative_url}}"> <source class="responsive-img-srcset" media="(max-width: 800px)" srcset="{{"/assets/img/2024-05-07-clml/binary_regression_information_metrics-800.webp" | relative_url}}"> <source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="{{"/assets/img/2024-05-07-clml/binary_regression_information_metrics-1400.webp" | relative_url}}"> <img class="img-fluid" src="{{"/assets/img/2024-05-07-clml/binary_regression_information_metrics.png" | relative_url}}" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"> </picture>
<figcaption class="caption" markdown="1">
*Information metrics for the three Bayesian linear regression models as a function of dataset size.* The joint marginal information does not indicate the best performing model. The conditional joint marginal information (conditioned on half the dataset size, predicting on the other half) only finds the best model after 4/5 of the data are observed. *Metrics are reported in bits (log base 2), five trials each.*
</figcaption>
Expand All @@ -949,7 +949,7 @@ include figure.html path="assets/img/2024-05-07-clml/binary_regression_condition
class="l-screen-inset img-fluid rounded z-depth-1"
{% endcomment %}

<figure markdown="1" class="l-body rounded z-depth-1"> <picture> <source class="responsive-img-srcset" media="(max-width: 480px)" srcset="/2024/assets/img/2024-05-07-clml/binary_regression_conditional_joint_marginal_information_decision_boundary-480.webp"> <source class="responsive-img-srcset" media="(max-width: 800px)" srcset="/2024/assets/img/2024-05-07-clml/binary_regression_conditional_joint_marginal_information_decision_boundary-800.webp"> <source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="/2024/assets/img/2024-05-07-clml/binary_regression_conditional_joint_marginal_information_decision_boundary-1400.webp"> <img class="img-fluid mx-auto" src="/2024/assets/img/2024-05-07-clml/binary_regression_conditional_joint_marginal_information_decision_boundary.png" style="max-width: 400px; display: block;" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"> </picture>
<figure markdown="1" class="l-body rounded z-depth-1"> <picture> <source class="responsive-img-srcset" media="(max-width: 480px)" srcset="{{"/assets/img/2024-05-07-clml/binary_regression_conditional_joint_marginal_information_decision_boundary-480.webp" | relative_url}}"> <source class="responsive-img-srcset" media="(max-width: 800px)" srcset="{{"/assets/img/2024-05-07-clml/binary_regression_conditional_joint_marginal_information_decision_boundary-800.webp" | relative_url}}"> <source class="responsive-img-srcset" media="(max-width: 1400px)" srcset="{{"/assets/img/2024-05-07-clml/binary_regression_conditional_joint_marginal_information_decision_boundary-1400.webp" | relative_url}}"> <img class="img-fluid mx-auto" src="{{"/assets/img/2024-05-07-clml/binary_regression_conditional_joint_marginal_information_decision_boundary.png" | relative_url}}" style="max-width: 400px; display: block;" width="auto" height="auto" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"> </picture>
<figcaption class="caption" markdown="1">
*Decision boundary for the best model amongst three ($$\phi_1$$, $$\phi_2$$, $$\phi_3$$) with the lowest conditional joint marginal cross-entropy/information, as a function of dataset size and held-back size.* The three models $$\phi_1$$, $$\phi_2$$, and $$\phi_3$$ correspond to different prior variances and noise levels. The white diagonal line shows where the conditional joint marginal information is computed using half the dataset size. In the region below this line, $$\phi_1$$ (blue) has the lowest conditional joint marginal information, while $$\phi_2$$ (orange) and $$\phi_3$$ (green) are preferred for different dataset and held-back sizes.
</figcaption>
Expand Down

0 comments on commit a5e7d69

Please sign in to comment.