Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: CDAT Migration - Fix large relative diff and x and y nan location mismatch for some variables in zonal_mean_xy set #790

Closed
tomvothecoder opened this issue Feb 15, 2024 · 1 comment · Fixed by #794
Labels
bug Bug fix (will increment patch version) cdat-migration-fy24 CDAT Migration FY24 Task

Comments

@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Feb 15, 2024

What happened?

Related to #752 (comment)

Source regression testing notebook: https://github.com/E3SM-Project/e3sm_diags/blob/cdat-migration-fy24/auxiliary_tools/cdat_regression_testing/654-zonal_mean_xy/debug_654-zonal_mean_xy_cdat_regression_test_netcdf_debug.ipynb

What did you expect to happen? Are there are possible answers you came across?

For most the variables (CLDTOT_TAU_1.3_9.4_ISSCP, etc.), it is most likely related to cosp_bin_sum().

Notice in the log output how the outputs for cdat_migration_fy24 are using

Minimal Complete Verifiable Example (MVCE)

No response

Relevant log output

Large relative difference

PminusE

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/COREv2_Flux/COREv2_Flux-PminusE-ANN-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/COREv2_Flux/COREv2_Flux-PminusE-ANN-global_ref.nc
var_key PminusE

Not equal to tolerance rtol=1e-05, atol=0

Mismatched elements: 169 / 180 (93.9%)
Max absolute difference: 2.68669262
Max relative difference: 0.99998843
 x: array([          nan,           nan,           nan,           nan,
                 nan,           nan,           nan,           nan,
                 nan,           nan,           nan, -1.362539e-06,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan, -0.117723,
       -0.275558, -0.151884, -0.083765, -0.021892,  0.034201,  0.038609,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/COREv2_Flux/COREv2_Flux-PminusE-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/COREv2_Flux/COREv2_Flux-PminusE-JJA-global_ref.nc
var_key PminusE

Not equal to tolerance rtol=1e-05, atol=0

Mismatched elements: 169 / 180 (93.9%)
Max absolute difference: 4.20649947
Max relative difference: 0.99998843
 x: array([          nan,           nan,           nan,           nan,
                 nan,           nan,           nan,           nan,
                 nan,           nan,           nan, -3.314214e-06,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan, -0.286348,
       -0.466065, -0.300971, -0.246707, -0.202872, -0.148632, -0.135121,...

x and y nan location mismatch

Possible cause:

  • Regridding using xesmf with unmapped_to_nan=False, when it should be unmapped_to_nan=True

ALBEDO, ALBEDOC, CLDTOT_TAU1.3_9.4_ISCCP, CLDTOT_TAU1.3_9.4_ISCCP, CLDTOT_TAU1.3_ISCCP, CLDTOT_TAU9.4_ISCCP, CLDLOW_TAU1.3_9.4_MISR, CLDLOW_TAU1.3_MISR, CLDLOW_TAU9.4_MISR, CLDTOT_TAU1.3_9.4_MISR, CLDTOT_TAU1.3_MISR, CLDTOT_TAU9.4_MISR, CLDHGH_TAU1.3_9.4_MODIS, CLDHGH_TAU1.3_MODIS, CLDHGH_TAU9.4_MODIS, CLDTOT_TAU1.3_9.4_MODIS, CLDTOT_TAU1.3_MODIS, CLDTOT_TAU9.4_MODIS

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/CERES-EBAF-TOA-v4.1/ceres_ebaf_toa_v4.1-ALBEDO-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/CERES-EBAF-TOA-v4.1/ceres_ebaf_toa_v4.1-ALBEDO-JJA-global_ref.nc
var_key ALBEDO

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([     inf,      inf,      inf,      inf,      inf,      inf,
            inf,      inf,      inf, 4.905806, 1.104448, 0.486332,
       1.070643, 0.9697  , 0.895325, 0.848754, 0.816032, 0.794203,...
 y: array([         nan,          nan,          nan,          nan,
                nan,          nan,          nan,          nan,
       2.497374e+06, 4.905807e+00, 1.104448e+00, 4.863323e-01,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/CERES-EBAF-TOA-v4.1/ceres_ebaf_toa_v4.1-ALBEDOC-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/CERES-EBAF-TOA-v4.1/ceres_ebaf_toa_v4.1-ALBEDOC-JJA-global_ref.nc
var_key ALBEDOC

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([     inf,      inf,      inf,      inf,      inf,      inf,
            inf,      inf,      inf, 2.820785, 2.415148, 2.439055,
       2.429912, 2.546782, 2.106862, 1.591239, 1.257344, 1.041096,...
 y: array([         nan,          nan,          nan,          nan,
                nan,          nan,          nan,          nan,
       2.353753e+04, 2.820784e+00, 2.415149e+00, 2.439055e+00,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU1.3_9.4_ISCCP-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU1.3_9.4_ISCCP-JJA-global_ref.nc
var_key CLDTOT_TAU1.3_9.4_ISCCP

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      , 47.524376, 49.217592, 40.503558,
       40.208695, 42.368686, 41.70946 , 44.171651, 44.821778, 45.508399,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan, 48.193733, 49.217592, 40.503558,
       40.208696, 42.368686, 41.70946 , 44.17165 , 44.821777, 45.5084  ,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU1.3_9.4_ISCCP-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU1.3_9.4_ISCCP-JJA-global_test.nc
var_key CLDTOT_TAU1.3_9.4_ISCCP

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  1.669266,  9.472262, 12.565807, 14.592382,
       17.258383, 18.448257, 19.743159, 20.801946, 22.11058 , 24.988214,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  4.925703,  9.472262, 12.565807, 14.592382,
       17.258383, 18.448257, 19.743159, 20.801945, 22.110581, 24.988214,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU1.3_ISCCP-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU1.3_ISCCP-JJA-global_ref.nc
var_key CLDTOT_TAU1.3_ISCCP

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      , 55.56152 , 60.006977, 66.614349,
       72.348068, 76.230691, 75.118569, 73.723432, 72.566653, 70.505745,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan, 56.344076, 60.006977, 66.61435 ,
       72.348068, 76.230692, 75.11857 , 73.723432, 72.566652, 70.505744,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU1.3_ISCCP-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU1.3_ISCCP-JJA-global_test.nc
var_key CLDTOT_TAU1.3_ISCCP

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  2.153515, 11.169407, 14.355866, 16.5108  ,
       19.725695, 21.38418 , 23.471586, 25.789955, 28.381688, 32.562917,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  6.354634, 11.169407, 14.355866, 16.5108  ,
       19.725695, 21.384179, 23.471586, 25.789955, 28.381688, 32.562917,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU9.4_ISCCP-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU9.4_ISCCP-JJA-global_ref.nc
var_key CLDTOT_TAU9.4_ISCCP

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  8.037144, 10.789386, 26.110793,
       32.139373, 33.862005, 33.40911 , 29.551782, 27.744875, 24.997345,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,  8.150343, 10.789385, 26.110793,
       32.139373, 33.862005, 33.40911 , 29.551782, 27.744875, 24.997345,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU9.4_ISCCP-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud ISCCP/ISCCPCOSP-CLDTOT_TAU9.4_ISCCP-JJA-global_test.nc
var_key CLDTOT_TAU9.4_ISCCP

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.484249,  1.697145,  1.790059,  1.918418,
        2.467312,  2.935922,  3.728426,  4.98801 ,  6.271108,  7.574703,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  1.428931,  1.697145,  1.790059,  1.918418,
        2.467312,  2.935922,  3.728426,  4.988009,  6.271108,  7.574703,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_9.4_MISR-ANN-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_9.4_MISR-ANN-global_ref.nc
var_key CLDLOW_TAU1.3_9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        1.533745,  3.139232,  3.324253,  3.274897,  4.989359,  6.846378,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
       29.236507, 28.109412, 28.533893, 27.251634, 26.469386, 26.97218 ,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_9.4_MISR-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_9.4_MISR-JJA-global_ref.nc
var_key CLDLOW_TAU1.3_9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_9.4_MISR-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_9.4_MISR-JJA-global_test.nc
var_key CLDLOW_TAU1.3_9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  1.198617,  7.309172,  9.268335,  9.669798,
       11.19571 , 11.517812, 11.814792, 12.461223, 13.276728, 15.05141 ,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  3.536903,  7.309172,  9.268335,  9.669798,
       11.19571 , 11.517812, 11.814792, 12.461223, 13.276728, 15.051411,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_MISR-ANN-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_MISR-ANN-global_ref.nc
var_key CLDLOW_TAU1.3_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        2.806976,  6.224152,  6.523888,  6.707491, 10.036236, 13.568533,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
       53.769157, 55.229167, 55.455046, 55.463217, 53.257188, 53.239823,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_MISR-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_MISR-JJA-global_ref.nc
var_key CLDLOW_TAU1.3_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_MISR-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU1.3_MISR-JJA-global_test.nc
var_key CLDLOW_TAU1.3_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  1.682866,  8.67263 , 10.420182, 10.854651,
       12.601834, 13.196861, 13.898068, 15.160259, 16.787511, 19.424786,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  4.965834,  8.67263 , 10.420182, 10.854651,
       12.601835, 13.196861, 13.898068, 15.160259, 16.787511, 19.424786,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU9.4_MISR-ANN-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU9.4_MISR-ANN-global_ref.nc
var_key CLDLOW_TAU9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        1.273231,  3.08492 ,  3.199634,  3.432593,  5.046877,  6.722155,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
       24.53265 , 27.119755, 26.921153, 28.211582, 26.787802, 26.267643,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU9.4_MISR-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU9.4_MISR-JJA-global_ref.nc
var_key CLDLOW_TAU9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU9.4_MISR-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDLOW_TAU9.4_MISR-JJA-global_test.nc
var_key CLDLOW_TAU9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.484249,  1.363458,  1.151847,  1.184853,
        1.406124,  1.679048,  2.083276,  2.699036,  3.510783,  4.373375,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  1.428931,  1.363458,  1.151847,  1.184853,
        1.406124,  1.679048,  2.083276,  2.699036,  3.510783,  4.373375,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_9.4_MISR-ANN-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_9.4_MISR-ANN-global_ref.nc
var_key CLDTOT_TAU1.3_9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        2.182318,  4.534118,  4.810252,  4.755759,  7.591126, 10.469784,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
       39.281719, 37.960055, 38.48202 , 37.21898 , 37.955631, 38.85693 ,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_9.4_MISR-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_9.4_MISR-JJA-global_ref.nc
var_key CLDTOT_TAU1.3_9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_9.4_MISR-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_9.4_MISR-JJA-global_test.nc
var_key CLDTOT_TAU1.3_9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  1.663766,  9.352386, 12.374152, 14.459126,
       17.169302, 18.363139, 19.672765, 20.744298, 22.064177, 24.981155,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  4.909472,  9.352386, 12.374152, 14.459126,
       17.169302, 18.363139, 19.672765, 20.744298, 22.064177, 24.981155,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_MISR-ANN-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_MISR-ANN-global_ref.nc
var_key CLDTOT_TAU1.3_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        4.052755,  8.979892,  9.465165, 10.054856, 15.900496, 21.680996,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
       72.94959 , 75.180491, 75.721322, 78.690177, 79.502482, 80.465552,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_MISR-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_MISR-JJA-global_ref.nc
var_key CLDTOT_TAU1.3_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_MISR-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU1.3_MISR-JJA-global_test.nc
var_key CLDTOT_TAU1.3_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  2.148015, 10.969964, 13.916991, 16.181666,
       19.454391, 21.16124 , 23.274337, 25.667828, 28.305911, 32.553575,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  6.338404, 10.969964, 13.916991, 16.181667,
       19.454391, 21.16124 , 23.274337, 25.667828, 28.305911, 32.553575,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU9.4_MISR-ANN-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU9.4_MISR-ANN-global_ref.nc
var_key CLDTOT_TAU9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        1.870437,  4.445774,  4.654913,  5.299097,  8.30937 , 11.211212,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
       33.667874, 37.220435, 37.239302, 41.471195, 41.546851, 41.608623,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU9.4_MISR-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU9.4_MISR-JJA-global_ref.nc
var_key CLDTOT_TAU9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU9.4_MISR-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MISR/MISRCOSP-CLDTOT_TAU9.4_MISR-JJA-global_test.nc
var_key CLDTOT_TAU9.4_MISR

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.484249,  1.617579,  1.542839,  1.72254 ,
        2.28509 ,  2.798101,  3.601572,  4.92353 ,  6.241734,  7.572421,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  1.428931,  1.617579,  1.542839,  1.72254 ,
        2.28509 ,  2.798101,  3.601572,  4.92353 ,  6.241734,  7.572421,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU1.3_9.4_MODIS-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU1.3_9.4_MODIS-JJA-global_ref.nc
var_key CLDHGH_TAU1.3_9.4_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  2.841666,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,  8.524998,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU1.3_9.4_MODIS-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU1.3_9.4_MODIS-JJA-global_test.nc
var_key CLDHGH_TAU1.3_9.4_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.204885,  0.682115,  1.070389,  1.753423,
        2.495639,  2.97833 ,  3.746221,  4.015793,  4.277098,  4.910514,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  0.604578,  0.682115,  1.070389,  1.753423,
        2.495639,  2.97833 ,  3.746221,  4.015793,  4.277098,  4.910514,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU1.3_MODIS-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU1.3_MODIS-JJA-global_ref.nc
var_key CLDHGH_TAU1.3_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      , 10.675877,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan, 32.027627,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU1.3_MODIS-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU1.3_MODIS-JJA-global_test.nc
var_key CLDHGH_TAU1.3_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.204885,  0.879539,  1.269787,  1.905216,
        2.764641,  3.309287,  4.264017,  4.998515,  5.625095,  6.605741,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  0.604578,  0.879539,  1.269787,  1.905216,
        2.764641,  3.309287,  4.264017,  4.998515,  5.625095,  6.605741,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU9.4_MODIS-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU9.4_MODIS-JJA-global_ref.nc
var_key CLDHGH_TAU9.4_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  7.834209,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan, 23.502629,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU9.4_MODIS-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDHGH_TAU9.4_MODIS-JJA-global_test.nc
var_key CLDHGH_TAU9.4_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.197424,  0.199398,  0.151793,
        0.269002,  0.330956,  0.517796,  0.982722,  1.347997,  1.695226,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  0.      ,  0.197424,  0.199398,  0.151793,
        0.269002,  0.330956,  0.517796,  0.982722,  1.347997,  1.695226,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU1.3_9.4_MODIS-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU1.3_9.4_MODIS-JJA-global_ref.nc
var_key CLDTOT_TAU1.3_9.4_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  8.471089,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan, 25.413268,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU1.3_9.4_MODIS-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU1.3_9.4_MODIS-JJA-global_test.nc
var_key CLDTOT_TAU1.3_9.4_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  1.663766,  9.298612, 12.20772 , 14.193612,
       16.819631, 17.998111, 19.324056, 20.295985, 21.57166 , 24.440842,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  4.909472,  9.298612, 12.20772 , 14.193612,
       16.819631, 17.998111, 19.324056, 20.295985, 21.57166 , 24.440842,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU1.3_MODIS-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU1.3_MODIS-JJA-global_ref.nc
var_key CLDTOT_TAU1.3_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      , 25.593424,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan, 76.780271,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU1.3_MODIS-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU1.3_MODIS-JJA-global_test.nc
var_key CLDTOT_TAU1.3_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  2.148015, 10.91249 , 13.742969, 15.900276,
       19.086355, 20.755151, 22.864176, 25.167742, 27.737386, 31.900404,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  6.338404, 10.91249 , 13.742969, 15.900276,
       19.086355, 20.755151, 22.864176, 25.167742, 27.737385, 31.900404,...

Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU9.4_MODIS-JJA-global_ref.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU9.4_MODIS-JJA-global_ref.nc
var_key CLDTOT_TAU9.4_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.      ,  0.      ,  0.      , 17.122334,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,       nan,       nan,       nan, 51.367002,...
Comparing:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/debug-654-zonal_mean_xy/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU9.4_MODIS-JJA-global_test.nc 
 /global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main/zonal_mean_xy/Cloud MODIS/MODISCOSP-CLDTOT_TAU9.4_MODIS-JJA-global_test.nc
var_key CLDTOT_TAU9.4_MODIS

Not equal to tolerance rtol=1e-05, atol=0

x and y nan location mismatch:
 x: array([ 0.      ,  0.      ,  0.      ,  0.      ,  0.      ,  0.      ,
        0.      ,  0.      ,  0.484249,  1.613878,  1.535249,  1.706663,
        2.266725,  2.75704 ,  3.54012 ,  4.871757,  6.165725,  7.459562,...
 y: array([      nan,       nan,       nan,       nan,       nan,       nan,
             nan,       nan,  1.428931,  1.613878,  1.535249,  1.706663,
        2.266725,  2.75704 ,  3.54012 ,  4.871757,  6.165725,  7.459562,...

Anything else we need to know?

No response

Environment

Latest cdat_migration_fy24 branch

@tomvothecoder tomvothecoder added the bug Bug fix (will increment patch version) label Feb 15, 2024
@tomvothecoder tomvothecoder changed the title [Bug]: CDAT Migration - Fix x and y nan location mismatch for some variables in zonal_mean_xy set [Bug]: CDAT Migration - Fix large relative diff and x and y nan location mismatch for some variables in zonal_mean_xy set Feb 15, 2024
@tomvothecoder tomvothecoder added the cdat-migration-fy24 CDAT Migration FY24 Task label Feb 26, 2024
@tomvothecoder
Copy link
Collaborator Author

Closed by #794

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug fix (will increment patch version) cdat-migration-fy24 CDAT Migration FY24 Task
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant