Vdedr¶

Verification of Radiance (VDEDR Diagnostic)¶

The Pikobs VDEDR Suite is a diagnostic tool designed to evaluate the effectiveness of bias corrections and highlight systematic differences in radiance data across experiments or datasets. By analyzing residual and raw biases, it provides essential insights into the accuracy and consistency of the radiance verification system.

Core Visualization: Bias and Error Analysis¶

This module produces three key plots to analyze biases and errors across datasets. The radiance channels serve as the y-axis in all plots, facilitating direct, channel-wise comparisons.

1. Bias Plot

Compares the residual bias (the difference between the average OMP values of the two datasets) and the raw bias (the difference between the bias-corrected OMP values). This plot highlights variations in bias measurements to aid in detecting dataset performance differences.

2. Delta Error Plot (OMP)

Shows the percentage difference in the Standard Deviation of OMP (% sigma) and the number of observations (% Nobs) between the two datasets. This evaluates relative changes in measurement errors and observation counts.

3. Delta Error Plot (OMA)

Similar to the OMP plot, but focuses on OMA (Observation minus Analysis). It assesses relative changes in OMA errors and observation counts, helping identify patterns specific to analysis deviations.

Interactive Web Environment¶

Beyond static plots, Pikobs automatically generates a Smart Web Viewer. This tool is designed for rapid multi-dimensional and comparative analysis:

Dynamic Filtering: The interface only displays valid combinations of metrics (OMP Bias, OMP Rel, OMA Rel), satellite, channel, and region available in your dataset.
Flip-Flop Mode: Allows instant toggling between two different plots (e.g., Control vs Experiment) to detect subtle changes without losing visual focus.
Access: The generated interactive viewer is linked directly here:

🔗 Explore the Viewer: Interactive Web Viewer

HPC Execution & Real-Time Monitoring¶

To maintain system stability on Environment and Climate Change Canada (ECCC) clusters, always execute diagnostic tasks using a dedicated compute node.

1. Initialize Interactive Session:

qsub -I -X -l select=4:ncpus=80:mpiprocs=80:ompthreads=1:mem=185gb -l place=scatter -l walltime=6:0:0

2. Generate Diagnostics Command:

python -c 'import pikobs; pikobs.vdedr.arg_call()' \
    --path_control_files      /home/dlo001/sites8/Data_pikobs/monitoring0/ \
    --control_name            g0 \
    --path_experience_files   a/home/dlo001/sites8/Data_pikobs/monitoring/ \
    --experience_name         g2 \
    --pathwork                /your/custom/output/path \
    --datestart               2026020100 \
    --dateend                 2026020600 \
    --region                  Monde Canada \
    --family                  to_amsua_allsky atms_allsky \
    --flags_criteria          iclear_sky cloudy_sky clear_cloudy_sky \
    --id_stn                  all \
    --n_cpu                   40

3. Expected Shell Output (Batch Monitoring):

When executing the command above, you will see the parallel processing progress in your terminal:

[PIKOBS] Initializing VDEDR Verification Engine...
[INFO] Detected 40 CPUs for parallel processing.
[DB] Connecting to Control: g0
[DB] Connecting to Experience: g2
[PARALLEL] Computing Bias and Delta Errors for family: to_amsua_allsky...
[PARALLEL] Generating plots for METOP-1...
[WEB] Compiling interactive web viewer data...
✅ Plot successfully saved: /your/custom/output/path/to_amsua_allsky/omp_bias_to_amsua_allsky_METOP-1_g0-g2_Monde_assimilee.png
✅ Advanced Web viewer successfully created at: /your/custom/output/path/pikobs_vdedr_viewer.html
[DONE] All verifications completed successfully.

Support and Issues¶

Pikobs is maintained for the ECCC community. For bug reports, feature requests, or technical issues, please use the official GitLab repository:

🔗 GitLab Issues: https://gitlab.science.gc.ca/dlo001/Pikobs

Document generated automatically for the Pikobs Atmospheric Science Project.