flags criteria

Flag Criteria for Filtering Elements

In data processing and analysis, flag criteria are essential for selecting and managing elements based on specific conditions or quality assessments. Pikobs uses a bitmask system to filter data effectively, focusing on elements that meet predefined standards or have undergone particular processing stages.

Standard Flag Criteria

When using the --flags_criteria parameter in Pikobs, the system checks specific bits to determine if an observation should be included.

all

No restrictions: Selects all available elements. No bitmask filters are applied.

assimilee

Assimilated elements: * Required Active Bits: BIT12 (4096) must be active (1). * Indicates that the observation has actively influenced the analysis. By default, this includes both clear and cloudy sky conditions.

bgckalt

Approved by Background Check: * Required Inactive Bits: BIT9 (512), BIT11 (2048), and BIT8 (256) must be inactive (0). * Removes observations rejected due to AO quality control, thinning/canal selection processes, and those on the blacklist.

bgckalt_qc

Approved by Background Check and QC-Var: * Required Inactive Bits: BIT9 (512) and BIT11 (2048) must be inactive (0).

monitoring

Monitoring filter: * Required Inactive Bits: BIT9 (512) and BIT7 (128) must be inactive (0). * Filters out observations rejected by AO quality control or satellite QC.

postalt

Post-Analysis filter: * Required Inactive Bits: BIT17 (131072), BIT9 (512), BIT11 (2048), and BIT8 (256) must be inactive (0). * Removes data rejected by QC-Var, AO quality control, thinning/canal processes, and blacklisted observations.

Sky Condition Filters (Radiance Data)

For satellite radiances, specific flags can be used to filter assimilated observations based on cloud conditions using BIT23 (Cloud-affected radiance).

clear_sky

Clear Sky Only:

• Required Active Bits: BIT12 (4096) must be active.

• Required Inactive Bits: BIT23 (8388608) must be strictly inactive (0).

• Filters assimilated radiances that are completely free of cloud contamination.

cloudy_sky

Cloudy Sky Only:

• Required Active Bits: Both BIT12 (4096) and BIT23 (8388608) must be active (1).

• Filters radiances specifically flagged as being affected by clouds that successfully passed assimilation checks.

clear_cloudy_sky

All-Sky (Clear + Cloudy):

• Required Active Bits: BIT12 (4096) must be active.

• This is the standard behavior for all-sky assimilation, accepting elements regardless of whether BIT23 is active or inactive.

Active Bits Description Reference

The following table defines all active bits, their decimal values, and their meaning within the quality control (QC) and assimilation processes:

Bitmask Definitions and Values

Bit Index	Decimal Value	Description / Rejection Reason
BIT0	1	Modified or generated by ADE.
BIT1	2	Element that exceeds a climatological extreme or fails consistency test.
BIT2	4	Erroneous element.
BIT3	8	Potentially erroneous element.
BIT4	16	Dubious element.
BIT5	32	Interpolated element, generated by DERIVATE.
BIT6	64	Corrected by DERIVATE sequence or bias correction.
BIT7	128	Rejected by satellite QC.
BIT8	256	Element rejected because it is on a blacklist.
BIT9	512	Element rejected by AO quality control (Background Check or QC-Var).
BIT10	1024	Generated by AO.
BIT11	2048	Rejected by a selection process (thinning or canal).
BIT12	4096	Assimilated (successfully used in analysis).
BIT13	8192	Comparison against test field, level 1.
BIT14	16384	Comparison against test field, level 2.
BIT15	32768	Comparison against test field, level 3.
BIT16	65536	Rejected by comparison against test field (Background Check).
BIT17	131072	Rejected by QC-Var.
BIT18	262144	Not used due to orography.
BIT19	524288	Not used due to land-sea mask.
BIT20	1048576	Aircraft position error detected by TrackQC.
BIT21	2097152	Inconsistency detected by a CQ process.
BIT22	4194304	Rejected due to transmittance above model top.
BIT23	8388608	Cloud-affected radiance.