NAME

  [1mtdda detect [22mâ Detect data that does not obey supplied constraints

SYNOPSIS

  [1mtdda detect [22m[‐h] [‐?] [‐7] [‐‐no‐config] [‐‐colour] [‐‐no‐colour]
              [‐epsilon EPSILON] [‐o REPORT_PATH] [‐a] [‐f]
              [‐t {strict,loose}] [‐‐write‐all‐records]
              [‐‐per‐constraint] [‐‐no‐per‐constraint]
              [‐‐no‐original‐fields] [‐‐original‐fields]
              [‐‐no‐output‐fields] [‐‐output‐fields [OUTPUT_FIELDS ...]]
              [‐r [REPORT ...]] [‐‐interleave] [‐‐no‐interleave]
              [‐‐index] [‐‐int] [‐‐key [KEY ...]]
              [‐‐verify‐required‐fields] [‐‐verify‐allowed‐fields]
              [‐‐no‐verify‐required‐fields] [‐‐no‐verify‐allowed‐fields]
              [‐‐varf] [‐‐no‐varf] [‐‐pandas] [‐‐polars]
              [‐‐backend BACKEND]
              [4mINPUT[24m [[4mCONSTRAINTS[24m [[4mOUTPUT[24m]]

POSITIONAL ARGUMENTS

  [4mINPUT[24m is one of:
    ‐ a CSV file or other flat file (e.g. [1m.csv[22m, [1m.txt[22m, [1m.psv[22m),
      optionally using [1m: [22mformat to specify flat‐file metadata
      (see the help for [1mtdda serial[22m)
    ‐ a data frame in a Parquet file ([1m.parquet[22m)
      e.g. from pandas, polars, R
    ‐ a table from PostgreSQL databases (e.g. [1mpostgres:tablename[22m)
    ‐ a table from MySQL databases (e.g. [1mmysql:tablename[22m)
    ‐ a table from SQLite databases (e.g. [1msqlite:tablename[22m)
    ‐ Standard input (stdin): Use [1m‐ [22mto read from stdin

  [4mCONSTRAINTS[24m, if provided, is a JSON [1m.tdda [22mfile containing
  constraints.

  If no constraints file is provided, a file with the same path as
  the input file, with a [1m.tdda [22mextension will be tried.

  [4mOUTPUT[24m specifies the destination for detected records.

  This is usually a file if the input was a file (e.g. a [1m.csv[0m
  file or a [1mparquet [22mfile), but does not have to be the same type.
  If the input is a database table, the output is always a database
  table in the same database.

DESCRIPTION

  The [1mtdda detect [22mcommand finds and reports data that fails to satisfy
  the constraints in the [4mCONSTRAINTS[24m file specified. It also performs all
  the same functions as [1mtdda verify[22m.

OPTIONS

  [1m‐h[22m, [1m‐‐help              [22mShow this help message and exit
  [1m‐?[22m, [1m‐‐?                 [22mSame as [1m‐h [22mor [1m‐‐help[0m
  [1m‐7[22m, [1m‐‐ascii             [22mReport without using special characters
  [1m‐N[22m, [1m‐‐no‐config         [22mSkip loading [1m˜/.tdda.toml[0m

  [1m‐‐colour                [22mUse colour in terminal output
  [1m‐‐no‐colour             [22mDo not use colour in terminal output

  [1m‐‐epsilon [4m[22mEPSILON[24m       Epsilon fuzziness (tolerance for comparisons)

  [1m‐a[22m, [1m‐‐all               [22mReport all fields, even if there are no
                          failures
  [1m‐f[22m, [1m‐‐fields            [22mReport only fields with failures

  [1m‐r[22m, [1m‐‐report [22m[[4mREPORT[24m ...]
                          Report formats to write, space‐separated.
                          Formats: [1mhtml[22m, [1mmd [22m([1mmarkdown[22m), [1mtxt [22m([1mtext[22m),
                          [1mjson[22m, [1myaml[22m, [1mtoml[22m.
                          The stem of the output file is taken from
                          [4mREPORT_PATH[24m if [1m‐o [22mis given, otherwise from
                          [4mOUTPUT[24m.
  [1m‐t[22m, [1m‐‐type_checking [22m{[4mstrict[24m,[4mloose[24m}
                          "loose" means consider all numeric types
                          equivalent
  [1m‐o[22m, [1m‐‐report‐path [4m[22mREPORT_PATH[0m
                          Stem path for report files (extension is
                          replaced by the format).

  [1m‐‐write‐all‐records   [22mInclude passing records
  [1m‐‐per‐constraint      [22mWrite one flag column per failing constraint in
                          addition to n_failures. Set by default.
  [1m‐‐no‐per‐constraint   [22mDo not write out any per‐constraint flag columns
  [1m‐‐no‐original‐fields  [22mDo not write out original fields columns
  [1m‐‐original‐fields     [22mWrite out original fields columns (default)
  [1m‐‐no‐output‐fields    [22mDo not write out any original fields in the output. By
                          default, all original columns will be included.
  [1m‐‐output‐fields [22m[[4mOUTPUT_FIELDS[24m ...]
                          Specify original columns to write out.
  [1m‐‐interleave          [22mInterleave ok columns with original fields.
  [1m‐‐no‐interleave       [22mDo not interleave ok columns with original fields.
  [1m‐‐index               [22mInclude a row‐number index in the output file when
                          detecting. Rows are usually numbered from 1,
                          unless the input file already has an index.
  [1m‐‐int                 [22mWrite out boolean fields as integers, with 1 for true
                          and 0 for false.
  [1m‐‐key [KEY ...]       [22mKey or key fields to use when reporting failures

  [1m‐‐verify‐required‐fields[22m, [1m‐‐vrf[0m
                          Force verify of required fields
  [1m‐‐verify‐allowed‐fields[22m, [1m‐‐vaf[0m
                          Force verify of allowed fields
  [1m‐‐no‐verify‐required‐fields[22m, [1m‐‐no‐vrf[0m
                          Force no verication of required fields
  [1m‐‐no‐verify‐allowed‐fields[22m, [1m‐‐no‐vaf[0m
                          Force no verification of allowed fields
  [1m‐‐varf[22m, [1m‐‐vraf          [22mForce verification of allowed and required
                          fields
  [1m‐‐no‐varf[22m, [1m‐‐no‐vraf    [22mForce no verification of allowed and required
                          fields

  [1m‐‐pandas[22m, [1m‐‐pd          [22mUse Pandas as DataFrame engine.
  [1m‐‐polars[22m, [1m‐‐pl          [22mUse Polars as DataFrame engine.
  [1m‐‐backend[22m, [1m‐B [4m[22mBACKEND[24m   Backend choice for Pandas
                          (when dataframe engine is Pandas)
                            [1mn [22mfor numpy_nullable *
                            [1ma [22mfor pyarrow
                            [1mo [22mfor original.

EXAMPLES

  The example data can be obtained by running [1mtdda examples[22m, which will
  create various directories, including [1mconstraints_examples[22m, containing
  source data for these examples.

  1) [1mtdda detect elements.parquet elements.tdda elements‐failures.parquet[0m

  This command reads data from [1melements.parquet[22m, checks it against the
  constraints in [1melements.tdda[22m, and writes records with one or more
  constraint failures to [1melements‐failures.parquet[22m.

  2) [1mtdda detect elements.parquet elements.tdda elements‐failures.parquet ‐r[0m
[1mhtml ‐o elements[0m

  As above, and also writes an HTML report to [1melements.html[22m.

  3) [1mtdda detect elements.parquet elements.tdda elements‐failures.parquet ‐r md[0m
[1mjson txt ‐o elements[0m

  As above, and also writes reports to [1melements.md[22m, [1melements.json[22m,
  and [1melements.txt[22m.

SEE ALSO

  [1mtdda verify[0m
  [1mtdda discover[0m
  [1mtdda serial[0m
















































