Dataframe 'Utilities'
Package ka_uts_dfr can be installed from PyPI or Anaconda.
To install with pip:
$ python -m pip install ka_uts_dfr
To install with conda:
$ conda install -c conda-forge ka_uts_dfr
This requires that the readme extra is installed:
$ python -m pip install ka_uts_dfr[readme]
The Modules of Package ka_uts_dfr could be classified into the following module classes:
Modules for Pandas Dataframe Name Type pddf.py Pandas Dataframe
The Module pddf.py contains a single static classes PdDf.
The static Class PdDf is used to manage Pandas Dataframes; it contains the subsequent methods.
Methods of static class PdDf Name Description sh_d_aod show dictionary of array of dictionaries. sh_d_pddf show dictionary of pandas dataframes. pivot_table create pandas dataframe pivot table. The pivot rules are defined by a pivot dictionary. filter Filter pandas dataframe. The filteris defined by filter dictionary set_ix_drop_col_filter set index and drop column filter format-leading_zeros format pandas dataframe columns with leading zeros format-as-date format pandas dataframe columns as date
Parameter of PdDf method sh_d_aod Name Type Description df TyPdDf Pandas Datafame key str Keyword arguments
Return Value of PdDf method sh_d_aod Name Type Description d_aod TyDoAoD dictionary of array of dictionaries
Parameter of PdDf method sh_d_pddf Name Type Description cls class current class df TyPdDf Pandas Datafame key str keyword arguments
Return Value of PdDf method sh_d_pddf Name Type Description d_df TyDoPdDf dictionary of pandas dataframes
Parameter of PdDf method pivot_table Name Type Description cls class current class df TyPdDf pandas datafame d_pv TyDic pivot table definition dictionary
Return Value of PdDf method pivot_table Name Type Description dfpv TyPdDf pandas dataframe pivot table
Parameter of PdDf method filter Name Type Description cls class current class df TyPdDf pandas datafame d_filter TyDic filter definition dictionary relation TyStr filter relation
Return Value of PdDf method filter Name Type Description df_new TyPdDf filtered pandas datafame
Parameter of PdDf method set_ix_drop_col_filter Name Type Description cls class current class df TyPdDf pandas datafame d_filter TyDic filter definition dictionary relation str filter relation
Return Value of PdDf method set_ix_drop_col_filter Name Type Description df_new TyPdDf filtered pandas datafame
Parameter of PdDf method format_leading_zeros Name Type Description cls class current class df TyPdDf pandas datafame d_filter TyDic filter definition dictionary relation str filter relation
Return Value of PdDf method format_leading_zeros Name Type Description df_new TyPdDf filtered pandas datafame
Parameter of PdDf method format_as_date Name Type Description cls class current class df TyPdDf pandas datafame d_filter TyDic filter definition dictionary relation str filter relation
Return Values of PdDf methodR ormat_as_date Name Type Description df_new TyPdDf filtered pandas datafame
Modules for Polars Dataframe Module Classes Name|Type Name Type Description pldf Polars Dataframe PdDf Static Manage Polars Dataframes
The Module pldf contains a single static class PLDF.
The static Class PlDf contains the subsequent methods.
pldf Methods Name Description filter Filter polars dataframe using the given statement. pivot Create polars dataframe pivot table. The pivot rules are defined by the given pivot dictionary. pivot_filter Filter polars dataframe using the given statement and create polars dataframe pivot table from filtered dataframe. The pivot rules are defined by the given pivot dictionary. to_aod create pandas dataframe pivot table. The pivot rules are defined by pivot dictionary to_doa create pandas dataframe pivot table. The pivot rules are defined by pivot dictionary
Parameter of PlDf method filter Name Type Description cls class current class df TyPdDf polars datafame stmt TyStmt filter statement
Return Value of PlDf method filter Name Type Description df_new TyPlDf filtered polars datafame
Parameter of P.Df method pivot Name Type Description cls class current class df TyPlDf polars datafame d_pv TyDic pivot table definition dictionary
Return value of PdDf method pivot Name Type Description dfpv TyPlDf polars dataframe pivot table
Parameter of PdDf method pivot_filter Name Type Description cls class current class df TyPlDf polars datafame d_pv TyDic pivot table definition dictionary stmt TyStmt filter statement
Return value of PlDf method pivot_gilter Name Type Description dfpv TyPlDf polars dataframe pivot table
Parameter of PdDf method to_aod Name Type Description df TyPlDf polars datafame
Return value of PlDf method to_aod Name Type Description aod TyAoD Array of Dictionaries
Parameter of PdDf method to_doa Name Type Description df TyPlDf polars datafame
Return value of PlDf method to_doa Name Type Description doa TyDoA Dictionary of Arrays
The Standard or user specifig logging is carried out by the log.py module of the logging package ka_uts_log using the configuration files ka_std_log.yml or ka_usr_log.yml in the configuration directory cfg of the logging package ka_uts_log. The Logging configuration of the logging package could be overriden by yaml files with the same names in the configuration directory cfg of the application packages.
Logging defines log file path names for the following log message types: .
Application parameter used in log naming Name Decription Values Example dir_dat Application data directory /otev/data tenant Application tenant name UMH package Application package name otev_xls_srr cmd Application command evupreg pid Process ID æevupreg log_ts_type Timestamp type used in logging files|ts, dt ts, dt ts log_sw_single_dir Enable single log directory or multiple log directories True, False True
Single or multiple Application log directories can be used for each message type:
Log types and directoriesg Log type Log directory long short multiple single debug dbqs dbqs logs info infs infs logs warning wrns wrns logs error errs errs logs critical crts crts logs
Naming conventions for logging file paths Type Directory File debug /<dir_dat>/<tenant>/RUN/<package>/<cmd>/<Log directory> <Log type>_<ts>_<pid>.log info /<dir_dat>/<tenant>/RUN/<package>/<cmd>/<Log directory> <Log type>_<ts>_<pid>.log warning /<dir_dat>/<tenant>/RUN/<package>/<cmd>/<Log directory> <Log type>_<ts>_<pid>.log error /<dir_dat>/<tenant>/RUN/<package>/<cmd>/<Log directory> <Log type>_<ts>_<pid>.log critical /<dir_dat>/<tenant>/RUN/<package>/<cmd>/<Log directory> <Log type>_<ts>_<pid>.log
Naming examples for logging file paths Type Directory File debug /data/otev/umh/RUN/otev_xls_srr/evupreg/logs debs_1737118199_9470.log info /data/otev/umh/RUN/otev_xls_srr/evupreg/logs infs_1737118199_9470.log warning /data/otev/umh/RUN/otev_xls_srr/evupreg/logs wrns_1737118199_9470.log error /data/otev/umh/RUN/otev_xls_srr/evupreg/logs errs_1737118199_9470.log critical /data/otev/umh/RUN/otev_xls_srr/evupreg/logs crts_1737118199_9470.log
Python package Name Definition Python package Python packages are directories that contains the special module __init__.py and other modules, packages files or directories. Python sub-package Python sub-packages are python packages which are contained in another pyhon package.
Python package sub-directories Name Definition Python package sub-directory Python packages sub-directories are directories contained in python packages. Special Python package sub-directory Special Python package sub-directories are python package sub-directories with with a special meaning
Special python sun-directories Name Description data Directory for package data files. cfg Directory for package configuration files.
Python package files Name Definition Python package files Python packages are files within a python package. Special python package files Special python package files are package files which are not modules and used as python marker files like __init__.py Python package module Python modules are files with suffix .py; they could be empty or contain python code; other modules can be imported into a module. Special python package module Special python modules like __init__.py or main.py are python modules with special names and functionality.
Overview °°°°°°°°
Special python package files Name Type Description py.typed Type checking marker file The py.typed file is a marker file used in Python packages to indicate that the package supports type checking. This is a part of the PEP 561 standard, which provides a standardized way to package and distribute type information in Python.
Overview °°°°°°°°
Special Python package modules Name Type Description __init__.py Package directory marker file The dunder (double underscore) module __init__.py is used to execute initialisation code or mark the directory it contains as a package. The Module enforces explicit imports and thus clear namespace use and call them with the dot notation. __main__.py entry point for the package The dunder module __main__.py serves as an entry point for the package. The module is executed when the package is called by the interpreter with the command python -m <package name>. __version__.py Version file The dunder module __version__.py consist of assignment statements used in Versioning.
Overview °°°°°°°°
Python elements Python method Python methods are python functions defined in python modules. Special python method Special python methods are python functions with special names and functionalities. Python class Python classes are defined in python modules. Python class method Python class methods are python methods defined python classes.
Overview °°°°°°°°
Special python methods Name Type Description __init__ class object constructor method The special method __init__ is called when an instance (object) of a class is created; instance attributes can be defined and initalized in the method.