tdlpackio
Introduction
tdlpackio is a Python package for reading and writing TDLPACK-formatted data contained in Fortran-based unformatted ("sequential") and direct-access ("random-access") files. tdlpackio provides a Cython extension module, tdlpacklib, for interfacing to the libtdlpack Fortran library, a subset of subroutines from the MOS-2000 (MOS2K) software system.
The TDLPACK data format is a GRIB-like binary data format that is exclusive to MOS2K Fortran-based sofftware system. This software system was developed at the NOAA/NWS/Meteorological Development Laboratory (MDL) and its primary purpose is to perform statistical post-processing of meteorological data. TDLPACK format is based on the World Meteorological Organizations (WMO) GRIdded Binary (GRIB), version 1, but with modifications to support the needs of MDL's statistical post-processing needs -- mainly the ability to store 1D (vector), datasets such as station observations, along with 2D gridded data.
The TDLPACK data format are contained in two types of Fortran-based files; sequential or random-access. Sequential files are variable length, record-based, and unformatted. Random-access files are fixed-length and direct-access. tdlpackio accommodates reading and writing of both types of TDLPACK files.
TDLPACK files can contain two other types of records: a station call letter record and trailer record.
A station call letter record can exist in both types of TDLPACK files and contains a stream of
alphanumeric characters (CHARACTER(LEN=8)). A trailer record exists to signal that either another station
call letter record is about to be read or we have reached the end of the file (EOF).
A trailer record is not written to a random-access file.
For more information on the MOS-2000 software system and TDLPACK format, user is referred to the official MOS-2000 documentation.
Verison 2.0 Refactor
tdlpackio v2.0 is a major factor of pytdlpack v1.x. The API, code design, and structure borrow heavily from grib2io v2. As stated, TDLPACK data can live in a sequential or random-access file, can contain vector or gridded data. tdlpackio v2 aims to provide data to the user in a consistent manner. When opening a TDLPACK file, records are lazily-indexed so that data are only read and unpacked only when necessary. tdlpackio performs the TDLPACK file reading natively in Python. The open class contains methods for reading and indexing. However, packing and unpacking TDLPACK data as well as writing to files are performed by the subroutines in libtdlpack.
Tutorials
The following Jupyter Notebooks are available as tutorials:
1# init for tdlpackio package 2from ._tdlpackio import * 3from ._tdlpackio import __doc__ 4 5from .version import version as __version__ 6 7__all__ = [ 8 "__version__", 9 "open", 10 "TdlpackRecord", 11 "TdlpackStationRecord", 12 "TdlpackTrailerRecord", 13 "TdlpackID", 14] 15 16try: 17 from . import __config__ 18 19 __version__ = __config__.tdlpackio_version 20 has_openmp_support = __config__.has_openmp_support 21 tdlpack_static = __config__.tdlpack_static 22 extra_objects = __config__.extra_objects 23except ImportError: 24 pass 25 26__tdlpacklib_version__ = "1.0.0" # For now... 27 28 29def show_config(): 30 """Print tdlpackio build configuration information.""" 31 print(f"tdlpackio version {__version__} Configuration:") 32 print(f"") 33 print(f"libtdlpack library version: {__tdlpacklib_version__}") 34 print(f"\tStatic library: {tdlpack_static}") 35 print(f"\tOpenMP support: {has_openmp_support}") 36 print(f"") 37 print(f"Static libs:") 38 for lib in extra_objects: 39 print(f"\t{lib}")
86class open: 87 """ 88 Open class for tdlpackio. 89 90 Parameters 91 ---------- 92 path : str 93 File name. 94 95 mode : {'r', 'w'}, default 'r' 96 File handle mode. 97 98 format : {'sequential', 'random-access'}, optional 99 File type when creating a new file. 100 101 ra_template : {'small', 'large'}, optional 102 Template used when creating random-access files. 103 """ 104 105 _filetype_map = { 106 "random-access": 1, 107 "sequential": 2, 108 } 109 110 def __init__(self, path, mode="r", format=None, ra_template=None): 111 if mode not in {"r", "w", "a"}: 112 raise ValueError(f"Invalid mode: {mode}") 113 if mode in {"r", "w"}: 114 mode = mode + "b" 115 elif mode == "a": 116 mode = "wb" 117 self.path = path 118 self.mode = mode 119 self.format = format 120 self.ra_template = ra_template 121 self._hasindex = False 122 self._index = {} 123 self._lun = -1 124 self.mode = mode 125 self.name = os.path.abspath(path) 126 self.records = 0 127 128 # Perform indexing on read 129 if "r" in self.mode: 130 self._filehandle = builtins.open(path, mode=mode) 131 self.filetype = self._get_tdlpack_file_type() 132 self._build_index() 133 134 elif "w" in self.mode: 135 self.bytes_written = 0 136 self.records_written = 0 137 self.filetype = format if format is not None else "sequential" 138 if self.filetype == "random-access": 139 ra_template = "small" if ra_template is None else ra_template 140 iret, self._lun = tdlpacklib.open_tdlpack_file( 141 self.name, self.mode, self._ifiletype, ra_template=ra_template 142 ) 143 elif self.filetype == "sequential": 144 self._filehandle = builtins.open(path, mode=mode) 145 iret, self._lun = tdlpacklib.open_tdlpack_file( 146 self.name, self.mode, self._ifiletype, ra_template=ra_template 147 ) 148 149 # Add self to file data store 150 _open_file_store[self.name] = self 151 152 @property 153 def _ifiletype(self): 154 """Return numeric filetype""" 155 return self._filetype_map[self.filetype] 156 157 @property 158 def size(self): 159 """Return the file size.""" 160 return os.path.getsize(self.name) 161 162 def __enter__(self): 163 """""" 164 return self 165 166 def __exit__(self, atype, value, traceback): 167 """""" 168 self.close() 169 170 def __iter__(self): 171 """""" 172 yield from self._index["record"] 173 174 def __repr__(self): 175 """""" 176 strings = [] 177 keys = list(self.__dict__.keys()) 178 for k in keys: 179 if not k.startswith("_"): 180 strings.append(f"{k} = {self.__dict__[k]}\n") 181 # Attach size property. 182 strings.append(f"size = {self.size}\n") 183 return "".join(strings) 184 185 def __getitem__(self, key): 186 """""" 187 if isinstance(key, slice): 188 return self._index["record"][key] 189 elif isinstance(key, int): 190 return self._index["record"][key] 191 else: 192 raise KeyError("Key must be an integer record number or a slice") 193 194 def _get_tdlpack_file_type(self): 195 """Determine the type of TDLPACK file""" 196 self._filehandle.seek(0) 197 a = struct.unpack(">i", self._filehandle.read(4))[0] 198 b = struct.unpack(">i", self._filehandle.read(4))[0] 199 self._filehandle.seek(0) 200 return "random-access" if [a, b] == [0, 4] else "sequential" 201 202 def _build_index(self): 203 """Record Indexer""" 204 # Initialize index dictionary 205 self._index["offset"] = [] 206 self._index["size"] = [] 207 self._index["type"] = [] 208 self._index["record"] = [] 209 210 if self.filetype == "random-access": 211 self._randomaccess_file_indexer() 212 elif self.filetype == "sequential": 213 self._sequential_file_indexer() 214 self._hasindex = True 215 216 def _randomaccess_file_indexer(self): 217 """Indexer for random-access TDLPACK files""" 218 # Read master key 219 version, nids, nwords, nkyrec, maxent, lastky = struct.unpack( 220 ">iiiiii", self._filehandle.read(24) 221 ) 222 nbytes = nwords * NBYPWD 223 last_key_check = [99999999] if lastky > 9999 else [9999, 99999999] 224 self.master_key = dict( 225 version=version, 226 nids=nids, 227 nwords=nwords, 228 nkyrec=nkyrec, 229 maxent=maxent, 230 lastky=lastky, 231 ) 232 self.key_records = [] 233 # Set file position to first key record 234 self._filehandle.seek(nbytes) 235 236 last_station_id_rec = -1 237 last_station_lat_rec = -1 238 last_station_lon_rec = -1 239 240 # Iterate over all key records 241 while True: 242 # Read key record "header" data 243 nkeys, prec_this_key, prec_next_key = struct.unpack( 244 ">iii", self._filehandle.read(12) 245 ) 246 self.key_records.append( 247 dict( 248 nkeys=nkeys, 249 prec_this_key=prec_this_key, 250 prec_next_key=prec_next_key, 251 ) 252 ) 253 254 ids = list() 255 nsize = list() 256 prec_begin = list() 257 258 # Read key record information 259 for i in range(nkeys): 260 id1, id2, id3, id4, nd, bprec = struct.unpack( 261 ">iiiiii", self._filehandle.read(24) 262 ) 263 ids.append([id1, id2, id3, id4]) 264 nsize.append(nd) 265 prec_begin.append(bprec) 266 267 # Using key record info, move around file to inventory TDLPACK data 268 for m, n, b in zip(ids, nsize, prec_begin): 269 # Disect prec_begin 270 prec1 = int(b / 1000.0) 271 nprec = int(b - (prec1 * 1000.0)) 272 273 # Offset to the data record 274 offset = (prec1 - 1) * nbytes 275 self._filehandle.seek(offset) 276 self._index["offset"].append(offset) 277 self._index["size"].append(n * NBYPWD) 278 279 # Determine record type. Since the 4-word ID is stored in the 280 # key record, we can use it to determine station call letter 281 # record or TDLPACK data record. 282 if m[0] == 400001000: 283 # Station ID record...not in TDLPACK format 284 rec = TdlpackStationRecord() 285 last_station_id_rec = self.records 286 rec._recnum = self.records 287 rec._source = self.name 288 rec._nsta_expected = int(n / 2) 289 self._index["record"].append(rec) 290 self._index["type"].append("station") 291 else: 292 if m[0] == 400006000: 293 last_station_lat_rec = self.records 294 elif m[0] == 400007000: 295 last_station_lon_rec = self.records 296 # TDLPACK data record 297 ipack = np.frombuffer( 298 self._filehandle.read(132), dtype=">i4" 299 ).astype(np.int32) 300 iret, is0, is1, is2, is4 = tdlpacklib.unpack_meta(ipack) 301 if np.all(is2 == 0): 302 is2 = None 303 rec = TdlpackRecord(is0, is1, is2, is4) 304 rec._recnum = self.records 305 rec._linked_station_id_record = last_station_id_rec 306 rec._linked_station_lat_record = last_station_lat_rec 307 rec._linked_station_lon_record = last_station_lon_rec 308 rec._source = self.name 309 shape = ( 310 (rec.ny, rec.nx) 311 if rec.type == "grid" 312 else (rec.numberOfPackedValues,) 313 ) 314 ndim = len(shape) 315 dtype = "float32" 316 rec._data = TdlpackRecordOnDiskArray( 317 shape, 318 ndim, 319 dtype, 320 self.filetype, 321 self._filehandle, 322 rec, 323 self._index["offset"][-1], 324 self._index["size"][-1], 325 ) 326 self._index["record"].append(rec) 327 self._index["type"].append("data") 328 self.records += 1 329 330 # Hold the record number of the last station ID record 331 if self._index["type"][-1] == "station": 332 _last_station_id_record = self.records # This should be OK. 333 334 # Break loop here, at last key record 335 if prec_next_key in last_key_check: 336 break 337 offset = (prec_next_key - 1) * nbytes 338 self._filehandle.seek(offset) 339 340 # Make key_records immutable 341 self.key_records = tuple(self.key_records) 342 343 def _sequential_file_indexer(self): 344 """Indexer for sequential TDLPACK files""" 345 last_station_id_rec = -1 346 last_station_lat_rec = -1 347 last_station_lon_rec = -1 348 349 # Iterate 350 while True: 351 try: 352 # First read 4-byte Fortran record header 353 pos = self._filehandle.tell() 354 fortran_header = struct.unpack(">i", self._filehandle.read(4))[0] 355 if fortran_header >= 132: 356 bytes_to_read = 132 357 else: 358 bytes_to_read = fortran_header 359 360 pos = self._filehandle.tell() 361 ioctet = np.frombuffer(self._filehandle.read(8), dtype=">i8").astype( 362 np.int64 363 )[0] 364 ipack = np.frombuffer( 365 self._filehandle.read(bytes_to_read - 8), dtype=">i4" 366 ).astype(np.int32) 367 _header = struct.unpack(">4s", ipack[0])[0].decode() 368 369 # Check to first 4 bytes of the data record to determine the data 370 # record type. 371 if _header == "PLDT": 372 if ipack[5] == 400006000: 373 last_station_lat_rec = self.records 374 elif ipack[5] == 400007000: 375 last_station_lon_rec = self.records 376 # TDLPACK data record 377 iret, is0, is1, is2, is4 = tdlpacklib.unpack_meta(ipack) 378 self._index["offset"].append(pos) 379 self._index["size"].append( 380 fortran_header 381 ) # Size given by Fortran header 382 if np.all(is2 == 0): 383 is2 = None 384 rec = TdlpackRecord(is0, is1, is2, is4) 385 rec._recnum = self.records 386 rec._linked_station_id_record = last_station_id_rec 387 rec._linked_station_lat_record = last_station_lat_rec 388 rec._linked_station_lon_record = last_station_lon_rec 389 rec._source = self.name 390 shape = ( 391 (rec.ny, rec.nx) 392 if rec.type == "grid" 393 else (rec.numberOfPackedValues,) 394 ) 395 ndim = len(shape) 396 dtype = "float32" 397 rec._data = TdlpackRecordOnDiskArray( 398 shape, 399 ndim, 400 dtype, 401 self.filetype, 402 self._filehandle, 403 rec, 404 self._index["offset"][-1], 405 self._index["size"][-1], 406 ) 407 self._index["record"].append(rec) 408 self._index["type"].append("data") 409 else: 410 if ioctet == 24 and ipack[4] == 9999: 411 # Trailer record 412 rec = TdlpackTrailerRecord() 413 rec._recnum = self.records 414 rec._source = self.name 415 self._index["offset"].append(pos) 416 self._index["size"].append(fortran_header) 417 self._index["type"].append("trailer") 418 self._index["record"].append(rec) 419 else: 420 # Station ID record 421 rec = TdlpackStationRecord() 422 last_station_id_rec = self.records 423 rec._recnum = self.records 424 rec._source = self.name 425 rec._nsta_expected = int(ioctet / 8) 426 self._index["offset"].append(pos) 427 self._index["size"].append(fortran_header) 428 self._index["type"].append("station") 429 self._index["record"].append(rec) 430 431 # At this point we have successfully identified a TDLPACK record from 432 # the file. Increment self.records and position the file pointer to 433 # now read the Fortran trailer. 434 self.records += 1 # Includes trailer records 435 self._filehandle.seek(fortran_header - bytes_to_read, 1) 436 fortran_trailer = struct.unpack(">i", self._filehandle.read(4))[0] 437 438 # Check Fortran header and trailer for the record. 439 if fortran_header != fortran_trailer: 440 raise IOError("Bad Fortran record.") 441 442 # Hold the record number of the last station ID record 443 if self._index["type"][-1] == "station": 444 _last_station_id_record = self.records # This should be OK. 445 446 except struct.error: 447 self._filehandle.seek(0) 448 break 449 450 def read(self, n): 451 """ 452 Read record from file. 453 454 Parameters 455 ---------- 456 n : int 457 Record number. 458 459 Returns 460 ------- 461 numpy.ndarray 462 Record data as a NumPy array. The returned dtype depends on the 463 record type: 464 465 - ``data`` or ``trailer`` : ``int32`` array. 466 - ``station`` : fixed-width byte string array (``S8``). 467 468 Notes 469 ----- 470 The file pointer is positioned using the internal index before reading. 471 Record size is determined from the file header (sequential) or index 472 (random-access). 473 """ 474 if "w" in self.mode: 475 pass # Remove this at some point.... 476 # Position file pointer to the beginning of the TDLPACK record. 477 self._filehandle.seek(self._index["offset"][n]) 478 if self.filetype == "sequential": 479 size = np.frombuffer(self._filehandle.read(8), dtype=">i8").astype( 480 np.int64 481 )[0] 482 elif self.filetype == "random-access": 483 size = self._index["size"][n] 484 485 if self._index["type"][n] in {"data", "trailer"}: 486 return np.frombuffer(self._filehandle.read(size), dtype=">i4").astype( 487 np.int32 488 ) 489 elif self._index["type"][n] == "station": 490 return np.frombuffer(self._filehandle.read(size), dtype="S8") 491 492 def write(self, record): 493 """ 494 Write record(s) to file. 495 496 Parameters 497 ---------- 498 record : TdlpackStationRecord, _TdlpackRecord, TdlpackTrailerRecord, or list 499 Record or list of records to write. 500 501 - ``TdlpackStationRecord`` : Station record. Station identifiers 502 are padded to ``NCHAR``. 503 - ``_TdlpackRecord`` : Packed data record. 504 - ``TdlpackTrailerRecord`` : Trailer record. 505 - ``list`` : List of supported record types. 506 507 Returns 508 ------- 509 None 510 511 Notes 512 ----- 513 Updates ``bytes_written``, ``records_written``, ``records``, and 514 ``_type_lastrecord_written``. 515 """ 516 if isinstance(record, list): 517 for rec in record: 518 self.write(rec) 519 return 520 521 nreplace, ncheck = 0, 0 522 523 if isinstance(record, TdlpackStationRecord): 524 # Adjust string length of each station to NCHAR. 525 stns = [s.ljust(NCHAR) for s in record.stations] 526 iret, self.bytes_written, self.records_written = ( 527 tdlpacklib.write_station_record( 528 self.name, 529 self._lun, 530 self._ifiletype, 531 stns, 532 self.bytes_written, 533 self.records_written, 534 nreplace, 535 ncheck, 536 ) 537 ) 538 539 elif issubclass(record.__class__, _TdlpackRecord): 540 iret, self.bytes_written, self.records_written = ( 541 tdlpacklib.write_tdlpack_record( 542 self.name, 543 self._lun, 544 self._ifiletype, 545 record._ipack, 546 self.bytes_written, 547 self.records_written, 548 nreplace, 549 ncheck, 550 ) 551 ) 552 553 elif isinstance(record, TdlpackTrailerRecord): 554 iret, self.bytes_written, self.records_written = ( 555 tdlpacklib.write_trailer_record( 556 self._lun, self._ifiletype, self.bytes_written, self.records_written 557 ) 558 ) 559 560 self._type_lastrecord_written = record.type 561 self.records += 1 562 563 def close(self): 564 """ 565 Close the file. 566 567 Returns 568 ------- 569 None 570 571 Notes 572 ----- 573 For write mode, a trailer record may be written for sequential files 574 if the last record type requires it. The underlying TDLpack file handle 575 is then closed. The file is removed from the internal open file store. 576 """ 577 if "r" in self.mode: 578 self._filehandle.close() 579 if "w" in self.mode: 580 if self.filetype == "sequential": 581 if self._type_lastrecord_written == "vector": 582 iret = tdlpacklib.write_trailer_record( 583 self._lun, 584 self._ifiletype, 585 self.bytes_written, 586 self.records_written, 587 ) 588 iret = tdlpacklib.close_tdlpack_file(self._lun, self._ifiletype) 589 if self.name in _open_file_store.keys(): 590 del _open_file_store[self.name] 591 592 def select(self, **kwargs): 593 """ 594 Select records by attribute. 595 596 Parameters 597 ---------- 598 **kwargs : dict 599 Keyword arguments specifying ``TdlpackRecord`` attributes and 600 values to match. 601 602 Returns 603 ------- 604 list 605 List of records matching all provided attribute/value pairs. 606 607 Notes 608 ----- 609 Selection is performed against records in the internal index. Records 610 must match all specified attributes to be included in the result. 611 """ 612 _id_keys = [ 613 "word1", 614 "word2", 615 "word3", 616 "word4", 617 "ccc", 618 "fff", 619 "b", 620 "dd", 621 "v", 622 "llll", 623 "uuuu", 624 "t", 625 "rr", 626 "o", 627 "hh", 628 "tau", 629 "thresh", 630 "i", 631 "s", 632 "g", 633 ] 634 635 # TODO: Add ability to process multiple values for each keyword (attribute) 636 idxs = [] 637 nkeys = len(kwargs.keys()) 638 for k, v in kwargs.items(): 639 if k in _id_keys: 640 for rec in self._index["record"]: 641 #if hasattr(rec, k) and getattr(rec, k) == v: 642 if getattr(rec.id, k) == v: 643 idxs.append(rec._recnum) 644 else: 645 for rec in self._index["record"]: 646 if hasattr(rec, k) and getattr(rec, k) == v: 647 idxs.append(rec._recnum) 648 idxs = np.array(idxs, dtype=">i4") 649 return [ 650 self._index["record"][i] 651 for i in [ 652 ii[0] 653 for ii in collections.Counter(idxs).most_common() 654 if ii[1] == nkeys 655 ] 656 ]
Open class for tdlpackio.
Parameters
- path (str): File name.
- mode ({'r', 'w'}, default 'r'): File handle mode.
- format ({'sequential', 'random-access'}, optional): File type when creating a new file.
- ra_template ({'small', 'large'}, optional): Template used when creating random-access files.
110 def __init__(self, path, mode="r", format=None, ra_template=None): 111 if mode not in {"r", "w", "a"}: 112 raise ValueError(f"Invalid mode: {mode}") 113 if mode in {"r", "w"}: 114 mode = mode + "b" 115 elif mode == "a": 116 mode = "wb" 117 self.path = path 118 self.mode = mode 119 self.format = format 120 self.ra_template = ra_template 121 self._hasindex = False 122 self._index = {} 123 self._lun = -1 124 self.mode = mode 125 self.name = os.path.abspath(path) 126 self.records = 0 127 128 # Perform indexing on read 129 if "r" in self.mode: 130 self._filehandle = builtins.open(path, mode=mode) 131 self.filetype = self._get_tdlpack_file_type() 132 self._build_index() 133 134 elif "w" in self.mode: 135 self.bytes_written = 0 136 self.records_written = 0 137 self.filetype = format if format is not None else "sequential" 138 if self.filetype == "random-access": 139 ra_template = "small" if ra_template is None else ra_template 140 iret, self._lun = tdlpacklib.open_tdlpack_file( 141 self.name, self.mode, self._ifiletype, ra_template=ra_template 142 ) 143 elif self.filetype == "sequential": 144 self._filehandle = builtins.open(path, mode=mode) 145 iret, self._lun = tdlpacklib.open_tdlpack_file( 146 self.name, self.mode, self._ifiletype, ra_template=ra_template 147 ) 148 149 # Add self to file data store 150 _open_file_store[self.name] = self
157 @property 158 def size(self): 159 """Return the file size.""" 160 return os.path.getsize(self.name)
Return the file size.
450 def read(self, n): 451 """ 452 Read record from file. 453 454 Parameters 455 ---------- 456 n : int 457 Record number. 458 459 Returns 460 ------- 461 numpy.ndarray 462 Record data as a NumPy array. The returned dtype depends on the 463 record type: 464 465 - ``data`` or ``trailer`` : ``int32`` array. 466 - ``station`` : fixed-width byte string array (``S8``). 467 468 Notes 469 ----- 470 The file pointer is positioned using the internal index before reading. 471 Record size is determined from the file header (sequential) or index 472 (random-access). 473 """ 474 if "w" in self.mode: 475 pass # Remove this at some point.... 476 # Position file pointer to the beginning of the TDLPACK record. 477 self._filehandle.seek(self._index["offset"][n]) 478 if self.filetype == "sequential": 479 size = np.frombuffer(self._filehandle.read(8), dtype=">i8").astype( 480 np.int64 481 )[0] 482 elif self.filetype == "random-access": 483 size = self._index["size"][n] 484 485 if self._index["type"][n] in {"data", "trailer"}: 486 return np.frombuffer(self._filehandle.read(size), dtype=">i4").astype( 487 np.int32 488 ) 489 elif self._index["type"][n] == "station": 490 return np.frombuffer(self._filehandle.read(size), dtype="S8")
Read record from file.
Parameters
- n (int): Record number.
Returns
numpy.ndarray: Record data as a NumPy array. The returned dtype depends on the record type:
dataortrailer:int32array.station: fixed-width byte string array (S8).
Notes
The file pointer is positioned using the internal index before reading. Record size is determined from the file header (sequential) or index (random-access).
492 def write(self, record): 493 """ 494 Write record(s) to file. 495 496 Parameters 497 ---------- 498 record : TdlpackStationRecord, _TdlpackRecord, TdlpackTrailerRecord, or list 499 Record or list of records to write. 500 501 - ``TdlpackStationRecord`` : Station record. Station identifiers 502 are padded to ``NCHAR``. 503 - ``_TdlpackRecord`` : Packed data record. 504 - ``TdlpackTrailerRecord`` : Trailer record. 505 - ``list`` : List of supported record types. 506 507 Returns 508 ------- 509 None 510 511 Notes 512 ----- 513 Updates ``bytes_written``, ``records_written``, ``records``, and 514 ``_type_lastrecord_written``. 515 """ 516 if isinstance(record, list): 517 for rec in record: 518 self.write(rec) 519 return 520 521 nreplace, ncheck = 0, 0 522 523 if isinstance(record, TdlpackStationRecord): 524 # Adjust string length of each station to NCHAR. 525 stns = [s.ljust(NCHAR) for s in record.stations] 526 iret, self.bytes_written, self.records_written = ( 527 tdlpacklib.write_station_record( 528 self.name, 529 self._lun, 530 self._ifiletype, 531 stns, 532 self.bytes_written, 533 self.records_written, 534 nreplace, 535 ncheck, 536 ) 537 ) 538 539 elif issubclass(record.__class__, _TdlpackRecord): 540 iret, self.bytes_written, self.records_written = ( 541 tdlpacklib.write_tdlpack_record( 542 self.name, 543 self._lun, 544 self._ifiletype, 545 record._ipack, 546 self.bytes_written, 547 self.records_written, 548 nreplace, 549 ncheck, 550 ) 551 ) 552 553 elif isinstance(record, TdlpackTrailerRecord): 554 iret, self.bytes_written, self.records_written = ( 555 tdlpacklib.write_trailer_record( 556 self._lun, self._ifiletype, self.bytes_written, self.records_written 557 ) 558 ) 559 560 self._type_lastrecord_written = record.type 561 self.records += 1
Write record(s) to file.
Parameters
record (TdlpackStationRecord, _TdlpackRecord, TdlpackTrailerRecord, or list): Record or list of records to write.
TdlpackStationRecord: Station record. Station identifiers are padded toNCHAR._TdlpackRecord: Packed data record.TdlpackTrailerRecord: Trailer record.list: List of supported record types.
Returns
- None
Notes
Updates bytes_written, records_written, records, and
_type_lastrecord_written.
563 def close(self): 564 """ 565 Close the file. 566 567 Returns 568 ------- 569 None 570 571 Notes 572 ----- 573 For write mode, a trailer record may be written for sequential files 574 if the last record type requires it. The underlying TDLpack file handle 575 is then closed. The file is removed from the internal open file store. 576 """ 577 if "r" in self.mode: 578 self._filehandle.close() 579 if "w" in self.mode: 580 if self.filetype == "sequential": 581 if self._type_lastrecord_written == "vector": 582 iret = tdlpacklib.write_trailer_record( 583 self._lun, 584 self._ifiletype, 585 self.bytes_written, 586 self.records_written, 587 ) 588 iret = tdlpacklib.close_tdlpack_file(self._lun, self._ifiletype) 589 if self.name in _open_file_store.keys(): 590 del _open_file_store[self.name]
Close the file.
Returns
- None
Notes
For write mode, a trailer record may be written for sequential files if the last record type requires it. The underlying TDLpack file handle is then closed. The file is removed from the internal open file store.
592 def select(self, **kwargs): 593 """ 594 Select records by attribute. 595 596 Parameters 597 ---------- 598 **kwargs : dict 599 Keyword arguments specifying ``TdlpackRecord`` attributes and 600 values to match. 601 602 Returns 603 ------- 604 list 605 List of records matching all provided attribute/value pairs. 606 607 Notes 608 ----- 609 Selection is performed against records in the internal index. Records 610 must match all specified attributes to be included in the result. 611 """ 612 _id_keys = [ 613 "word1", 614 "word2", 615 "word3", 616 "word4", 617 "ccc", 618 "fff", 619 "b", 620 "dd", 621 "v", 622 "llll", 623 "uuuu", 624 "t", 625 "rr", 626 "o", 627 "hh", 628 "tau", 629 "thresh", 630 "i", 631 "s", 632 "g", 633 ] 634 635 # TODO: Add ability to process multiple values for each keyword (attribute) 636 idxs = [] 637 nkeys = len(kwargs.keys()) 638 for k, v in kwargs.items(): 639 if k in _id_keys: 640 for rec in self._index["record"]: 641 #if hasattr(rec, k) and getattr(rec, k) == v: 642 if getattr(rec.id, k) == v: 643 idxs.append(rec._recnum) 644 else: 645 for rec in self._index["record"]: 646 if hasattr(rec, k) and getattr(rec, k) == v: 647 idxs.append(rec._recnum) 648 idxs = np.array(idxs, dtype=">i4") 649 return [ 650 self._index["record"][i] 651 for i in [ 652 ii[0] 653 for ii in collections.Counter(idxs).most_common() 654 if ii[1] == nkeys 655 ] 656 ]
Select records by attribute.
Parameters
- **kwargs (dict):
Keyword arguments specifying
TdlpackRecordattributes and values to match.
Returns
- list: List of records matching all provided attribute/value pairs.
Notes
Selection is performed against records in the internal index. Records must match all specified attributes to be included in the result.
659class TdlpackRecord: 660 """ 661 Creation class for TDLPACK record objects. 662 663 This class dynamically constructs and returns an instance of a 664 ``_TdlpackRecord`` subclass based on the provided section arrays and 665 optional keyword arguments. Record classes are cached by type to avoid 666 repeated class construction. 667 668 Parameters 669 ---------- 670 is0 : numpy.ndarray, optional 671 Section 0 array. Default is zero-initialized ``int32`` array of size ``ND7``. 672 is1 : numpy.ndarray, optional 673 Section 1 array. Default is zero-initialized ``int32`` array of size ``ND7``. 674 is2 : numpy.ndarray, optional 675 Section 2 array. Default is zero-initialized ``int32`` array of size ``ND7``. 676 is4 : numpy.ndarray, optional 677 Section 4 array. Default is zero-initialized ``int32`` array of size ``ND7``. 678 *args : tuple 679 Additional positional arguments passed to the constructed record class. 680 **kwargs : dict 681 Optional keyword arguments. The following key is recognized: 682 683 - ``type`` : str, optional 684 Record type. Default is ``"vector"``. If ``is2`` contains any 685 non-zero values, the type is automatically set to ``"grid"``. 686 687 Returns 688 ------- 689 _TdlpackRecord 690 Instance of a dynamically generated subclass of ``_TdlpackRecord``. 691 692 Notes 693 ----- 694 - Record subclasses are created dynamically and cached in 695 ``_record_class_store`` using the record type as the key. 696 - For ``"grid"`` records, ``templates.GridDefinitionSection`` is added 697 as a base class and ``is1[1]`` is set to indicate the presence of a 698 grid definition section. 699 """ 700 701 def __new__( 702 self, 703 is0: np.array = np.zeros((ND7), dtype=np.int32), 704 is1: np.array = np.zeros((ND7), dtype=np.int32), 705 is2: np.array = np.zeros((ND7), dtype=np.int32), 706 is4: np.array = np.zeros((ND7), dtype=np.int32), 707 *args, 708 **kwargs, 709 ): 710 711 bases = list() 712 713 rectype = "vector" 714 if "type" in kwargs.keys(): 715 rectype = kwargs["type"] 716 717 if bool(np.any(is2)): 718 rectype = "grid" 719 720 if rectype == "grid": 721 bases.append(templates.GridDefinitionSection) 722 is1[1] = 1 # Flag in is1 to state that a grid definition section exists 723 724 try: 725 Record = _record_class_store[rectype] 726 except KeyError: 727 728 @dataclass(init=False, repr=False) 729 class Record(_TdlpackRecord, *bases): 730 pass 731 732 _record_class_store[rectype] = Record 733 734 return Record(is0, is1, is2, is4, *args)
Creation class for TDLPACK record objects.
This class dynamically constructs and returns an instance of a
_TdlpackRecord subclass based on the provided section arrays and
optional keyword arguments. Record classes are cached by type to avoid
repeated class construction.
Parameters
- is0 (numpy.ndarray, optional):
Section 0 array. Default is zero-initialized
int32array of sizeND7. - is1 (numpy.ndarray, optional):
Section 1 array. Default is zero-initialized
int32array of sizeND7. - is2 (numpy.ndarray, optional):
Section 2 array. Default is zero-initialized
int32array of sizeND7. - is4 (numpy.ndarray, optional):
Section 4 array. Default is zero-initialized
int32array of sizeND7. - *args (tuple): Additional positional arguments passed to the constructed record class.
**kwargs (dict): Optional keyword arguments. The following key is recognized:
type: str, optional Record type. Default is"vector". Ifis2contains any non-zero values, the type is automatically set to"grid".
Returns
- _TdlpackRecord: Instance of a dynamically generated subclass of
_TdlpackRecord.
Notes
- Record subclasses are created dynamically and cached in
_record_class_storeusing the record type as the key. - For
"grid"records,templates.GridDefinitionSectionis added as a base class andis1[1]is set to indicate the presence of a grid definition section.
1129@dataclass 1130class TdlpackStationRecord: 1131 """ 1132 TDLPACK Station Record class 1133 """ 1134 1135 stations_in: InitVar[Optional[Iterable[str]]] = None 1136 1137 type: str = field(init=False, repr=False, default="station") 1138 stations: ClassVar[templates.Stations] = templates.Stations() 1139 1140 # Private class variable holding the list 1141 _stations: Optional[list[str]] = field(init=False, repr=False, default=None) 1142 1143 def __post_init__(self, stations_in): 1144 self._nsta_expected = 0 1145 self._recnum = -1 1146 self._source = None 1147 self._stations = None 1148 self._type = "station" 1149 self.id = TdlpackID([400001000, 0, 0, 0], self) 1150 1151 if stations_in is not None: 1152 self.stations = stations_in 1153 1154 def __str__(self): 1155 return ( 1156 f"{self._recnum}:d=0000000000:" 1157 f"STATION CALL LETTER RECORD:{self.numberOfStations}" 1158 ) 1159 1160 @property 1161 def numberOfStations(self): 1162 if self._source is not None and ( 1163 isinstance(self._stations, list) or self._stations is None 1164 ): 1165 return self._nsta_expected 1166 return 0 if self.stations is None else len(self.stations) 1167 1168 @property 1169 def data(self): 1170 pass 1171 1172 def pack(self): 1173 pass
TDLPACK Station Record class
1176@dataclass 1177class TdlpackTrailerRecord: 1178 """ 1179 TDLPACK Trailer Record class 1180 """ 1181 1182 type: str = field(init=False, repr=False, default="trailer") 1183 1184 def __post_init__(self): 1185 """""" 1186 self._recnum = -1 1187 self._type = "trailer" 1188 self.id = TdlpackID([0, 0, 0, 0], self) 1189 1190 def __str__(self): 1191 """""" 1192 return f"{self._recnum}:d=0000000000:TRAILER RECORD" 1193 1194 @property 1195 def data(self): 1196 """""" 1197 pass 1198 1199 def pack(self): 1200 """""" 1201 pass
TDLPACK Trailer Record class
1204class TdlpackID: 1205 """ 1206 TDLPACK variable ID class 1207 """ 1208 1209 __slots__ = ("_id", "_rec") 1210 1211 def __init__(self, id, linked_rec=None): 1212 """ 1213 Initialize TDLPACK variable ID 1214 1215 Parameters 1216 ---------- 1217 id : list of ints 1218 The 4-word TDLPACK variable ID 1219 linked_rec : TdlpackRecord, optional 1220 TDLPACK record object. This optional argument provides a mechanism 1221 for updating the TDLPACK variable ID in the TdlpackRecord is1 array. 1222 """ 1223 self._id = utils.parse_id(id) 1224 self._rec = linked_rec 1225 1226 def __eq__(self, value): 1227 if isinstance(value, list) or isinstance(value, tuple): 1228 return ( 1229 self.word1 == value[0] 1230 and self.word2 == value[1] 1231 and self.word3 == value[2] 1232 and self.word4 == value[3] 1233 ) 1234 else: 1235 return False 1236 1237 def __format__(self, spec: str) -> str: 1238 if not spec: 1239 spec = "basic" 1240 return self.format(spec) 1241 1242 def __repr__(self): 1243 return repr(utils.unparse_id(self._id)) 1244 1245 @classmethod 1246 def from_string(cls, idstr): 1247 """ 1248 Create a TdlpackID object from a TDLPACK ID string. 1249 1250 Parameters 1251 ---------- 1252 idstr : str 1253 String containing the TDLPACK ID with leading zeros and delimited 1254 by a non-numeric character. 1255 1256 Returns 1257 ------- 1258 An instance of TdlpackID. 1259 """ 1260 delim = idstr[9] 1261 if {idstr[19], idstr[29]} != {delim, delim}: 1262 raise ValueError(f"Invalid TDLPACK ID string format") 1263 return cls( 1264 [ 1265 int(i.lstrip("0")) if len(i.lstrip("0")) > 0 else 0 1266 for i in idstr.split(delim) 1267 ] 1268 ) 1269 1270 def format(self, style: str = "basic") -> str: 1271 """ 1272 Format the TDLPACK ID identifier as a string. 1273 1274 This method returns a string representation of the identifier in one 1275 of several supported formats commonly used in MOS-2000 workflows. 1276 1277 Parameters 1278 ---------- 1279 style : {'basic', 'b', 'mos', 'm', 'parsed', 'p'}, optional 1280 Output format style (case-insensitive): 1281 1282 - ``"basic"`` or ``"b"`` 1283 Four-word identifier. Each word is printed as a zero-padded 1284 integer field. 1285 1286 - ``"mos"`` or ``"m"`` 1287 MOS-style identifier consisting of the first three words 1288 followed by the ISG components and the threshold value 1289 formatted in scientific notation (``.0000e±00``). 1290 1291 - ``"parsed"`` or ``"p"`` 1292 Identifier parsed into its individual components as defined 1293 by the internal ID mapping. All components are printed as 1294 zero-padded integers except the threshold value, which is 1295 printed as a floating-point value with ``F13.6`` formatting. 1296 1297 Returns 1298 ------- 1299 str 1300 String representation of the identifier in the requested format. 1301 1302 Notes 1303 ----- 1304 The MOS-style threshold representation removes the leading zero 1305 from scientific notation (e.g., ``0.0000e+00`` → ``.0000e+00``) 1306 to match legacy MOS-2000 formatting conventions. 1307 1308 Examples 1309 -------- 1310 Using the ``format`` method: 1311 1312 >>> rec.format("basic") 1313 '001000008 000000500 000000000 0000000000' 1314 1315 >>> rec.format("mos") 1316 '001000008 000000500 000000000 000 .0000e+00' 1317 1318 >>> rec.format("parsed") 1319 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1320 1321 Using Python's format protocol (``__format__``): 1322 1323 >>> f"{rec.id:basic}" 1324 '001000008 000000500 000000000 0000000000' 1325 1326 >>> f"{rec.id:mos}" 1327 '001000008 000000500 000000000 000 .0000e+00' 1328 1329 >>> f"{rec.id:parsed}" 1330 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1331 """ 1332 style = style.lower() 1333 1334 if style in {"basic", "b"}: 1335 return ( 1336 f"{str(self.word1).zfill(9)} " 1337 f"{str(self.word2).zfill(9)} " 1338 f"{str(self.word3).zfill(9)} " 1339 f"{str(self.word4).zfill(10)}" 1340 ) 1341 elif style in {"mos", "m"}: 1342 thresh = f"{self.thresh:.4e}" 1343 if thresh.startswith("0"): 1344 thresh = thresh[1:] 1345 elif thresh.startswith("-0"): 1346 thresh = "-" + thresh[2:] 1347 return ( 1348 f"{str(self.word1).zfill(9)} " 1349 f"{str(self.word2).zfill(9)} " 1350 f"{str(self.word3).zfill(9)} " 1351 f"{self.i}{self.s}{self.g} {thresh}" 1352 ) 1353 elif style in {"parsed", "p"}: 1354 parsed = "" 1355 for k, v in self._id.items(): 1356 if "thresh" not in k: 1357 parsed += f"{str(v).zfill(len(k))} " 1358 else: 1359 parsed += f"{v:13.6f}" 1360 return parsed 1361 1362 raise ValueError(f"Unknown TdlpackID format style: {style!r}") 1363 1364 def to_dict(self): 1365 """ 1366 Return TDLPACK variable ID as dict. 1367 1368 Returns 1369 ------- 1370 String of the 4-word TDLPACK variable ID. 1371 """ 1372 return self._id 1373 1374 def to_string(self, delim=" "): 1375 """ 1376 Return TDLPACK variable ID as string. 1377 1378 Parameters 1379 ---------- 1380 delim : str, optional 1381 Delimiter character between each TDLPACK variable ID word. 1382 1383 Returns 1384 ------- 1385 String of the 4-word TDLPACK variable ID. 1386 """ 1387 strlen = (9, 9, 9, 10) 1388 return delim.join( 1389 [str(i).zfill(l) for l, i in zip(strlen, utils.unparse_id(self._id))] 1390 ) 1391 1392 @property 1393 def word1(self): 1394 """ 1395 First ID word. 1396 1397 Returns 1398 ------- 1399 int 1400 First parsed ID word. 1401 """ 1402 return utils.unparse_id(self._id)[0] 1403 1404 @word1.setter 1405 def word1(self, value): 1406 """ 1407 Set first ID word. 1408 1409 Parameters 1410 ---------- 1411 value : int 1412 New value. 1413 1414 Notes 1415 ----- 1416 Updates internal ID and ``is1[8]`` if record is attached. 1417 """ 1418 newid = utils.unparse_id(self._id) 1419 newid[0] = value 1420 self._id = utils.parse_id(newid) 1421 if self._rec is not None: 1422 self._rec.is1[8] = newid[0] 1423 1424 @property 1425 def word2(self): 1426 """ 1427 Second ID word. 1428 1429 Returns 1430 ------- 1431 int 1432 Second parsed ID word. 1433 """ 1434 return utils.unparse_id(self._id)[1] 1435 1436 @word2.setter 1437 def word2(self, value): 1438 """ 1439 Set second ID word. 1440 1441 Parameters 1442 ---------- 1443 value : int 1444 New value. 1445 1446 Notes 1447 ----- 1448 Updates internal ID and ``is1[9]`` if record is attached. 1449 """ 1450 newid = utils.unparse_id(self._id) 1451 newid[1] = value 1452 self._id = utils.parse_id(newid) 1453 if self._rec is not None: 1454 self._rec.is1[9] = newid[1] 1455 1456 @property 1457 def word3(self): 1458 """ 1459 Third ID word. 1460 1461 Returns 1462 ------- 1463 int 1464 Third parsed ID word. 1465 """ 1466 return utils.unparse_id(self._id)[2] 1467 1468 @word3.setter 1469 def word3(self, value): 1470 """ 1471 Set third ID word. 1472 1473 Parameters 1474 ---------- 1475 value : int 1476 New value. 1477 1478 Notes 1479 ----- 1480 Updates internal ID and ``is1[10]`` if record is attached. 1481 """ 1482 newid = utils.unparse_id(self._id) 1483 newid[2] = value 1484 self._id = utils.parse_id(newid) 1485 if self._rec is not None: 1486 self._rec.is1[10] = newid[2] 1487 1488 @property 1489 def word4(self): 1490 """ 1491 Fourth ID word. 1492 1493 Returns 1494 ------- 1495 int 1496 Fourth parsed ID word. 1497 """ 1498 return utils.unparse_id(self._id)[3] 1499 1500 @word4.setter 1501 def word4(self, value): 1502 """ 1503 Set fourth ID word. 1504 1505 Parameters 1506 ---------- 1507 value : int 1508 New value. 1509 1510 Notes 1511 ----- 1512 Updates internal ID and ``is1[11]`` if record is attached. 1513 """ 1514 newid = utils.unparse_id(self._id) 1515 newid[3] = value 1516 self._id = utils.parse_id(newid) 1517 if self._rec is not None: 1518 self._rec.is1[11] = newid[3] 1519 1520 @property 1521 def ccc(self): 1522 """ 1523 CCC identifier component. 1524 1525 Returns 1526 ------- 1527 int 1528 """ 1529 return self._id["ccc"] 1530 1531 @ccc.setter 1532 def ccc(self, value): 1533 """ 1534 Set CCC identifier component. 1535 1536 Parameters 1537 ---------- 1538 value : int 1539 1540 Notes 1541 ----- 1542 Updates ``is1[8]`` if record is attached. 1543 """ 1544 self._id["ccc"] = value 1545 if self._rec is not None: 1546 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1547 1548 @property 1549 def fff(self): 1550 """ 1551 FFF identifier component. 1552 1553 Returns 1554 ------- 1555 int 1556 """ 1557 return self._id["fff"] 1558 1559 @fff.setter 1560 def fff(self, value): 1561 """ 1562 Set FFF identifier component. 1563 1564 Parameters 1565 ---------- 1566 value : int 1567 1568 Notes 1569 ----- 1570 Updates ``is1[8]`` if record is attached. 1571 """ 1572 self._id["fff"] = value 1573 if self._rec is not None: 1574 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1575 1576 @property 1577 def cccfff(self): 1578 """ 1579 Combined CCCFFF identifier. 1580 1581 Returns 1582 ------- 1583 int 1584 Integer representation of ``word1 / 1000``. 1585 """ 1586 return int(self.word1 / 1000) 1587 1588 @property 1589 def b(self): 1590 """ 1591 B identifier component. 1592 1593 Returns 1594 ------- 1595 int 1596 """ 1597 return self._id["b"] 1598 1599 @b.setter 1600 def b(self, value): 1601 """ 1602 Set B identifier component. 1603 1604 Parameters 1605 ---------- 1606 value : int 1607 1608 Notes 1609 ----- 1610 Updates ``is1[8]`` if record is attached. 1611 """ 1612 self._id["b"] = value 1613 if self._rec is not None: 1614 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1615 1616 @property 1617 def dd(self): 1618 """ 1619 DD identifier component. 1620 1621 Returns 1622 ------- 1623 int 1624 """ 1625 return self._id["dd"] 1626 1627 @dd.setter 1628 def dd(self, value): 1629 """ 1630 Set DD identifier component. 1631 1632 Parameters 1633 ---------- 1634 value : int 1635 1636 Notes 1637 ----- 1638 Updates ``is1[8]`` and ``is1[14]`` if record is attached. 1639 """ 1640 self._id["dd"] = value 1641 if self._rec is not None: 1642 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1643 self._rec.is1[14] = value 1644 1645 @property 1646 def v(self): 1647 """ 1648 V identifier component. 1649 1650 Returns 1651 ------- 1652 int 1653 """ 1654 return self._id["v"] 1655 1656 @v.setter 1657 def v(self, value): 1658 """ 1659 Set V identifier component. 1660 1661 Parameters 1662 ---------- 1663 value : int 1664 1665 Notes 1666 ----- 1667 Updates ``is1[9]`` if record is attached. 1668 """ 1669 self._id["v"] = value 1670 if self._rec is not None: 1671 self._rec.is1[9] = utils.unparse_id(self._id)[1] 1672 1673 @property 1674 def llll(self): 1675 """ 1676 LLLL identifier component. 1677 1678 Returns 1679 ------- 1680 int 1681 """ 1682 return self._id["llll"] 1683 1684 @llll.setter 1685 def llll(self, value): 1686 """ 1687 Set LLLL identifier component. 1688 1689 Parameters 1690 ---------- 1691 value : int 1692 1693 Notes 1694 ----- 1695 Updates ``is1[9]`` if record is attached. 1696 """ 1697 self._id["llll"] = value 1698 if self._rec is not None: 1699 self._rec.is1[9] = utils.unparse_id(self._id)[1] 1700 1701 @property 1702 def uuuu(self): 1703 """ 1704 UUUU identifier component. 1705 1706 Returns 1707 ------- 1708 int 1709 """ 1710 return self._id["uuuu"] 1711 1712 @uuuu.setter 1713 def uuuu(self, value): 1714 """ 1715 Set UUUU identifier component. 1716 1717 Parameters 1718 ---------- 1719 value : int 1720 1721 Notes 1722 ----- 1723 Updates ``is1[9]`` if record is attached. 1724 """ 1725 self._id["uuuu"] = value 1726 if self._rec is not None: 1727 self._rec.is1[9] = utils.unparse_id(self._id)[1] 1728 1729 @property 1730 def t(self): 1731 """ 1732 T identifier component. 1733 1734 Returns 1735 ------- 1736 int 1737 """ 1738 return self._id["t"] 1739 1740 @t.setter 1741 def t(self, value): 1742 """ 1743 Set T identifier component. 1744 1745 Parameters 1746 ---------- 1747 value : int 1748 1749 Notes 1750 ----- 1751 Updates ``is1[10]`` if record is attached. 1752 """ 1753 self._id["t"] = value 1754 if self._rec is not None: 1755 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1756 1757 @property 1758 def rr(self): 1759 """ 1760 RR identifier component. 1761 1762 Returns 1763 ------- 1764 int 1765 """ 1766 return self._id["rr"] 1767 1768 @rr.setter 1769 def rr(self, value): 1770 """ 1771 Set RR identifier component. 1772 1773 Parameters 1774 ---------- 1775 value : int 1776 1777 Notes 1778 ----- 1779 Updates ``is1[10]`` if record is attached. 1780 """ 1781 self._id["rr"] = value 1782 if self._rec is not None: 1783 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1784 1785 @property 1786 def o(self): 1787 """ 1788 O identifier component. 1789 1790 Returns 1791 ------- 1792 int 1793 """ 1794 return self._id["o"] 1795 1796 @o.setter 1797 def o(self, value): 1798 """ 1799 Set O identifier component. 1800 1801 Parameters 1802 ---------- 1803 value : int 1804 1805 Notes 1806 ----- 1807 Updates ``is1[10]`` if record is attached. 1808 """ 1809 self._id["o"] = value 1810 if self._rec is not None: 1811 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1812 1813 @property 1814 def hh(self): 1815 """ 1816 HH identifier component. 1817 1818 Returns 1819 ------- 1820 int 1821 """ 1822 return self._id["hh"] 1823 1824 @hh.setter 1825 def hh(self, value): 1826 """ 1827 Set HH identifier component. 1828 1829 Parameters 1830 ---------- 1831 value : int 1832 1833 Notes 1834 ----- 1835 Updates ``is1[10]`` if record is attached. 1836 """ 1837 self._id["hh"] = value 1838 if self._rec is not None: 1839 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1840 1841 @property 1842 def tau(self): 1843 """ 1844 Forecast hour (tau). 1845 1846 Returns 1847 ------- 1848 int 1849 """ 1850 return self._id["tau"] 1851 1852 @tau.setter 1853 def tau(self, value): 1854 """ 1855 Set forecast hour (tau). 1856 1857 Parameters 1858 ---------- 1859 value : int 1860 1861 Notes 1862 ----- 1863 Updates ``is1[10]`` and ``is1[12]`` if record is attached. 1864 """ 1865 self._id["tau"] = value 1866 if self._rec is not None: 1867 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1868 self._rec.is1[12] = value 1869 1870 @property 1871 def thresh(self): 1872 """ 1873 Threshold identifier component. 1874 1875 Returns 1876 ------- 1877 int 1878 """ 1879 return self._id["thresh"] 1880 1881 @thresh.setter 1882 def thresh(self, value): 1883 """ 1884 Set threshold identifier component. 1885 1886 Parameters 1887 ---------- 1888 value : int 1889 1890 Notes 1891 ----- 1892 Updates ``is1[11]`` if record is attached. 1893 """ 1894 self._id["thresh"] = value 1895 if self._rec is not None: 1896 self._rec.is1[11] = utils.unparse_id(self._id)[3] 1897 1898 @property 1899 def i(self): 1900 """ 1901 I identifier component. 1902 1903 Returns 1904 ------- 1905 int 1906 """ 1907 return self._id["i"] 1908 1909 @i.setter 1910 def i(self, value): 1911 """ 1912 Set I identifier component. 1913 1914 Parameters 1915 ---------- 1916 value : int 1917 1918 Notes 1919 ----- 1920 Updates ``is1[11]`` if record is attached. 1921 """ 1922 self._id["i"] = value 1923 if self._rec is not None: 1924 self._rec.is1[11] = utils.unparse_id(self._id)[3] 1925 1926 @property 1927 def s(self): 1928 """ 1929 S identifier component. 1930 1931 Returns 1932 ------- 1933 int 1934 """ 1935 return self._id["s"] 1936 1937 @s.setter 1938 def s(self, value): 1939 """ 1940 Set S identifier component. 1941 1942 Parameters 1943 ---------- 1944 value : int 1945 1946 Notes 1947 ----- 1948 Updates ``is1[11]`` if record is attached. 1949 """ 1950 self._id["s"] = value 1951 if self._rec is not None: 1952 self._rec.is1[11] = utils.unparse_id(self._id)[3] 1953 1954 @property 1955 def g(self): 1956 """ 1957 G identifier component. 1958 1959 Returns 1960 ------- 1961 int 1962 """ 1963 return self._id["g"] 1964 1965 @g.setter 1966 def g(self, value): 1967 """ 1968 Set G identifier component. 1969 1970 Parameters 1971 ---------- 1972 value : int 1973 1974 Notes 1975 ----- 1976 Updates ``is1[11]`` if record is attached. 1977 """ 1978 self._id["g"] = value 1979 if self._rec is not None: 1980 self._rec.is1[11] = utils.unparse_id(self._id)[3]
TDLPACK variable ID class
1211 def __init__(self, id, linked_rec=None): 1212 """ 1213 Initialize TDLPACK variable ID 1214 1215 Parameters 1216 ---------- 1217 id : list of ints 1218 The 4-word TDLPACK variable ID 1219 linked_rec : TdlpackRecord, optional 1220 TDLPACK record object. This optional argument provides a mechanism 1221 for updating the TDLPACK variable ID in the TdlpackRecord is1 array. 1222 """ 1223 self._id = utils.parse_id(id) 1224 self._rec = linked_rec
Initialize TDLPACK variable ID
Parameters
- id (list of ints): The 4-word TDLPACK variable ID
- linked_rec (TdlpackRecord, optional): TDLPACK record object. This optional argument provides a mechanism for updating the TDLPACK variable ID in the TdlpackRecord is1 array.
1245 @classmethod 1246 def from_string(cls, idstr): 1247 """ 1248 Create a TdlpackID object from a TDLPACK ID string. 1249 1250 Parameters 1251 ---------- 1252 idstr : str 1253 String containing the TDLPACK ID with leading zeros and delimited 1254 by a non-numeric character. 1255 1256 Returns 1257 ------- 1258 An instance of TdlpackID. 1259 """ 1260 delim = idstr[9] 1261 if {idstr[19], idstr[29]} != {delim, delim}: 1262 raise ValueError(f"Invalid TDLPACK ID string format") 1263 return cls( 1264 [ 1265 int(i.lstrip("0")) if len(i.lstrip("0")) > 0 else 0 1266 for i in idstr.split(delim) 1267 ] 1268 )
Create a TdlpackID object from a TDLPACK ID string.
Parameters
- idstr (str): String containing the TDLPACK ID with leading zeros and delimited by a non-numeric character.
Returns
- An instance of TdlpackID.
1270 def format(self, style: str = "basic") -> str: 1271 """ 1272 Format the TDLPACK ID identifier as a string. 1273 1274 This method returns a string representation of the identifier in one 1275 of several supported formats commonly used in MOS-2000 workflows. 1276 1277 Parameters 1278 ---------- 1279 style : {'basic', 'b', 'mos', 'm', 'parsed', 'p'}, optional 1280 Output format style (case-insensitive): 1281 1282 - ``"basic"`` or ``"b"`` 1283 Four-word identifier. Each word is printed as a zero-padded 1284 integer field. 1285 1286 - ``"mos"`` or ``"m"`` 1287 MOS-style identifier consisting of the first three words 1288 followed by the ISG components and the threshold value 1289 formatted in scientific notation (``.0000e±00``). 1290 1291 - ``"parsed"`` or ``"p"`` 1292 Identifier parsed into its individual components as defined 1293 by the internal ID mapping. All components are printed as 1294 zero-padded integers except the threshold value, which is 1295 printed as a floating-point value with ``F13.6`` formatting. 1296 1297 Returns 1298 ------- 1299 str 1300 String representation of the identifier in the requested format. 1301 1302 Notes 1303 ----- 1304 The MOS-style threshold representation removes the leading zero 1305 from scientific notation (e.g., ``0.0000e+00`` → ``.0000e+00``) 1306 to match legacy MOS-2000 formatting conventions. 1307 1308 Examples 1309 -------- 1310 Using the ``format`` method: 1311 1312 >>> rec.format("basic") 1313 '001000008 000000500 000000000 0000000000' 1314 1315 >>> rec.format("mos") 1316 '001000008 000000500 000000000 000 .0000e+00' 1317 1318 >>> rec.format("parsed") 1319 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1320 1321 Using Python's format protocol (``__format__``): 1322 1323 >>> f"{rec.id:basic}" 1324 '001000008 000000500 000000000 0000000000' 1325 1326 >>> f"{rec.id:mos}" 1327 '001000008 000000500 000000000 000 .0000e+00' 1328 1329 >>> f"{rec.id:parsed}" 1330 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1331 """ 1332 style = style.lower() 1333 1334 if style in {"basic", "b"}: 1335 return ( 1336 f"{str(self.word1).zfill(9)} " 1337 f"{str(self.word2).zfill(9)} " 1338 f"{str(self.word3).zfill(9)} " 1339 f"{str(self.word4).zfill(10)}" 1340 ) 1341 elif style in {"mos", "m"}: 1342 thresh = f"{self.thresh:.4e}" 1343 if thresh.startswith("0"): 1344 thresh = thresh[1:] 1345 elif thresh.startswith("-0"): 1346 thresh = "-" + thresh[2:] 1347 return ( 1348 f"{str(self.word1).zfill(9)} " 1349 f"{str(self.word2).zfill(9)} " 1350 f"{str(self.word3).zfill(9)} " 1351 f"{self.i}{self.s}{self.g} {thresh}" 1352 ) 1353 elif style in {"parsed", "p"}: 1354 parsed = "" 1355 for k, v in self._id.items(): 1356 if "thresh" not in k: 1357 parsed += f"{str(v).zfill(len(k))} " 1358 else: 1359 parsed += f"{v:13.6f}" 1360 return parsed 1361 1362 raise ValueError(f"Unknown TdlpackID format style: {style!r}")
Format the TDLPACK ID identifier as a string.
This method returns a string representation of the identifier in one of several supported formats commonly used in MOS-2000 workflows.
Parameters
style ({'basic', 'b', 'mos', 'm', 'parsed', 'p'}, optional): Output format style (case-insensitive):
"basic"or"b"Four-word identifier. Each word is printed as a zero-padded integer field."mos"or"m"MOS-style identifier consisting of the first three words followed by the ISG components and the threshold value formatted in scientific notation (.0000e±00)."parsed"or"p"Identifier parsed into its individual components as defined by the internal ID mapping. All components are printed as zero-padded integers except the threshold value, which is printed as a floating-point value withF13.6formatting.
Returns
- str: String representation of the identifier in the requested format.
Notes
The MOS-style threshold representation removes the leading zero
from scientific notation (e.g., 0.0000e+00 → .0000e+00)
to match legacy MOS-2000 formatting conventions.
Examples
Using the format method:
>>> rec.format("basic")
'001000008 000000500 000000000 0000000000'
>>> rec.format("mos")
'001000008 000000500 000000000 000 .0000e+00'
>>> rec.format("parsed")
'001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000'
Using Python's format protocol (__format__):
>>> f"{rec.id:basic}"
'001000008 000000500 000000000 0000000000'
>>> f"{rec.id:mos}"
'001000008 000000500 000000000 000 .0000e+00'
>>> f"{rec.id:parsed}"
'001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000'
1364 def to_dict(self): 1365 """ 1366 Return TDLPACK variable ID as dict. 1367 1368 Returns 1369 ------- 1370 String of the 4-word TDLPACK variable ID. 1371 """ 1372 return self._id
Return TDLPACK variable ID as dict.
Returns
- String of the 4-word TDLPACK variable ID.
1374 def to_string(self, delim=" "): 1375 """ 1376 Return TDLPACK variable ID as string. 1377 1378 Parameters 1379 ---------- 1380 delim : str, optional 1381 Delimiter character between each TDLPACK variable ID word. 1382 1383 Returns 1384 ------- 1385 String of the 4-word TDLPACK variable ID. 1386 """ 1387 strlen = (9, 9, 9, 10) 1388 return delim.join( 1389 [str(i).zfill(l) for l, i in zip(strlen, utils.unparse_id(self._id))] 1390 )
Return TDLPACK variable ID as string.
Parameters
- delim (str, optional): Delimiter character between each TDLPACK variable ID word.
Returns
- String of the 4-word TDLPACK variable ID.
1392 @property 1393 def word1(self): 1394 """ 1395 First ID word. 1396 1397 Returns 1398 ------- 1399 int 1400 First parsed ID word. 1401 """ 1402 return utils.unparse_id(self._id)[0]
First ID word.
Returns
- int: First parsed ID word.
1424 @property 1425 def word2(self): 1426 """ 1427 Second ID word. 1428 1429 Returns 1430 ------- 1431 int 1432 Second parsed ID word. 1433 """ 1434 return utils.unparse_id(self._id)[1]
Second ID word.
Returns
- int: Second parsed ID word.
1456 @property 1457 def word3(self): 1458 """ 1459 Third ID word. 1460 1461 Returns 1462 ------- 1463 int 1464 Third parsed ID word. 1465 """ 1466 return utils.unparse_id(self._id)[2]
Third ID word.
Returns
- int: Third parsed ID word.
1488 @property 1489 def word4(self): 1490 """ 1491 Fourth ID word. 1492 1493 Returns 1494 ------- 1495 int 1496 Fourth parsed ID word. 1497 """ 1498 return utils.unparse_id(self._id)[3]
Fourth ID word.
Returns
- int: Fourth parsed ID word.
1520 @property 1521 def ccc(self): 1522 """ 1523 CCC identifier component. 1524 1525 Returns 1526 ------- 1527 int 1528 """ 1529 return self._id["ccc"]
CCC identifier component.
Returns
- int
1548 @property 1549 def fff(self): 1550 """ 1551 FFF identifier component. 1552 1553 Returns 1554 ------- 1555 int 1556 """ 1557 return self._id["fff"]
FFF identifier component.
Returns
- int
1576 @property 1577 def cccfff(self): 1578 """ 1579 Combined CCCFFF identifier. 1580 1581 Returns 1582 ------- 1583 int 1584 Integer representation of ``word1 / 1000``. 1585 """ 1586 return int(self.word1 / 1000)
Combined CCCFFF identifier.
Returns
- int: Integer representation of
word1 / 1000.
1588 @property 1589 def b(self): 1590 """ 1591 B identifier component. 1592 1593 Returns 1594 ------- 1595 int 1596 """ 1597 return self._id["b"]
B identifier component.
Returns
- int
1616 @property 1617 def dd(self): 1618 """ 1619 DD identifier component. 1620 1621 Returns 1622 ------- 1623 int 1624 """ 1625 return self._id["dd"]
DD identifier component.
Returns
- int
1645 @property 1646 def v(self): 1647 """ 1648 V identifier component. 1649 1650 Returns 1651 ------- 1652 int 1653 """ 1654 return self._id["v"]
V identifier component.
Returns
- int
1673 @property 1674 def llll(self): 1675 """ 1676 LLLL identifier component. 1677 1678 Returns 1679 ------- 1680 int 1681 """ 1682 return self._id["llll"]
LLLL identifier component.
Returns
- int
1701 @property 1702 def uuuu(self): 1703 """ 1704 UUUU identifier component. 1705 1706 Returns 1707 ------- 1708 int 1709 """ 1710 return self._id["uuuu"]
UUUU identifier component.
Returns
- int
1729 @property 1730 def t(self): 1731 """ 1732 T identifier component. 1733 1734 Returns 1735 ------- 1736 int 1737 """ 1738 return self._id["t"]
T identifier component.
Returns
- int
1757 @property 1758 def rr(self): 1759 """ 1760 RR identifier component. 1761 1762 Returns 1763 ------- 1764 int 1765 """ 1766 return self._id["rr"]
RR identifier component.
Returns
- int
1785 @property 1786 def o(self): 1787 """ 1788 O identifier component. 1789 1790 Returns 1791 ------- 1792 int 1793 """ 1794 return self._id["o"]
O identifier component.
Returns
- int
1813 @property 1814 def hh(self): 1815 """ 1816 HH identifier component. 1817 1818 Returns 1819 ------- 1820 int 1821 """ 1822 return self._id["hh"]
HH identifier component.
Returns
- int
1841 @property 1842 def tau(self): 1843 """ 1844 Forecast hour (tau). 1845 1846 Returns 1847 ------- 1848 int 1849 """ 1850 return self._id["tau"]
Forecast hour (tau).
Returns
- int
1870 @property 1871 def thresh(self): 1872 """ 1873 Threshold identifier component. 1874 1875 Returns 1876 ------- 1877 int 1878 """ 1879 return self._id["thresh"]
Threshold identifier component.
Returns
- int
1898 @property 1899 def i(self): 1900 """ 1901 I identifier component. 1902 1903 Returns 1904 ------- 1905 int 1906 """ 1907 return self._id["i"]
I identifier component.
Returns
- int