tdlpackio
Introduction
tdlpackio is a Python package for reading and writing TDLPACK-formatted data contained in Fortran-based unformatted ("sequential") and direct-access ("random-access") files. tdlpackio provides a Cython extension module, tdlpacklib, for interfacing to the libtdlpack Fortran library, a subset of subroutines from the MOS-2000 (MOS2K) software system.
The TDLPACK data format is a GRIB-like binary data format that is exclusive to MOS2K Fortran-based sofftware system. This software system was developed at the NOAA/NWS/Meteorological Development Laboratory (MDL) and its primary purpose is to perform statistical post-processing of meteorological data. TDLPACK format is based on the World Meteorological Organizations (WMO) GRIdded Binary (GRIB), version 1, but with modifications to support the needs of MDL's statistical post-processing needs -- mainly the ability to store 1D (vector), datasets such as station observations, along with 2D gridded data.
The TDLPACK data format are contained in two types of Fortran-based files; sequential or random-access. Sequential files are variable length, record-based, and unformatted. Random-access files are fixed-length and direct-access. tdlpackio accommodates reading and writing of both types of TDLPACK files.
TDLPACK files can contain two other types of records: a station call letter record and trailer record.
A station call letter record can exist in both types of TDLPACK files and contains a stream of
alphanumeric characters (CHARACTER(LEN=8)). A trailer record exists to signal that either another station
call letter record is about to be read or we have reached the end of the file (EOF).
A trailer record is not written to a random-access file.
For more information on the MOS-2000 software system and TDLPACK format, user is referred to the official MOS-2000 documentation.
Verison 2.0 Refactor
tdlpackio v2.0 is a major factor of pytdlpack v1.x. The API, code design, and structure borrow heavily from grib2io v2. As stated, TDLPACK data can live in a sequential or random-access file, can contain vector or gridded data. tdlpackio v2 aims to provide data to the user in a consistent manner. When opening a TDLPACK file, records are lazily-indexed so that data are only read and unpacked only when necessary. tdlpackio performs the TDLPACK file reading natively in Python. The open class contains methods for reading and indexing. However, packing and unpacking TDLPACK data as well as writing to files are performed by the subroutines in libtdlpack.
Tutorials
The following Jupyter Notebooks are available as tutorials:
1from __future__ import annotations 2 3# init for tdlpackio package 4from ._tdlpackio import * 5from ._tdlpackio import __doc__ 6 7from .grids import GRIDS, TdlpackGridDef, get_grid, has_grid 8from .version import version as __version__ 9 10__all__ = [ 11 "__version__", 12 "open", 13 "TdlpackRecord", 14 "TdlpackStationRecord", 15 "TdlpackTrailerRecord", 16 "TdlpackID", 17 "TdlpackGridDef", 18 "GRIDS", 19 "get_grid", 20 "has_grid", 21] 22 23try: 24 from . import __config__ 25 26 __version__ = __config__.tdlpackio_version 27 has_openmp_support = __config__.has_openmp_support 28 tdlpack_static = __config__.tdlpack_static 29 extra_objects = __config__.extra_objects 30except ImportError: 31 has_openmp_support = None 32 tdlpack_static = None 33 extra_objects = [] 34 35__tdlpacklib_version__ = "1.0.0" # For now... 36 37 38def show_config(): 39 """Print tdlpackio build configuration information.""" 40 print(f"tdlpackio version {__version__} Configuration:") 41 print("") 42 print(f"libtdlpack library version: {__tdlpacklib_version__}") 43 print(f"\tStatic library: {tdlpack_static}") 44 print(f"\tOpenMP support: {has_openmp_support}") 45 print("") 46 print("Static libs:") 47 for lib in extra_objects: 48 print(f"\t{lib}")
89class open: 90 """ 91 Open class for tdlpackio. 92 93 Parameters 94 ---------- 95 path : str 96 File name. 97 98 mode : {'r', 'w'}, default 'r' 99 File handle mode. 100 101 format : {'sequential', 'random-access'}, optional 102 File type when creating a new file. 103 104 ra_template : {'small', 'large'}, optional 105 Template used when creating random-access files. 106 """ 107 108 _filetype_map = { 109 "random-access": 1, 110 "sequential": 2, 111 } 112 113 def __init__(self, path, mode="r", format=None, ra_template=None): 114 if mode not in {"r", "w", "a"}: 115 raise ValueError(f"Invalid mode: {mode}") 116 if mode in {"r", "w"}: 117 mode = mode + "b" 118 elif mode == "a": 119 mode = "wb" 120 self.path = path 121 self.mode = mode 122 self.format = format 123 self.ra_template = ra_template 124 self._hasindex = False 125 self._index = {} 126 self._lun = -1 127 self.mode = mode 128 self.name = os.path.abspath(path) 129 self.records = 0 130 131 # Perform indexing on read 132 if "r" in self.mode: 133 self._filehandle = builtins.open(path, mode=mode) 134 self.filetype = self._get_tdlpack_file_type() 135 self._build_index() 136 137 elif "w" in self.mode: 138 self.bytes_written = 0 139 self.records_written = 0 140 self.filetype = format if format is not None else "sequential" 141 if self.filetype == "random-access": 142 ra_template = "small" if ra_template is None else ra_template 143 iret, self._lun = tdlpacklib.open_tdlpack_file(self.name, self.mode, self._ifiletype, ra_template=ra_template) 144 elif self.filetype == "sequential": 145 self._filehandle = builtins.open(path, mode=mode) 146 iret, self._lun = tdlpacklib.open_tdlpack_file(self.name, self.mode, self._ifiletype, ra_template=ra_template) 147 148 # Add self to file data store 149 _open_file_store[self.name] = self 150 151 @property 152 def _ifiletype(self): 153 """Return numeric filetype""" 154 return self._filetype_map[self.filetype] 155 156 @property 157 def size(self): 158 """Return the file size.""" 159 return os.path.getsize(self.name) 160 161 def __enter__(self): 162 """""" 163 return self 164 165 def __exit__(self, atype, value, traceback): 166 """""" 167 self.close() 168 169 def __iter__(self): 170 """""" 171 yield from self._index["record"] 172 173 def __repr__(self): 174 """""" 175 strings = [] 176 keys = list(self.__dict__.keys()) 177 for k in keys: 178 if not k.startswith("_"): 179 strings.append(f"{k} = {self.__dict__[k]}\n") 180 # Attach size property. 181 strings.append(f"size = {self.size}\n") 182 return "".join(strings) 183 184 def __getitem__(self, key): 185 """""" 186 if isinstance(key, slice): 187 return self._index["record"][key] 188 elif isinstance(key, int): 189 return self._index["record"][key] 190 else: 191 raise KeyError("Key must be an integer record number or a slice") 192 193 def _get_tdlpack_file_type(self): 194 """Determine the type of TDLPACK file""" 195 self._filehandle.seek(0) 196 a = struct.unpack(">i", self._filehandle.read(4))[0] 197 b = struct.unpack(">i", self._filehandle.read(4))[0] 198 self._filehandle.seek(0) 199 return "random-access" if [a, b] == [0, 4] else "sequential" 200 201 def _build_index(self): 202 """Record Indexer""" 203 # Initialize index dictionary 204 self._index["offset"] = [] 205 self._index["size"] = [] 206 self._index["type"] = [] 207 self._index["record"] = [] 208 209 if self.filetype == "random-access": 210 self._randomaccess_file_indexer() 211 elif self.filetype == "sequential": 212 self._sequential_file_indexer() 213 self._hasindex = True 214 215 def _randomaccess_file_indexer(self): 216 """Indexer for random-access TDLPACK files""" 217 # Read master key 218 version, nids, nwords, nkyrec, maxent, lastky = struct.unpack(">iiiiii", self._filehandle.read(24)) 219 nbytes = nwords * NBYPWD 220 last_key_check = [99999999] if lastky > 9999 else [9999, 99999999] 221 self.master_key = dict( 222 version=version, 223 nids=nids, 224 nwords=nwords, 225 nkyrec=nkyrec, 226 maxent=maxent, 227 lastky=lastky, 228 ) 229 self.key_records = [] 230 # Set file position to first key record 231 self._filehandle.seek(nbytes) 232 233 last_station_id_rec = -1 234 last_station_lat_rec = -1 235 last_station_lon_rec = -1 236 237 # Iterate over all key records 238 while True: 239 # Read key record "header" data 240 nkeys, prec_this_key, prec_next_key = struct.unpack(">iii", self._filehandle.read(12)) 241 self.key_records.append( 242 dict( 243 nkeys=nkeys, 244 prec_this_key=prec_this_key, 245 prec_next_key=prec_next_key, 246 ) 247 ) 248 249 ids = list() 250 nsize = list() 251 prec_begin = list() 252 253 # Read key record information 254 for i in range(nkeys): 255 id1, id2, id3, id4, nd, bprec = struct.unpack(">iiiiii", self._filehandle.read(24)) 256 ids.append([id1, id2, id3, id4]) 257 nsize.append(nd) 258 prec_begin.append(bprec) 259 260 # Using key record info, move around file to inventory TDLPACK data 261 for m, n, b in zip(ids, nsize, prec_begin): 262 # Disect prec_begin 263 prec1 = int(b / 1000.0) 264 265 # Offset to the data record 266 offset = (prec1 - 1) * nbytes 267 self._filehandle.seek(offset) 268 self._index["offset"].append(offset) 269 self._index["size"].append(n * NBYPWD) 270 271 # Determine record type. Since the 4-word ID is stored in the 272 # key record, we can use it to determine station call letter 273 # record or TDLPACK data record. 274 if m[0] == 400001000: 275 # Station ID record...not in TDLPACK format 276 rec = TdlpackStationRecord() 277 last_station_id_rec = self.records 278 rec._recnum = self.records 279 rec._source = self.name 280 rec._nsta_expected = int(n / 2) 281 self._index["record"].append(rec) 282 self._index["type"].append("station") 283 else: 284 if m[0] == 400006000: 285 last_station_lat_rec = self.records 286 elif m[0] == 400007000: 287 last_station_lon_rec = self.records 288 # TDLPACK data record 289 ipack = np.frombuffer(self._filehandle.read(132), dtype=">i4").astype(np.int32) 290 iret, is0, is1, is2, is4 = tdlpacklib.unpack_meta(ipack) 291 if np.all(is2 == 0): 292 is2 = None 293 rec = TdlpackRecord(is0, is1, is2, is4) 294 rec._recnum = self.records 295 rec._linked_station_id_record = last_station_id_rec 296 rec._linked_station_lat_record = last_station_lat_rec 297 rec._linked_station_lon_record = last_station_lon_rec 298 rec._source = self.name 299 shape = (rec.ny, rec.nx) if rec.type == "grid" else (rec.numberOfPackedValues,) 300 ndim = len(shape) 301 dtype = "float32" 302 rec._data = TdlpackRecordOnDiskArray( 303 shape, 304 ndim, 305 dtype, 306 self.filetype, 307 self._filehandle, 308 rec, 309 self._index["offset"][-1], 310 self._index["size"][-1], 311 ) 312 self._index["record"].append(rec) 313 self._index["type"].append("data") 314 self.records += 1 315 316 # Hold the record number of the last station ID record 317 if self._index["type"][-1] == "station": 318 _last_station_id_record = self.records # This should be OK. 319 320 # Break loop here, at last key record 321 if prec_next_key in last_key_check: 322 break 323 offset = (prec_next_key - 1) * nbytes 324 self._filehandle.seek(offset) 325 326 # Make key_records immutable 327 self.key_records = tuple(self.key_records) 328 329 def _sequential_file_indexer(self): 330 """Indexer for sequential TDLPACK files""" 331 last_station_id_rec = -1 332 last_station_lat_rec = -1 333 last_station_lon_rec = -1 334 335 # Iterate 336 while True: 337 try: 338 # First read 4-byte Fortran record header 339 pos = self._filehandle.tell() 340 fortran_header = struct.unpack(">i", self._filehandle.read(4))[0] 341 if fortran_header >= 132: 342 bytes_to_read = 132 343 else: 344 bytes_to_read = fortran_header 345 346 pos = self._filehandle.tell() 347 ioctet = np.frombuffer(self._filehandle.read(8), dtype=">i8").astype(np.int64)[0] 348 ipack = np.frombuffer(self._filehandle.read(bytes_to_read - 8), dtype=">i4").astype(np.int32) 349 _header = struct.unpack(">4s", ipack[0])[0].decode() 350 351 # Check to first 4 bytes of the data record to determine the data 352 # record type. 353 if _header == "PLDT": 354 if ipack[5] == 400006000: 355 last_station_lat_rec = self.records 356 elif ipack[5] == 400007000: 357 last_station_lon_rec = self.records 358 # TDLPACK data record 359 iret, is0, is1, is2, is4 = tdlpacklib.unpack_meta(ipack) 360 self._index["offset"].append(pos) 361 self._index["size"].append(fortran_header) # Size given by Fortran header 362 if np.all(is2 == 0): 363 is2 = None 364 rec = TdlpackRecord(is0, is1, is2, is4) 365 rec._recnum = self.records 366 rec._linked_station_id_record = last_station_id_rec 367 rec._linked_station_lat_record = last_station_lat_rec 368 rec._linked_station_lon_record = last_station_lon_rec 369 rec._source = self.name 370 shape = (rec.ny, rec.nx) if rec.type == "grid" else (rec.numberOfPackedValues,) 371 ndim = len(shape) 372 dtype = "float32" 373 rec._data = TdlpackRecordOnDiskArray( 374 shape, 375 ndim, 376 dtype, 377 self.filetype, 378 self._filehandle, 379 rec, 380 self._index["offset"][-1], 381 self._index["size"][-1], 382 ) 383 self._index["record"].append(rec) 384 self._index["type"].append("data") 385 else: 386 if ioctet == 24 and ipack[4] == 9999: 387 # Trailer record 388 rec = TdlpackTrailerRecord() 389 rec._recnum = self.records 390 rec._source = self.name 391 self._index["offset"].append(pos) 392 self._index["size"].append(fortran_header) 393 self._index["type"].append("trailer") 394 self._index["record"].append(rec) 395 else: 396 # Station ID record 397 rec = TdlpackStationRecord() 398 last_station_id_rec = self.records 399 rec._recnum = self.records 400 rec._source = self.name 401 rec._nsta_expected = int(ioctet / 8) 402 self._index["offset"].append(pos) 403 self._index["size"].append(fortran_header) 404 self._index["type"].append("station") 405 self._index["record"].append(rec) 406 407 # At this point we have successfully identified a TDLPACK record from 408 # the file. Increment self.records and position the file pointer to 409 # now read the Fortran trailer. 410 self.records += 1 # Includes trailer records 411 self._filehandle.seek(fortran_header - bytes_to_read, 1) 412 fortran_trailer = struct.unpack(">i", self._filehandle.read(4))[0] 413 414 # Check Fortran header and trailer for the record. 415 if fortran_header != fortran_trailer: 416 raise IOError("Bad Fortran record.") 417 418 # Hold the record number of the last station ID record 419 if self._index["type"][-1] == "station": 420 _last_station_id_record = self.records # This should be OK. 421 422 except struct.error: 423 self._filehandle.seek(0) 424 break 425 426 def read(self, n): 427 """ 428 Read record from file. 429 430 Parameters 431 ---------- 432 n : int 433 Record number. 434 435 Returns 436 ------- 437 numpy.ndarray 438 Record data as a NumPy array. The returned dtype depends on the 439 record type: 440 441 - ``data`` or ``trailer`` : ``int32`` array. 442 - ``station`` : fixed-width byte string array (``S8``). 443 444 Notes 445 ----- 446 The file pointer is positioned using the internal index before reading. 447 Record size is determined from the file header (sequential) or index 448 (random-access). 449 """ 450 if "w" in self.mode: 451 pass # Remove this at some point.... 452 # Position file pointer to the beginning of the TDLPACK record. 453 self._filehandle.seek(self._index["offset"][n]) 454 if self.filetype == "sequential": 455 size = np.frombuffer(self._filehandle.read(8), dtype=">i8").astype(np.int64)[0] 456 elif self.filetype == "random-access": 457 size = self._index["size"][n] 458 459 if self._index["type"][n] in {"data", "trailer"}: 460 return np.frombuffer(self._filehandle.read(size), dtype=">i4").astype(np.int32) 461 elif self._index["type"][n] == "station": 462 return np.frombuffer(self._filehandle.read(size), dtype="S8") 463 464 def write(self, record): 465 """ 466 Write record(s) to file. 467 468 Parameters 469 ---------- 470 record : TdlpackStationRecord, _TdlpackRecord, TdlpackTrailerRecord, or list 471 Record or list of records to write. 472 473 - ``TdlpackStationRecord`` : Station record. Station identifiers 474 are padded to ``NCHAR``. 475 - ``_TdlpackRecord`` : Packed data record. 476 - ``TdlpackTrailerRecord`` : Trailer record. 477 - ``list`` : List of supported record types. 478 479 Returns 480 ------- 481 None 482 483 Notes 484 ----- 485 Updates ``bytes_written``, ``records_written``, ``records``, and 486 ``_type_lastrecord_written``. 487 """ 488 if isinstance(record, list): 489 for rec in record: 490 self.write(rec) 491 return 492 493 nreplace, ncheck = 0, 0 494 495 if isinstance(record, TdlpackStationRecord): 496 # Adjust string length of each station to NCHAR. 497 stns = [s.ljust(NCHAR) for s in record.stations] 498 iret, self.bytes_written, self.records_written = tdlpacklib.write_station_record( 499 self.name, 500 self._lun, 501 self._ifiletype, 502 stns, 503 self.bytes_written, 504 self.records_written, 505 nreplace, 506 ncheck, 507 ) 508 509 elif issubclass(record.__class__, _TdlpackRecord): 510 iret, self.bytes_written, self.records_written = tdlpacklib.write_tdlpack_record( 511 self.name, 512 self._lun, 513 self._ifiletype, 514 record._ipack, 515 self.bytes_written, 516 self.records_written, 517 nreplace, 518 ncheck, 519 ) 520 521 elif isinstance(record, TdlpackTrailerRecord): 522 iret, self.bytes_written, self.records_written = tdlpacklib.write_trailer_record( 523 self._lun, self._ifiletype, self.bytes_written, self.records_written 524 ) 525 526 self._type_lastrecord_written = record.type 527 self.records += 1 528 529 def close(self): 530 """ 531 Close the file. 532 533 Returns 534 ------- 535 None 536 537 Notes 538 ----- 539 For write mode, a trailer record may be written for sequential files 540 if the last record type requires it. The underlying TDLpack file handle 541 is then closed. The file is removed from the internal open file store. 542 """ 543 if "r" in self.mode: 544 self._filehandle.close() 545 if "w" in self.mode: 546 if self.filetype == "sequential": 547 if self._type_lastrecord_written == "vector": 548 iret = tdlpacklib.write_trailer_record( 549 self._lun, 550 self._ifiletype, 551 self.bytes_written, 552 self.records_written, 553 ) 554 iret = tdlpacklib.close_tdlpack_file(self._lun, self._ifiletype) 555 if iret != 0: 556 raise ValueError("return from tdlpacklib.close_tdlpack_file is non-zero") 557 if self.name in _open_file_store.keys(): 558 del _open_file_store[self.name] 559 560 def select(self, **kwargs): 561 """ 562 Select records by attribute. 563 564 Parameters 565 ---------- 566 **kwargs : dict 567 Keyword arguments specifying ``TdlpackRecord`` attributes and 568 values to match. 569 570 Returns 571 ------- 572 list 573 List of records matching all provided attribute/value pairs. 574 575 Notes 576 ----- 577 Selection is performed against records in the internal index. Records 578 must match all specified attributes to be included in the result. 579 """ 580 _id_keys = [ 581 "word1", 582 "word2", 583 "word3", 584 "word4", 585 "ccc", 586 "fff", 587 "b", 588 "dd", 589 "v", 590 "llll", 591 "uuuu", 592 "t", 593 "rr", 594 "o", 595 "hh", 596 "tau", 597 "thresh", 598 "i", 599 "s", 600 "g", 601 ] 602 603 # TODO: Add ability to process multiple values for each keyword (attribute) 604 idxs = [] 605 nkeys = len(kwargs.keys()) 606 for k, v in kwargs.items(): 607 if k in _id_keys: 608 for rec in self._index["record"]: 609 # if hasattr(rec, k) and getattr(rec, k) == v: 610 if getattr(rec.id, k) == v: 611 idxs.append(rec._recnum) 612 else: 613 for rec in self._index["record"]: 614 if hasattr(rec, k) and getattr(rec, k) == v: 615 idxs.append(rec._recnum) 616 idxs = np.array(idxs, dtype=">i4") 617 return [self._index["record"][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]]
Open class for tdlpackio.
Parameters
- path (str): File name.
- mode ({'r', 'w'}, default 'r'): File handle mode.
- format ({'sequential', 'random-access'}, optional): File type when creating a new file.
- ra_template ({'small', 'large'}, optional): Template used when creating random-access files.
113 def __init__(self, path, mode="r", format=None, ra_template=None): 114 if mode not in {"r", "w", "a"}: 115 raise ValueError(f"Invalid mode: {mode}") 116 if mode in {"r", "w"}: 117 mode = mode + "b" 118 elif mode == "a": 119 mode = "wb" 120 self.path = path 121 self.mode = mode 122 self.format = format 123 self.ra_template = ra_template 124 self._hasindex = False 125 self._index = {} 126 self._lun = -1 127 self.mode = mode 128 self.name = os.path.abspath(path) 129 self.records = 0 130 131 # Perform indexing on read 132 if "r" in self.mode: 133 self._filehandle = builtins.open(path, mode=mode) 134 self.filetype = self._get_tdlpack_file_type() 135 self._build_index() 136 137 elif "w" in self.mode: 138 self.bytes_written = 0 139 self.records_written = 0 140 self.filetype = format if format is not None else "sequential" 141 if self.filetype == "random-access": 142 ra_template = "small" if ra_template is None else ra_template 143 iret, self._lun = tdlpacklib.open_tdlpack_file(self.name, self.mode, self._ifiletype, ra_template=ra_template) 144 elif self.filetype == "sequential": 145 self._filehandle = builtins.open(path, mode=mode) 146 iret, self._lun = tdlpacklib.open_tdlpack_file(self.name, self.mode, self._ifiletype, ra_template=ra_template) 147 148 # Add self to file data store 149 _open_file_store[self.name] = self
156 @property 157 def size(self): 158 """Return the file size.""" 159 return os.path.getsize(self.name)
Return the file size.
426 def read(self, n): 427 """ 428 Read record from file. 429 430 Parameters 431 ---------- 432 n : int 433 Record number. 434 435 Returns 436 ------- 437 numpy.ndarray 438 Record data as a NumPy array. The returned dtype depends on the 439 record type: 440 441 - ``data`` or ``trailer`` : ``int32`` array. 442 - ``station`` : fixed-width byte string array (``S8``). 443 444 Notes 445 ----- 446 The file pointer is positioned using the internal index before reading. 447 Record size is determined from the file header (sequential) or index 448 (random-access). 449 """ 450 if "w" in self.mode: 451 pass # Remove this at some point.... 452 # Position file pointer to the beginning of the TDLPACK record. 453 self._filehandle.seek(self._index["offset"][n]) 454 if self.filetype == "sequential": 455 size = np.frombuffer(self._filehandle.read(8), dtype=">i8").astype(np.int64)[0] 456 elif self.filetype == "random-access": 457 size = self._index["size"][n] 458 459 if self._index["type"][n] in {"data", "trailer"}: 460 return np.frombuffer(self._filehandle.read(size), dtype=">i4").astype(np.int32) 461 elif self._index["type"][n] == "station": 462 return np.frombuffer(self._filehandle.read(size), dtype="S8")
Read record from file.
Parameters
- n (int): Record number.
Returns
numpy.ndarray: Record data as a NumPy array. The returned dtype depends on the record type:
dataortrailer:int32array.station: fixed-width byte string array (S8).
Notes
The file pointer is positioned using the internal index before reading. Record size is determined from the file header (sequential) or index (random-access).
464 def write(self, record): 465 """ 466 Write record(s) to file. 467 468 Parameters 469 ---------- 470 record : TdlpackStationRecord, _TdlpackRecord, TdlpackTrailerRecord, or list 471 Record or list of records to write. 472 473 - ``TdlpackStationRecord`` : Station record. Station identifiers 474 are padded to ``NCHAR``. 475 - ``_TdlpackRecord`` : Packed data record. 476 - ``TdlpackTrailerRecord`` : Trailer record. 477 - ``list`` : List of supported record types. 478 479 Returns 480 ------- 481 None 482 483 Notes 484 ----- 485 Updates ``bytes_written``, ``records_written``, ``records``, and 486 ``_type_lastrecord_written``. 487 """ 488 if isinstance(record, list): 489 for rec in record: 490 self.write(rec) 491 return 492 493 nreplace, ncheck = 0, 0 494 495 if isinstance(record, TdlpackStationRecord): 496 # Adjust string length of each station to NCHAR. 497 stns = [s.ljust(NCHAR) for s in record.stations] 498 iret, self.bytes_written, self.records_written = tdlpacklib.write_station_record( 499 self.name, 500 self._lun, 501 self._ifiletype, 502 stns, 503 self.bytes_written, 504 self.records_written, 505 nreplace, 506 ncheck, 507 ) 508 509 elif issubclass(record.__class__, _TdlpackRecord): 510 iret, self.bytes_written, self.records_written = tdlpacklib.write_tdlpack_record( 511 self.name, 512 self._lun, 513 self._ifiletype, 514 record._ipack, 515 self.bytes_written, 516 self.records_written, 517 nreplace, 518 ncheck, 519 ) 520 521 elif isinstance(record, TdlpackTrailerRecord): 522 iret, self.bytes_written, self.records_written = tdlpacklib.write_trailer_record( 523 self._lun, self._ifiletype, self.bytes_written, self.records_written 524 ) 525 526 self._type_lastrecord_written = record.type 527 self.records += 1
Write record(s) to file.
Parameters
record (TdlpackStationRecord, _TdlpackRecord, TdlpackTrailerRecord, or list): Record or list of records to write.
TdlpackStationRecord: Station record. Station identifiers are padded toNCHAR._TdlpackRecord: Packed data record.TdlpackTrailerRecord: Trailer record.list: List of supported record types.
Returns
- None
Notes
Updates bytes_written, records_written, records, and
_type_lastrecord_written.
529 def close(self): 530 """ 531 Close the file. 532 533 Returns 534 ------- 535 None 536 537 Notes 538 ----- 539 For write mode, a trailer record may be written for sequential files 540 if the last record type requires it. The underlying TDLpack file handle 541 is then closed. The file is removed from the internal open file store. 542 """ 543 if "r" in self.mode: 544 self._filehandle.close() 545 if "w" in self.mode: 546 if self.filetype == "sequential": 547 if self._type_lastrecord_written == "vector": 548 iret = tdlpacklib.write_trailer_record( 549 self._lun, 550 self._ifiletype, 551 self.bytes_written, 552 self.records_written, 553 ) 554 iret = tdlpacklib.close_tdlpack_file(self._lun, self._ifiletype) 555 if iret != 0: 556 raise ValueError("return from tdlpacklib.close_tdlpack_file is non-zero") 557 if self.name in _open_file_store.keys(): 558 del _open_file_store[self.name]
Close the file.
Returns
- None
Notes
For write mode, a trailer record may be written for sequential files if the last record type requires it. The underlying TDLpack file handle is then closed. The file is removed from the internal open file store.
560 def select(self, **kwargs): 561 """ 562 Select records by attribute. 563 564 Parameters 565 ---------- 566 **kwargs : dict 567 Keyword arguments specifying ``TdlpackRecord`` attributes and 568 values to match. 569 570 Returns 571 ------- 572 list 573 List of records matching all provided attribute/value pairs. 574 575 Notes 576 ----- 577 Selection is performed against records in the internal index. Records 578 must match all specified attributes to be included in the result. 579 """ 580 _id_keys = [ 581 "word1", 582 "word2", 583 "word3", 584 "word4", 585 "ccc", 586 "fff", 587 "b", 588 "dd", 589 "v", 590 "llll", 591 "uuuu", 592 "t", 593 "rr", 594 "o", 595 "hh", 596 "tau", 597 "thresh", 598 "i", 599 "s", 600 "g", 601 ] 602 603 # TODO: Add ability to process multiple values for each keyword (attribute) 604 idxs = [] 605 nkeys = len(kwargs.keys()) 606 for k, v in kwargs.items(): 607 if k in _id_keys: 608 for rec in self._index["record"]: 609 # if hasattr(rec, k) and getattr(rec, k) == v: 610 if getattr(rec.id, k) == v: 611 idxs.append(rec._recnum) 612 else: 613 for rec in self._index["record"]: 614 if hasattr(rec, k) and getattr(rec, k) == v: 615 idxs.append(rec._recnum) 616 idxs = np.array(idxs, dtype=">i4") 617 return [self._index["record"][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]]
Select records by attribute.
Parameters
- **kwargs (dict):
Keyword arguments specifying
TdlpackRecordattributes and values to match.
Returns
- list: List of records matching all provided attribute/value pairs.
Notes
Selection is performed against records in the internal index. Records must match all specified attributes to be included in the result.
620class TdlpackRecord: 621 """ 622 Creation class for TDLPACK record objects. 623 624 This class dynamically constructs and returns an instance of a 625 ``_TdlpackRecord`` subclass based on the provided section arrays and 626 optional keyword arguments. Record classes are cached by type to avoid 627 repeated class construction. 628 629 Parameters 630 ---------- 631 is0 : numpy.ndarray, optional 632 Section 0 array. Default is zero-initialized ``int32`` array of size ``ND7``. 633 is1 : numpy.ndarray, optional 634 Section 1 array. Default is zero-initialized ``int32`` array of size ``ND7``. 635 is2 : numpy.ndarray, optional 636 Section 2 array. Default is zero-initialized ``int32`` array of size ``ND7``. 637 is4 : numpy.ndarray, optional 638 Section 4 array. Default is zero-initialized ``int32`` array of size ``ND7``. 639 *args : tuple 640 Additional positional arguments passed to the constructed record class. 641 **kwargs : dict 642 Optional keyword arguments. The following key is recognized: 643 644 - ``type`` : str, optional 645 Record type. Default is ``"vector"``. If ``is2`` contains any 646 non-zero values, the type is automatically set to ``"grid"``. 647 - ``grid`` : str, optional 648 Predefined grid. Valid values can be found in ``tdlpackio.grids.GRIDS``. 649 This will force the type to be ``"grid"`` and override values in the ``is2`` 650 argument. 651 652 Returns 653 ------- 654 _TdlpackRecord 655 Instance of a dynamically generated subclass of ``_TdlpackRecord``. 656 657 Notes 658 ----- 659 - Record subclasses are created dynamically and cached in 660 ``_record_class_store`` using the record type as the key. 661 - For ``"grid"`` records, ``templates.GridDefinitionSection`` is added 662 as a base class and ``is1[1]`` is set to indicate the presence of a 663 grid definition section. 664 """ 665 666 def __new__( 667 self, 668 is0: np.array = np.zeros((ND7), dtype=np.int32), 669 is1: np.array = np.zeros((ND7), dtype=np.int32), 670 is2: np.array = np.zeros((ND7), dtype=np.int32), 671 is4: np.array = np.zeros((ND7), dtype=np.int32), 672 *args, 673 **kwargs, 674 ): 675 676 rectype = "vector" 677 if "type" in kwargs.keys(): 678 rectype = kwargs["type"] 679 if rectype not in {"grid", "vector"}: 680 raise ValueError('Invalid "type" argument. Expected "grid" or "vector".') 681 682 if bool(np.any(is2)): 683 rectype = "grid" 684 685 if "grid" in kwargs.keys(): 686 rectype = "grid" 687 if kwargs["grid"] not in grids.GRIDS.keys(): 688 raise ValueError('Invalid "grid" argument.') 689 is2 = grids.get_grid(kwargs["grid"]).to_is2() 690 691 bases = list() 692 if rectype == "grid": 693 bases.append(templates.GridDefinitionSection) 694 is1[1] = 1 # Flag in is1 to state that a grid definition section exists 695 696 try: 697 Record = _record_class_store[rectype] 698 except KeyError: 699 700 @dataclass(init=False, repr=False) 701 class Record(_TdlpackRecord, *bases): 702 pass 703 704 _record_class_store[rectype] = Record 705 706 return Record(is0, is1, is2, is4, *args)
Creation class for TDLPACK record objects.
This class dynamically constructs and returns an instance of a
_TdlpackRecord subclass based on the provided section arrays and
optional keyword arguments. Record classes are cached by type to avoid
repeated class construction.
Parameters
- is0 (numpy.ndarray, optional):
Section 0 array. Default is zero-initialized
int32array of sizeND7. - is1 (numpy.ndarray, optional):
Section 1 array. Default is zero-initialized
int32array of sizeND7. - is2 (numpy.ndarray, optional):
Section 2 array. Default is zero-initialized
int32array of sizeND7. - is4 (numpy.ndarray, optional):
Section 4 array. Default is zero-initialized
int32array of sizeND7. - *args (tuple): Additional positional arguments passed to the constructed record class.
**kwargs (dict): Optional keyword arguments. The following key is recognized:
type: str, optional Record type. Default is"vector". Ifis2contains any non-zero values, the type is automatically set to"grid".grid: str, optional Predefined grid. Valid values can be found intdlpackio.grids.GRIDS. This will force the type to be"grid"and override values in theis2argument.
Returns
- _TdlpackRecord: Instance of a dynamically generated subclass of
_TdlpackRecord.
Notes
- Record subclasses are created dynamically and cached in
_record_class_storeusing the record type as the key. - For
"grid"records,templates.GridDefinitionSectionis added as a base class andis1[1]is set to indicate the presence of a grid definition section.
1059@dataclass 1060class TdlpackStationRecord: 1061 """ 1062 TDLPACK Station Record class 1063 """ 1064 1065 stations_in: InitVar[Optional[Iterable[str]]] = None 1066 1067 type: str = field(init=False, repr=False, default="station") 1068 stations: ClassVar[templates.Stations] = templates.Stations() 1069 1070 # Private class variable holding the list 1071 _stations: Optional[list[str]] = field(init=False, repr=False, default=None) 1072 1073 def __post_init__(self, stations_in): 1074 self._nsta_expected = 0 1075 self._recnum = -1 1076 self._source = None 1077 self._stations = None 1078 self._type = "station" 1079 self.id = TdlpackID([400001000, 0, 0, 0], self) 1080 1081 if stations_in is not None: 1082 self.stations = stations_in 1083 1084 def __str__(self): 1085 return f"{self._recnum}:d=0000000000:STATION CALL LETTER RECORD:{self.numberOfStations}" 1086 1087 @property 1088 def numberOfStations(self): 1089 if self._source is not None and (isinstance(self._stations, list) or self._stations is None): 1090 return self._nsta_expected 1091 return 0 if self.stations is None else len(self.stations) 1092 1093 @property 1094 def data(self): 1095 pass 1096 1097 def pack(self): 1098 pass
TDLPACK Station Record class
1101@dataclass 1102class TdlpackTrailerRecord: 1103 """ 1104 TDLPACK Trailer Record class 1105 """ 1106 1107 type: str = field(init=False, repr=False, default="trailer") 1108 1109 def __post_init__(self): 1110 """""" 1111 self._recnum = -1 1112 self._type = "trailer" 1113 self.id = TdlpackID([0, 0, 0, 0], self) 1114 1115 def __str__(self): 1116 """""" 1117 return f"{self._recnum}:d=0000000000:TRAILER RECORD" 1118 1119 @property 1120 def data(self): 1121 """""" 1122 pass 1123 1124 def pack(self): 1125 """""" 1126 pass
TDLPACK Trailer Record class
1129class TdlpackID: 1130 """ 1131 TDLPACK variable ID class 1132 """ 1133 1134 __slots__ = ("_id", "_rec") 1135 1136 def __init__(self, id=[0, 0, 0, 0], linked_rec=None): 1137 """ 1138 Initialize TDLPACK variable ID 1139 1140 Parameters 1141 ---------- 1142 id : list of ints 1143 The 4-word TDLPACK variable ID 1144 linked_rec : TdlpackRecord, optional 1145 TDLPACK record object. This optional argument provides a mechanism 1146 for updating the TDLPACK variable ID in the TdlpackRecord is1 array. 1147 """ 1148 self._id = utils.parse_id(id) 1149 self._rec = linked_rec 1150 1151 def __eq__(self, value): 1152 if isinstance(value, list) or isinstance(value, tuple): 1153 return self.word1 == value[0] and self.word2 == value[1] and self.word3 == value[2] and self.word4 == value[3] 1154 else: 1155 return False 1156 1157 def __format__(self, spec: str) -> str: 1158 if not spec: 1159 spec = "basic" 1160 return self.format(spec) 1161 1162 def __repr__(self): 1163 return repr(utils.unparse_id(self._id)) 1164 1165 def __setitem__(self, key, value): 1166 self._id[key] = value 1167 1168 def __getitem__(self, key): 1169 return self._id[key] 1170 1171 @classmethod 1172 def from_string(cls, idstr): 1173 """ 1174 Create a TdlpackID object from a TDLPACK ID string. 1175 1176 Parameters 1177 ---------- 1178 idstr : str 1179 String containing the TDLPACK ID with leading zeros and delimited 1180 by a non-numeric character. 1181 1182 Returns 1183 ------- 1184 An instance of TdlpackID. 1185 """ 1186 delim = idstr[9] 1187 if {idstr[19], idstr[29]} != {delim, delim}: 1188 raise ValueError(f"Invalid TDLPACK ID string format") 1189 return cls([int(i.lstrip("0")) if len(i.lstrip("0")) > 0 else 0 for i in idstr.split(delim)]) 1190 1191 def format(self, style: str = "basic") -> str: 1192 """ 1193 Format the TDLPACK ID identifier as a string. 1194 1195 This method returns a string representation of the identifier in one 1196 of several supported formats commonly used in MOS-2000 workflows. 1197 1198 Parameters 1199 ---------- 1200 style : {'basic', 'b', 'mos', 'm', 'parsed', 'p'}, optional 1201 Output format style (case-insensitive): 1202 1203 - ``"basic"`` or ``"b"`` 1204 Four-word identifier. Each word is printed as a zero-padded 1205 integer field. 1206 1207 - ``"mos"`` or ``"m"`` 1208 MOS-style identifier consisting of the first three words 1209 followed by the ISG components and the threshold value 1210 formatted in scientific notation (``.0000e±00``). 1211 1212 - ``"parsed"`` or ``"p"`` 1213 Identifier parsed into its individual components as defined 1214 by the internal ID mapping. All components are printed as 1215 zero-padded integers except the threshold value, which is 1216 printed as a floating-point value with ``F13.6`` formatting. 1217 1218 Returns 1219 ------- 1220 str 1221 String representation of the identifier in the requested format. 1222 1223 Notes 1224 ----- 1225 The MOS-style threshold representation removes the leading zero 1226 from scientific notation (e.g., ``0.0000e+00`` → ``.0000e+00``) 1227 to match legacy MOS-2000 formatting conventions. 1228 1229 Examples 1230 -------- 1231 Using the ``format`` method: 1232 1233 >>> rec.format("basic") 1234 '001000008 000000500 000000000 0000000000' 1235 1236 >>> rec.format("mos") 1237 '001000008 000000500 000000000 000 .0000e+00' 1238 1239 >>> rec.format("parsed") 1240 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1241 1242 Using Python's format protocol (``__format__``): 1243 1244 >>> f"{rec.id:basic}" 1245 '001000008 000000500 000000000 0000000000' 1246 1247 >>> f"{rec.id:mos}" 1248 '001000008 000000500 000000000 000 .0000e+00' 1249 1250 >>> f"{rec.id:parsed}" 1251 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1252 """ 1253 style = style.lower() 1254 1255 if style in {"basic", "b"}: 1256 return f"{str(self.word1).zfill(9)} {str(self.word2).zfill(9)} {str(self.word3).zfill(9)} {str(self.word4).zfill(10)}" 1257 elif style in {"mos", "m"}: 1258 thresh = f"{self.thresh:.4e}" 1259 if thresh.startswith("0"): 1260 thresh = thresh[1:] 1261 elif thresh.startswith("-0"): 1262 thresh = "-" + thresh[2:] 1263 return f"{str(self.word1).zfill(9)} {str(self.word2).zfill(9)} {str(self.word3).zfill(9)} {self.i}{self.s}{self.g} {thresh}" 1264 elif style in {"parsed", "p"}: 1265 parsed = "" 1266 for k, v in self._id.items(): 1267 if "thresh" not in k: 1268 parsed += f"{str(v).zfill(len(k))} " 1269 else: 1270 parsed += f"{v:13.6f}" 1271 return parsed 1272 1273 raise ValueError(f"Unknown TdlpackID format style: {style!r}") 1274 1275 def to_dict(self): 1276 """ 1277 Return TDLPACK variable ID as dict. 1278 1279 Returns 1280 ------- 1281 String of the 4-word TDLPACK variable ID. 1282 """ 1283 return self._id 1284 1285 def to_string(self, delim=" "): 1286 """ 1287 Return TDLPACK variable ID as string. 1288 1289 Parameters 1290 ---------- 1291 delim : str, optional 1292 Delimiter character between each TDLPACK variable ID word. 1293 1294 Returns 1295 ------- 1296 String of the 4-word TDLPACK variable ID. 1297 """ 1298 strlen = (9, 9, 9, 10) 1299 return delim.join([str(i).zfill(word) for word, i in zip(strlen, utils.unparse_id(self._id))]) 1300 1301 @property 1302 def word1(self): 1303 """ 1304 First ID word. 1305 1306 Returns 1307 ------- 1308 int 1309 First parsed ID word. 1310 """ 1311 return utils.unparse_id(self._id)[0] 1312 1313 @word1.setter 1314 def word1(self, value): 1315 """ 1316 Set first ID word. 1317 1318 Parameters 1319 ---------- 1320 value : int 1321 New value. 1322 1323 Notes 1324 ----- 1325 Updates internal ID and ``is1[8]`` if record is attached. 1326 """ 1327 newid = utils.unparse_id(self._id) 1328 newid[0] = value 1329 self._id = utils.parse_id(newid) 1330 if self._rec is not None: 1331 self._rec.is1[8] = newid[0] 1332 1333 @property 1334 def word2(self): 1335 """ 1336 Second ID word. 1337 1338 Returns 1339 ------- 1340 int 1341 Second parsed ID word. 1342 """ 1343 return utils.unparse_id(self._id)[1] 1344 1345 @word2.setter 1346 def word2(self, value): 1347 """ 1348 Set second ID word. 1349 1350 Parameters 1351 ---------- 1352 value : int 1353 New value. 1354 1355 Notes 1356 ----- 1357 Updates internal ID and ``is1[9]`` if record is attached. 1358 """ 1359 newid = utils.unparse_id(self._id) 1360 newid[1] = value 1361 self._id = utils.parse_id(newid) 1362 if self._rec is not None: 1363 self._rec.is1[9] = newid[1] 1364 1365 @property 1366 def word3(self): 1367 """ 1368 Third ID word. 1369 1370 Returns 1371 ------- 1372 int 1373 Third parsed ID word. 1374 """ 1375 return utils.unparse_id(self._id)[2] 1376 1377 @word3.setter 1378 def word3(self, value): 1379 """ 1380 Set third ID word. 1381 1382 Parameters 1383 ---------- 1384 value : int 1385 New value. 1386 1387 Notes 1388 ----- 1389 Updates internal ID and ``is1[10]`` if record is attached. 1390 """ 1391 newid = utils.unparse_id(self._id) 1392 newid[2] = value 1393 self._id = utils.parse_id(newid) 1394 if self._rec is not None: 1395 self._rec.is1[10] = newid[2] 1396 1397 @property 1398 def word4(self): 1399 """ 1400 Fourth ID word. 1401 1402 Returns 1403 ------- 1404 int 1405 Fourth parsed ID word. 1406 """ 1407 return utils.unparse_id(self._id)[3] 1408 1409 @word4.setter 1410 def word4(self, value): 1411 """ 1412 Set fourth ID word. 1413 1414 Parameters 1415 ---------- 1416 value : int 1417 New value. 1418 1419 Notes 1420 ----- 1421 Updates internal ID and ``is1[11]`` if record is attached. 1422 """ 1423 newid = utils.unparse_id(self._id) 1424 newid[3] = value 1425 self._id = utils.parse_id(newid) 1426 if self._rec is not None: 1427 self._rec.is1[11] = newid[3] 1428 1429 @property 1430 def ccc(self): 1431 """ 1432 CCC identifier component. 1433 1434 Returns 1435 ------- 1436 int 1437 """ 1438 return self._id["ccc"] 1439 1440 @ccc.setter 1441 def ccc(self, value): 1442 """ 1443 Set CCC identifier component. 1444 1445 Parameters 1446 ---------- 1447 value : int 1448 1449 Notes 1450 ----- 1451 Updates ``is1[8]`` if record is attached. 1452 """ 1453 self._id["ccc"] = value 1454 if self._rec is not None: 1455 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1456 1457 @property 1458 def fff(self): 1459 """ 1460 FFF identifier component. 1461 1462 Returns 1463 ------- 1464 int 1465 """ 1466 return self._id["fff"] 1467 1468 @fff.setter 1469 def fff(self, value): 1470 """ 1471 Set FFF identifier component. 1472 1473 Parameters 1474 ---------- 1475 value : int 1476 1477 Notes 1478 ----- 1479 Updates ``is1[8]`` if record is attached. 1480 """ 1481 self._id["fff"] = value 1482 if self._rec is not None: 1483 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1484 1485 @property 1486 def cccfff(self): 1487 """ 1488 Combined CCCFFF identifier. 1489 1490 Returns 1491 ------- 1492 int 1493 Integer representation of ``word1 / 1000``. 1494 """ 1495 return int(self.word1 / 1000) 1496 1497 @property 1498 def b(self): 1499 """ 1500 B identifier component. 1501 1502 Returns 1503 ------- 1504 int 1505 """ 1506 return self._id["b"] 1507 1508 @b.setter 1509 def b(self, value): 1510 """ 1511 Set B identifier component. 1512 1513 Parameters 1514 ---------- 1515 value : int 1516 1517 Notes 1518 ----- 1519 Updates ``is1[8]`` if record is attached. 1520 """ 1521 self._id["b"] = value 1522 if self._rec is not None: 1523 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1524 1525 @property 1526 def dd(self): 1527 """ 1528 DD identifier component. 1529 1530 Returns 1531 ------- 1532 int 1533 """ 1534 return self._id["dd"] 1535 1536 @dd.setter 1537 def dd(self, value): 1538 """ 1539 Set DD identifier component. 1540 1541 Parameters 1542 ---------- 1543 value : int 1544 1545 Notes 1546 ----- 1547 Updates ``is1[8]`` and ``is1[14]`` if record is attached. 1548 """ 1549 self._id["dd"] = value 1550 if self._rec is not None: 1551 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1552 self._rec.is1[14] = value 1553 1554 @property 1555 def v(self): 1556 """ 1557 V identifier component. 1558 1559 Returns 1560 ------- 1561 int 1562 """ 1563 return self._id["v"] 1564 1565 @v.setter 1566 def v(self, value): 1567 """ 1568 Set V identifier component. 1569 1570 Parameters 1571 ---------- 1572 value : int 1573 1574 Notes 1575 ----- 1576 Updates ``is1[9]`` if record is attached. 1577 """ 1578 self._id["v"] = value 1579 if self._rec is not None: 1580 self._rec.is1[9] = utils.unparse_id(self._id)[1] 1581 1582 @property 1583 def llll(self): 1584 """ 1585 LLLL identifier component. 1586 1587 Returns 1588 ------- 1589 int 1590 """ 1591 return self._id["llll"] 1592 1593 @llll.setter 1594 def llll(self, value): 1595 """ 1596 Set LLLL identifier component. 1597 1598 Parameters 1599 ---------- 1600 value : int 1601 1602 Notes 1603 ----- 1604 Updates ``is1[9]`` if record is attached. 1605 """ 1606 self._id["llll"] = value 1607 if self._rec is not None: 1608 self._rec.is1[9] = utils.unparse_id(self._id)[1] 1609 1610 @property 1611 def uuuu(self): 1612 """ 1613 UUUU identifier component. 1614 1615 Returns 1616 ------- 1617 int 1618 """ 1619 return self._id["uuuu"] 1620 1621 @uuuu.setter 1622 def uuuu(self, value): 1623 """ 1624 Set UUUU identifier component. 1625 1626 Parameters 1627 ---------- 1628 value : int 1629 1630 Notes 1631 ----- 1632 Updates ``is1[9]`` if record is attached. 1633 """ 1634 self._id["uuuu"] = value 1635 if self._rec is not None: 1636 self._rec.is1[9] = utils.unparse_id(self._id)[1] 1637 1638 @property 1639 def t(self): 1640 """ 1641 T identifier component. 1642 1643 Returns 1644 ------- 1645 int 1646 """ 1647 return self._id["t"] 1648 1649 @t.setter 1650 def t(self, value): 1651 """ 1652 Set T identifier component. 1653 1654 Parameters 1655 ---------- 1656 value : int 1657 1658 Notes 1659 ----- 1660 Updates ``is1[10]`` if record is attached. 1661 """ 1662 self._id["t"] = value 1663 if self._rec is not None: 1664 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1665 1666 @property 1667 def rr(self): 1668 """ 1669 RR identifier component. 1670 1671 Returns 1672 ------- 1673 int 1674 """ 1675 return self._id["rr"] 1676 1677 @rr.setter 1678 def rr(self, value): 1679 """ 1680 Set RR identifier component. 1681 1682 Parameters 1683 ---------- 1684 value : int 1685 1686 Notes 1687 ----- 1688 Updates ``is1[10]`` if record is attached. 1689 """ 1690 self._id["rr"] = value 1691 if self._rec is not None: 1692 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1693 1694 @property 1695 def o(self): 1696 """ 1697 O identifier component. 1698 1699 Returns 1700 ------- 1701 int 1702 """ 1703 return self._id["o"] 1704 1705 @o.setter 1706 def o(self, value): 1707 """ 1708 Set O identifier component. 1709 1710 Parameters 1711 ---------- 1712 value : int 1713 1714 Notes 1715 ----- 1716 Updates ``is1[10]`` if record is attached. 1717 """ 1718 self._id["o"] = value 1719 if self._rec is not None: 1720 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1721 1722 @property 1723 def hh(self): 1724 """ 1725 HH identifier component. 1726 1727 Returns 1728 ------- 1729 int 1730 """ 1731 return self._id["hh"] 1732 1733 @hh.setter 1734 def hh(self, value): 1735 """ 1736 Set HH identifier component. 1737 1738 Parameters 1739 ---------- 1740 value : int 1741 1742 Notes 1743 ----- 1744 Updates ``is1[10]`` if record is attached. 1745 """ 1746 self._id["hh"] = value 1747 if self._rec is not None: 1748 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1749 1750 @property 1751 def tau(self): 1752 """ 1753 Forecast hour (tau). 1754 1755 Returns 1756 ------- 1757 int 1758 """ 1759 return self._id["tau"] 1760 1761 @tau.setter 1762 def tau(self, value): 1763 """ 1764 Set forecast hour (tau). 1765 1766 Parameters 1767 ---------- 1768 value : int 1769 1770 Notes 1771 ----- 1772 Updates ``is1[10]`` and ``is1[12]`` if record is attached. 1773 """ 1774 self._id["tau"] = value 1775 if self._rec is not None: 1776 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1777 self._rec.is1[12] = value 1778 1779 @property 1780 def thresh(self): 1781 """ 1782 Threshold identifier component. 1783 1784 Returns 1785 ------- 1786 int 1787 """ 1788 return self._id["thresh"] 1789 1790 @thresh.setter 1791 def thresh(self, value): 1792 """ 1793 Set threshold identifier component. 1794 1795 Parameters 1796 ---------- 1797 value : int 1798 1799 Notes 1800 ----- 1801 Updates ``is1[11]`` if record is attached. 1802 """ 1803 self._id["thresh"] = value 1804 if self._rec is not None: 1805 self._rec.is1[11] = utils.unparse_id(self._id)[3] 1806 1807 @property 1808 def i(self): 1809 """ 1810 I identifier component. 1811 1812 Returns 1813 ------- 1814 int 1815 """ 1816 return self._id["i"] 1817 1818 @i.setter 1819 def i(self, value): 1820 """ 1821 Set I identifier component. 1822 1823 Parameters 1824 ---------- 1825 value : int 1826 1827 Notes 1828 ----- 1829 Updates ``is1[11]`` if record is attached. 1830 """ 1831 self._id["i"] = value 1832 if self._rec is not None: 1833 self._rec.is1[11] = utils.unparse_id(self._id)[3] 1834 1835 @property 1836 def s(self): 1837 """ 1838 S identifier component. 1839 1840 Returns 1841 ------- 1842 int 1843 """ 1844 return self._id["s"] 1845 1846 @s.setter 1847 def s(self, value): 1848 """ 1849 Set S identifier component. 1850 1851 Parameters 1852 ---------- 1853 value : int 1854 1855 Notes 1856 ----- 1857 Updates ``is1[11]`` if record is attached. 1858 """ 1859 self._id["s"] = value 1860 if self._rec is not None: 1861 self._rec.is1[11] = utils.unparse_id(self._id)[3] 1862 1863 @property 1864 def g(self): 1865 """ 1866 G identifier component. 1867 1868 Returns 1869 ------- 1870 int 1871 """ 1872 return self._id["g"] 1873 1874 @g.setter 1875 def g(self, value): 1876 """ 1877 Set G identifier component. 1878 1879 Parameters 1880 ---------- 1881 value : int 1882 1883 Notes 1884 ----- 1885 Updates ``is1[11]`` if record is attached. 1886 """ 1887 self._id["g"] = value 1888 if self._rec is not None: 1889 self._rec.is1[11] = utils.unparse_id(self._id)[3]
TDLPACK variable ID class
1136 def __init__(self, id=[0, 0, 0, 0], linked_rec=None): 1137 """ 1138 Initialize TDLPACK variable ID 1139 1140 Parameters 1141 ---------- 1142 id : list of ints 1143 The 4-word TDLPACK variable ID 1144 linked_rec : TdlpackRecord, optional 1145 TDLPACK record object. This optional argument provides a mechanism 1146 for updating the TDLPACK variable ID in the TdlpackRecord is1 array. 1147 """ 1148 self._id = utils.parse_id(id) 1149 self._rec = linked_rec
Initialize TDLPACK variable ID
Parameters
- id (list of ints): The 4-word TDLPACK variable ID
- linked_rec (TdlpackRecord, optional): TDLPACK record object. This optional argument provides a mechanism for updating the TDLPACK variable ID in the TdlpackRecord is1 array.
1171 @classmethod 1172 def from_string(cls, idstr): 1173 """ 1174 Create a TdlpackID object from a TDLPACK ID string. 1175 1176 Parameters 1177 ---------- 1178 idstr : str 1179 String containing the TDLPACK ID with leading zeros and delimited 1180 by a non-numeric character. 1181 1182 Returns 1183 ------- 1184 An instance of TdlpackID. 1185 """ 1186 delim = idstr[9] 1187 if {idstr[19], idstr[29]} != {delim, delim}: 1188 raise ValueError(f"Invalid TDLPACK ID string format") 1189 return cls([int(i.lstrip("0")) if len(i.lstrip("0")) > 0 else 0 for i in idstr.split(delim)])
Create a TdlpackID object from a TDLPACK ID string.
Parameters
- idstr (str): String containing the TDLPACK ID with leading zeros and delimited by a non-numeric character.
Returns
- An instance of TdlpackID.
1191 def format(self, style: str = "basic") -> str: 1192 """ 1193 Format the TDLPACK ID identifier as a string. 1194 1195 This method returns a string representation of the identifier in one 1196 of several supported formats commonly used in MOS-2000 workflows. 1197 1198 Parameters 1199 ---------- 1200 style : {'basic', 'b', 'mos', 'm', 'parsed', 'p'}, optional 1201 Output format style (case-insensitive): 1202 1203 - ``"basic"`` or ``"b"`` 1204 Four-word identifier. Each word is printed as a zero-padded 1205 integer field. 1206 1207 - ``"mos"`` or ``"m"`` 1208 MOS-style identifier consisting of the first three words 1209 followed by the ISG components and the threshold value 1210 formatted in scientific notation (``.0000e±00``). 1211 1212 - ``"parsed"`` or ``"p"`` 1213 Identifier parsed into its individual components as defined 1214 by the internal ID mapping. All components are printed as 1215 zero-padded integers except the threshold value, which is 1216 printed as a floating-point value with ``F13.6`` formatting. 1217 1218 Returns 1219 ------- 1220 str 1221 String representation of the identifier in the requested format. 1222 1223 Notes 1224 ----- 1225 The MOS-style threshold representation removes the leading zero 1226 from scientific notation (e.g., ``0.0000e+00`` → ``.0000e+00``) 1227 to match legacy MOS-2000 formatting conventions. 1228 1229 Examples 1230 -------- 1231 Using the ``format`` method: 1232 1233 >>> rec.format("basic") 1234 '001000008 000000500 000000000 0000000000' 1235 1236 >>> rec.format("mos") 1237 '001000008 000000500 000000000 000 .0000e+00' 1238 1239 >>> rec.format("parsed") 1240 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1241 1242 Using Python's format protocol (``__format__``): 1243 1244 >>> f"{rec.id:basic}" 1245 '001000008 000000500 000000000 0000000000' 1246 1247 >>> f"{rec.id:mos}" 1248 '001000008 000000500 000000000 000 .0000e+00' 1249 1250 >>> f"{rec.id:parsed}" 1251 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1252 """ 1253 style = style.lower() 1254 1255 if style in {"basic", "b"}: 1256 return f"{str(self.word1).zfill(9)} {str(self.word2).zfill(9)} {str(self.word3).zfill(9)} {str(self.word4).zfill(10)}" 1257 elif style in {"mos", "m"}: 1258 thresh = f"{self.thresh:.4e}" 1259 if thresh.startswith("0"): 1260 thresh = thresh[1:] 1261 elif thresh.startswith("-0"): 1262 thresh = "-" + thresh[2:] 1263 return f"{str(self.word1).zfill(9)} {str(self.word2).zfill(9)} {str(self.word3).zfill(9)} {self.i}{self.s}{self.g} {thresh}" 1264 elif style in {"parsed", "p"}: 1265 parsed = "" 1266 for k, v in self._id.items(): 1267 if "thresh" not in k: 1268 parsed += f"{str(v).zfill(len(k))} " 1269 else: 1270 parsed += f"{v:13.6f}" 1271 return parsed 1272 1273 raise ValueError(f"Unknown TdlpackID format style: {style!r}")
Format the TDLPACK ID identifier as a string.
This method returns a string representation of the identifier in one of several supported formats commonly used in MOS-2000 workflows.
Parameters
style ({'basic', 'b', 'mos', 'm', 'parsed', 'p'}, optional): Output format style (case-insensitive):
"basic"or"b"Four-word identifier. Each word is printed as a zero-padded integer field."mos"or"m"MOS-style identifier consisting of the first three words followed by the ISG components and the threshold value formatted in scientific notation (.0000e±00)."parsed"or"p"Identifier parsed into its individual components as defined by the internal ID mapping. All components are printed as zero-padded integers except the threshold value, which is printed as a floating-point value withF13.6formatting.
Returns
- str: String representation of the identifier in the requested format.
Notes
The MOS-style threshold representation removes the leading zero
from scientific notation (e.g., 0.0000e+00 → .0000e+00)
to match legacy MOS-2000 formatting conventions.
Examples
Using the format method:
>>> rec.format("basic")
'001000008 000000500 000000000 0000000000'
>>> rec.format("mos")
'001000008 000000500 000000000 000 .0000e+00'
>>> rec.format("parsed")
'001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000'
Using Python's format protocol (__format__):
>>> f"{rec.id:basic}"
'001000008 000000500 000000000 0000000000'
>>> f"{rec.id:mos}"
'001000008 000000500 000000000 000 .0000e+00'
>>> f"{rec.id:parsed}"
'001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000'
1275 def to_dict(self): 1276 """ 1277 Return TDLPACK variable ID as dict. 1278 1279 Returns 1280 ------- 1281 String of the 4-word TDLPACK variable ID. 1282 """ 1283 return self._id
Return TDLPACK variable ID as dict.
Returns
- String of the 4-word TDLPACK variable ID.
1285 def to_string(self, delim=" "): 1286 """ 1287 Return TDLPACK variable ID as string. 1288 1289 Parameters 1290 ---------- 1291 delim : str, optional 1292 Delimiter character between each TDLPACK variable ID word. 1293 1294 Returns 1295 ------- 1296 String of the 4-word TDLPACK variable ID. 1297 """ 1298 strlen = (9, 9, 9, 10) 1299 return delim.join([str(i).zfill(word) for word, i in zip(strlen, utils.unparse_id(self._id))])
Return TDLPACK variable ID as string.
Parameters
- delim (str, optional): Delimiter character between each TDLPACK variable ID word.
Returns
- String of the 4-word TDLPACK variable ID.
1301 @property 1302 def word1(self): 1303 """ 1304 First ID word. 1305 1306 Returns 1307 ------- 1308 int 1309 First parsed ID word. 1310 """ 1311 return utils.unparse_id(self._id)[0]
First ID word.
Returns
- int: First parsed ID word.
1333 @property 1334 def word2(self): 1335 """ 1336 Second ID word. 1337 1338 Returns 1339 ------- 1340 int 1341 Second parsed ID word. 1342 """ 1343 return utils.unparse_id(self._id)[1]
Second ID word.
Returns
- int: Second parsed ID word.
1365 @property 1366 def word3(self): 1367 """ 1368 Third ID word. 1369 1370 Returns 1371 ------- 1372 int 1373 Third parsed ID word. 1374 """ 1375 return utils.unparse_id(self._id)[2]
Third ID word.
Returns
- int: Third parsed ID word.
1397 @property 1398 def word4(self): 1399 """ 1400 Fourth ID word. 1401 1402 Returns 1403 ------- 1404 int 1405 Fourth parsed ID word. 1406 """ 1407 return utils.unparse_id(self._id)[3]
Fourth ID word.
Returns
- int: Fourth parsed ID word.
1429 @property 1430 def ccc(self): 1431 """ 1432 CCC identifier component. 1433 1434 Returns 1435 ------- 1436 int 1437 """ 1438 return self._id["ccc"]
CCC identifier component.
Returns
- int
1457 @property 1458 def fff(self): 1459 """ 1460 FFF identifier component. 1461 1462 Returns 1463 ------- 1464 int 1465 """ 1466 return self._id["fff"]
FFF identifier component.
Returns
- int
1485 @property 1486 def cccfff(self): 1487 """ 1488 Combined CCCFFF identifier. 1489 1490 Returns 1491 ------- 1492 int 1493 Integer representation of ``word1 / 1000``. 1494 """ 1495 return int(self.word1 / 1000)
Combined CCCFFF identifier.
Returns
- int: Integer representation of
word1 / 1000.
1497 @property 1498 def b(self): 1499 """ 1500 B identifier component. 1501 1502 Returns 1503 ------- 1504 int 1505 """ 1506 return self._id["b"]
B identifier component.
Returns
- int
1525 @property 1526 def dd(self): 1527 """ 1528 DD identifier component. 1529 1530 Returns 1531 ------- 1532 int 1533 """ 1534 return self._id["dd"]
DD identifier component.
Returns
- int
1554 @property 1555 def v(self): 1556 """ 1557 V identifier component. 1558 1559 Returns 1560 ------- 1561 int 1562 """ 1563 return self._id["v"]
V identifier component.
Returns
- int
1582 @property 1583 def llll(self): 1584 """ 1585 LLLL identifier component. 1586 1587 Returns 1588 ------- 1589 int 1590 """ 1591 return self._id["llll"]
LLLL identifier component.
Returns
- int
1610 @property 1611 def uuuu(self): 1612 """ 1613 UUUU identifier component. 1614 1615 Returns 1616 ------- 1617 int 1618 """ 1619 return self._id["uuuu"]
UUUU identifier component.
Returns
- int
1638 @property 1639 def t(self): 1640 """ 1641 T identifier component. 1642 1643 Returns 1644 ------- 1645 int 1646 """ 1647 return self._id["t"]
T identifier component.
Returns
- int
1666 @property 1667 def rr(self): 1668 """ 1669 RR identifier component. 1670 1671 Returns 1672 ------- 1673 int 1674 """ 1675 return self._id["rr"]
RR identifier component.
Returns
- int
1694 @property 1695 def o(self): 1696 """ 1697 O identifier component. 1698 1699 Returns 1700 ------- 1701 int 1702 """ 1703 return self._id["o"]
O identifier component.
Returns
- int
1722 @property 1723 def hh(self): 1724 """ 1725 HH identifier component. 1726 1727 Returns 1728 ------- 1729 int 1730 """ 1731 return self._id["hh"]
HH identifier component.
Returns
- int
1750 @property 1751 def tau(self): 1752 """ 1753 Forecast hour (tau). 1754 1755 Returns 1756 ------- 1757 int 1758 """ 1759 return self._id["tau"]
Forecast hour (tau).
Returns
- int
1779 @property 1780 def thresh(self): 1781 """ 1782 Threshold identifier component. 1783 1784 Returns 1785 ------- 1786 int 1787 """ 1788 return self._id["thresh"]
Threshold identifier component.
Returns
- int
1807 @property 1808 def i(self): 1809 """ 1810 I identifier component. 1811 1812 Returns 1813 ------- 1814 int 1815 """ 1816 return self._id["i"]
I identifier component.
Returns
- int
20@dataclass(frozen=True, slots=True) 21class TdlpackGridDef: 22 """Definition of a TDLPACK grid. 23 24 Stored longitude convention is MOS-2000: 25 west longitude is positive. 26 """ 27 28 mapProjection: int 29 nx: int 30 ny: int 31 latitudeLowerLeft: float 32 longitudeLowerLeft: float 33 standardLatitude: float 34 orientationLongitude: float 35 gridLength: float 36 37 # Array mapping 38 IS2_INDEX: ClassVar[dict[str, int]] = { 39 "mapProjection": 1, 40 "nx": 2, 41 "ny": 3, 42 "latitudeLowerLeft": 4, 43 "longitudeLowerLeft": 5, 44 "orientationLongitude": 6, 45 "gridLength": 7, 46 "standardLatitude": 8, 47 } 48 49 # Attribute scale factors used to convert to whole integers. 50 SCALE_FACTOR: ClassVar[dict[str, int]] = { 51 "mapProjection": 1, 52 "nx": 1, 53 "ny": 1, 54 "latitudeLowerLeft": 10_000, 55 "longitudeLowerLeft": 10_000, 56 "orientationLongitude": 10_000, 57 "gridLength": 1000, 58 "standardLatitude": 10_000, 59 } 60 61 def scaled_value(self, name: str) -> int: 62 """Return one grid attribute scaled to an integer.""" 63 value = getattr(self, name) 64 scale = self.SCALE_FACTOR[name] 65 return int(round(value * scale)) 66 67 def to_is2(self) -> list[int]: 68 """Return the grid definition as a scaled integer array.""" 69 is2 = np.zeros((_tdlpackio.ND7), dtype=np.int32) 70 71 for name, index in self.IS2_INDEX.items(): 72 is2[index] = self.scaled_value(name) 73 74 return is2 75 76 def to_int_dict(self) -> dict[str, int]: 77 """Return scaled integer values keyed by attribute name.""" 78 return {name: self.scaled_value(name) for name in self.IS2_INDEX}
Definition of a TDLPACK grid.
Stored longitude convention is MOS-2000: west longitude is positive.
61 def scaled_value(self, name: str) -> int: 62 """Return one grid attribute scaled to an integer.""" 63 value = getattr(self, name) 64 scale = self.SCALE_FACTOR[name] 65 return int(round(value * scale))
Return one grid attribute scaled to an integer.
67 def to_is2(self) -> list[int]: 68 """Return the grid definition as a scaled integer array.""" 69 is2 = np.zeros((_tdlpackio.ND7), dtype=np.int32) 70 71 for name, index in self.IS2_INDEX.items(): 72 is2[index] = self.scaled_value(name) 73 74 return is2
Return the grid definition as a scaled integer array.
218def get_grid( 219 name: str, 220 *, 221 lon_format: Literal["mos2k", "standard", "0_360"] = "mos2k", 222) -> TdlpackGridDef: 223 """ 224 Return a grid definition by name. 225 226 Notes 227 ----- 228 Internally, all grid definitions are stored using the MOS-2000 229 longitude convention (west longitude is positive). 230 231 Parameters 232 ---------- 233 name : str 234 Grid name. 235 lon_format : {"mos2k", "standard", "0_360"}, optional 236 Longitude convention to return: 237 238 - "mos2k": West longitude positive (native storage format) 239 - "standard": East-positive, range [-180, 180] 240 - "0_360": East-positive, range [0, 360) 241 242 Returns 243 ------- 244 TdlpackGridDef 245 Grid definition using the requested longitude convention. 246 """ 247 try: 248 grid = GRIDS[name.lower()] 249 except KeyError as exc: 250 raise KeyError(f"Unknown grid: {name!r}") from exc 251 252 # Native format (no copy needed) 253 if lon_format == "mos2k": 254 return grid 255 256 if lon_format == "standard": 257 return replace( 258 grid, 259 longitudeLowerLeft=_mos2k_to_standard(grid.longitudeLowerLeft), 260 orientationLongitude=_mos2k_to_standard(grid.orientationLongitude), 261 ) 262 263 if lon_format == "0_360": 264 return replace( 265 grid, 266 longitudeLowerLeft=_mos2k_to_0_360(grid.longitudeLowerLeft), 267 orientationLongitude=_mos2k_to_0_360(grid.orientationLongitude), 268 ) 269 270 raise ValueError(f"Invalid lon_format: {lon_format!r}. Expected one of {{'mos2k', 'standard', '0_360'}}.")
Return a grid definition by name.
Notes
Internally, all grid definitions are stored using the MOS-2000 longitude convention (west longitude is positive).
Parameters
- name (str): Grid name.
lon_format ({"mos2k", "standard", "0_360"}, optional): Longitude convention to return:
- "mos2k": West longitude positive (native storage format)
- "standard": East-positive, range [-180, 180]
- "0_360": East-positive, range [0, 360)
Returns
- TdlpackGridDef: Grid definition using the requested longitude convention.
273def has_grid(name: str) -> bool: 274 """Return True if a grid name is known.""" 275 return name.lower() in GRIDS
Return True if a grid name is known.