tdlpackio
Introduction
tdlpackio is a Python package for reading and writing TDLPACK-formatted data contained in Fortran-based unformatted ("sequential") and direct-access ("random-access") files. tdlpackio provides a Cython extension module, tdlpacklib, for interfacing to the libtdlpack Fortran library, a subset of subroutines from the MOS-2000 (MOS2K) software system.
The TDLPACK data format is a GRIB-like binary data format that is exclusive to MOS2K Fortran-based sofftware system. This software system was developed at the NOAA/NWS/Meteorological Development Laboratory (MDL) and its primary purpose is to perform statistical post-processing of meteorological data. TDLPACK format is based on the World Meteorological Organizations (WMO) GRIdded Binary (GRIB), version 1, but with modifications to support the needs of MDL's statistical post-processing needs -- mainly the ability to store 1D (vector), datasets such as station observations, along with 2D gridded data.
The TDLPACK data format are contained in two types of Fortran-based files; sequential or random-access. Sequential files are variable length, record-based, and unformatted. Random-access files are fixed-length and direct-access. tdlpackio accommodates reading and writing of both types of TDLPACK files.
TDLPACK files can contain two other types of records: a station call letter record and trailer record.
A station call letter record can exist in both types of TDLPACK files and contains a stream of
alphanumeric characters (CHARACTER(LEN=8)). A trailer record exists to signal that either another station
call letter record is about to be read or we have reached the end of the file (EOF).
A trailer record is not written to a random-access file.
For more information on the MOS-2000 software system and TDLPACK format, user is referred to the official MOS-2000 documentation.
Verison 2.0 Refactor
tdlpackio v2.0 is a major factor of pytdlpack v1.x. The API, code design, and structure borrow heavily from grib2io v2. As stated, TDLPACK data can live in a sequential or random-access file, can contain vector or gridded data. tdlpackio v2 aims to provide data to the user in a consistent manner. When opening a TDLPACK file, records are lazily-indexed so that data are only read and unpacked only when necessary. tdlpackio performs the TDLPACK file reading natively in Python. The open class contains methods for reading and indexing. However, packing and unpacking TDLPACK data as well as writing to files are performed by the subroutines in libtdlpack.
Tutorials
The following Jupyter Notebooks are available as tutorials:
1from __future__ import annotations 2 3# init for tdlpackio package 4from ._tdlpackio import * 5from ._tdlpackio import __doc__ 6 7from .grids import GRIDS, TdlpackGridDef, get_grid, has_grid 8from .version import version as __version__ 9 10__all__ = [ 11 "__version__", 12 "open", 13 "TdlpackRecord", 14 "TdlpackStationRecord", 15 "TdlpackTrailerRecord", 16 "TdlpackID", 17 "TdlpackGridDef", 18 "GRIDS", 19 "get_grid", 20 "has_grid", 21] 22 23try: 24 from . import __config__ 25 26 __version__ = __config__.tdlpackio_version 27 has_openmp_support = __config__.has_openmp_support 28 tdlpack_static = __config__.tdlpack_static 29 extra_objects = __config__.extra_objects 30except ImportError: 31 has_openmp_support = None 32 tdlpack_static = None 33 extra_objects = [] 34 35__tdlpacklib_version__ = "1.0.0" # For now... 36 37 38def show_config(): 39 """Print tdlpackio build configuration information.""" 40 print(f"tdlpackio version {__version__} Configuration:") 41 print("") 42 print(f"libtdlpack library version: {__tdlpacklib_version__}") 43 print(f"\tStatic library: {tdlpack_static}") 44 print(f"\tOpenMP support: {has_openmp_support}") 45 print("") 46 print("Static libs:") 47 for lib in extra_objects: 48 print(f"\t{lib}")
89class open: 90 """ 91 Open class for tdlpackio. 92 93 Parameters 94 ---------- 95 path : str 96 File name. 97 98 mode : {'r', 'w'}, default 'r' 99 File handle mode. 100 101 format : {'sequential', 'random-access'}, optional 102 File type when creating a new file. 103 104 ra_template : {'small', 'large'}, optional 105 Template used when creating random-access files. 106 """ 107 108 _filetype_map = { 109 "random-access": 1, 110 "sequential": 2, 111 } 112 113 def __init__(self, path, mode="r", format=None, ra_template=None): 114 if mode not in {"r", "w", "a"}: 115 raise ValueError(f"Invalid mode: {mode}") 116 if mode in {"r", "w"}: 117 mode = mode + "b" 118 elif mode == "a": 119 mode = "wb" 120 self.path = path 121 self.mode = mode 122 self.format = format 123 self.ra_template = ra_template 124 self._hasindex = False 125 self._index = {} 126 self._lun = -1 127 self.mode = mode 128 self.name = os.path.abspath(path) 129 self.records = 0 130 131 # Perform indexing on read 132 if "r" in self.mode: 133 self._filehandle = builtins.open(path, mode=mode) 134 self.filetype = self._get_tdlpack_file_type() 135 self._build_index() 136 137 elif "w" in self.mode: 138 self.bytes_written = 0 139 self.records_written = 0 140 self.filetype = format if format is not None else "sequential" 141 if self.filetype == "random-access": 142 ra_template = "small" if ra_template is None else ra_template 143 iret, self._lun = tdlpacklib.open_tdlpack_file(self.name, self.mode, self._ifiletype, ra_template=ra_template) 144 elif self.filetype == "sequential": 145 self._filehandle = builtins.open(path, mode=mode) 146 iret, self._lun = tdlpacklib.open_tdlpack_file(self.name, self.mode, self._ifiletype, ra_template=ra_template) 147 148 # Add self to file data store 149 _open_file_store[self.name] = self 150 151 @property 152 def _ifiletype(self): 153 """Return numeric filetype""" 154 return self._filetype_map[self.filetype] 155 156 @property 157 def size(self): 158 """Return the file size.""" 159 return os.path.getsize(self.name) 160 161 def __enter__(self): 162 """""" 163 return self 164 165 def __exit__(self, atype, value, traceback): 166 """""" 167 self.close() 168 169 def __iter__(self): 170 """""" 171 yield from self._index["record"] 172 173 def __repr__(self): 174 """""" 175 strings = [] 176 keys = list(self.__dict__.keys()) 177 for k in keys: 178 if not k.startswith("_"): 179 strings.append(f"{k} = {self.__dict__[k]}\n") 180 # Attach size property. 181 strings.append(f"size = {self.size}\n") 182 return "".join(strings) 183 184 def __getitem__(self, key): 185 """""" 186 if isinstance(key, slice): 187 return self._index["record"][key] 188 elif isinstance(key, int): 189 return self._index["record"][key] 190 else: 191 raise KeyError("Key must be an integer record number or a slice") 192 193 def _get_tdlpack_file_type(self): 194 """Determine the type of TDLPACK file""" 195 self._filehandle.seek(0) 196 a = struct.unpack(">i", self._filehandle.read(4))[0] 197 b = struct.unpack(">i", self._filehandle.read(4))[0] 198 self._filehandle.seek(0) 199 return "random-access" if [a, b] == [0, 4] else "sequential" 200 201 def _build_index(self): 202 """Record Indexer""" 203 # Initialize index dictionary 204 self._index["offset"] = [] 205 self._index["size"] = [] 206 self._index["type"] = [] 207 self._index["record"] = [] 208 209 if self.filetype == "random-access": 210 self._randomaccess_file_indexer() 211 elif self.filetype == "sequential": 212 self._sequential_file_indexer() 213 self._hasindex = True 214 215 def _randomaccess_file_indexer(self): 216 """Indexer for random-access TDLPACK files""" 217 # Read master key 218 version, nids, nwords, nkyrec, maxent, lastky = struct.unpack(">iiiiii", self._filehandle.read(24)) 219 nbytes = nwords * NBYPWD 220 last_key_check = [99999999] if lastky > 9999 else [9999, 99999999] 221 self.master_key = dict( 222 version=version, 223 nids=nids, 224 nwords=nwords, 225 nkyrec=nkyrec, 226 maxent=maxent, 227 lastky=lastky, 228 ) 229 self.key_records = [] 230 # Set file position to first key record 231 self._filehandle.seek(nbytes) 232 233 last_station_id_rec = -1 234 last_station_lat_rec = -1 235 last_station_lon_rec = -1 236 237 # Iterate over all key records 238 while True: 239 # Read key record "header" data 240 nkeys, prec_this_key, prec_next_key = struct.unpack(">iii", self._filehandle.read(12)) 241 self.key_records.append( 242 dict( 243 nkeys=nkeys, 244 prec_this_key=prec_this_key, 245 prec_next_key=prec_next_key, 246 ) 247 ) 248 249 ids = list() 250 nsize = list() 251 prec_begin = list() 252 253 # Read key record information 254 for i in range(nkeys): 255 id1, id2, id3, id4, nd, bprec = struct.unpack(">iiiiii", self._filehandle.read(24)) 256 ids.append([id1, id2, id3, id4]) 257 nsize.append(nd) 258 prec_begin.append(bprec) 259 260 # Using key record info, move around file to inventory TDLPACK data 261 for m, n, b in zip(ids, nsize, prec_begin): 262 # Disect prec_begin 263 prec1 = int(b / 1000.0) 264 265 # Offset to the data record 266 offset = (prec1 - 1) * nbytes 267 self._filehandle.seek(offset) 268 self._index["offset"].append(offset) 269 self._index["size"].append(n * NBYPWD) 270 271 # Determine record type. Since the 4-word ID is stored in the 272 # key record, we can use it to determine station call letter 273 # record or TDLPACK data record. 274 if m[0] == 400001000: 275 # Station ID record...not in TDLPACK format 276 rec = TdlpackStationRecord() 277 last_station_id_rec = self.records 278 rec._recnum = self.records 279 rec._source = self.name 280 rec._nsta_expected = int(n / 2) 281 self._index["record"].append(rec) 282 self._index["type"].append("station") 283 else: 284 if m[0] == 400006000: 285 last_station_lat_rec = self.records 286 elif m[0] == 400007000: 287 last_station_lon_rec = self.records 288 # TDLPACK data record 289 ipack = np.frombuffer(self._filehandle.read(132), dtype=">i4").astype(np.int32) 290 iret, is0, is1, is2, is4 = tdlpacklib.unpack_meta(ipack) 291 if np.all(is2 == 0): 292 is2 = None 293 rec = TdlpackRecord(is0, is1, is2, is4) 294 rec._recnum = self.records 295 rec._linked_station_id_record = last_station_id_rec 296 rec._linked_station_lat_record = last_station_lat_rec 297 rec._linked_station_lon_record = last_station_lon_rec 298 rec._source = self.name 299 shape = (rec.ny, rec.nx) if rec.type == "grid" else (rec.numberOfPackedValues,) 300 ndim = len(shape) 301 dtype = "float32" 302 rec._data = TdlpackRecordOnDiskArray( 303 shape, 304 ndim, 305 dtype, 306 self.filetype, 307 self._filehandle, 308 rec, 309 self._index["offset"][-1], 310 self._index["size"][-1], 311 ) 312 self._index["record"].append(rec) 313 self._index["type"].append("data") 314 self.records += 1 315 316 # Hold the record number of the last station ID record 317 if self._index["type"][-1] == "station": 318 _last_station_id_record = self.records # This should be OK. 319 320 # Break loop here, at last key record 321 if prec_next_key in last_key_check: 322 break 323 offset = (prec_next_key - 1) * nbytes 324 self._filehandle.seek(offset) 325 326 # Make key_records immutable 327 self.key_records = tuple(self.key_records) 328 329 def _sequential_file_indexer(self): 330 """Indexer for sequential TDLPACK files""" 331 last_station_id_rec = -1 332 last_station_lat_rec = -1 333 last_station_lon_rec = -1 334 335 # Iterate 336 while True: 337 try: 338 # First read 4-byte Fortran record header 339 pos = self._filehandle.tell() 340 fortran_header = struct.unpack(">i", self._filehandle.read(4))[0] 341 if fortran_header >= 132: 342 bytes_to_read = 132 343 else: 344 bytes_to_read = fortran_header 345 346 pos = self._filehandle.tell() 347 ioctet = np.frombuffer(self._filehandle.read(8), dtype=">i8").astype(np.int64)[0] 348 ipack = np.frombuffer(self._filehandle.read(bytes_to_read - 8), dtype=">i4").astype(np.int32) 349 _header = struct.unpack(">4s", ipack[0])[0].decode() 350 351 # Check to first 4 bytes of the data record to determine the data 352 # record type. 353 if _header == "PLDT": 354 if ipack[5] == 400006000: 355 last_station_lat_rec = self.records 356 elif ipack[5] == 400007000: 357 last_station_lon_rec = self.records 358 # TDLPACK data record 359 iret, is0, is1, is2, is4 = tdlpacklib.unpack_meta(ipack) 360 self._index["offset"].append(pos) 361 self._index["size"].append(fortran_header) # Size given by Fortran header 362 if np.all(is2 == 0): 363 is2 = None 364 rec = TdlpackRecord(is0, is1, is2, is4) 365 rec._recnum = self.records 366 rec._linked_station_id_record = last_station_id_rec 367 rec._linked_station_lat_record = last_station_lat_rec 368 rec._linked_station_lon_record = last_station_lon_rec 369 rec._source = self.name 370 shape = (rec.ny, rec.nx) if rec.type == "grid" else (rec.numberOfPackedValues,) 371 ndim = len(shape) 372 dtype = "float32" 373 rec._data = TdlpackRecordOnDiskArray( 374 shape, 375 ndim, 376 dtype, 377 self.filetype, 378 self._filehandle, 379 rec, 380 self._index["offset"][-1], 381 self._index["size"][-1], 382 ) 383 self._index["record"].append(rec) 384 self._index["type"].append("data") 385 else: 386 if ioctet == 24 and ipack[4] == 9999: 387 # Trailer record 388 rec = TdlpackTrailerRecord() 389 rec._recnum = self.records 390 rec._source = self.name 391 self._index["offset"].append(pos) 392 self._index["size"].append(fortran_header) 393 self._index["type"].append("trailer") 394 self._index["record"].append(rec) 395 else: 396 # Station ID record 397 rec = TdlpackStationRecord() 398 last_station_id_rec = self.records 399 rec._recnum = self.records 400 rec._source = self.name 401 rec._nsta_expected = int(ioctet / 8) 402 self._index["offset"].append(pos) 403 self._index["size"].append(fortran_header) 404 self._index["type"].append("station") 405 self._index["record"].append(rec) 406 407 # At this point we have successfully identified a TDLPACK record from 408 # the file. Increment self.records and position the file pointer to 409 # now read the Fortran trailer. 410 self.records += 1 # Includes trailer records 411 self._filehandle.seek(fortran_header - bytes_to_read, 1) 412 fortran_trailer = struct.unpack(">i", self._filehandle.read(4))[0] 413 414 # Check Fortran header and trailer for the record. 415 if fortran_header != fortran_trailer: 416 raise IOError("Bad Fortran record.") 417 418 # Hold the record number of the last station ID record 419 if self._index["type"][-1] == "station": 420 _last_station_id_record = self.records # This should be OK. 421 422 except struct.error: 423 self._filehandle.seek(0) 424 break 425 426 def read(self, n): 427 """ 428 Read record from file. 429 430 Parameters 431 ---------- 432 n : int 433 Record number. 434 435 Returns 436 ------- 437 numpy.ndarray 438 Record data as a NumPy array. The returned dtype depends on the 439 record type: 440 441 - ``data`` or ``trailer`` : ``int32`` array. 442 - ``station`` : fixed-width byte string array (``S8``). 443 444 Notes 445 ----- 446 The file pointer is positioned using the internal index before reading. 447 Record size is determined from the file header (sequential) or index 448 (random-access). 449 """ 450 if "w" in self.mode: 451 pass # Remove this at some point.... 452 # Position file pointer to the beginning of the TDLPACK record. 453 self._filehandle.seek(self._index["offset"][n]) 454 if self.filetype == "sequential": 455 size = np.frombuffer(self._filehandle.read(8), dtype=">i8").astype(np.int64)[0] 456 elif self.filetype == "random-access": 457 size = self._index["size"][n] 458 459 if self._index["type"][n] in {"data", "trailer"}: 460 return np.frombuffer(self._filehandle.read(size), dtype=">i4").astype(np.int32) 461 elif self._index["type"][n] == "station": 462 return np.frombuffer(self._filehandle.read(size), dtype="S8") 463 464 def write(self, record): 465 """ 466 Write record(s) to file. 467 468 Parameters 469 ---------- 470 record : TdlpackStationRecord, _TdlpackRecord, TdlpackTrailerRecord, or list 471 Record or list of records to write. 472 473 - ``TdlpackStationRecord`` : Station record. Station identifiers 474 are padded to ``NCHAR``. 475 - ``_TdlpackRecord`` : Packed data record. 476 - ``TdlpackTrailerRecord`` : Trailer record. 477 - ``list`` : List of supported record types. 478 479 Returns 480 ------- 481 None 482 483 Notes 484 ----- 485 Updates ``bytes_written``, ``records_written``, ``records``, and 486 ``_type_lastrecord_written``. 487 """ 488 if isinstance(record, list): 489 for rec in record: 490 self.write(rec) 491 return 492 493 nreplace, ncheck = 0, 0 494 495 if isinstance(record, TdlpackStationRecord): 496 # Adjust string length of each station to NCHAR. 497 stns = [s.ljust(NCHAR) for s in record.stations] 498 iret, self.bytes_written, self.records_written = tdlpacklib.write_station_record( 499 self.name, 500 self._lun, 501 self._ifiletype, 502 stns, 503 self.bytes_written, 504 self.records_written, 505 nreplace, 506 ncheck, 507 ) 508 509 elif issubclass(record.__class__, _TdlpackRecord): 510 iret, self.bytes_written, self.records_written = tdlpacklib.write_tdlpack_record( 511 self.name, 512 self._lun, 513 self._ifiletype, 514 record._ipack, 515 self.bytes_written, 516 self.records_written, 517 nreplace, 518 ncheck, 519 ) 520 521 elif isinstance(record, TdlpackTrailerRecord): 522 iret, self.bytes_written, self.records_written = tdlpacklib.write_trailer_record( 523 self._lun, self._ifiletype, self.bytes_written, self.records_written 524 ) 525 526 self._type_lastrecord_written = record.type 527 self.records += 1 528 529 def close(self): 530 """ 531 Close the file. 532 533 Returns 534 ------- 535 None 536 537 Notes 538 ----- 539 For write mode, a trailer record may be written for sequential files 540 if the last record type requires it. The underlying TDLpack file handle 541 is then closed. The file is removed from the internal open file store. 542 """ 543 if "r" in self.mode: 544 self._filehandle.close() 545 if "w" in self.mode: 546 if self.filetype == "sequential": 547 if self._type_lastrecord_written == "vector": 548 iret = tdlpacklib.write_trailer_record( 549 self._lun, 550 self._ifiletype, 551 self.bytes_written, 552 self.records_written, 553 ) 554 iret = tdlpacklib.close_tdlpack_file(self._lun, self._ifiletype) 555 if iret != 0: 556 raise ValueError("return from tdlpacklib.close_tdlpack_file is non-zero") 557 if self.name in _open_file_store.keys(): 558 del _open_file_store[self.name] 559 560 def select(self, **kwargs): 561 """ 562 Select records by attribute. 563 564 Parameters 565 ---------- 566 **kwargs : dict 567 Keyword arguments specifying ``TdlpackRecord`` attributes and 568 values to match. 569 570 Returns 571 ------- 572 list 573 List of records matching all provided attribute/value pairs. 574 575 Notes 576 ----- 577 Selection is performed against records in the internal index. Records 578 must match all specified attributes to be included in the result. 579 """ 580 _id_keys = [ 581 "word1", 582 "word2", 583 "word3", 584 "word4", 585 "ccc", 586 "fff", 587 "b", 588 "dd", 589 "v", 590 "llll", 591 "uuuu", 592 "t", 593 "rr", 594 "o", 595 "hh", 596 "tau", 597 "thresh", 598 "i", 599 "s", 600 "g", 601 ] 602 603 # TODO: Add ability to process multiple values for each keyword (attribute) 604 idxs = [] 605 nkeys = len(kwargs.keys()) 606 for k, v in kwargs.items(): 607 if k in _id_keys: 608 for rec in self._index["record"]: 609 # if hasattr(rec, k) and getattr(rec, k) == v: 610 if getattr(rec.id, k) == v: 611 idxs.append(rec._recnum) 612 else: 613 for rec in self._index["record"]: 614 if hasattr(rec, k) and getattr(rec, k) == v: 615 idxs.append(rec._recnum) 616 idxs = np.array(idxs, dtype=">i4") 617 return [self._index["record"][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]]
Open class for tdlpackio.
Parameters
- path (str): File name.
- mode ({'r', 'w'}, default 'r'): File handle mode.
- format ({'sequential', 'random-access'}, optional): File type when creating a new file.
- ra_template ({'small', 'large'}, optional): Template used when creating random-access files.
113 def __init__(self, path, mode="r", format=None, ra_template=None): 114 if mode not in {"r", "w", "a"}: 115 raise ValueError(f"Invalid mode: {mode}") 116 if mode in {"r", "w"}: 117 mode = mode + "b" 118 elif mode == "a": 119 mode = "wb" 120 self.path = path 121 self.mode = mode 122 self.format = format 123 self.ra_template = ra_template 124 self._hasindex = False 125 self._index = {} 126 self._lun = -1 127 self.mode = mode 128 self.name = os.path.abspath(path) 129 self.records = 0 130 131 # Perform indexing on read 132 if "r" in self.mode: 133 self._filehandle = builtins.open(path, mode=mode) 134 self.filetype = self._get_tdlpack_file_type() 135 self._build_index() 136 137 elif "w" in self.mode: 138 self.bytes_written = 0 139 self.records_written = 0 140 self.filetype = format if format is not None else "sequential" 141 if self.filetype == "random-access": 142 ra_template = "small" if ra_template is None else ra_template 143 iret, self._lun = tdlpacklib.open_tdlpack_file(self.name, self.mode, self._ifiletype, ra_template=ra_template) 144 elif self.filetype == "sequential": 145 self._filehandle = builtins.open(path, mode=mode) 146 iret, self._lun = tdlpacklib.open_tdlpack_file(self.name, self.mode, self._ifiletype, ra_template=ra_template) 147 148 # Add self to file data store 149 _open_file_store[self.name] = self
156 @property 157 def size(self): 158 """Return the file size.""" 159 return os.path.getsize(self.name)
Return the file size.
426 def read(self, n): 427 """ 428 Read record from file. 429 430 Parameters 431 ---------- 432 n : int 433 Record number. 434 435 Returns 436 ------- 437 numpy.ndarray 438 Record data as a NumPy array. The returned dtype depends on the 439 record type: 440 441 - ``data`` or ``trailer`` : ``int32`` array. 442 - ``station`` : fixed-width byte string array (``S8``). 443 444 Notes 445 ----- 446 The file pointer is positioned using the internal index before reading. 447 Record size is determined from the file header (sequential) or index 448 (random-access). 449 """ 450 if "w" in self.mode: 451 pass # Remove this at some point.... 452 # Position file pointer to the beginning of the TDLPACK record. 453 self._filehandle.seek(self._index["offset"][n]) 454 if self.filetype == "sequential": 455 size = np.frombuffer(self._filehandle.read(8), dtype=">i8").astype(np.int64)[0] 456 elif self.filetype == "random-access": 457 size = self._index["size"][n] 458 459 if self._index["type"][n] in {"data", "trailer"}: 460 return np.frombuffer(self._filehandle.read(size), dtype=">i4").astype(np.int32) 461 elif self._index["type"][n] == "station": 462 return np.frombuffer(self._filehandle.read(size), dtype="S8")
Read record from file.
Parameters
- n (int): Record number.
Returns
numpy.ndarray: Record data as a NumPy array. The returned dtype depends on the record type:
dataortrailer:int32array.station: fixed-width byte string array (S8).
Notes
The file pointer is positioned using the internal index before reading. Record size is determined from the file header (sequential) or index (random-access).
464 def write(self, record): 465 """ 466 Write record(s) to file. 467 468 Parameters 469 ---------- 470 record : TdlpackStationRecord, _TdlpackRecord, TdlpackTrailerRecord, or list 471 Record or list of records to write. 472 473 - ``TdlpackStationRecord`` : Station record. Station identifiers 474 are padded to ``NCHAR``. 475 - ``_TdlpackRecord`` : Packed data record. 476 - ``TdlpackTrailerRecord`` : Trailer record. 477 - ``list`` : List of supported record types. 478 479 Returns 480 ------- 481 None 482 483 Notes 484 ----- 485 Updates ``bytes_written``, ``records_written``, ``records``, and 486 ``_type_lastrecord_written``. 487 """ 488 if isinstance(record, list): 489 for rec in record: 490 self.write(rec) 491 return 492 493 nreplace, ncheck = 0, 0 494 495 if isinstance(record, TdlpackStationRecord): 496 # Adjust string length of each station to NCHAR. 497 stns = [s.ljust(NCHAR) for s in record.stations] 498 iret, self.bytes_written, self.records_written = tdlpacklib.write_station_record( 499 self.name, 500 self._lun, 501 self._ifiletype, 502 stns, 503 self.bytes_written, 504 self.records_written, 505 nreplace, 506 ncheck, 507 ) 508 509 elif issubclass(record.__class__, _TdlpackRecord): 510 iret, self.bytes_written, self.records_written = tdlpacklib.write_tdlpack_record( 511 self.name, 512 self._lun, 513 self._ifiletype, 514 record._ipack, 515 self.bytes_written, 516 self.records_written, 517 nreplace, 518 ncheck, 519 ) 520 521 elif isinstance(record, TdlpackTrailerRecord): 522 iret, self.bytes_written, self.records_written = tdlpacklib.write_trailer_record( 523 self._lun, self._ifiletype, self.bytes_written, self.records_written 524 ) 525 526 self._type_lastrecord_written = record.type 527 self.records += 1
Write record(s) to file.
Parameters
record (TdlpackStationRecord, _TdlpackRecord, TdlpackTrailerRecord, or list): Record or list of records to write.
TdlpackStationRecord: Station record. Station identifiers are padded toNCHAR._TdlpackRecord: Packed data record.TdlpackTrailerRecord: Trailer record.list: List of supported record types.
Returns
- None
Notes
Updates bytes_written, records_written, records, and
_type_lastrecord_written.
529 def close(self): 530 """ 531 Close the file. 532 533 Returns 534 ------- 535 None 536 537 Notes 538 ----- 539 For write mode, a trailer record may be written for sequential files 540 if the last record type requires it. The underlying TDLpack file handle 541 is then closed. The file is removed from the internal open file store. 542 """ 543 if "r" in self.mode: 544 self._filehandle.close() 545 if "w" in self.mode: 546 if self.filetype == "sequential": 547 if self._type_lastrecord_written == "vector": 548 iret = tdlpacklib.write_trailer_record( 549 self._lun, 550 self._ifiletype, 551 self.bytes_written, 552 self.records_written, 553 ) 554 iret = tdlpacklib.close_tdlpack_file(self._lun, self._ifiletype) 555 if iret != 0: 556 raise ValueError("return from tdlpacklib.close_tdlpack_file is non-zero") 557 if self.name in _open_file_store.keys(): 558 del _open_file_store[self.name]
Close the file.
Returns
- None
Notes
For write mode, a trailer record may be written for sequential files if the last record type requires it. The underlying TDLpack file handle is then closed. The file is removed from the internal open file store.
560 def select(self, **kwargs): 561 """ 562 Select records by attribute. 563 564 Parameters 565 ---------- 566 **kwargs : dict 567 Keyword arguments specifying ``TdlpackRecord`` attributes and 568 values to match. 569 570 Returns 571 ------- 572 list 573 List of records matching all provided attribute/value pairs. 574 575 Notes 576 ----- 577 Selection is performed against records in the internal index. Records 578 must match all specified attributes to be included in the result. 579 """ 580 _id_keys = [ 581 "word1", 582 "word2", 583 "word3", 584 "word4", 585 "ccc", 586 "fff", 587 "b", 588 "dd", 589 "v", 590 "llll", 591 "uuuu", 592 "t", 593 "rr", 594 "o", 595 "hh", 596 "tau", 597 "thresh", 598 "i", 599 "s", 600 "g", 601 ] 602 603 # TODO: Add ability to process multiple values for each keyword (attribute) 604 idxs = [] 605 nkeys = len(kwargs.keys()) 606 for k, v in kwargs.items(): 607 if k in _id_keys: 608 for rec in self._index["record"]: 609 # if hasattr(rec, k) and getattr(rec, k) == v: 610 if getattr(rec.id, k) == v: 611 idxs.append(rec._recnum) 612 else: 613 for rec in self._index["record"]: 614 if hasattr(rec, k) and getattr(rec, k) == v: 615 idxs.append(rec._recnum) 616 idxs = np.array(idxs, dtype=">i4") 617 return [self._index["record"][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]]
Select records by attribute.
Parameters
- **kwargs (dict):
Keyword arguments specifying
TdlpackRecordattributes and values to match.
Returns
- list: List of records matching all provided attribute/value pairs.
Notes
Selection is performed against records in the internal index. Records must match all specified attributes to be included in the result.
620class TdlpackRecord: 621 """ 622 Creation class for TDLPACK record objects. 623 624 This class dynamically constructs and returns an instance of a 625 ``_TdlpackRecord`` subclass based on the provided section arrays and 626 optional keyword arguments. Record classes are cached by type to avoid 627 repeated class construction. 628 629 Parameters 630 ---------- 631 is0 : numpy.ndarray, optional 632 Section 0 array. Default is zero-initialized ``int32`` array of size ``ND7``. 633 is1 : numpy.ndarray, optional 634 Section 1 array. Default is zero-initialized ``int32`` array of size ``ND7``. 635 is2 : numpy.ndarray, optional 636 Section 2 array. Default is zero-initialized ``int32`` array of size ``ND7``. 637 is4 : numpy.ndarray, optional 638 Section 4 array. Default is zero-initialized ``int32`` array of size ``ND7``. 639 *args : tuple 640 Additional positional arguments passed to the constructed record class. 641 **kwargs : dict 642 Optional keyword arguments. The following key is recognized: 643 644 - ``type`` : str, optional 645 Record type. Default is ``"vector"``. If ``is2`` contains any 646 non-zero values, the type is automatically set to ``"grid"``. 647 - ``grid`` : str, optional 648 Predefined grid. Valid values can be found in ``tdlpackio.grids.GRIDS``. 649 This will force the type to be ``"grid"`` and override values in the ``is2`` 650 argument. 651 652 Returns 653 ------- 654 _TdlpackRecord 655 Instance of a dynamically generated subclass of ``_TdlpackRecord``. 656 657 Notes 658 ----- 659 - Record subclasses are created dynamically and cached in 660 ``_record_class_store`` using the record type as the key. 661 - For ``"grid"`` records, ``templates.GridDefinitionSection`` is added 662 as a base class and ``is1[1]`` is set to indicate the presence of a 663 grid definition section. 664 """ 665 666 def __new__( 667 self, 668 is0: np.array = np.zeros((ND7), dtype=np.int32), 669 is1: np.array = np.zeros((ND7), dtype=np.int32), 670 is2: np.array = np.zeros((ND7), dtype=np.int32), 671 is4: np.array = np.zeros((ND7), dtype=np.int32), 672 *args, 673 **kwargs, 674 ): 675 676 rectype = "vector" 677 if "type" in kwargs.keys(): 678 rectype = kwargs["type"] 679 if rectype not in {"grid", "vector"}: 680 raise ValueError('Invalid "type" argument. Expected "grid" or "vector".') 681 682 if bool(np.any(is2)): 683 rectype = "grid" 684 685 if "grid" in kwargs.keys(): 686 rectype = "grid" 687 if kwargs["grid"] not in grids.GRIDS.keys(): 688 raise ValueError('Invalid "grid" argument.') 689 is2 = grids.get_grid(kwargs["grid"]).to_is2() 690 691 bases = list() 692 if rectype == "grid": 693 bases.append(templates.GridDefinitionSection) 694 is1[1] = 1 # Flag in is1 to state that a grid definition section exists 695 696 try: 697 Record = _record_class_store[rectype] 698 except KeyError: 699 700 @dataclass(init=False, repr=False) 701 class Record(_TdlpackRecord, *bases): 702 pass 703 704 _record_class_store[rectype] = Record 705 706 return Record(is0, is1, is2, is4, *args)
Creation class for TDLPACK record objects.
This class dynamically constructs and returns an instance of a
_TdlpackRecord subclass based on the provided section arrays and
optional keyword arguments. Record classes are cached by type to avoid
repeated class construction.
Parameters
- is0 (numpy.ndarray, optional):
Section 0 array. Default is zero-initialized
int32array of sizeND7. - is1 (numpy.ndarray, optional):
Section 1 array. Default is zero-initialized
int32array of sizeND7. - is2 (numpy.ndarray, optional):
Section 2 array. Default is zero-initialized
int32array of sizeND7. - is4 (numpy.ndarray, optional):
Section 4 array. Default is zero-initialized
int32array of sizeND7. - *args (tuple): Additional positional arguments passed to the constructed record class.
**kwargs (dict): Optional keyword arguments. The following key is recognized:
type: str, optional Record type. Default is"vector". Ifis2contains any non-zero values, the type is automatically set to"grid".grid: str, optional Predefined grid. Valid values can be found intdlpackio.grids.GRIDS. This will force the type to be"grid"and override values in theis2argument.
Returns
- _TdlpackRecord: Instance of a dynamically generated subclass of
_TdlpackRecord.
Notes
- Record subclasses are created dynamically and cached in
_record_class_storeusing the record type as the key. - For
"grid"records,templates.GridDefinitionSectionis added as a base class andis1[1]is set to indicate the presence of a grid definition section.
1059@dataclass 1060class TdlpackStationRecord: 1061 """ 1062 TDLPACK Station Record class 1063 """ 1064 1065 stations_in: InitVar[Optional[Iterable[str]]] = None 1066 1067 type: str = field(init=False, repr=False, default="station") 1068 stations: ClassVar[templates.Stations] = templates.Stations() 1069 1070 # Private class variable holding the list 1071 _stations: Optional[list[str]] = field(init=False, repr=False, default=None) 1072 1073 def __post_init__(self, stations_in): 1074 self._nsta_expected = 0 1075 self._recnum = -1 1076 self._source = None 1077 self._stations = None 1078 self._type = "station" 1079 self.id = TdlpackID([400001000, 0, 0, 0], self) 1080 1081 if stations_in is not None: 1082 self.stations = stations_in 1083 1084 def __str__(self): 1085 return f"{self._recnum}:d=0000000000:STATION CALL LETTER RECORD:{self.numberOfStations}" 1086 1087 @property 1088 def numberOfStations(self): 1089 if self._source is not None and (isinstance(self._stations, list) or self._stations is None): 1090 return self._nsta_expected 1091 return 0 if self.stations is None else len(self.stations) 1092 1093 @property 1094 def data(self): 1095 pass 1096 1097 def pack(self): 1098 pass
TDLPACK Station Record class
1101@dataclass 1102class TdlpackTrailerRecord: 1103 """ 1104 TDLPACK Trailer Record class 1105 """ 1106 1107 type: str = field(init=False, repr=False, default="trailer") 1108 1109 def __post_init__(self): 1110 """""" 1111 self._recnum = -1 1112 self._type = "trailer" 1113 self.id = TdlpackID([0, 0, 0, 0], self) 1114 1115 def __str__(self): 1116 """""" 1117 return f"{self._recnum}:d=0000000000:TRAILER RECORD" 1118 1119 @property 1120 def data(self): 1121 """""" 1122 pass 1123 1124 def pack(self): 1125 """""" 1126 pass
TDLPACK Trailer Record class
1129class TdlpackID: 1130 """ 1131 TDLPACK variable ID class 1132 """ 1133 1134 __slots__ = ("_id", "_rec") 1135 1136 def __init__(self, id, linked_rec=None): 1137 """ 1138 Initialize TDLPACK variable ID 1139 1140 Parameters 1141 ---------- 1142 id : list of ints 1143 The 4-word TDLPACK variable ID 1144 linked_rec : TdlpackRecord, optional 1145 TDLPACK record object. This optional argument provides a mechanism 1146 for updating the TDLPACK variable ID in the TdlpackRecord is1 array. 1147 """ 1148 self._id = utils.parse_id(id) 1149 self._rec = linked_rec 1150 1151 def __eq__(self, value): 1152 if isinstance(value, list) or isinstance(value, tuple): 1153 return self.word1 == value[0] and self.word2 == value[1] and self.word3 == value[2] and self.word4 == value[3] 1154 else: 1155 return False 1156 1157 def __format__(self, spec: str) -> str: 1158 if not spec: 1159 spec = "basic" 1160 return self.format(spec) 1161 1162 def __repr__(self): 1163 return repr(utils.unparse_id(self._id)) 1164 1165 @classmethod 1166 def from_string(cls, idstr): 1167 """ 1168 Create a TdlpackID object from a TDLPACK ID string. 1169 1170 Parameters 1171 ---------- 1172 idstr : str 1173 String containing the TDLPACK ID with leading zeros and delimited 1174 by a non-numeric character. 1175 1176 Returns 1177 ------- 1178 An instance of TdlpackID. 1179 """ 1180 delim = idstr[9] 1181 if {idstr[19], idstr[29]} != {delim, delim}: 1182 raise ValueError(f"Invalid TDLPACK ID string format") 1183 return cls([int(i.lstrip("0")) if len(i.lstrip("0")) > 0 else 0 for i in idstr.split(delim)]) 1184 1185 def format(self, style: str = "basic") -> str: 1186 """ 1187 Format the TDLPACK ID identifier as a string. 1188 1189 This method returns a string representation of the identifier in one 1190 of several supported formats commonly used in MOS-2000 workflows. 1191 1192 Parameters 1193 ---------- 1194 style : {'basic', 'b', 'mos', 'm', 'parsed', 'p'}, optional 1195 Output format style (case-insensitive): 1196 1197 - ``"basic"`` or ``"b"`` 1198 Four-word identifier. Each word is printed as a zero-padded 1199 integer field. 1200 1201 - ``"mos"`` or ``"m"`` 1202 MOS-style identifier consisting of the first three words 1203 followed by the ISG components and the threshold value 1204 formatted in scientific notation (``.0000e±00``). 1205 1206 - ``"parsed"`` or ``"p"`` 1207 Identifier parsed into its individual components as defined 1208 by the internal ID mapping. All components are printed as 1209 zero-padded integers except the threshold value, which is 1210 printed as a floating-point value with ``F13.6`` formatting. 1211 1212 Returns 1213 ------- 1214 str 1215 String representation of the identifier in the requested format. 1216 1217 Notes 1218 ----- 1219 The MOS-style threshold representation removes the leading zero 1220 from scientific notation (e.g., ``0.0000e+00`` → ``.0000e+00``) 1221 to match legacy MOS-2000 formatting conventions. 1222 1223 Examples 1224 -------- 1225 Using the ``format`` method: 1226 1227 >>> rec.format("basic") 1228 '001000008 000000500 000000000 0000000000' 1229 1230 >>> rec.format("mos") 1231 '001000008 000000500 000000000 000 .0000e+00' 1232 1233 >>> rec.format("parsed") 1234 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1235 1236 Using Python's format protocol (``__format__``): 1237 1238 >>> f"{rec.id:basic}" 1239 '001000008 000000500 000000000 0000000000' 1240 1241 >>> f"{rec.id:mos}" 1242 '001000008 000000500 000000000 000 .0000e+00' 1243 1244 >>> f"{rec.id:parsed}" 1245 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1246 """ 1247 style = style.lower() 1248 1249 if style in {"basic", "b"}: 1250 return f"{str(self.word1).zfill(9)} {str(self.word2).zfill(9)} {str(self.word3).zfill(9)} {str(self.word4).zfill(10)}" 1251 elif style in {"mos", "m"}: 1252 thresh = f"{self.thresh:.4e}" 1253 if thresh.startswith("0"): 1254 thresh = thresh[1:] 1255 elif thresh.startswith("-0"): 1256 thresh = "-" + thresh[2:] 1257 return f"{str(self.word1).zfill(9)} {str(self.word2).zfill(9)} {str(self.word3).zfill(9)} {self.i}{self.s}{self.g} {thresh}" 1258 elif style in {"parsed", "p"}: 1259 parsed = "" 1260 for k, v in self._id.items(): 1261 if "thresh" not in k: 1262 parsed += f"{str(v).zfill(len(k))} " 1263 else: 1264 parsed += f"{v:13.6f}" 1265 return parsed 1266 1267 raise ValueError(f"Unknown TdlpackID format style: {style!r}") 1268 1269 def to_dict(self): 1270 """ 1271 Return TDLPACK variable ID as dict. 1272 1273 Returns 1274 ------- 1275 String of the 4-word TDLPACK variable ID. 1276 """ 1277 return self._id 1278 1279 def to_string(self, delim=" "): 1280 """ 1281 Return TDLPACK variable ID as string. 1282 1283 Parameters 1284 ---------- 1285 delim : str, optional 1286 Delimiter character between each TDLPACK variable ID word. 1287 1288 Returns 1289 ------- 1290 String of the 4-word TDLPACK variable ID. 1291 """ 1292 strlen = (9, 9, 9, 10) 1293 return delim.join([str(i).zfill(word) for word, i in zip(strlen, utils.unparse_id(self._id))]) 1294 1295 @property 1296 def word1(self): 1297 """ 1298 First ID word. 1299 1300 Returns 1301 ------- 1302 int 1303 First parsed ID word. 1304 """ 1305 return utils.unparse_id(self._id)[0] 1306 1307 @word1.setter 1308 def word1(self, value): 1309 """ 1310 Set first ID word. 1311 1312 Parameters 1313 ---------- 1314 value : int 1315 New value. 1316 1317 Notes 1318 ----- 1319 Updates internal ID and ``is1[8]`` if record is attached. 1320 """ 1321 newid = utils.unparse_id(self._id) 1322 newid[0] = value 1323 self._id = utils.parse_id(newid) 1324 if self._rec is not None: 1325 self._rec.is1[8] = newid[0] 1326 1327 @property 1328 def word2(self): 1329 """ 1330 Second ID word. 1331 1332 Returns 1333 ------- 1334 int 1335 Second parsed ID word. 1336 """ 1337 return utils.unparse_id(self._id)[1] 1338 1339 @word2.setter 1340 def word2(self, value): 1341 """ 1342 Set second ID word. 1343 1344 Parameters 1345 ---------- 1346 value : int 1347 New value. 1348 1349 Notes 1350 ----- 1351 Updates internal ID and ``is1[9]`` if record is attached. 1352 """ 1353 newid = utils.unparse_id(self._id) 1354 newid[1] = value 1355 self._id = utils.parse_id(newid) 1356 if self._rec is not None: 1357 self._rec.is1[9] = newid[1] 1358 1359 @property 1360 def word3(self): 1361 """ 1362 Third ID word. 1363 1364 Returns 1365 ------- 1366 int 1367 Third parsed ID word. 1368 """ 1369 return utils.unparse_id(self._id)[2] 1370 1371 @word3.setter 1372 def word3(self, value): 1373 """ 1374 Set third ID word. 1375 1376 Parameters 1377 ---------- 1378 value : int 1379 New value. 1380 1381 Notes 1382 ----- 1383 Updates internal ID and ``is1[10]`` if record is attached. 1384 """ 1385 newid = utils.unparse_id(self._id) 1386 newid[2] = value 1387 self._id = utils.parse_id(newid) 1388 if self._rec is not None: 1389 self._rec.is1[10] = newid[2] 1390 1391 @property 1392 def word4(self): 1393 """ 1394 Fourth ID word. 1395 1396 Returns 1397 ------- 1398 int 1399 Fourth parsed ID word. 1400 """ 1401 return utils.unparse_id(self._id)[3] 1402 1403 @word4.setter 1404 def word4(self, value): 1405 """ 1406 Set fourth ID word. 1407 1408 Parameters 1409 ---------- 1410 value : int 1411 New value. 1412 1413 Notes 1414 ----- 1415 Updates internal ID and ``is1[11]`` if record is attached. 1416 """ 1417 newid = utils.unparse_id(self._id) 1418 newid[3] = value 1419 self._id = utils.parse_id(newid) 1420 if self._rec is not None: 1421 self._rec.is1[11] = newid[3] 1422 1423 @property 1424 def ccc(self): 1425 """ 1426 CCC identifier component. 1427 1428 Returns 1429 ------- 1430 int 1431 """ 1432 return self._id["ccc"] 1433 1434 @ccc.setter 1435 def ccc(self, value): 1436 """ 1437 Set CCC identifier component. 1438 1439 Parameters 1440 ---------- 1441 value : int 1442 1443 Notes 1444 ----- 1445 Updates ``is1[8]`` if record is attached. 1446 """ 1447 self._id["ccc"] = value 1448 if self._rec is not None: 1449 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1450 1451 @property 1452 def fff(self): 1453 """ 1454 FFF identifier component. 1455 1456 Returns 1457 ------- 1458 int 1459 """ 1460 return self._id["fff"] 1461 1462 @fff.setter 1463 def fff(self, value): 1464 """ 1465 Set FFF identifier component. 1466 1467 Parameters 1468 ---------- 1469 value : int 1470 1471 Notes 1472 ----- 1473 Updates ``is1[8]`` if record is attached. 1474 """ 1475 self._id["fff"] = value 1476 if self._rec is not None: 1477 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1478 1479 @property 1480 def cccfff(self): 1481 """ 1482 Combined CCCFFF identifier. 1483 1484 Returns 1485 ------- 1486 int 1487 Integer representation of ``word1 / 1000``. 1488 """ 1489 return int(self.word1 / 1000) 1490 1491 @property 1492 def b(self): 1493 """ 1494 B identifier component. 1495 1496 Returns 1497 ------- 1498 int 1499 """ 1500 return self._id["b"] 1501 1502 @b.setter 1503 def b(self, value): 1504 """ 1505 Set B identifier component. 1506 1507 Parameters 1508 ---------- 1509 value : int 1510 1511 Notes 1512 ----- 1513 Updates ``is1[8]`` if record is attached. 1514 """ 1515 self._id["b"] = value 1516 if self._rec is not None: 1517 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1518 1519 @property 1520 def dd(self): 1521 """ 1522 DD identifier component. 1523 1524 Returns 1525 ------- 1526 int 1527 """ 1528 return self._id["dd"] 1529 1530 @dd.setter 1531 def dd(self, value): 1532 """ 1533 Set DD identifier component. 1534 1535 Parameters 1536 ---------- 1537 value : int 1538 1539 Notes 1540 ----- 1541 Updates ``is1[8]`` and ``is1[14]`` if record is attached. 1542 """ 1543 self._id["dd"] = value 1544 if self._rec is not None: 1545 self._rec.is1[8] = utils.unparse_id(self._id)[0] 1546 self._rec.is1[14] = value 1547 1548 @property 1549 def v(self): 1550 """ 1551 V identifier component. 1552 1553 Returns 1554 ------- 1555 int 1556 """ 1557 return self._id["v"] 1558 1559 @v.setter 1560 def v(self, value): 1561 """ 1562 Set V identifier component. 1563 1564 Parameters 1565 ---------- 1566 value : int 1567 1568 Notes 1569 ----- 1570 Updates ``is1[9]`` if record is attached. 1571 """ 1572 self._id["v"] = value 1573 if self._rec is not None: 1574 self._rec.is1[9] = utils.unparse_id(self._id)[1] 1575 1576 @property 1577 def llll(self): 1578 """ 1579 LLLL identifier component. 1580 1581 Returns 1582 ------- 1583 int 1584 """ 1585 return self._id["llll"] 1586 1587 @llll.setter 1588 def llll(self, value): 1589 """ 1590 Set LLLL identifier component. 1591 1592 Parameters 1593 ---------- 1594 value : int 1595 1596 Notes 1597 ----- 1598 Updates ``is1[9]`` if record is attached. 1599 """ 1600 self._id["llll"] = value 1601 if self._rec is not None: 1602 self._rec.is1[9] = utils.unparse_id(self._id)[1] 1603 1604 @property 1605 def uuuu(self): 1606 """ 1607 UUUU identifier component. 1608 1609 Returns 1610 ------- 1611 int 1612 """ 1613 return self._id["uuuu"] 1614 1615 @uuuu.setter 1616 def uuuu(self, value): 1617 """ 1618 Set UUUU identifier component. 1619 1620 Parameters 1621 ---------- 1622 value : int 1623 1624 Notes 1625 ----- 1626 Updates ``is1[9]`` if record is attached. 1627 """ 1628 self._id["uuuu"] = value 1629 if self._rec is not None: 1630 self._rec.is1[9] = utils.unparse_id(self._id)[1] 1631 1632 @property 1633 def t(self): 1634 """ 1635 T identifier component. 1636 1637 Returns 1638 ------- 1639 int 1640 """ 1641 return self._id["t"] 1642 1643 @t.setter 1644 def t(self, value): 1645 """ 1646 Set T identifier component. 1647 1648 Parameters 1649 ---------- 1650 value : int 1651 1652 Notes 1653 ----- 1654 Updates ``is1[10]`` if record is attached. 1655 """ 1656 self._id["t"] = value 1657 if self._rec is not None: 1658 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1659 1660 @property 1661 def rr(self): 1662 """ 1663 RR identifier component. 1664 1665 Returns 1666 ------- 1667 int 1668 """ 1669 return self._id["rr"] 1670 1671 @rr.setter 1672 def rr(self, value): 1673 """ 1674 Set RR identifier component. 1675 1676 Parameters 1677 ---------- 1678 value : int 1679 1680 Notes 1681 ----- 1682 Updates ``is1[10]`` if record is attached. 1683 """ 1684 self._id["rr"] = value 1685 if self._rec is not None: 1686 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1687 1688 @property 1689 def o(self): 1690 """ 1691 O identifier component. 1692 1693 Returns 1694 ------- 1695 int 1696 """ 1697 return self._id["o"] 1698 1699 @o.setter 1700 def o(self, value): 1701 """ 1702 Set O identifier component. 1703 1704 Parameters 1705 ---------- 1706 value : int 1707 1708 Notes 1709 ----- 1710 Updates ``is1[10]`` if record is attached. 1711 """ 1712 self._id["o"] = value 1713 if self._rec is not None: 1714 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1715 1716 @property 1717 def hh(self): 1718 """ 1719 HH identifier component. 1720 1721 Returns 1722 ------- 1723 int 1724 """ 1725 return self._id["hh"] 1726 1727 @hh.setter 1728 def hh(self, value): 1729 """ 1730 Set HH identifier component. 1731 1732 Parameters 1733 ---------- 1734 value : int 1735 1736 Notes 1737 ----- 1738 Updates ``is1[10]`` if record is attached. 1739 """ 1740 self._id["hh"] = value 1741 if self._rec is not None: 1742 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1743 1744 @property 1745 def tau(self): 1746 """ 1747 Forecast hour (tau). 1748 1749 Returns 1750 ------- 1751 int 1752 """ 1753 return self._id["tau"] 1754 1755 @tau.setter 1756 def tau(self, value): 1757 """ 1758 Set forecast hour (tau). 1759 1760 Parameters 1761 ---------- 1762 value : int 1763 1764 Notes 1765 ----- 1766 Updates ``is1[10]`` and ``is1[12]`` if record is attached. 1767 """ 1768 self._id["tau"] = value 1769 if self._rec is not None: 1770 self._rec.is1[10] = utils.unparse_id(self._id)[2] 1771 self._rec.is1[12] = value 1772 1773 @property 1774 def thresh(self): 1775 """ 1776 Threshold identifier component. 1777 1778 Returns 1779 ------- 1780 int 1781 """ 1782 return self._id["thresh"] 1783 1784 @thresh.setter 1785 def thresh(self, value): 1786 """ 1787 Set threshold identifier component. 1788 1789 Parameters 1790 ---------- 1791 value : int 1792 1793 Notes 1794 ----- 1795 Updates ``is1[11]`` if record is attached. 1796 """ 1797 self._id["thresh"] = value 1798 if self._rec is not None: 1799 self._rec.is1[11] = utils.unparse_id(self._id)[3] 1800 1801 @property 1802 def i(self): 1803 """ 1804 I identifier component. 1805 1806 Returns 1807 ------- 1808 int 1809 """ 1810 return self._id["i"] 1811 1812 @i.setter 1813 def i(self, value): 1814 """ 1815 Set I identifier component. 1816 1817 Parameters 1818 ---------- 1819 value : int 1820 1821 Notes 1822 ----- 1823 Updates ``is1[11]`` if record is attached. 1824 """ 1825 self._id["i"] = value 1826 if self._rec is not None: 1827 self._rec.is1[11] = utils.unparse_id(self._id)[3] 1828 1829 @property 1830 def s(self): 1831 """ 1832 S identifier component. 1833 1834 Returns 1835 ------- 1836 int 1837 """ 1838 return self._id["s"] 1839 1840 @s.setter 1841 def s(self, value): 1842 """ 1843 Set S identifier component. 1844 1845 Parameters 1846 ---------- 1847 value : int 1848 1849 Notes 1850 ----- 1851 Updates ``is1[11]`` if record is attached. 1852 """ 1853 self._id["s"] = value 1854 if self._rec is not None: 1855 self._rec.is1[11] = utils.unparse_id(self._id)[3] 1856 1857 @property 1858 def g(self): 1859 """ 1860 G identifier component. 1861 1862 Returns 1863 ------- 1864 int 1865 """ 1866 return self._id["g"] 1867 1868 @g.setter 1869 def g(self, value): 1870 """ 1871 Set G identifier component. 1872 1873 Parameters 1874 ---------- 1875 value : int 1876 1877 Notes 1878 ----- 1879 Updates ``is1[11]`` if record is attached. 1880 """ 1881 self._id["g"] = value 1882 if self._rec is not None: 1883 self._rec.is1[11] = utils.unparse_id(self._id)[3]
TDLPACK variable ID class
1136 def __init__(self, id, linked_rec=None): 1137 """ 1138 Initialize TDLPACK variable ID 1139 1140 Parameters 1141 ---------- 1142 id : list of ints 1143 The 4-word TDLPACK variable ID 1144 linked_rec : TdlpackRecord, optional 1145 TDLPACK record object. This optional argument provides a mechanism 1146 for updating the TDLPACK variable ID in the TdlpackRecord is1 array. 1147 """ 1148 self._id = utils.parse_id(id) 1149 self._rec = linked_rec
Initialize TDLPACK variable ID
Parameters
- id (list of ints): The 4-word TDLPACK variable ID
- linked_rec (TdlpackRecord, optional): TDLPACK record object. This optional argument provides a mechanism for updating the TDLPACK variable ID in the TdlpackRecord is1 array.
1165 @classmethod 1166 def from_string(cls, idstr): 1167 """ 1168 Create a TdlpackID object from a TDLPACK ID string. 1169 1170 Parameters 1171 ---------- 1172 idstr : str 1173 String containing the TDLPACK ID with leading zeros and delimited 1174 by a non-numeric character. 1175 1176 Returns 1177 ------- 1178 An instance of TdlpackID. 1179 """ 1180 delim = idstr[9] 1181 if {idstr[19], idstr[29]} != {delim, delim}: 1182 raise ValueError(f"Invalid TDLPACK ID string format") 1183 return cls([int(i.lstrip("0")) if len(i.lstrip("0")) > 0 else 0 for i in idstr.split(delim)])
Create a TdlpackID object from a TDLPACK ID string.
Parameters
- idstr (str): String containing the TDLPACK ID with leading zeros and delimited by a non-numeric character.
Returns
- An instance of TdlpackID.
1185 def format(self, style: str = "basic") -> str: 1186 """ 1187 Format the TDLPACK ID identifier as a string. 1188 1189 This method returns a string representation of the identifier in one 1190 of several supported formats commonly used in MOS-2000 workflows. 1191 1192 Parameters 1193 ---------- 1194 style : {'basic', 'b', 'mos', 'm', 'parsed', 'p'}, optional 1195 Output format style (case-insensitive): 1196 1197 - ``"basic"`` or ``"b"`` 1198 Four-word identifier. Each word is printed as a zero-padded 1199 integer field. 1200 1201 - ``"mos"`` or ``"m"`` 1202 MOS-style identifier consisting of the first three words 1203 followed by the ISG components and the threshold value 1204 formatted in scientific notation (``.0000e±00``). 1205 1206 - ``"parsed"`` or ``"p"`` 1207 Identifier parsed into its individual components as defined 1208 by the internal ID mapping. All components are printed as 1209 zero-padded integers except the threshold value, which is 1210 printed as a floating-point value with ``F13.6`` formatting. 1211 1212 Returns 1213 ------- 1214 str 1215 String representation of the identifier in the requested format. 1216 1217 Notes 1218 ----- 1219 The MOS-style threshold representation removes the leading zero 1220 from scientific notation (e.g., ``0.0000e+00`` → ``.0000e+00``) 1221 to match legacy MOS-2000 formatting conventions. 1222 1223 Examples 1224 -------- 1225 Using the ``format`` method: 1226 1227 >>> rec.format("basic") 1228 '001000008 000000500 000000000 0000000000' 1229 1230 >>> rec.format("mos") 1231 '001000008 000000500 000000000 000 .0000e+00' 1232 1233 >>> rec.format("parsed") 1234 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1235 1236 Using Python's format protocol (``__format__``): 1237 1238 >>> f"{rec.id:basic}" 1239 '001000008 000000500 000000000 0000000000' 1240 1241 >>> f"{rec.id:mos}" 1242 '001000008 000000500 000000000 000 .0000e+00' 1243 1244 >>> f"{rec.id:parsed}" 1245 '001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000' 1246 """ 1247 style = style.lower() 1248 1249 if style in {"basic", "b"}: 1250 return f"{str(self.word1).zfill(9)} {str(self.word2).zfill(9)} {str(self.word3).zfill(9)} {str(self.word4).zfill(10)}" 1251 elif style in {"mos", "m"}: 1252 thresh = f"{self.thresh:.4e}" 1253 if thresh.startswith("0"): 1254 thresh = thresh[1:] 1255 elif thresh.startswith("-0"): 1256 thresh = "-" + thresh[2:] 1257 return f"{str(self.word1).zfill(9)} {str(self.word2).zfill(9)} {str(self.word3).zfill(9)} {self.i}{self.s}{self.g} {thresh}" 1258 elif style in {"parsed", "p"}: 1259 parsed = "" 1260 for k, v in self._id.items(): 1261 if "thresh" not in k: 1262 parsed += f"{str(v).zfill(len(k))} " 1263 else: 1264 parsed += f"{v:13.6f}" 1265 return parsed 1266 1267 raise ValueError(f"Unknown TdlpackID format style: {style!r}")
Format the TDLPACK ID identifier as a string.
This method returns a string representation of the identifier in one of several supported formats commonly used in MOS-2000 workflows.
Parameters
style ({'basic', 'b', 'mos', 'm', 'parsed', 'p'}, optional): Output format style (case-insensitive):
"basic"or"b"Four-word identifier. Each word is printed as a zero-padded integer field."mos"or"m"MOS-style identifier consisting of the first three words followed by the ISG components and the threshold value formatted in scientific notation (.0000e±00)."parsed"or"p"Identifier parsed into its individual components as defined by the internal ID mapping. All components are printed as zero-padded integers except the threshold value, which is printed as a floating-point value withF13.6formatting.
Returns
- str: String representation of the identifier in the requested format.
Notes
The MOS-style threshold representation removes the leading zero
from scientific notation (e.g., 0.0000e+00 → .0000e+00)
to match legacy MOS-2000 formatting conventions.
Examples
Using the format method:
>>> rec.format("basic")
'001000008 000000500 000000000 0000000000'
>>> rec.format("mos")
'001000008 000000500 000000000 000 .0000e+00'
>>> rec.format("parsed")
'001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000'
Using Python's format protocol (__format__):
>>> f"{rec.id:basic}"
'001000008 000000500 000000000 0000000000'
>>> f"{rec.id:mos}"
'001000008 000000500 000000000 000 .0000e+00'
>>> f"{rec.id:parsed}"
'001 000 0 08 0 0000 0500 0 00 0 00 000 0 0 0 0.000000'
1269 def to_dict(self): 1270 """ 1271 Return TDLPACK variable ID as dict. 1272 1273 Returns 1274 ------- 1275 String of the 4-word TDLPACK variable ID. 1276 """ 1277 return self._id
Return TDLPACK variable ID as dict.
Returns
- String of the 4-word TDLPACK variable ID.
1279 def to_string(self, delim=" "): 1280 """ 1281 Return TDLPACK variable ID as string. 1282 1283 Parameters 1284 ---------- 1285 delim : str, optional 1286 Delimiter character between each TDLPACK variable ID word. 1287 1288 Returns 1289 ------- 1290 String of the 4-word TDLPACK variable ID. 1291 """ 1292 strlen = (9, 9, 9, 10) 1293 return delim.join([str(i).zfill(word) for word, i in zip(strlen, utils.unparse_id(self._id))])
Return TDLPACK variable ID as string.
Parameters
- delim (str, optional): Delimiter character between each TDLPACK variable ID word.
Returns
- String of the 4-word TDLPACK variable ID.
1295 @property 1296 def word1(self): 1297 """ 1298 First ID word. 1299 1300 Returns 1301 ------- 1302 int 1303 First parsed ID word. 1304 """ 1305 return utils.unparse_id(self._id)[0]
First ID word.
Returns
- int: First parsed ID word.
1327 @property 1328 def word2(self): 1329 """ 1330 Second ID word. 1331 1332 Returns 1333 ------- 1334 int 1335 Second parsed ID word. 1336 """ 1337 return utils.unparse_id(self._id)[1]
Second ID word.
Returns
- int: Second parsed ID word.
1359 @property 1360 def word3(self): 1361 """ 1362 Third ID word. 1363 1364 Returns 1365 ------- 1366 int 1367 Third parsed ID word. 1368 """ 1369 return utils.unparse_id(self._id)[2]
Third ID word.
Returns
- int: Third parsed ID word.
1391 @property 1392 def word4(self): 1393 """ 1394 Fourth ID word. 1395 1396 Returns 1397 ------- 1398 int 1399 Fourth parsed ID word. 1400 """ 1401 return utils.unparse_id(self._id)[3]
Fourth ID word.
Returns
- int: Fourth parsed ID word.
1423 @property 1424 def ccc(self): 1425 """ 1426 CCC identifier component. 1427 1428 Returns 1429 ------- 1430 int 1431 """ 1432 return self._id["ccc"]
CCC identifier component.
Returns
- int
1451 @property 1452 def fff(self): 1453 """ 1454 FFF identifier component. 1455 1456 Returns 1457 ------- 1458 int 1459 """ 1460 return self._id["fff"]
FFF identifier component.
Returns
- int
1479 @property 1480 def cccfff(self): 1481 """ 1482 Combined CCCFFF identifier. 1483 1484 Returns 1485 ------- 1486 int 1487 Integer representation of ``word1 / 1000``. 1488 """ 1489 return int(self.word1 / 1000)
Combined CCCFFF identifier.
Returns
- int: Integer representation of
word1 / 1000.
1491 @property 1492 def b(self): 1493 """ 1494 B identifier component. 1495 1496 Returns 1497 ------- 1498 int 1499 """ 1500 return self._id["b"]
B identifier component.
Returns
- int
1519 @property 1520 def dd(self): 1521 """ 1522 DD identifier component. 1523 1524 Returns 1525 ------- 1526 int 1527 """ 1528 return self._id["dd"]
DD identifier component.
Returns
- int
1548 @property 1549 def v(self): 1550 """ 1551 V identifier component. 1552 1553 Returns 1554 ------- 1555 int 1556 """ 1557 return self._id["v"]
V identifier component.
Returns
- int
1576 @property 1577 def llll(self): 1578 """ 1579 LLLL identifier component. 1580 1581 Returns 1582 ------- 1583 int 1584 """ 1585 return self._id["llll"]
LLLL identifier component.
Returns
- int
1604 @property 1605 def uuuu(self): 1606 """ 1607 UUUU identifier component. 1608 1609 Returns 1610 ------- 1611 int 1612 """ 1613 return self._id["uuuu"]
UUUU identifier component.
Returns
- int
1632 @property 1633 def t(self): 1634 """ 1635 T identifier component. 1636 1637 Returns 1638 ------- 1639 int 1640 """ 1641 return self._id["t"]
T identifier component.
Returns
- int
1660 @property 1661 def rr(self): 1662 """ 1663 RR identifier component. 1664 1665 Returns 1666 ------- 1667 int 1668 """ 1669 return self._id["rr"]
RR identifier component.
Returns
- int
1688 @property 1689 def o(self): 1690 """ 1691 O identifier component. 1692 1693 Returns 1694 ------- 1695 int 1696 """ 1697 return self._id["o"]
O identifier component.
Returns
- int
1716 @property 1717 def hh(self): 1718 """ 1719 HH identifier component. 1720 1721 Returns 1722 ------- 1723 int 1724 """ 1725 return self._id["hh"]
HH identifier component.
Returns
- int
1744 @property 1745 def tau(self): 1746 """ 1747 Forecast hour (tau). 1748 1749 Returns 1750 ------- 1751 int 1752 """ 1753 return self._id["tau"]
Forecast hour (tau).
Returns
- int
1773 @property 1774 def thresh(self): 1775 """ 1776 Threshold identifier component. 1777 1778 Returns 1779 ------- 1780 int 1781 """ 1782 return self._id["thresh"]
Threshold identifier component.
Returns
- int
1801 @property 1802 def i(self): 1803 """ 1804 I identifier component. 1805 1806 Returns 1807 ------- 1808 int 1809 """ 1810 return self._id["i"]
I identifier component.
Returns
- int
20@dataclass(frozen=True, slots=True) 21class TdlpackGridDef: 22 """Definition of a TDLPACK grid. 23 24 Stored longitude convention is MOS-2000: 25 west longitude is positive. 26 """ 27 28 mapProjection: int 29 nx: int 30 ny: int 31 latitudeLowerLeft: float 32 longitudeLowerLeft: float 33 standardLatitude: float 34 orientationLongitude: float 35 gridLength: float 36 37 # Array mapping 38 IS2_INDEX: ClassVar[dict[str, int]] = { 39 "mapProjection": 1, 40 "nx": 2, 41 "ny": 3, 42 "latitudeLowerLeft": 4, 43 "longitudeLowerLeft": 5, 44 "orientationLongitude": 6, 45 "gridLength": 7, 46 "standardLatitude": 8, 47 } 48 49 # Attribute scale factors used to convert to whole integers. 50 SCALE_FACTOR: ClassVar[dict[str, int]] = { 51 "mapProjection": 1, 52 "nx": 1, 53 "ny": 1, 54 "latitudeLowerLeft": 10_000, 55 "longitudeLowerLeft": 10_000, 56 "orientationLongitude": 10_000, 57 "gridLength": 1000, 58 "standardLatitude": 10_000, 59 } 60 61 def scaled_value(self, name: str) -> int: 62 """Return one grid attribute scaled to an integer.""" 63 value = getattr(self, name) 64 scale = self.SCALE_FACTOR[name] 65 return int(round(value * scale)) 66 67 def to_is2(self) -> list[int]: 68 """Return the grid definition as a scaled integer array.""" 69 is2 = np.zeros((_tdlpackio.ND7), dtype=np.int32) 70 71 for name, index in self.IS2_INDEX.items(): 72 is2[index] = self.scaled_value(name) 73 74 return is2 75 76 def to_int_dict(self) -> dict[str, int]: 77 """Return scaled integer values keyed by attribute name.""" 78 return {name: self.scaled_value(name) for name in self.IS2_INDEX}
Definition of a TDLPACK grid.
Stored longitude convention is MOS-2000: west longitude is positive.
61 def scaled_value(self, name: str) -> int: 62 """Return one grid attribute scaled to an integer.""" 63 value = getattr(self, name) 64 scale = self.SCALE_FACTOR[name] 65 return int(round(value * scale))
Return one grid attribute scaled to an integer.
67 def to_is2(self) -> list[int]: 68 """Return the grid definition as a scaled integer array.""" 69 is2 = np.zeros((_tdlpackio.ND7), dtype=np.int32) 70 71 for name, index in self.IS2_INDEX.items(): 72 is2[index] = self.scaled_value(name) 73 74 return is2
Return the grid definition as a scaled integer array.
218def get_grid( 219 name: str, 220 *, 221 lon_format: Literal["mos2k", "standard", "0_360"] = "mos2k", 222) -> TdlpackGridDef: 223 """ 224 Return a grid definition by name. 225 226 Notes 227 ----- 228 Internally, all grid definitions are stored using the MOS-2000 229 longitude convention (west longitude is positive). 230 231 Parameters 232 ---------- 233 name : str 234 Grid name. 235 lon_format : {"mos2k", "standard", "0_360"}, optional 236 Longitude convention to return: 237 238 - "mos2k": West longitude positive (native storage format) 239 - "standard": East-positive, range [-180, 180] 240 - "0_360": East-positive, range [0, 360) 241 242 Returns 243 ------- 244 TdlpackGridDef 245 Grid definition using the requested longitude convention. 246 """ 247 try: 248 grid = GRIDS[name.lower()] 249 except KeyError as exc: 250 raise KeyError(f"Unknown grid: {name!r}") from exc 251 252 # Native format (no copy needed) 253 if lon_format == "mos2k": 254 return grid 255 256 if lon_format == "standard": 257 return replace( 258 grid, 259 longitudeLowerLeft=_mos2k_to_standard(grid.longitudeLowerLeft), 260 orientationLongitude=_mos2k_to_standard(grid.orientationLongitude), 261 ) 262 263 if lon_format == "0_360": 264 return replace( 265 grid, 266 longitudeLowerLeft=_mos2k_to_0_360(grid.longitudeLowerLeft), 267 orientationLongitude=_mos2k_to_0_360(grid.orientationLongitude), 268 ) 269 270 raise ValueError(f"Invalid lon_format: {lon_format!r}. Expected one of {{'mos2k', 'standard', '0_360'}}.")
Return a grid definition by name.
Notes
Internally, all grid definitions are stored using the MOS-2000 longitude convention (west longitude is positive).
Parameters
- name (str): Grid name.
lon_format ({"mos2k", "standard", "0_360"}, optional): Longitude convention to return:
- "mos2k": West longitude positive (native storage format)
- "standard": East-positive, range [-180, 180]
- "0_360": East-positive, range [0, 360)
Returns
- TdlpackGridDef: Grid definition using the requested longitude convention.
273def has_grid(name: str) -> bool: 274 """Return True if a grid name is known.""" 275 return name.lower() in GRIDS
Return True if a grid name is known.