Measured Data Format file reader module for version 4.x.
With Unix and Windows for python 2.6+ and 3.2+
Author: | Aymeric Rateau |
---|
Created on Thu Dec 10 12:57:28 2013
Bases: dict
DATA class is organizing record classes itself made of recordchannel. This class inherits from dict. Keys are corresponding to channel group recordID A DATA class corresponds to a data block, a dict of record classes (one per channel group) Each record class contains a list of recordchannel class representing the structure of channel record.
Attributes
fid | (io.open) file identifier |
pointerToData | (int) position of Data block in mdf file |
type | (str) ‘sorted’ or ‘unsorted’ data block |
Methods
addRecord(record) | Adds a new record in DATA class dict |
read(channelList, zip=None) | Reads data block |
load(record, zip=None, nameList=None) | Reads sorted data block from record definition |
readRecord(recordID, buf, channelList=None): | read record from a buffer |
Adds a new record in DATA class dict.
Parameters: | record class
|
---|
Reads data block from record definition
Parameters: | record class
zip : bool, optional
nameList : list of str, optional
|
---|---|
Returns: | numpy recarray of data |
Reads data block
Parameters: | channelList : list of str
zip : bool, optional
|
---|
read record from a buffer
Parameters: | recordID : int
buf : str
channelList : list of str
|
---|
DATABlock converts raw data into arrays
Parameters: | record : class
parent_block : class
channelList : list of str, optional
sortedFlag : bool, optional
|
---|---|
Returns: | a recarray containing the channels data |
Notes
This function will read DTBlock, RDBlock, DZBlock (compressed), RDBlock (VLSD), sorted or unsorted
append new field in a recarray
Parameters: | rec : numpy recarray name : str
arr : numpy array to be appended numpy_dtype : numpy dtype, optional
|
---|---|
Returns: | numpy recarray appended |
function returning numpy style string from channel data type and number of bits
Parameters: | signalDataType : int
numberOfBits : int
|
---|---|
Returns: | dataType : str
|
Converts number of bits into number of aligned bytes
Parameters: | nBits : int
|
---|---|
Returns: | number of equivalent bytes |
modifies name of field in a recarray
Parameters: | arr : numpy recarray old_name : str
new_name : str
|
---|---|
Returns: | numpy recarray with modified field name |
converts specific channel from raw to physical data according to CCBlock information
Parameters: | channelName : dict
channelName : str
convert_tables : bool
multiProc : bool, default False
Q : Queue class, default None
|
---|---|
Returns: | dict
|
Adds ‘_title’ to channel name for numpy.core.records methods.
function returning C format string from channel data type and number of bits
Parameters: | signalDataType : int
numberOfBits : int
|
---|---|
Returns: | dataType : str
|
Makes all strings in a list having same length by appending spaces strings.
Parameters: | buf : list of str |
---|---|
Returns: | list of str elements all having same length |
apply formula conversion to data
Parameters: | vect : numpy 1D array
cc_val : mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |
invalid_bytes class to handle invalid bytes in record if existing
Attributes
name | (str) Name of channel |
signalDataType | (int) signal type according to specification |
bitCount | (int) number of bits used to store channel record |
nBytes | (int) number of bytes (1 byte = 8 bits) taken by channel record |
dataFormat | (str) numpy dtype as string |
Format : | C format understood by fread |
CFormat | (struct class instance) struct instance to convert from C Format |
byteOffset | (int) position of channel record in complete record in bytes |
bitOffset | (int) bit position of channel value inside byte in case of channel having bit count below 8 |
RecordFormat | (list of str) dtype format used for numpy.core.records functions ((name,name_title),str_stype) |
channelType | (int) channel type |
posByteBeg | (int) start position in number of bit of channel record in complete record |
posByteEnd | (int) end position in number of bit of channel record in complete record |
posBitBeg | (int) start position in number of bit of channel record in complete record |
posBitEnd | (int) end position in number of bit of channel record in complete record |
VLSD_CG_Flag | (bool) flag when Channel Group VLSD is used |
data | (int) pointer to data block linked to a channel (VLSD, MLSD) |
Methods
__init__(info, dataGroup, channelGroup, recordIDsize) | constructor |
channel_validity(channelName) | returns channel validity bit array |
extract channel validity bits
Parameters: | channelName : str
|
---|
apply linear conversion to data
Parameters: | vect : numpy 1D array
cc_val : mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |
Bases: dict
mdf file reader class from version 4.0 to 4.1
Attributes
fileName | (str) file name |
VersionNumber | (int) mdf file version number |
masterChannelList | (dict) Represents data structure: a key per master channel with corresponding value containing a list of channels One key or master channel represents then a data group having same sampling interval. |
multiProc | (bool) Flag to request channel conversion multi processed for performance improvement. One thread per data group. |
convertAfterRead | (bool) flag to convert raw data to physical just after read |
filterChannelNames | (bool) flag to filter long channel names from its module names separated by ‘.’ |
author | (str) |
organisation | (str) |
project | (str) |
subject | (str) |
comment | (str) |
time | (str) |
date | (str) |
Methods
read4( fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True) | Reads mdf 4.x file data and stores it in dict |
getChannelData4(channelName) | Returns channel numpy array |
convertChannel4(channelName) | converts specific channel from raw to physical data according to CCBlock information |
convertAllChannel4() | Converts all channels from raw data to converted data according to CCBlock information |
Converts all channels from raw data to converted data according to CCBlock information Converted data will take more memory.
converts specific channel from raw to physical data according to CCBlock information
Parameters: | channelName : str
|
---|
Returns channel numpy array
Parameters: | channelName : str
|
---|
Notes
This method is the safest to get channel data as numpy array from ‘data’ dict key might contain raw data
Reads mdf 4.x file data and stores it in dict
Parameters: | fileName : str, optional
info : mdfinfo4.info4 class
multiProc : bool
channelList : list of str, optional
convertAfterRead : bool, optional
|
---|
Put raw data from buf to a dict L and processes nested nBit channels
Parameters: | Q : multiprocessing.Queue, optional
buf : DATA class
info : info class
dataGroup : int
channelList : list of str, optional
multiProc : bool
|
---|---|
Returns: | Q : multiprocessing.Queue
L : dict
|
apply rational conversion to data
Parameters: | vect : numpy 1D array
cc_val : mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |
Bases: list
record class lists recordchannel classes, it is representing a channel group
Attributes
CGrecordLength | (int) length of record corresponding of channel group in Byte CG Block information |
recordLength | (int) length of record as understood by program based on C datatypes |
numberOfRecords | (int) number of records in data block |
recordID | (int) recordID corresponding to channel group |
recordIDsize | (int) size of recordID |
recordIDCFormat | (str) record identifier C format string as understood by fread |
dataGroup | (int:) data group number |
channelGroup | (int) channel group number |
numpyDataRecordFormat | (list) list of numpy (dtype) for each channel |
dataRecordName | (list) list of channel names used for recarray attribute definition |
master | (dict) define name and number of master channel |
recordToChannelMatching | (dict) helps to identify nested bits in byte |
channelNames | (list) channel names to be stored, useful for low memory consumption but slow |
Flags | (bool) channel flags as from specification |
VLSD_CG | (dict) dict of Channel Group VLSD, key being recordID |
VLSD | (list of recordChannel) list of recordChannel being VLSD |
MLSD | (dict) copy from info[‘MLSD’] if existing |
byte_aligned | (Bool) flag for byte aligned record |
invalid_channel | (Default None) invalid_byte class if existing in record otherwise None |
Methods
addChannel(info, channelNumber) | |
loadInfo(info) | |
readSortedRecord(fid, pointer, channelList=None) | |
readRecordBuf(buf, channelList=None) | |
readBitarray(bita, channelList=None) |
add a channel in class
Parameters: | info : mdfinfo4.info4 class channelNumber : int
|
---|
gathers records related from info class
Parameters: | info : mdfinfo4.info4 class |
---|
reads stream of record bytes using bitarray module needed for not byte aligned data
Parameters: | bitarray : stream
channelList : List of str, optional
|
---|---|
Returns: | rec : numpy recarray
|
read stream of record bytes
Parameters: | buf : stream
channelList : list of str, optional
|
---|---|
Returns: | rec : dict
|
reads record, only one channel group per datagroup Parameters —————- fid : float
file identifier
Returns: | rec : numpy recarray
|
---|
Notes
If channelList is None, read data using numpy.core.records.fromfile that is rather quick. However, in case of large file, you can use channelList to load only interesting channels or only one channel on demand, but be aware it might be much slower.
recordChannel class gathers all about channel structure in a record
Attributes
name | (str) Name of channel |
channelNumber | (int) channel number corresponding to mdfinfo3.info3 class |
signalDataType | (int) signal type according to specification |
bitCount | (int) number of bits used to store channel record |
nBytes | (int) number of bytes (1 byte = 8 bits) taken by channel record |
dataFormat | (str) numpy dtype as string |
Format : | C format understood by fread |
CFormat | (struct class instance) struct instance to convert from C Format |
byteOffset | (int) position of channel record in complete record in bytes |
bitOffset | (int) bit position of channel value inside byte in case of channel having bit count below 8 |
RecordFormat | (list of str) dtype format used for numpy.core.records functions ((name,name_title),str_stype) |
channelType | (int) channel type |
posByteBeg | (int) start position in number of byte of channel record in complete record |
posByteEnd | (int) end position in number of byte of channel record in complete record |
posBitBeg | (int) start position in number of bit of channel record in complete record |
posBitEnd | (int) end position in number of bit of channel record in complete record |
VLSD_CG_Flag | (bool) flag when Channel Group VLSD is used |
data | (int) pointer to data block linked to a channel (VLSD, MLSD) |
Methods
__init__(info, dataGroup, channelGroup, channelNumber, recordIDsize) | constructor |
__str__() | to print class attributes |
apply text to text conversion to data
Parameters: | vect : numpy 1D array
cc_ref : cc_ref from mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |
apply text to value conversion to data
Parameters: | vect : numpy 1D array
cc_val : cc_val from mdfinfo4.info4 conversion block (‘CCBlock’) dict cc_ref : cc_ref from mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |
apply value range to text conversion to data
Parameters: | vect : numpy 1D array
cc_val : cc_val from mdfinfo4.info4 conversion block (‘CCBlock’) dict cc_ref : cc_ref from mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |
apply value range to value table conversion to data
Parameters: | vect : numpy 1D array
cc_val : mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |
apply value to text conversion to data
Parameters: | vect : numpy 1D array
cc_val : cc_val from mdfinfo4.info4 conversion block (‘CCBlock’) dict cc_ref : cc_ref from mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |
apply value to value table with interpolation conversion to data
Parameters: | vect : numpy 1D array
cc_val : mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |
apply value to value table without interpolation conversion to data
Parameters: | vect : numpy 1D array
cc_val : mdfinfo4.info4 conversion block (‘CCBlock’) dict |
---|---|
Returns: | converted data to physical value |