PyDSDL usage¶
The entirety of the library API is exposed at the top level as pydsdl.*
.
There are no usable submodules.
You can find a practical usage example in the Nunavut code generation library that uses PyDSDL as the frontend.
The main function¶
-
pydsdl.
read_namespace
(root_namespace_directory: str, lookup_directories: Optional[Union[str, Iterable[str]]] = None, print_output_handler: Optional[Callable[[str, int, str], None]] = None, allow_unregulated_fixed_port_id: bool = False) → List[pydsdl._serializable._composite.CompositeType]¶ This function is the main entry point of the library. It reads all DSDL definitions from the specified root namespace directory and produces the annotated AST.
- Parameters
root_namespace_directory – The path of the root namespace directory that will be read. For example, “dsdl/uavcan” to read the “uavcan” namespace.
lookup_directories – List of other namespace directories containing data type definitions that are referred to from the target root namespace. For example, if you are reading a vendor-specific namespace, the list of lookup directories should always include a path to the standard root namespace
uavcan
, otherwise the types defined in the vendor-specific namespace won’t be able to use data types from the standard namespace.print_output_handler – If provided, this callable will be invoked when a
@print
directive is encountered or when the frontend needs to emit a diagnostic; the arguments are: path, line number (1-based), text. If not provided, no output will be produced except for the standard Python logging subsystem (but@print
expressions will be evaluated anyway, and a failed evaluation will be a fatal error).allow_unregulated_fixed_port_id – Do not reject unregulated fixed port identifiers. As demanded by the specification, the frontend rejects unregulated fixed port ID by default. This is a dangerous feature that must not be used unless you understand the risks. Please read https://uavcan.org/guide.
- Returns
A list of
pydsdl.CompositeType
sorted lexicographically by full data type name, then by major version (newest version first), then by minor version (newest version first). The ordering guarantee allows the caller to always find the newest version simply by picking the first matching occurrence.- Raises
pydsdl.FrontendError
,OSError
if directories do not exist or inaccessible;ValueError
/TypeError
if the arguments are invalid.
Type model¶
-
class
pydsdl.
Any
¶ Bases:
abc.ABC
This abstract class represents an arbitrary intrinsic DSDL expression value. Both serializable types and expression types derive from this common ancestor.
Per the DSDL data model, a serializable type is also a value. Serializable types have the suffix
Type
because their instances represent not DSDL values but DSDL types.-
TYPE_NAME
: str = None¶ The DSDL-name of the data type implemented by the class, as defined in Specification.
-
-
class
pydsdl.
Primitive
¶ Bases:
pydsdl._expression._any.Any
-
abstract property
native_value
¶ Yields an appropriate Python-native representation of the contained value, like
fractions.Fraction
,str
, etc. Specializations define covariant return types.
-
abstract property
-
class
pydsdl.
Boolean
(value: bool = False)¶ Bases:
pydsdl._expression._primitive.Primitive
-
TYPE_NAME
= 'bool'¶
-
property
native_value
¶
-
-
class
pydsdl.
Rational
(value: Union[int, fractions.Fraction])¶ Bases:
pydsdl._expression._primitive.Primitive
-
TYPE_NAME
= 'rational'¶
-
__init__
(value: Union[int, fractions.Fraction])¶
-
property
native_value
¶
-
-
class
pydsdl.
String
(value: str)¶ Bases:
pydsdl._expression._primitive.Primitive
-
TYPE_NAME
= 'string'¶
-
property
native_value
¶
-
-
class
pydsdl.
Container
¶ Bases:
pydsdl._expression._any.Any
-
abstract property
element_type
¶
-
abstract
__iter__
() → Iterator[Any]¶
-
abstract property
-
class
pydsdl.
Set
(elements: Iterable[pydsdl._expression._any.Any])¶ Bases:
pydsdl._expression._container.Container
-
TYPE_NAME
= 'set'¶
-
__init__
(elements: Iterable[pydsdl._expression._any.Any])¶
-
__iter__
() → Iterator[Any]¶
-
property
element_type
¶
-
-
class
pydsdl.
SerializableType
¶ Bases:
pydsdl._expression._any.Any
Instances are immutable. Invoking
__str__()
on a data type returns its uniform normalized definition, e.g.,uavcan.node.Heartbeat.1.0[<=36]
,truncated float16[<=36]
.-
TYPE_NAME
= 'metaserializable'¶
-
BITS_PER_BYTE
= 8¶ This is dictated by the UAVCAN Specification.
-
abstract property
bit_length_set
¶ A set of all possible bit length values of the serialized representations of this type. Refer to the specification for the background. The returned set is guaranteed to be non-empty. See
pydsdl.BitLengthSet
.
-
abstract property
alignment_requirement
¶ Serialized representations of this type are required/guaranteed to be aligned such that their offset from the beginning of the containing serialized representation, in bits, is a multiple of this value, in bits. Alignment of a type whose alignment requirement is X bits is facilitated by injecting
[0, X)
zero padding bits before the serialized representation of the type.For any element
L
of the bit length set of a type whose alignment requirement isA
,L % A = 0
. I.e., the length of a serialized representation of the type is always a multiple of its alignment requirement.This value is always a non-negative integer power of two. The alignment of one is a degenerate case denoting no alignment.
-
-
class
pydsdl.
PrimitiveType
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶ Bases:
pydsdl._serializable._serializable.SerializableType
-
MAX_BIT_LENGTH
= 64¶
-
BITS_IN_BYTE
= 8¶
-
__init__
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶
-
property
bit_length_set
¶
-
property
bit_length
¶ This is a shortcut for
next(iter(x.bit_length_set))
, because the bit length set of a primitive type always contains exactly one element (i.e., primitive types are fixed-length).
-
property
standard_bit_length
¶ The term “standard length” here means that values of such bit length are commonly used in modern computer microarchitectures, such as
uint8
,float64
,int32
, and so on. Booleans are excluded. More precisely, a primitive is said to be “standard length” when the following hold:bit_length >= 8 2**ceil(log2(bit_length)) == bit_length.
-
property
cast_mode
¶
-
property
alignment_requirement
¶
-
-
class
pydsdl.
BooleanType
(cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶ Bases:
pydsdl._serializable._primitive.PrimitiveType
-
__init__
(cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶
-
-
class
pydsdl.
ArithmeticType
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶ Bases:
pydsdl._serializable._primitive.PrimitiveType
-
__init__
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶
-
abstract property
inclusive_value_range
¶
-
-
class
pydsdl.
IntegerType
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶ Bases:
pydsdl._serializable._primitive.ArithmeticType
-
__init__
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶
-
abstract property
inclusive_value_range
¶
-
-
class
pydsdl.
SignedIntegerType
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶ Bases:
pydsdl._serializable._primitive.IntegerType
-
__init__
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶
-
property
inclusive_value_range
¶
-
-
class
pydsdl.
UnsignedIntegerType
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶ Bases:
pydsdl._serializable._primitive.IntegerType
-
__init__
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶
-
property
inclusive_value_range
¶
-
-
class
pydsdl.
FloatType
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶ Bases:
pydsdl._serializable._primitive.ArithmeticType
-
__init__
(bit_length: int, cast_mode: pydsdl._serializable._primitive.PrimitiveType.CastMode)¶
-
property
inclusive_value_range
¶
-
-
class
pydsdl.
VoidType
(bit_length: int)¶ Bases:
pydsdl._serializable._serializable.SerializableType
-
MAX_BIT_LENGTH
= 64¶
-
property
bit_length_set
¶
-
property
bit_length
¶ This is a shortcut for
next(iter(x.bit_length_set))
, because the bit length set of a void type always contains exactly one element (i.e., void types are fixed-length).
-
property
alignment_requirement
¶
-
-
class
pydsdl.
ArrayType
(element_type: pydsdl._serializable._serializable.SerializableType, capacity: int)¶ Bases:
pydsdl._serializable._serializable.SerializableType
-
__init__
(element_type: pydsdl._serializable._serializable.SerializableType, capacity: int)¶
-
property
element_type
¶
-
property
capacity
¶ The (maximum) number of elements in the (variable-length) array.
-
property
string_like
¶ True if the array might contain a text string, in which case it is termed to be “string-like”. A string-like array is a variable-length array of
uint8
. See https://github.com/UAVCAN/specification/issues/51.
-
property
alignment_requirement
¶ The alignment requirement of an array equals that of its element type. The length of the serialized representation of any type is a multiple of its alignment requirement; therefore, every element is always placed such that its alignment requirement is satisfied.
-
-
class
pydsdl.
FixedLengthArrayType
(element_type: pydsdl._serializable._serializable.SerializableType, capacity: int)¶ Bases:
pydsdl._serializable._array.ArrayType
-
__init__
(element_type: pydsdl._serializable._serializable.SerializableType, capacity: int)¶
-
property
bit_length_set
¶
-
enumerate_elements_with_offsets
(base_offset: Optional[pydsdl._bit_length_set.BitLengthSet] = None) → Iterator[Tuple[int, pydsdl._bit_length_set.BitLengthSet]]¶ This is a convenience method for code generation. Its behavior mimics that of
pydsdl.StructureType.iterate_fields_with_offsets()
, except that we iterate over indexes instead of fields.- Parameters
base_offset – The base offset to add to each element. If not supplied, assumed to be
{0}
. The base offset will be implicitly padded out toalignment_requirement
.- Returns
For an N-element array, an iterator over N elements, where each element is a tuple of the index of the array element (zero-based) and its offset as a bit length set.
-
-
class
pydsdl.
VariableLengthArrayType
(element_type: pydsdl._serializable._serializable.SerializableType, capacity: int)¶ Bases:
pydsdl._serializable._array.ArrayType
-
__init__
(element_type: pydsdl._serializable._serializable.SerializableType, capacity: int)¶
-
property
bit_length_set
¶
-
property
string_like
¶ See the base class.
-
property
length_field_type
¶ The unsigned integer type of the implicit array length field. Note that the set of valid length values is a subset of that of the returned type.
-
-
class
pydsdl.
CompositeType
(name: str, version: pydsdl._serializable._composite.Version, attributes: Iterable[pydsdl._serializable._attribute.Attribute], deprecated: bool, fixed_port_id: Optional[int], source_file_path: str, parent_service: Optional[ServiceType] = None)¶ Bases:
pydsdl._serializable._serializable.SerializableType
This is the most interesting type in the library because it represents an actual DSDL definition upon its interpretation. This is an abstract class with several specializations.
-
MAX_NAME_LENGTH
= 50¶
-
MAX_VERSION_NUMBER
= 255¶
-
NAME_COMPONENT_SEPARATOR
= '.'¶
-
__init__
(name: str, version: pydsdl._serializable._composite.Version, attributes: Iterable[pydsdl._serializable._attribute.Attribute], deprecated: bool, fixed_port_id: Optional[int], source_file_path: str, parent_service: Optional[ServiceType] = None)¶
-
property
full_name
¶ The full name, e.g.,
uavcan.node.Heartbeat
.
-
property
name_components
¶ Components of the full name as a list, e.g.,
['uavcan', 'node', 'Heartbeat']
.
-
property
short_name
¶ The last component of the full name, e.g.,
Heartbeat
ofuavcan.node.Heartbeat
.
-
property
full_namespace
¶ The full name without the short name, e.g.,
uavcan.node
foruavcan.node.Heartbeat
.
-
property
root_namespace
¶ The first component of the full name, e.g.,
uavcan
ofuavcan.node.Heartbeat
.
-
property
version
¶ The version numbers of the type, e.g.,
(1, 0)
ofuavcan.node.Heartbeat.1.0
.
-
property
extent
¶ The amount of memory, in bits, that needs to be allocated in order to store a serialized representation of this type or any of its minor versions under the same major version. This value is always at least as large as the sum of maximum bit lengths of all fields padded to one byte. If the type is final, its extent equals
max(bit_length_set)
.
-
property
bit_length_set
¶ The bit length set of a composite is always aligned at
alignment_requirement
. For a final type this is the true bit length set computed by aggregating the fields and padding the result toalignment_requirement
. That is, final types expose their internal structure; for example, a type that contains a single field of typeuint32[2]
would have a single entry in the bit length set:{64}
.
-
property
deprecated
¶ Whether the definition is marked
@deprecated
.
-
property
attributes
¶
-
property
fields
¶
-
property
fields_except_padding
¶
-
property
constants
¶
-
property
fixed_port_id
¶
-
property
has_fixed_port_id
¶
-
property
source_file_path
¶ For synthesized types such as service request/response sections, this property is defined as an empty string.
-
property
alignment_requirement
¶
-
property
parent_service
¶ pydsdl.ServiceType
contains two special fields of this type:request
andresponse
. For them this property points to the parent service instance; otherwise it’s None.
-
abstract
iterate_fields_with_offsets
(base_offset: Optional[pydsdl._bit_length_set.BitLengthSet] = None) → Iterator[Tuple[pydsdl._serializable._attribute.Field, pydsdl._bit_length_set.BitLengthSet]]¶ Iterates over every field (not attribute – constants are excluded) of the data type, yielding it together with its offset, where the offset is represented as
pydsdl.BitLengthSet
. The offset of each field is added to the base offset, which may be specified by the caller; if not specified, the base offset is assumed to be{0}
.The objective of this method is to allow code generators to easily implement fully unrolled serialization and deserialization routines, where “unrolled” means that upon encountering another (nested) composite type, the serialization routine would not delegate its serialization to the serialization routine of the encountered type, but instead would serialize it in-place, as if the field of that type was replaced with its own fields in-place. The lack of delegation has very important performance implications: when the serialization routine does not delegate serialization of the nested types, it can perform infinitely deep field alignment analysis, thus being able to reliably statically determine whether each field of the type, including nested types at arbitrarily deep levels of nesting, is aligned relative to the origin of the serialized representation of the outermost type. As a result, the code generator will be able to avoid unnecessary reliance on slow bit-level copy routines replacing them instead with much faster byte-level copy (like
memcpy()
) or even plain memory aliasing.When invoked on a tagged union type, the method yields the same offset for every field (since that’s how tagged unions are serialized), where the offset equals the bit length of the implicit union tag (plus the base offset, of course, if provided).
Please refer to the usage examples to see how this feature can be used.
- Parameters
base_offset – Assume the specified base offset; assume zero offset if the parameter is not provided. The base offset will be implicitly padded out to
alignment_requirement
.- Returns
A generator of
(Field, BitLengthSet)
. Each instance ofpydsdl.BitLengthSet
yielded by the generator is a dedicated copy, meaning that the consumer can mutate the returned instances arbitrarily without affecting future values. It is guaranteed that each yielded instance is non-empty.
-
__getitem__
(attribute_name: str) → pydsdl._serializable._attribute.Attribute¶ Allows the caller to retrieve an attribute by name. Padding fields are not accessible via this interface because they don’t have names. Raises
KeyError
if there is no such attribute.
-
-
class
pydsdl.
UnionType
(name: str, version: pydsdl._serializable._composite.Version, attributes: Iterable[pydsdl._serializable._attribute.Attribute], deprecated: bool, fixed_port_id: Optional[int], source_file_path: str, parent_service: Optional[ServiceType] = None)¶ Bases:
pydsdl._serializable._composite.CompositeType
A message type that is marked
@union
.-
MIN_NUMBER_OF_VARIANTS
= 2¶
-
__init__
(name: str, version: pydsdl._serializable._composite.Version, attributes: Iterable[pydsdl._serializable._attribute.Attribute], deprecated: bool, fixed_port_id: Optional[int], source_file_path: str, parent_service: Optional[ServiceType] = None)¶
-
property
bit_length_set
¶
-
property
number_of_variants
¶
-
property
tag_field_type
¶ The unsigned integer type of the implicit union tag field. Note that the set of valid tag values is a subset of that of the returned type.
-
iterate_fields_with_offsets
(base_offset: Optional[pydsdl._bit_length_set.BitLengthSet] = None) → Iterator[Tuple[pydsdl._serializable._attribute.Field, pydsdl._bit_length_set.BitLengthSet]]¶ See the base class.
-
static
aggregate_bit_length_sets
(field_types: Sequence[pydsdl._serializable._serializable.SerializableType]) → pydsdl.BitLengthSet¶ Computes the bit length set for a tagged union type given the type of each of its variants. The final padding is not applied.
Unions are easy to handle because when serialized, a union is essentially just a single field prefixed with a fixed-length integer tag. So we just build a full set of combinations and then add the tag length to each element.
Observe that unions are not defined for less than 2 elements; however, this function tries to be generic by properly handling those cases as well, even though they are not permitted by the specification. For zero fields, the function yields
{0}
; for one field, the function yields the BLS of the field itself.
-
-
class
pydsdl.
StructureType
(name: str, version: pydsdl._serializable._composite.Version, attributes: Iterable[pydsdl._serializable._attribute.Attribute], deprecated: bool, fixed_port_id: Optional[int], source_file_path: str, parent_service: Optional[ServiceType] = None)¶ Bases:
pydsdl._serializable._composite.CompositeType
A message type that is NOT marked
@union
.-
iterate_fields_with_offsets
(base_offset: Optional[pydsdl._bit_length_set.BitLengthSet] = None) → Iterator[Tuple[pydsdl._serializable._attribute.Field, pydsdl._bit_length_set.BitLengthSet]]¶ See the base class.
-
property
bit_length_set
¶
-
static
aggregate_bit_length_sets
(field_types: Sequence[pydsdl._serializable._serializable.SerializableType]) → pydsdl.BitLengthSet¶ Computes the bit length set for a structure type given the type of each of its fields. The final padding is not applied.
-
-
class
pydsdl.
DelimitedType
(inner: pydsdl._serializable._composite.CompositeType, extent: Optional[int])¶ Bases:
pydsdl._serializable._composite.CompositeType
Composites that are not final are wrapped into this container. It is a decorator over a composite type instance that injects the extent, bit length set, and field iteration logic that is specific to delimited (appendable, non-final) types.
Most of the attributes are copied from the wrapped type (e.g., name, fixed port-ID, attributes, etc.), except for those that relate to the bit layout.
Non-final composites are serialized into delimited opaque containers like
uint8[<=(extent + 7) // 8]
, where the implicit length prefix is of typedelimiter_header_type
. Their bit length set is also computed as if it was an array as declared above, in order to prevent the containing definitions from making assumptions about the offsets of the following fields that might not survive the evolution of the type (e.g., version 1 may be 64 bits long, version 2 might be 56 bits long, then version 3 could grow to 96 bits, unpredictable).-
DEFAULT_EXTENT_MULTIPLIER
= Fraction(3, 2)¶ If the extent is not specified explicitly, it is computed by multiplying the extent of the inner type by this.
-
__init__
(inner: pydsdl._serializable._composite.CompositeType, extent: Optional[int])¶
-
property
inner_type
¶ The appendable type that is serialized inside this delimited container. Its bit length set, extent, and other layout-specific entities are computed as if it was a final type.
-
property
extent
¶ The extent of a delimited type can be specified explicitly via
@extent
(provided that it is not less than the minimum); otherwise, it defaults tofloor(inner_type.extent * 3/2)
padded to byte.Optional optimization hint: if the objective is to allocate buffer memory for constructing a new serialized representation locally, then it may be beneficial to use the extent of the inner type rather than this one because it may be smaller. This is not safe for deserialization, of course.
-
property
bit_length_set
¶ For a non-final type, not many guarantees about the bit length set can be provided, because the type may be mutated in the next minor revision. Therefore, a synthetic bit length set is constructed that is merely a list of all possible bit lengths plus the delimiter header. For example, a type that contains a single field of type
uint32[2]
would have the bit length set of{h, h+8, h+16, ..., h+56, h+64}
whereh
is the length of the delimiter header.
-
property
delimiter_header_type
¶ The type of the integer prefix field that encodes the size of the serialized representation [in bytes] of the
inner_type
.
-
iterate_fields_with_offsets
(base_offset: Optional[pydsdl._bit_length_set.BitLengthSet] = None) → Iterator[Tuple[pydsdl._serializable._attribute.Field, pydsdl._bit_length_set.BitLengthSet]]¶ Delegates the call to the inner type, but with the base offset increased by the size of the delimiter header.
-
-
class
pydsdl.
ServiceType
(name: str, version: pydsdl._serializable._composite.Version, request_params: pydsdl._serializable._composite.ServiceType.SchemaParams, response_params: pydsdl._serializable._composite.ServiceType.SchemaParams, deprecated: bool, fixed_port_id: Optional[int], source_file_path: str)¶ Bases:
pydsdl._serializable._composite.CompositeType
A service (not message) type. Unlike message types, it can’t be serialized directly.
There are exactly two pseudo-fields:
request
andresponse
, which contain the request and the response structure of the service type, respectively.-
class
SchemaParams
(attributes: Iterable[pydsdl._serializable._attribute.Attribute], extent: Optional[int], is_final: bool, is_union: bool)¶ Bases:
object
A trivial helper dataclass used for constructing new instances.
-
__init__
(attributes: Iterable[pydsdl._serializable._attribute.Attribute], extent: Optional[int], is_final: bool, is_union: bool)¶
-
construct_composite
(name: str, version: pydsdl._serializable._composite.Version, deprecated: bool, parent_service: pydsdl._serializable._composite.ServiceType) → pydsdl._serializable._composite.CompositeType¶
-
-
__init__
(name: str, version: pydsdl._serializable._composite.Version, request_params: pydsdl._serializable._composite.ServiceType.SchemaParams, response_params: pydsdl._serializable._composite.ServiceType.SchemaParams, deprecated: bool, fixed_port_id: Optional[int], source_file_path: str)¶
-
property
bit_length_set
¶
-
property
request_type
¶ The type of the request schema.
-
property
response_type
¶ The type of the response schema.
-
iterate_fields_with_offsets
(base_offset: Optional[pydsdl._bit_length_set.BitLengthSet] = None) → Iterator[Tuple[pydsdl._serializable._attribute.Field, pydsdl._bit_length_set.BitLengthSet]]¶ Always raises a
TypeError
.
-
class
-
class
pydsdl.
Attribute
(data_type: pydsdl._serializable._serializable.SerializableType, name: str)¶ Bases:
pydsdl._expression._any.Any
-
__init__
(data_type: pydsdl._serializable._serializable.SerializableType, name: str)¶
-
property
data_type
¶
-
property
name
¶ For padding fields this is an empty string.
-
-
class
pydsdl.
Field
(data_type: pydsdl._serializable._serializable.SerializableType, name: str)¶
-
class
pydsdl.
PaddingField
(data_type: pydsdl._serializable._void.VoidType)¶ Bases:
pydsdl._serializable._attribute.Field
-
__init__
(data_type: pydsdl._serializable._void.VoidType)¶
-
-
class
pydsdl.
Constant
(data_type: pydsdl._serializable._serializable.SerializableType, name: str, value: pydsdl._expression._any.Any)¶ Bases:
pydsdl._serializable._attribute.Attribute
-
__init__
(data_type: pydsdl._serializable._serializable.SerializableType, name: str, value: pydsdl._expression._any.Any)¶
-
property
value
¶ The result of evaluating the constant initialization expression. The value is guaranteed to be compliant with the constant’s own type – it is checked at the evaluation time. The compliance rules are defined in the Specification.
-
Exceptions¶
-
exception
pydsdl.
FrontendError
(text: str, path: Optional[str] = None, line: Optional[int] = None)¶ Bases:
Exception
This is the root exception type for all custom exceptions defined in the library. This type itself is not expected to be particularly useful to the library user; please refer to the direct descendants instead.
-
set_error_location_if_unknown
(path: Optional[str] = None, line: Optional[int] = None) → None¶ Entries that are already known will be left unchanged. This is useful when propagating exceptions through recursive instances, e.g., when processing nested definitions.
-
property
path
¶ Source file path where the error has occurred, if known.
-
property
line
¶ Source file line number (first line numbered 1) where the error has occurred, if known. The path is always known if the line number is set.
-
property
text
¶
-
-
exception
pydsdl.
InvalidDefinitionError
(text: str, path: Optional[str] = None, line: Optional[int] = None)¶ Bases:
pydsdl.FrontendError
This exception type is used to point out mistakes and errors in DSDL definitions. This type is inherited by a dozen of specialized exception types; however, the class hierarchy beneath this type may be unstable and should not be relied upon by the application directly.
-
exception
pydsdl.
InternalError
(text: Optional[str] = None, path: Optional[str] = None, line: Optional[int] = None, culprit: Optional[Exception] = None)¶ Bases:
pydsdl.FrontendError
This exception is used to report internal errors in the front end itself that prevented it from processing the definitions. Every occurrence should be reported to the developers.
Ancillary members¶
-
class
pydsdl.
BitLengthSet
(values: Optional[Union[Iterable[int], int]] = None)¶ Bases:
object
This type represents the Bit Length Set as defined in the Specification. It is used for representing bit offsets of fields and bit lengths of serialized representations.
Instances are comparable between each other, with plain integers, and with native sets of integers. The methods do not mutate the instance they are invoked on; instead, the result is returned as a new instance, excepting the in-place
__ixx__()
operator overloads.-
__init__
(values: Optional[Union[Iterable[int], int]] = None)¶ Accepts any iterable that yields integers (like another bit length set) or a single integer, in which case it will result in the set containing only the one specified integer. The source container is always deep-copied.
>>> BitLengthSet() BitLengthSet() >>> len(BitLengthSet()) == 0 True >>> BitLengthSet(1) BitLengthSet({1}) >>> BitLengthSet({1, 2, 3}) BitLengthSet({1, 2, 3})
-
is_aligned_at
(bit_length: int) → bool¶ Checks whether all of the contained offset values match the specified alignment goal. An empty bit length set is considered to have infinite alignment.
>>> BitLengthSet(64).is_aligned_at(32) True >>> BitLengthSet(48).is_aligned_at(32) False >>> BitLengthSet(48).is_aligned_at(16) True >>> BitLengthSet().is_aligned_at(123456) True
-
is_aligned_at_byte
() → bool¶ A shorthand for
is_aligned_at()
using the standard byte size as prescribed by the Specification.>>> BitLengthSet(32).is_aligned_at_byte() True >>> BitLengthSet(33).is_aligned_at_byte() False
-
pad_to_alignment
(bit_length: int) → pydsdl.BitLengthSet¶ Pad each element in the set such that the set becomes aligned at the specified alignment goal. After this transformation is applied, elements may become up to
bit_length-1
bits larger. The argument shall be a positive integer, otherwise it’s aValueError
.>>> BitLengthSet({0, 1, 2, 3, 4, 5, 6, 7, 8}).pad_to_alignment(1) # Alignment to 1 is a no-op. BitLengthSet({0, 1, 2, 3, 4, 5, 6, 7, 8}) >>> BitLengthSet({0, 1, 2, 3, 4, 5, 6, 7, 8}).pad_to_alignment(2) BitLengthSet({0, 2, 4, 6, 8}) >>> BitLengthSet({0, 1, 5, 7}).pad_to_alignment(2) BitLengthSet({0, 2, 6, 8}) >>> BitLengthSet({0, 1, 2, 3, 4, 5, 6, 7, 8}).pad_to_alignment(3) BitLengthSet({0, 3, 6, 9}) >>> BitLengthSet({0, 1, 2, 3, 4, 5, 6, 7, 8}).pad_to_alignment(8) BitLengthSet({0, 8}) >>> BitLengthSet({0, 9}).pad_to_alignment(8) BitLengthSet({0, 16}) >>> from random import randint >>> alignment = randint(1, 64) >>> BitLengthSet(randint(1, 1000) for _ in range(100)).pad_to_alignment(alignment).is_aligned_at(alignment) True
-
elementwise_sum_k_multicombinations
(k: int) → pydsdl.BitLengthSet¶ This is a special case of
elementwise_sum_cartesian_product()
. The original object is not modified.One can replace this method with the aforementioned general case method and the behavior would not change; however, we need this special case method for performance reasons. When dealing with arrays (either fixed- or variable-length), usage of this method instead of the generic one yields significantly better performance, since the computational complexity of k-selections is much lower than that of the Cartesian product.
>>> BitLengthSet(1).elementwise_sum_k_multicombinations(1) BitLengthSet({1}) >>> BitLengthSet({1, 2, 3}).elementwise_sum_k_multicombinations(1) BitLengthSet({1, 2, 3}) >>> BitLengthSet({1, 2, 3}).elementwise_sum_k_multicombinations(2) BitLengthSet({2, 3, 4, 5, 6})
-
static
elementwise_sum_cartesian_product
(sets: Iterable[Union[Iterable[int], int]]) → pydsdl.BitLengthSet¶ This operation is fundamental for bit length and bit offset (which are, generally, the same thing) computation.
The basic background is explained in the specification. The idea is that the bit offset of a given entity in a data type definition of the structure category (or, in other words, the bit length set of serialized representations of the preceding entities, which is the same thing, assuming that the data type is of the structure category) is a function of bit length sets of each preceding entity. Various combinations of bit lengths of the preceding entities are possible, which can be expressed through the Cartesian product over the bit length sets of the preceding entities. Since in a type of the structure category entities are arranged as an ordered sequence of a fixed length (meaning that entities can’t be added or removed), the resulting bit length (offset) is computed by elementwise summation of each element of the Cartesian product.
This method is not applicable for the tagged union type category, since a tagged union holds exactly one value at any moment; therefore, the bit length set of a tagged union is simply a union of bit length sets of each entity that can be contained in the union, plus the length of the implicit union tag field.
From the standpoint of bit length combination analysis, fixed-length arrays are a special case of structures, because they also contain a fixed ordered sequence of fields, where all fields are of the same type. The method defined for structures applies to fixed-length arrays, but one should be aware that it may be computationally suboptimal, since the fact that all array elements are of the same type allows us to replace the computationally expensive Cartesian product with k-multicombinations (k-selections).
In the context of bit length analysis, variable-length arrays do not require any special treatment, since a variable-length array with the capacity of N elements can be modeled as a tagged union containing N fixed arrays of length from 1 to N, plus one empty field (representing the case of an empty variable-length array).
>>> BitLengthSet.elementwise_sum_cartesian_product([1, 2, 10]) BitLengthSet({13}) >>> BitLengthSet.elementwise_sum_cartesian_product([{1, 2}, {4, 5}]) BitLengthSet({5, 6, 7}) >>> BitLengthSet.elementwise_sum_cartesian_product([{1, 2, 3}, {4, 5, 6}]) BitLengthSet({5, 6, 7, 8, 9}) >>> BitLengthSet.elementwise_sum_cartesian_product([{1, 2, 3}, {4, 5, 6}, {7, 8, 9}]) BitLengthSet({12, 13, 14, 15, 16, 17, 18})
-
__eq__
(other: Any) → bool¶ Whether the current set equals the other. The other may be a bit length set, an integer, or a native
typing.Set[int]
.
-
__bool__
() → bool¶ Evaluates to True unless empty.
>>> assert not BitLengthSet() >>> assert not BitLengthSet({}) >>> assert BitLengthSet(0) >>> assert BitLengthSet({1, 2, 3})
-
__add__
(other: Any) → pydsdl.BitLengthSet¶ This operation models the addition of a new object to a serialized representation; i.e., it is an alias for
elementwise_sum_cartesian_product([self, other])
. The result is stored into a new instance which is returned.If the argument is a bit length set, an elementwise sum set of the Cartesian product of the argument set with the current set will be computed, and the result will be returned as a new set (self is not modified). One can easily see that if the argument is a set of one value (or a scalar), this method will result in the addition of said scalar to every element of the original set.
SPECIAL CASE: if the current set is empty at the time of invocation, it will be assumed to be equal
{0}
.The other may be a bit length set, an integer, or a native
typing.Set[int]
.>>> BitLengthSet() + BitLengthSet() BitLengthSet() >>> BitLengthSet(4) + BitLengthSet(3) BitLengthSet({7}) >>> BitLengthSet({4, 91}) + 3 BitLengthSet({7, 94}) >>> BitLengthSet({4, 91}) + {5, 7} BitLengthSet({9, 11, 96, 98})
-
__radd__
(other: Any) → pydsdl.BitLengthSet¶ See
__add__()
.>>> {1, 2, 3} + BitLengthSet({4, 5, 6}) BitLengthSet({5, 6, 7, 8, 9}) >>> 1 + BitLengthSet({2, 5, 7}) BitLengthSet({3, 6, 8})
-
__iadd__
(other: Any) → pydsdl.BitLengthSet¶ See
__add__()
.>>> a = BitLengthSet({1, 2, 3}) >>> a += {4, 5, 6} >>> a BitLengthSet({5, 6, 7, 8, 9})
-
__or__
(other: Any) → pydsdl.BitLengthSet¶ Creates and returns a new set that is a union of this set with another bit length set.
>>> a = BitLengthSet() >>> a = a | BitLengthSet({1, 2, 3}) >>> a BitLengthSet({1, 2, 3}) >>> a = a | {3, 4, 5} >>> a BitLengthSet({1, 2, 3, 4, 5}) >>> a | 6 BitLengthSet({1, 2, 3, 4, 5, 6})
-
__ror__
(other: Any) → pydsdl.BitLengthSet¶ See
__or__()
.>>> {1, 2, 3} | BitLengthSet({4, 5, 6}) BitLengthSet({1, 2, 3, 4, 5, 6}) >>> 1 | BitLengthSet({2, 5, 7}) BitLengthSet({1, 2, 5, 7})
-
__ior__
(other: Any) → pydsdl.BitLengthSet¶ See
__or__()
.>>> a = BitLengthSet({4, 5, 6}) >>> a |= {1, 2, 3} >>> a BitLengthSet({1, 2, 3, 4, 5, 6})
-
__str__
() → str¶ Always yields a sorted representation for the ease of human consumption.
>>> str(BitLengthSet()) '{}' >>> str(BitLengthSet({918, 16, 7, 42})) '{7, 16, 42, 918}'
-
__repr__
() → str¶ >>> BitLengthSet() BitLengthSet() >>> BitLengthSet({918, 16, 7, 42}) BitLengthSet({7, 16, 42, 918})
-
__hash__
= None¶
-