.. _pkt: :mod:`baldaquin.pkt` --- Binary packets ======================================= This module contains all the facilities to deal with binary packet data---by `packet` we mean one piece one unit of binary data, and a packet can typically be imagined as the elementary unit of information outputted by the hardware that is seen from the DAQ side, containing one (in the simplest case) or more events. The module provides the :class:`AbstractPacket ` abstract class as a base for all the packet classes. Subclass should implement the following interfaces * a ``data`` property, returning the underlying binary buffer (typically a ``bytes`` object); * a ``fields`` property, i.e., a tuple of string with the names of all the fields that have to be extracted from the data when a class instance is unpacked; * the ``__len__()`` dunder method, returning the size of the data in bytes; * the ``__iter__()`` dunder method, that makes the class iterable; * a ``pack()`` method, packing all the fields into the corresponding data; * an ``unpack()`` method, unpacking the data into the corresponding fields, with the understanding that ``pack()`` and ``unpack()`` should be guardanteed to roundtrip. From a DAQ standpoint, the main use of concrete packet classes should be something along the lines of >>> packet = Packet.unpack(data) That is: you have a piece of binary data from the hardware, you know the layout of the packet, you can unpack it in the form a useful data structure that is easy to work with, plot, write to file, and alike. Being able to go the other way around (i.e., initialize a packet from its fields) is useful from a testing standpoint, and that is the very reason for provinding the ``pack()`` interface, that does things in this direction. .. warning:: We have not put much thought, yet, into support for variable-size packets, and the interfaces might change as we actually implement and use them. At this time the user should feel comfortable in using the :class:`FixedSizePacketBasePacket ` base class and the associated :meth:`packetclass ` decorator. In addition, the :class:`AbstractPacket ` provides placeholders for helping redirecting packet buffers to text sink. More specifically: * ``text_header()`` is meant to return a sensible header for a text output file containing packets; * ``to_text()`` is meant to provide a sensible text representation of the single packet, appropriate to write the packet to disk. Fixed-size packets ------------------ In its simplest incarnation, a packet is just a simple set of number packed in binary format with a well-defined layout and with a fixed size. This module provides the :meth:`packetclass ` decorator and the :class:`FixedSizePacketBasePacket ` base class to define concrete fixed-size packet structures. The :meth:`packetclass ` decorator is loosely inspired by the Python ``dataclass`` decorator, and what it does is essentially providing a class constructor based on class annotations. The basic contract is that for any annotation in the form of >>> field_name: Format a new attribute with the given ``field_name`` is added to the class, with the :class:`Format ` specifying the type of the field in the packet layout, according to the rules in the Python `struct `_. If the format charater is not supported, a ``ValueError`` is raised. Additionally, if a value is provided to the class annotation >>> field_name: format_char = value the value of the corresponding attribute is checked at runtime, and a :class:`FieldMismatchError ` exception is raised if the two do not match. (This is useful, e.g., when a packet has a fixed header that need to be checked within the event loop.) Finally, a ``layout`` class attribute can be optionally specified to control the byte order, size and alignment of the packet, according to the :class:`Layout ` enum. If no layout is specified, ``@`` (native order and size) is assumed. If the layout character is not supported a ``ValueError`` is raised. The :class:`FixedSizePacketBasePacket ` base class complement the decorator and implements the protocol defined by the :class:`AbstractPacket ` abstract class. For instance, the following snippet .. code-block:: @packetclass class Trigger(FixedSizePacketBase): layout = Layout.BIG_ENDIAN header: Format.UNSIGNED_CHAR = 0xff pin_number: Format.UNSIGNED_CHAR timestamp: Format.UNSIGNED_LONG_LONG defines a fully fledged packet class with three fields (big endian, standard size), where the header is required to be ``0xff`` (this is automatically checked at runtime) and that can be used as advertised: >>> packet = Trigger(0xff, 1, 15426782) >>> print(packet) >>> Trigger(header=255, pin_number=1, timestamp=15426782, >>> data=b'\xff\x01\x00\x00\x00\x00\x00\xebd\xde', _format=>BBQ) >>> print(len(packet)) >>> 10 >>> print(isinstance(packet, AbstractPacket)) >>> True (you will notice that when you create a packet from the constructor, the binary representation is automatically calculated using the ``pack()`` interface). And, of course, in real life (as opposed to unit-testing) you will almost always find yourself unpacking things, i.e., >>> packet = Trigger.unpack(b'\xff\x01\x00\x00\x00\x00\x00\xebd\xde') >>> print(packet) >>> Trigger(header=255, pin_number=1, timestamp=15426782, >>> data=b'\xff\x01\x00\x00\x00\x00\x00\xebd\xde', _format=>BBQ) (i.e., you have binary data from your hardware, and you can seamlessly turned into a useful data structure that you can interact with.) Packet objects defined in this way are as frozen as Python allows---you can't modify the values of the basic underlying fields once an instance has been created >>> packet.pin_number = 0 >>> AttributeError: Cannot modify Trigger.pin_number' and this is done with the goal of preserving the correspondence between the binary paylod and the unpacked field values at runtime. You can define new fields, though, and the ``AbstractPacket`` protocol, just as plain Python ``dataclasses``, provides a ``__post_init__()`` hook which is called at the end of the constructor (and is doing nothing by default). This is useful, e.g., for converting digitized values into the corresponding physical values. Say, for instance, that the ``timestamp`` in our simple ``Trigger`` class is the the number of microseconds since the last reset latched with an onboard counter, and we want to convert them to seconds. This can be achieved by something along the lines of .. code-block:: @packetclass class Trigger(FixedSizePacketBase): layout = Layout.BIG_ENDIAN header: Format.UNSIGNED_CHAR = 0xff pin_number: Format.UNSIGNED_CHAR microseconds: Format.UNSIGNED_LONG_LONG def __post_init__(self): self.seconds = self.microseconds / 1000000 with the understanding that >>> packet = Trigger(0xff, 1, 15426782) >>> print(packet.seconds) >>> 15.426782 Note the :class:`FixedSizePacketBasePacket ` base class provides a sensible implementation of the :meth:`text_header() ` and :meth:`to_text() ` hooks, although in practical situations one is often better off re-implementing them for the specific application at hand. Reading packet files ~~~~~~~~~~~~~~~~~~~~ In order to ease the packet I/O, the module provides a :class:`PacketFile ` class to interface with binary files containing packets. The :meth:`open() ` method supports the context manager protocol, and the class itself supports the iterator protocol. The basic use semantics is >>> with PacketFile(PacketClass).open(file_path) as input_file: >>> for packet in input_file: >>> print(packet) For application where a given post-processing requires to put in memory all the packets in the file (e.g., when it is necessary to combine adjacent packets in more complex, high-level quantities), the :meth:`read_all() ` method is provided. (It goes without saying, this comes with all the caveats of putting a potentially large amount of information in memory.) Module documentation -------------------- .. automodule:: baldaquin.pkt