Proposal: A "universal" Data Format


  • Admin

    Author Pablo Klaschka (@pklaschka)
    Version Rev. 0 (original version)
    Date 2020-07-25
    Est. time for implementation 2 weeks

    This RFC is a proposal for a “universal” DF that could get used by specialized SMs that don’t want to distribute a specialized FEM additionally.

    The primary goal would be to create a data format that, on a high level, describes the data it contains (which can contain several datasets in multiple other data formats).

    We don’t want to do this on a low-level, describing the actual layout of the visualization (i.e., basically, generating the necessary HTML code). The aim is to create a data structure that simply describes what its content describes and leaves it up to the interpreting FEM to display it in a manner that’s suitable for the device that presents that data).

    Requirements

    The Data Format must support:

    • definitions of what the data shows (is it a list of CPUs or hard drives?)
    • common high-level information structures (lists, tables, key-value-pair-data, numbers with units, etc.)
    • hierarchical data (a table-like data set could contain, in one of its fields, a list; this might get visualized in a popup when an Expand button in that cell gets clicked)
    • two or more datasets that belong to the same hierarchical “position” but are, in some sense, independent. A possible visualization might be tabs for the different datasets.

    How it could look like

    {
      type: 'universal', // indicate to clients that this is a universal DF dataset
      data: [...] // the data itself
    }
    

    Each dataset is represented by an object. On its own root level, defines its type (based on a pre-defined set of types like parallel, list, or table, cf. the Requirements above) as type, necessary “Metadata” (the value’s name, unit, …) and its actual data as data property.

    Datasets can, in the hierarchical data, can contain other datasets (a table cell a number with a unit, tabs tables, and lists, etc.)

    The DF’s root data field can be of any dataset type (even “just” a string 😉)

    Tooling Requirements

    Typescript definitions (and possibly implementations) of the data set types would need to exist as a library to make working with such an abstract data format feasible.

    A FEM that, at least at some raw level, interprets the data format and visualizes it accordingly would also be required.

    Final thoughts

    The discrepancy between SMs and FEMs is, as of right now, one of the biggest difficulties of the SerSta ecosystem. This RFC proposes adding a concept that

    1. is compatible with the current specifications
    2. (because of 1.) keeps the option of individual FEMs when required (and thus, doesn’t take away from the modularity of the Server State system)
    3. To some extend, solves this discrepancy between SMs and FEMs by encouraging a cross-platform-compatible standard, when possible