FL011 Partial files named `data`

    Partial files named data #

    Problem ID Manufacturer Affected Firmware Fixed in firmware Affected Hardware Status
    FL011 Frontier Labs 3.12+ N/A BAR-LT Intended behavior

    Related to FL003.

    It appears FL now write temporary files to partial extension-less data files.

    It looks like most of the file is a valid audio file:

    screenshot of header

    Open questions:

    • Does this occur for WAVE recordings too?
    • Can these files be repaired?
      • Yes: For FLAC files they can go through a partial FLAC transcoding leaving a WAVE trailing fragment at the end of the file. EMU handles this case.
    • The logs will usually make some note about an error, but don’t mention the data file specifically.

    Notes:

    There are various problems that can occur with these files. We’ve seen examples of:

    1. partial files that are just empty
    2. partial files that have a RIFF WAVE header for which their data chunks are wholly empty (data chunk size is 44 bytes)
      • this case in particular is an example of FL003 wrapped in FL011.
    3. partial files that contain FLAC frames for the whole file (almost fully recoverable)
      • these files are usually produced after transcoding, but fail when they are renamed (due to duplicate filenames created by clock syncs)
    4. partial files that contain FLAC frames for the first part of the file, but have other data (WAVE samples???) in the rest of the file!
      • these files are produced when the sensor is interrupted during the transcoding step

    Status #

    Intended behavior: The sensor produces these data files while recording. In the event of an unexpected interruption to recording the data file is not renamed to it’s final name thus omitting it from most filename based filters.

    There is still potential to repair these files.

    We considered merging FL003 with FL011 but although related the production of data partial files is a unique behavior.

    Status with the vendor #

    FL has verified this behavior is intentional.

    The suggestion from FL003 to append a .partial suffix to normal file names was not adopted.

    Effects on common tools #

    Acoustic Workbench (Ecosounds, A2O) #

    Can repair and process some of these files, particularly cases 3 & 4 from the above list.

    ffmpeg/ffprobe #

    ffprobe seems to process the file with no issues

    Depending on the fault ffprobe will process as much of the file as it can read before failure. Regardless though, reading or converting some cases of the files will produce corrupt output.

    EMU #

    Emu can partially repair these partial files. See https://github.com/QutEcoacoustics/emu/blob/master/docs/fixes.md#fixing-partial-data-files-fl011.

    Examples #

    https://connectqutedu.sharepoint.com/sites/QUTEcoacousticsAnon/Shared%20Documents/Forms/AllItems.aspx?id=%2Fsites%2FQUTEcoacousticsAnon%2FShared%20Documents%2Fknown%2Dproblems%2Ffrontier%5Flabs%2FFL011