APLOSE detections

[Gautzilla]

🖼️ Annotation results are coming to OSEkit!

How does it work

The idea is to be able to load APLOSE results csv into OSEkit as a collection of new Annotation objects:

annotations = Annotation.from_csv(Path("aplose_results.csv"))

The annotations could then be filtered thanks to their properties :

verified_annotations = [
    a for a in annotations if all(v.is_validated for v in a.verifications)
]

Using the Annotation with OSEkit

The Annotation class inherits from the Event class: its begin and end properties can be used to access the audio corresponding to the annotation box easily, e.g.:

from osekit.core.audio_data import AudioData
from osekit.core.audio_dataset import AudioDataset
from osekit.core.audio_file import AudioFile

afs = [AudioFile(...)] # Or even project.origin_files with the Public API

annotation_audio_dataset = AudioDataset(
    [
        AudioData.from_files(
            files=afs,
            begin=annotation.begin,
            end=annotation.end,
        )
        for annotation in annotations
    ]
)

# Then you could e.g. plot each spectrogram with SpectroDataset.from_audio_dataset

[coveralls]

coverage: 99.003% (+0.04%) from 98.961% — Gautzilla:annotation into Project-OSmOSE:main

[ElodieENSTA]

The column names hasn't change on APLOSE side

[Gautzilla]

damn good catch, I didn't see that my refactor affected also the dict keys!

I really should write an integration test before messing with everything 🥸

[MaelleTtrt]

Detection.from_csv() failed when I tried to read this file : path_aplose = Path(r"C:\Users\torterma\Desktop\cetiroise\CETIROISE_F5_25072023_Annotation_results_standardized.csv")

  File "C:\Users\torterma\Documents\Projets_Osmose\Git\OSEkit\.venv\Lib\site-packages\IPython\core\interactiveshell.py", line 3699, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<ipython-input-11-eae37ed4ec6f>", line 1, in <module>
    detections = Detection.from_csv(Path(path_aplose))
  File "C:\Users\torterma\Documents\Projets_Osmose\Git\OSEkit\src\osekit\core\detection.py", line 409, in from_csv
    return [cls.from_dict(record) for record in records]
            ~~~~~~~~~~~~~^^^^^^^^
  File "C:\Users\torterma\Documents\Projets_Osmose\Git\OSEkit\src\osekit\core\detection.py", line 320, in from_dict
    confidence_indicator = ConfidenceIndicator.from_relative_level_string(
        label=row["confidence_indicator_label"],
        relative_level_string=row["confidence_indicator_level"],
    )
  File "C:\Users\torterma\Documents\Projets_Osmose\Git\OSEkit\src\osekit\core\detection.py", line 178, in from_relative_level_string
    level, maximum_level = map(int, relative_level_string.split("/"))
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'split'```


I think it's because the `confidence_indicator_level` column is empty. In this case, I think that `Detection.from_csv()` should return `None` in the `confidence_indicator_level` column.

[Gautzilla]

Good catch, I indeed didn't notice the (if exists) part in the csv file description, I'll fix that shortyl!

[mathieudpnt]

@Gautzilla I have an error when using a yolo-aplose-formatted csv because i do not have all the required columns

annotations = Detection.from_csv(Path(r"C:\Users\dupontma2\Downloads\check\C16D1_ST7189_yolo_v2_prediction.csv"))
Traceback (most recent call last):
  File "C:\Users\dupontma2\Documents\Git\OSmOSE\OSEkit\.venv\Lib\site-packages\IPython\core\interactiveshell.py", line 3701, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<ipython-input-5-a298c7373fb5>", line 1, in <module>
    annotations = Detection.from_csv(Path(r"C:\Users\dupontma2\Downloads\check\C16D1_ST7189_yolo_v2_prediction.csv"))
  File "C:\Users\dupontma2\Documents\Git\OSmOSE\OSEkit\src\osekit\core\detection.py", line 413, in from_csv
    return [cls.from_dict(record) for record in records]
            ~~~~~~~~~~~~~^^^^^^^^
  File "C:\Users\dupontma2\Documents\Git\OSmOSE\OSEkit\src\osekit\core\detection.py", line 303, in from_dict
    detection_id=row["annotation_id"],
                 ~~~^^^^^^^^^^^^^^^^^
KeyError: 'annotation_id'

We need to specify which column are in deed mandatory, to me that would be:

• start_frequency
• end_frequency
• start_datetime
• end_datetime
• annotation
• annotator

@ElodieENSTA what do you think ?

[ElodieENSTA]

We need to specify which column are in deed mandatory, to me that would be:

• filename,
• start_time,
• end_time,
• start_frequency,
• end_frequency,
• annotation,
• annotator,
• start_datetime,
• end_datetime,
• type,
• confidence.

@ElodieENSTA what do you think ?

I don't think the filename is mandatory with continuous datasets, since you have the timestamps. So start/end relative time should not be mandatory as well.
I'm renaming start/end frequency to min/max frequency (more appropriate), but still need to do so on annotation import ^^
confidence as a float between 0 and 1?

[Gautzilla]

We need to specify which column are in deed mandatory, to me that would be:

• filename,
• start_time,
• end_time,
• start_frequency,
• end_frequency,
• annotation,
• annotator,
• start_datetime,
• end_datetime,
• type,
• confidence.

@ElodieENSTA what do you think ?

I don't think the filename is mandatory with continuous datasets, since you have the timestamps. So start/end relative time should not be mandatory as well. I'm renaming start/end frequency to min/max frequency (more appropriate), but still need to do so on annotation import ^^ confidence as a float between 0 and 1?

Maybe the bare minimum should only be what a detector outputs at bare minimum?

• Start time (absolute)
• End time (absolute)
• Start frequency
• End frequency
• Label (?)
• Confidence value (?)

Not sure which parameters truly are mandatory in every detector output tho

[mathieudpnt]

@Gautzilla I have an error when using a yolo-aplose-formatted csv because i do not have all the required columns

annotations = Detection.from_csv(Path(r"C:\Users\dupontma2\Downloads\check\C16D1_ST7189_yolo_v2_prediction.csv"))
Traceback (most recent call last):
  File "C:\Users\dupontma2\Documents\Git\OSmOSE\OSEkit\.venv\Lib\site-packages\IPython\core\interactiveshell.py", line 3701, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<ipython-input-5-a298c7373fb5>", line 1, in <module>
    annotations = Detection.from_csv(Path(r"C:\Users\dupontma2\Downloads\check\C16D1_ST7189_yolo_v2_prediction.csv"))
  File "C:\Users\dupontma2\Documents\Git\OSmOSE\OSEkit\src\osekit\core\detection.py", line 413, in from_csv
    return [cls.from_dict(record) for record in records]
            ~~~~~~~~~~~~~^^^^^^^^
  File "C:\Users\dupontma2\Documents\Git\OSmOSE\OSEkit\src\osekit\core\detection.py", line 303, in from_dict
    detection_id=row["annotation_id"],
                 ~~~^^^^^^^^^^^^^^^^^
KeyError: 'annotation_id'

@Gautzilla I still get this error when loading a custom APLOSE formatted csv that does not contains certains columns like annotation_id

Also 2 points:

• We should set fill=False automatically in to_rectangle method as we would always use this setting I think
• make it possible that from_csv method accepts Path | list[Path] to automatically concatenate Detection instances from several csv

[Gautzilla]

@ElodieENSTA The only mandatory columns are now min_frequency, max_frequency, start_datetime and end_datetime, as shown in this sample csv file.

@mathieudpnt

• Detection.to_rectangle fill parameter is now defaulted to False.
• Detection.from_csv() now accepts lists of csv files which detections are concatenated

[mathieudpnt]

LGTM, I would however prefer a better str method as mentionned in the review.

Also, I would add one more feature to easily add a label above the detection rectangle like so

EDIT, As discussed, label feature will be implemented in another future PR

[mathieudpnt]

I would prefer another str method that's relevant to detections without an detection_id maybe something like

def __repr__(self) -> str:
    return str(f"{self.metadata.detection_id} - {self.begin} - {self.end} - {self.frequency_bounds}")

>>> detection = Detection(...)
>>> print(detection)
>>> None - 2023-02-11 12:03:50+01:00 - 2023-02-11 12:04:00+01:00 - FrequencyBounds(min=0, max=64000)

?

[mathieudpnt]

Another thing! It would be nice to have a color_mapdict that stores the APLOSE app label colors ! For instance the 3 first color codes on APLOSE app are

color_map = [
    "#A33F73",
    "#EC8E02",
    "#12B0BA",
]

[ElodieENSTA]

Another thing! It would be nice to have a color_mapdict that stores the APLOSE app label colors ! For instance the 3 first color codes on APLOSE app are

color_map = [
    "#A33F73",
    "#EC8E02",
    "#12B0BA",
]

I don't exaclty get why this will be usefull, but if you need it, all the colors are here : https://github.com/Project-OSmOSE/APLOSE/blob/chore/base-ui-with-storybook/frontend/src/css/annotation.scss

[mathieudpnt]

Another thing! It would be nice to have a color_mapdict that stores the APLOSE app label colors ! For instance the 3 first color codes on APLOSE app are

color_map = [
    "#A33F73",
    "#EC8E02",
    "#12B0BA",
]

I don't exaclty get why this will be usefull, but if you need it, all the colors are here : https://github.com/Project-OSmOSE/APLOSE/blob/chore/base-ui-with-storybook/frontend/src/css/annotation.scss

It's mainly for convenience! I am quite familiar with APLOSE default colors so I might as well implement it as default colors here 🤷