Skip to content

DataAplose simplification#103

Open
mathieudpnt wants to merge 16 commits into
Project-OSmOSE:mainfrom
mathieudpnt:refacto_DataAplose
Open

DataAplose simplification#103
mathieudpnt wants to merge 16 commits into
Project-OSmOSE:mainfrom
mathieudpnt:refacto_DataAplose

Conversation

@mathieudpnt

@mathieudpnt mathieudpnt commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

I simplified the process, now user only has to declare a dict of arguments and use DataAplose from_dict method. The rest of the process should remain the same.

Example:

from post_processing.dataclass.data_aplose import DataAplose

config = [
    {
        "detection_file": ...,
        "filename_format": ...,
    },
    {
        "detection_file": ...,
        "filename_format": ...,
    },
]

data = DataAplose.from_dict(config)

config can be either a dict or a list of dict.

A dict must contains the following arguments:

  • detection_file: Path -> path to an APLOSE-formatted result csv file
  • filename_format: str -> string to parse datetimes from filename column from the detection file

Others non-mandatory arguments can be given:

  • timebin_new: Timedelta | None = None
  • start_datetime: Timestamp | None = None
  • end_datetime: Timestamp | None = None
  • annotator: str | list[str] | None = None
  • annotation: str | list[str] | None = None
  • type: str | None = None
  • recording_file: Path | None = None
  • user_selection: str = "all"
  • min_frequency: float | None = None
  • max_frequency: float | None = None
  • confidence: float | None = None

@mathieudpnt mathieudpnt requested a review from MaelleTtrt June 17, 2026 14:41
@mathieudpnt

Copy link
Copy Markdown
Collaborator Author

I also updated the utils script names, got rid of "_utils"

example:
glider_utils -> glider

from post_processing.utils import glider

@mathieudpnt mathieudpnt requested a review from naiscd June 17, 2026 15:25
@mathieudpnt

Copy link
Copy Markdown
Collaborator Author

I also (finally) renamed the package to disclose 🪩

@mathieudpnt

mathieudpnt commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator Author

and finally i update RecordingPeriod class, now to display the effort, user has to provide in the configuration dict recording_file which corresponds to the Path of a recording_period CSV file:

config ={
        "detection_file": Path(...),
        "filename_format": "...",
        "recording_file": Path(...),
}

data= DataAplose.from_dict(config)

fig, ax = plt.subplots(1, 1)
ax = data.set_ax(...)
data.plot(
    ...
    effort=True,
)
plt.tight_layout()
plt.show()

@MaelleTtrt

Copy link
Copy Markdown
Member

I have an issue when building a DataAplose from a configuration containing multiple concatenated files. Calling data.plot(..., effort=True) raises an error.
From debugging, the error seems to come from RecordingPeriod.from_config, which expects config.recording_file. However, in my case config is a list of configs, and recording_file is only defined in the first element of that list.

I’m not sure what the intended behavior should be here:

  • Should recording_file be provided in every config item when concatenating datasets?
  • Or should plot(effort=True) handle the case where self.config is a list and retrieve the recording information accordingly? Because maybe there could be multiple recording-file that should be concatenated. This is not my case as I already concatenated the recording_files into one csv.

@MaelleTtrt

Copy link
Copy Markdown
Member

The start_datetime and end_datetime specified in the config are different from the one present in the DataAplose built from this config. The one present in DataAplose are the start_datetime and end_datetime of the detections contained in data.df.
I think start_datetime and end_datetime of DataAplose should be the one specified by the user in the config.

@mathieudpnt

Copy link
Copy Markdown
Collaborator Author

@MaelleTtrt I have fixed DataAplose start_datetime and end_datetime, if specified in a config dict, the Timestamps will stick to these values, else it will take the min/max Timestamps of the DataFrame.

Also, i have created a concat method to DataAploseConfig to handle a recording_file even if a list of dict is provided for the conf. Let me know if this works for you

@mathieudpnt

Copy link
Copy Markdown
Collaborator Author

While we're at it, i changed every mention of annotation to label 👀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants