Add lines database from Francie Cashman

[brittonsmith]

This adds the lines database file updated by @fcashman, including a handful of very recent additions. The is the lines file used by the FOGGIE project. Adding this here will allow for ending the project-specific version of trident. There are extra commits here because I have extracted the full history of this file to allow it and its provenance to exist independently of the default lines file and so I could isolate just this file for the PR.

Also, the default file had one duplicate line, which I removed.

There are still a few things to be worked out that I was hoping to start a discussion about here. Firstly, there should probably be some documentation that makes it clear how these two lists are different. From what I understand, the default lines.txt is mainly populated by data from NIST. I'd be willing to add a note to the header file of lines.txt. As well, perhaps that file should be renamed lines_nist.txt or something. I would be happy to do that and thread that name through the code so that the default still works. Finally, I would add a brief discussion to the narrative documentation.

@chummels, would this be acceptable to you?

[chummels]

Thanks for the PR. I have been wanting to finally tweak and merge the long-standing PR that integrates the linetools package into trident (#174 ), which would provide a different mechanism for looking up individual lines. Specifically, it seems like that package tracks the line data that observers use regularly, which ultimately ensures consistency between what observers and theorists are assuming. Do you (or Francie) happen to know the relative agreement between her line list and the one in the linetools package? I think it uses her lists.

Either way, I figured I'd preserve the backwards compatibility of using the lines.txt method. So this PR could work in that framework. On the other hand, if we're now going to have a few different options for looking up line data: two lines.txt versions and the linetools package, maybe this is worth considering more broadly on the best way to specify things? I'm happy to have a discussion about this, though.

As far as the original lines.txt, some of those data came from NIST, but others came from private sources like Jess Werk and Chuck Steidel. So it's sort of an amalgam of different datasets.

[coveralls]

Coverage Report for CI Build 0

Coverage remained the same at 76.053%

Details

• Coverage remained the same as the base build.
• Patch coverage: No coverable lines changed in this PR.
• No coverage regressions found.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	2351
Covered Lines:	1788
Line Coverage:	76.05%
Coverage Strength:	0.76 hits per line

---

💛 - Coveralls

[brittonsmith]

I looked briefly and linetools appears to have all of Cashman et al. (2017). However, there are piecemeal updates to this list from other sources like Boisse & Bergeron (2019) and Cashman et al. (2024) that I could not find any evidence for. My two cents: I think there is merit to being able have control over the line data being used and not being dependent on another package if a user prefers not to be. I'd rather see the option to use one's own file stick around, even if the default option ultimately moves to linetools.

Ok, what would you like to see before this is merged? It sounds like keeping the default lines.txt file named that way is your preference, totally fine. Do you want any narrative documentation on how to change line lists or a brief discussion of how these two are different?

[fcashman]

Hi all. My group is currently putting together an update to Cashman+2017. We're aiming to publish an updated line list every 7-10 years. The P II and Ni II values were flagged in the 2017 paper as being particularly in need, so folks were quick to publish on those with updated values. Linetools uses my paper and Morton, but linetools hasn't had an update to its line list in quite some time. I also share the opinion that is powerful to have total control over the line data being used and quite like the current set up. My group is also working on updated data for Si II, which I'd be happy to submit after it's been peer-reviewed.

[chummels]

Thank you both for the detailed responses. This brings up some interesting points.

OK, so I agree with you both that we should have a mechanism for the user to specify the line data that they desire. I think this could be to continue with the current paradigm of specifying a text file.

A few relevant thoughts (no, I'm not asking Britton to do this in this PR, just trying to think out the best way for all of these relevant issues to get addressed):

• Is there a better way to specify line data than the admittedly simplistic text file format that we already have? One could nominally come up with an HDF5 or dictionary format or something, but maybe there isn't really any benefit to that. Especially now that I'm building emission lines into Trident, it raises the question of how to do this most generally. Just trying to consider things for the future.

• As Britton is suggesting in this PR, we can potentially have a few different line lists that users could select to use: the original lines.txt, this new one from from Francie, and I don't know if there are others that different groups might prefer? I realize that the details of us getting the line values in the existing lines.txt is hazy, which is not so great for reproducibility. I guess I'm OK with changing the defaults to the new one from Francie if there is broad agreement that this is "the best" currently? It will break the answer tests and such, and it might even break backwards compatibility since all of the current codebase just looks for lines.txt. I'm just trying to think about the best way forward to ensure people get the most accurate results.

• Another aspect to consider is how do we ensure that users continue to get the most up-to-date values in their line data. I realize you guys are fine because you have Francie on the team, so you just update your line list into the future, but it is useful to have up to date line data universally accessible to all trident users. In an ideal world, it would be through a package like linetools, which is kept up to date as new data comes online. But it appears that this is not happening. I guess maybe we could call this new linelist lines_cashman_2026.txt or more simply cashman_2026.txt, something like that, allowing for updated values in the future? But when there are updates, do we also shift the defaults to that? Hmmm. Maybe the best way is to create a value in the trident config file that points to one's default line list, and when there are updates that we think are really relevant and good, we update the config file to point to the new data as default? But again, this breaks tests, and might even override what the user wants in their own config. Not sure.

• Now I'm not even sure if it's worthwhile to add the linetools functionality, since it hasn't been updated. Maybe it makes more sense to add in a function to linetools natively that just spits out their current line list into a format expected by trident, so if people want to use the linetools values in Trident, it can easily do so. Of course it won't automatically update the values with linetools when it updates its line lists, but it sounds like there aren't a lot of updates on that end so maybe that's OK.

• Lastly, one consideration is to ensure reproducibility of results between groups. Most of the time, when people cite Trident, they just say they used Trident with whichever ion balance table, since largely the line list was fixed. Perhaps we should now recommend that people specify what version of Trident they're using, which line list, and which ion balance table, just to ensure that things are fully specified and reproducible in the long term. Maybe I'll just make a special function that Trident spits out that relevant data whenever a spectrum or ProjectionPlot is generated? Not sure.

Anyway, that was a long comment. I realize we could just merge this PR and be done with it, but I am just seeking feedback from other users of the code to ensure we do the best thing for the long-term accuracy and reproducibility of the results in trident. If you two (or any other users!) have comments or opinions on these thoughts, I'd love to hear them.

[brittonsmith]

@chummels, thanks for the detailed response. I agree some things should be worked out before this is merged. I've got thoughts on most of your points.

• Is there a better way to specify line data than the admittedly simplistic text file format that we already have?

I think the current method works very well. File size is not a concern and ascii is most easily human inspectable. It's simple to flip through and find relevant information. Plus, git can more easily track the change history of text files than binary formats. With binary, you don't get a diff.

• ...I'm just trying to think about the best way forward to ensure people get the most accurate results.

I'm a huge fan of backward compatibility. As well, keeping the tests passing in trident has always been a challenge. I think the current line list file should continue to be the default, although renaming it something more informative might be a good idea: lines_default.txt, lines_legacy.txt? I think it would be good to add a header explaining that it's the original file along with some info about the main sources. Then, add a section in the narrative docs called something like "Data Files" that lists the available possibilities and pointers to documentation on how to select them. This could also include the ion balance tables.

• Another aspect to consider is how do we ensure that users continue to get the most up-to-date values in their line data.

I think that's really what this PR is all about, specifically making the most direct connection between researchers like Francie and those that want to consume that output. The whole intention here is to not confine the work to a fork, but to get it into public releases, now and into the future. I would like to merge this PR and then sunset foggie-trident.

• Now I'm not even sure if it's worthwhile to add the linetools functionality, since it hasn't been updated.

I wouldn't want it as the default option for the following reasons: it's only install from source; hasn't had regular releases; it's an added layer of complication if you want to update it and use that right away in trident. Still it's a useful package and I think there is merit to it as an optional feature since it covers a lot of literature and more lines overall. Perhaps a function to output a line list from linetools in trident format would bridge these two things in a useful way.

• Lastly, one consideration is to ensure reproducibility of results between groups.

Citation functionality is a great idea but will take time. I think you can get a lot of this with a few easy things:

• do a major release for every line list update so version becomes a tracker of the line list
• add guidance to the trident citation page with suggested wording that includes the items you mention

For this specific PR, I am willing to add a docs section on the line list file options and make sure there is something showing how to change the one used at runtime. Would this be sufficient to get it merged?