Investigate multiple mappings in trait mapping database

[apriltuesday]

While looking into #427 we found cases of traits being mapped to multiple terms, one more precise than the other. We should confirm whether these are correct and implement processing to ensure these are fixed automatically or curated in future rounds.

Note that currently, existing trait mappings aren't processed if they aren't actively used in ClinVar releases, meaning that obsolete or redundant terms can linger in the mappings file. As part of this we can revisit this decision if needed.

[apriltuesday]

I wrote a small script to go through our current mappings to filter out obsolete and multiple mappings. The script in its current form is here and the output is as follows:

15972 ontology mappings loaded for ontology EFO
Removed 1886 obsolete mappings
Removed 716 multiple mappings
13370 mappings remaining

The obsolete and multiple mappings are in a spreadsheet here. Note that:

• Obsolete mappings are filtered out before multiples - i.e. all multiple mappings in the spreadsheet are current EFO terms.
• Mappings for terms that are in use by ClinVar are kept up-to-date by the automated pipeline + curation. This means most of the obsolete mappings in the spreadsheet are for terms not present in ClinVar anymore.

Questions to decide:

• Should we regularly remove unused obsolete mappings? If so, at what point in the process?
• How should we clean up the multiple mappings? Is there some automation we can do to minimise manual effort?
• Should we continue to allow multiple mappings in cases where they are necessary (i.e. multiple concepts within a ClinVar term, see SOP)?
  • If so, we might need to clean up inappropriate multiple mappings regularly as well. If not, we can just disallow duplicate terms in the mappings file.

@tcezard @Dona094 Have a look at the spreadsheet and let me know what you think, we'll probably schedule a discussion and then I will finalise the script.

[apriltuesday]

Obsolete mappings:

• Move to a separate file as part of curation spreadsheet generation

Multiple mappings:

• New spreadsheet export and automated mappings should overwrite any ClinVar terms present in old mappings
• Old mappings should still be kept if the term does not occur in the new mappings
• Curator should delete lines containing duplicate terms that are not required, these will be updated
• Multiple mappings will still be supported

Future investigations for cleaning up old mappings:

• Check how many mappings in latest_mappings are used in latest evidence strings / latest ClinVar
• How many are bypassing curation completely (high confidence from Zooma or exact match from OLS) - especially multiple mappings coming back from Zooma

[apriltuesday]

I've done some additional investigations into how we use previous trait mappings. The complete notebook can be viewed here.

In summary, there are a couple reasons why we should continue to track previous mappings (i.e., mappings not from the most recent curated or automated set):

1. The set of trait names that we curate is different from the set of trait names that we use to annotate. In particular we curate only the preferred trait names, but when we generate evidence we use all trait names. This has several ramifications and is discussed more in Investigate use of all trait names vs. preferred trait names in OT evidence generation #384, but for the purposes of this ticket it means that we currently require the previous mappings when we generate evidence, because the latest automated/curated mappings don't contain all trait names.
2. It's unclear whether Zooma consistently reports high confidence for mappings we've submitted to them in the past. This might be due to a past or present bug in Zooma or some intended behaviour we aren't knowledgeable about. Either way, it means that mappings that are automated might at some point need to go through curation again, and in general it means we might not want to rely entirely on Zooma for reliably storing past mappings.

My feeling right now is that we could go ahead with changes to clean up obsolete and multiple mappings, but we might want to hold off on a completely new strategy of updating the latest mappings until we have a better handle on these two issues.

@tcezard Let me know what you think.

[apriltuesday]

Further investigation found that the process to create the latest mappings file was actually handling multiple mappings correctly; #495 reimplements the logic in a more readable fashion and adds unit tests.

In fact, it looks like many multiple mappings are originating from automated mappings - either multiple exact label matches from OLS (i.e. arguably duplicate terms within EFO), or multiple high-confidence results from ZOOMA. The former could be addressed by a process of providing feedback to EFO, but the latter is trickier to deal with since we don't know if we can actually delete our previous mappings from ZOOMA.

The conclusion from this is that we should rely less on ZOOMA as our source of truth and instead use a process we have better control to manage our mappings. This will be done in a separate issue.