Add affiliation-country filter for DOI queries via ROR-to-country mapping

[Copilot]

Issue - https://github.com/datacite/product-backlog/issues/674
https://github.com/datacite/product-backlog/issues/671

Implements /dois?affiliation-country=US,GB query parameter to filter DOIs by countries inferred from ROR affiliations in creators/contributors metadata.

Implementation

ROR-to-country resolution

• ROR_TO_COUNTRIES mapping loaded from app/resources/ror_to_countries.json (populated via S3 workflow)
• get_countries_from_ror(ror_id) helper in Rorable concern normalizes ROR IDs and returns ISO 3166-1 alpha-2 codes

Elasticsearch indexing

• New affiliation_countries keyword field in DOI mapping
• Doi#affiliation_countries traverses creators/contributors, extracts ROR affiliations, resolves to countries, deduplicates
• Private extract_countries_from_people(people) helper eliminates code duplication

Query filtering

• affiliation_country parameter in Doi.query parses comma-separated values, normalizes to uppercase, applies ES terms filter
• Pagination links include parameter passthrough
• OpenAPI spec documents parameter

S3 sync workflow

• Extended update-ror-mappings.yml to download/update ror_to_countries.json with semantic JSON comparison

Usage

# Single country
GET /dois?affiliation-country=US

# Multiple (OR logic)
GET /dois?affiliation-country=US,GB,DE

# Case-insensitive
GET /dois?affiliation-country=us,gb  # normalized to uppercase

Pattern consistency

Follows established approach from PR #1437 (funder mappings): static JSON resource, boot-time constant loading, Rorable helper, indexed field, comma-separated filter syntax.

Testing

• Unit tests for get_countries_from_ror (various ROR formats, edge cases)
• Integration tests for affiliation_countries (creator/contributor aggregation, deduplication)
• Placeholder JSON contains test fixtures; production data populated by workflow

Post-merge steps

1. Trigger update-ror-mappings workflow to populate production ROR-to-country data
2. Reindex DOIs to populate affiliation_countries field

Original prompt

Implement new REST API filter affiliation-country for /dois in datacite/lupo, backed by Elasticsearch/OpenSearch indexing.

Background

• Product requirement: filter DOI records by associated country inferred from ROR IDs in creators/contributors affiliations.
• Reuse approach from PR Adds funded-by and include-funder-child-organizations URL parameters #1437 (funder mappings): store static mapping JSON in app/resources, load at boot, derive indexed fields, and filter via ES query.

Requirements

1. Elasticsearch mapping

• Add a new multi-value keyword field affiliation_countries to the DOI Elasticsearch mapping (in app/models/doi.rb mapping block).

2. ROR-to-country mapping file loading

• Add new constant ROR_TO_COUNTRIES loaded from app/resources/ror_to_countries.json in config/initializers/load_ror_data.rb.
• Add placeholder app/resources/ror_to_countries.json file (minimal valid JSON) so app boots/tests run even before workflow updates populate it.

3. ROR helper

• Extend app/models/concerns/rorable.rb with get_countries_from_ror(ror_id) that:
  • normalizes ROR ID using existing ror_from_url helper
  • looks up ROR_TO_COUNTRIES[normalized_ror]
  • returns unique uppercase ISO3166 alpha2 codes (array)

4. DOI indexing

• Add Doi#affiliation_countries method that:
  • traverses creators and contributors
  • reads affiliation objects where affiliationIdentifierScheme == "ROR" and affiliationIdentifier is present
  • resolves each affiliationIdentifier ROR id to countries via get_countries_from_ror
  • dedupes results
• Ensure affiliation_countries is included in Doi#as_indexed_json output under key "affiliation_countries".

5. REST API filtering

• Add a new URL query parameter for /dois: affiliation-country.
• Implement filter handling in DOI query filter builder in app/models/doi.rb:
  • parse comma-separated values
  • trim whitespace
  • case-insensitive input; normalize to uppercase
  • apply Elasticsearch terms filter on affiliation_countries

6. Pagination link passthrough

• Update app/controllers/datacite_dois_controller.rb pagination next link query param construction to include affiliation-country.

7. OpenAPI documentation

• Update openapi.yaml to document affiliation-country parameter alongside other /dois filters.

8. Workflow update for S3 mapping refresh

• Update .github/workflows/update-ror-mappings.yml to also download and update ror_to_countries.json from the same S3 prefix (ror_funder_mapping/ror_to_countries.json) and include it in the semantic JSON compare/update logic.

Notes

• Keep names as recommended: ES/index field affiliation_countries, API param affiliation-country, mapping file ror_to_countries.json, constant ROR_TO_COUNTRIES.
• Do not add new VCR cassettes; minimal tests/doc updates are acceptable. The requester will run VCR locally and follow up.

Related reference

• PR pattern reference: Adds funded-by and include-funder-child-organizations URL parameters #1437

This pull request was created from Copilot chat.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.