diff --git a/docs/operator/configuration_parameters.md b/docs/operator/configuration_parameters.md index 5ec5f34e4a2..31b1c076c4d 100644 --- a/docs/operator/configuration_parameters.md +++ b/docs/operator/configuration_parameters.md @@ -495,43 +495,52 @@ a setting already specified in the Configuration File. - ***VO***: _(Optional)_ Internal short VO name. No default. ## RSE settings -The RSE settings are set separately using `rucio.RSEClient.update_rse` or `rucio rse update`, and specifies RSE configuration used by the Rucio instance. +The RSE settings are set separately using `rucio.RSEClient.update_rse` or `rucio rse update`, and specify RSE configuration used by the Rucio instance. Mutable settings are `deterministic`, `rse_type`, `staging_area`, `volatile`, `qos_class`, `availability_delete`, `availability_read`, `availability_write`, `city`, `country_name`, `latitude`, `longitude`, `region_code`, and `time_zone`. +Geographic fields (`city`, `country_name`, `latitude`, `longitude`, `region_code`, `time_zone`) can also be set at RSE creation time via API parameters. Read more about RSEs [here](started/concepts/rucio_storage_element.md) and how to set them up [here](operator/configuration.md#creating-new-rses). - **availability_delete**: Boolean. Control if this RSE allows deletions by the Reaper daemon using any protocol. Default: `True`. - **availability_read**: Boolean. Control if this RSE allows reads using any protocol. Default: `True`. - **availability_write**: Boolean. Control if this RSE allows writes using any protocol. Default: `True`. -- **credentials**: Dictionary[String, Any]: Specify token credentials used for accessing this RSE if it is in a commercial cloud. No default. -- **delete_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for deletions. Default: `1`. -- **deterministic**: Boolean: Controls if the RSE is allowed to generate paths based solely on the DID (scope:filename). More info about non-deterministic RSEs [here](started/concepts/replica_workflow.md#replica-paths-on-storage). Default: `True`. -- **domain**: List[String]: Specifies the locations this RSE may be accessed by clients from. Cannot be changed. -- **id**: String: Identification string of the RSE. Cannot be changed. -- **lfn2pfn_algorithm**: String: Name of the algorithm in the configured policy package to be used for creating replica paths. Cannot be changed. If no lfn2pfn_algorithm attribute is set, then the setting defaults to lfn2pfn_algorithm_default in the configured policy package. Default: `default`. -- **qos_class**: String: No functionality in modern Rucio. No default. -- **read_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for reads. Default: `1`. -- **rse**: String: The name of the Rucio Storage Element as given at creation time. Cannot be changed. -- **rse_type**: String: Specify `DISK` or `TAPE` for control of [QoS](operator/qos_rse_config.md). Default: `DISK`. -- **sign_url**: Optional[str]. Enable cloud support for this storage element. No default. -- **staging_area**: Boolean.Specify if this RSE is a disk buffer to a tertiary storage backend, subject to additional constraints (specifically a lifetime for rules on this RSE must be defined). Default: `False`. -- **third_party_copy_read_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for TPC reads. Default `1`. -- **third_party_copy_write_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for TPC writes. Default `1`. -- **verify_checksum**: Boolean: Specifies if the RSE has support for checksum verification. Default: `True`. +- **city**: Optional[String]. City where the RSE is located. Used for geographic RSE selection. No default. +- **continent**: Optional[String]. Continent where the RSE is located. No default. +- **country_name**: Optional[String]. Country where the RSE is located. Used for geographic RSE selection. No default. +- **credentials**: Optional[Dictionary[String, Any]]. Specify token credentials used for accessing this RSE if it is in a commercial cloud. No default. +- **deterministic**: Boolean. Controls if the RSE is allowed to generate paths based solely on the DID (scope:filename). More info about non-deterministic RSEs [here](started/concepts/replica_workflow.md#replica-paths-on-storage). Default: `True`. +- **domain**: List[String]. Specifies the locations this RSE may be accessed by clients from. Cannot be changed. +- **id**: String. Identification string of the RSE. Cannot be changed. +- **latitude**: Optional[Float]. Latitude coordinate of the RSE location. Used for geographic calculations. No default. +- **longitude**: Optional[Float]. Longitude coordinate of the RSE location. Used for geographic calculations. No default. +- **protocols**: List[RSEProtocolDict]. List of protocol configurations for accessing this RSE. Cannot be changed directly; use protocol-specific methods. +- **qos_class**: Optional[String]. No functionality in modern Rucio. No default. +- **region_code**: Optional[String]. Region code where the RSE is located (e.g., state or province). No default. +- **rse**: String. The name of the Rucio Storage Element as given at creation time. Cannot be changed. +- **rse_type**: String. Specify `DISK` or `TAPE` for control of [QoS](operator/qos_rse_config.md). Default: `DISK`. +- **sign_url**: Optional[String]. Enable cloud support for this storage element. No default. +- **staging_area**: Boolean. Specify if this RSE is a disk buffer to a tertiary storage backend, subject to additional constraints (specifically a lifetime for rules on this RSE must be defined). Default: `False`. +- **time_zone**: Optional[String]. Time zone of the RSE location (e.g., "Europe/Zurich"). No default. +- **verify_checksum**: Boolean. Specifies if the RSE has support for checksum verification. Default: `True`. - **volatile**: Boolean. Specifies if the RSE is cache storage. Subject to volatile RSE restrictions detailed [here](https://rucio.github.io/documentation/operator/qos_rse_config). Default: `False`. -- **write_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for writes. Default: `1`. + +The following fields are compatibility fields that may be present in the settings dictionary: +- **protocol**: RSEProtocolDict. A single protocol configuration (for backward compatibility). +- **prefix**: String. Protocol-specific prefix path. +- **scheme**: String. Protocol scheme (e.g., `https`, `gsiftp`). +- **hostname**: String. Hostname of the storage endpoint. ## RSE attributes -The RSE Attributes are set separately using `rucio.RSEClient.add_rse_attribute` or `rucio rse attribute add`. -and only contains information about the specific RSE's for the Rucio instance. +The RSE Attributes are set separately using `rucio.RSEClient.add_rse_attribute` or `rucio rse attribute add`, +and only contain information about the specific RSEs for the Rucio instance. Read more about RSEs [here](started/concepts/rucio_storage_element.md) and how to set them up [here](operator/configuration.md#creating-new-rses). - **associated_sites**: String. Separated by commas. Used for chaining of subscriptions so that transfers to one RSE will also be mirrored to associated_sites. No default. -- **archive_timeout**: Integer: Only used for transfers with a tape destination. Controls the number of seconds the FTS3 transfer manager will wait for the tape archival of the file to go `FAILED` or `FINISHED`. No default. -- **auto_approve_bytes**: String: Upper limit for the size in bytes of a DID for which rules will be automatically approved. Example: `500GB`. No default. -- **auto_approve_files**: Integer: Upper limit for the number of files covered by a rule which will be automatically approved. No default. -- **available_for_multihop**: Boolean. If True, allow to use this RSE as an intermediate hop in a multi-hop transfer. Default: `False`. +- **archive_timeout**: Integer. Only used for transfers with a tape destination. Controls the number of seconds the FTS3 transfer manager will wait for the tape archival of the file to go `FAILED` or `FINISHED`. No default. +- **auto_approve_bytes**: String. Upper limit for the size in bytes of a DID for which rules will be automatically approved. Example: `500GB`. No default. +- **auto_approve_files**: Integer. Upper limit for the number of files covered by a rule which will be automatically approved. No default. +- **available_for_multihop**: Boolean. If True, allow to use this RSE as an intermediate hop in a multi-hop transfer. Default: `False`. - **block_manual_approval**: Boolean. Disable manual rule approval for this RSE. Default: `False`. - **bittorrent_tracker_addr**: String. Used to configure the URL of the bittorrent tracker API when using the torrent transfer manager. No Default. - **checksum_key**: String. Used to specify an alternate RSE attribute to search for supported checksums beyond those with global support (ADLER32, MD5). Default: `supported_checksums`. @@ -541,15 +550,15 @@ and how to set them up [here](operator/configuration.md#creating-new-rses). - **fts**: String. Specify the REST API URL of the FTS3 transfer manager. No default. - **greedyDeletion**: Boolean. Allow files without a rule locking them to be deleted by a Reaper Daemon. Default behavior only marks a file for deletion when there is no space on an RSE for a new required file. Default: `False`. - **group_by_rse_attribute**: String. Control the RSE attribute (such as `country`) which transfer source RSEs will be grouped by when determining an appropriate transfer source. Default: `UNKNOWN`. +- **lfn2pfn_algorithm**: String. Name of the algorithm to be used for generating paths to files on **deterministic RSEs**. Must be defined in the configured policy package. If not set, defaults to `lfn2pfn_algorithm_default` from the `[policy]` section of the config file. Common values: `identity`, `hash`. Default: `default`. Note: This attribute is also included in the RSE settings dictionary when protocols are retrieved. - **globus_endpoint_id**: String. Specify the REST API URL of the Globus transfer manager. No default. - **hop_penalty**: Integer. Usage cost of this RSE as an intermediate in [multihop transfers](operator_transfers/transfers_overview.md). Overrides the global `transfers/hop_penalty` configuration for this particular RSE. Requires `available_for_multihop` attribute is True on the RSE. No default. - **is_object_store**: Boolean. Control the auditor daemon's behavior. Instead of dumping all files, list them by date. Default: `False`. -- **istape**: Boolean. Default: `False`. -- **lfn2pfn_algorithm**: String. Name of the algorithm to be used for generating paths to files on the storage. Must be defined in the configured policy package. Default: `default`. +- **istape**: Boolean. Default: `False`. Historically used within RSE expressions. Not needed by Rucio itself. Refer to `rse_type` setting instead. - **mock**: Boolean. Default: `False`. - **multihop_tombstone_delay**: Integer. Delay before a multihop transfer intermediate rule is to be deleted. Defined in seconds. Default: `7200`. -- **naming_convention**: String. Name of the algorithm in the configured policy package which is to be used to validate DIDs on this RSE. Default: `None`. +- **naming_convention**: String. Name of the `non_deterministic_pfn` algorithm in the configured policy package used to generate physical filenames for **non-deterministic RSEs** (typically tape systems). Unlike `lfn2pfn_algorithm` (used for deterministic RSEs), this algorithm can use file metadata, dataset membership, and other contextual information to construct filenames. Only applies to replications (not direct uploads). Default: `None`. - **oidc_support**: Boolean. Specifies that the RSE supports OIDC authentication for FTS3 transfers. Default: `False`. - **overwrite_when_only_on_disk**: Boolean. On a `TAPE` RSE, controls if a file can be overwritten. A file may only be overwritten if it has not yet been written to the tape backend. Default: `False`. - **overwrite**: Boolean. Controls if a file can be overwritten on the RSE. Default: `True` for `rse_type: DISK`. `False` for `rse_type: TAPE`. diff --git a/docs/operator/rse_configuration.md b/docs/operator/rse_configuration.md new file mode 100644 index 00000000000..f20807a086e --- /dev/null +++ b/docs/operator/rse_configuration.md @@ -0,0 +1,637 @@ +# RSE Configuration Guide + +This section provides an overview of Rucio Storage Elements (RSEs) from an operator's perspective, covering setup, configuration, and best practices. + +The Rucio Storage Elements (RSEs) are the fundamental storage endpoints in a Rucio deployment. They represent physical or logical storage locations where data files are stored. + +## RSE Configuration Concepts +Before diving into the setup methods, it's important to understand the main concepts that define RSE configuration: + +### RSE Settings vs Attributes + +Rucio distinguishes between **RSE settings** (properties of the RSE itself) and **RSE attributes** (key-value metadata). + +**RSE Settings** (configured via `rucio rse update` or `RSEClient.update_rse()`): +- `rse_type`: Defines whether the RSE is `DISK` or `TAPE` (defaults to `DISK`) +- `verify_checksum`: Boolean controlling checksum verification +- `deterministic`: Boolean controlling whether RSE uses deterministic file paths +- Geographic fields: `city`, `country_name`, `latitude`, `longitude`, `region_code`, `time_zone` + +More can be found in the [RSE Settings configuration page](configuration_parameters/#rse-settings). + +**RSE Attributes** (configured via `rucio rse attribute add` or `RSEClient.add_rse_attribute()`): +- `lfn2pfn_algorithm`: Algorithm name for generating physical paths on **deterministic RSEs** (defaults to `lfn2pfn_algorithm_default` from policy config if not set) +- `naming_convention`: Algorithm name for generating physical file names on **non-deterministic RSEs** (typically tape systems) +- `fts`: FTS endpoint URL (required only if using FTS for third-party transfers) +- `archive_timeout`: Timeout for tape archival completion +- Storage characteristics, transfer policies, monitoring endpoints + +:::danger[Important Configuration Notes] +1. **RSE Type**: Set via `rucio rse update --rse RSE_NAME --rse-type TAPE` (not as an attribute). Defaults to `DISK` if omitted. +2. **FTS Endpoint**: Only required if your deployment uses FTS for transfers. Set as an attribute: `rucio rse attribute add --rse RSE_NAME --key fts --value https://fts.example.com:8446` +3. **LFN2PFN Algorithm**: Must be set as an RSE attribute during initial setup. This is effectively immutable and should be chosen carefully based on whether your RSE is deterministic. +::: + +An exhaustive list of RSE attributes can be found in the [configuration parameters page](configuration_parameters/#rse-attributes). + +### Protocols + +Protocols define the methods clients use to access data on an RSE. Each protocol specifies: +- **Scheme**: The access protocol (e.g., `https`, `root`, `davs`, see [RSE Protocols Documentation](https://rucio.cern.ch/documentation/html/transfer_protocols/posix.html) for more information) +- **Hostname/Port**: Network endpoint for the storage system +- **Prefix**: Base path for file storage +- **Implementation**: Python class handling protocol operations +- **Domains**: Separate configurations for local (LAN) and remote (WAN) access +- **Operations**: Supported operations (read, write, delete, third-party copy) + +Multiple protocols can be configured for a single RSE, allowing different access methods for different use cases. + +### Account Limits + +Account limits (also called quotas) control how much storage space a Rucio account can consume on a specific RSE. Limits are: +- Set per account per RSE +- Specified in bytes (or as `infinity` for unlimited) +- Enforced by Rucio's replication rules and placement decisions + +Without appropriate limits, accounts cannot create replicas on an RSE. + +--- + +## RSE Setup Methods + +There are two primary methods for setting up RSEs in Rucio: + +1. **Command Line Interface (CLI)**: Using the `rucio` CLI tool +2. **Python API**: Using the Rucio core Python library + +Both methods achieve the same result. Choose the method that best fits your workflow. + +### Method 1: Using the Rucio CLI + +The CLI approach is straightforward and suitable for interactive setup or shell scripts. + +**Basic workflow:** + +```bash +# 1. Add the RSE +rucio rse add RSE_NAME + +# 2. Set RSE type +rucio rse update RSE_NAME --key rse_type --value DISK # or TAPE + +# 3. Add attributes +rucio rse attribute add RSE_NAME --key attribute_name --value attribute_value + +# 4. Add protocol(s) +rucio rse protocol add --host hostname --scheme scheme --prefix /path \ + --port port --impl implementation \ + --domain-json '{"wan": {...}, "lan": {...}}' \ + RSE_NAME + +# 5. Set account limits +rucio account limit add account_name --rse RSE_NAME --bytes quota +``` + +### Method 2: Using the Python Client API + +The Python Client API is the recommended approach for automation, complex configurations, or when integrating with other Python tools. + +**Basic workflow:** + +```python +#!/usr/bin/env python +from rucio.client.rseclient import RSEClient +from rucio.client.accountlimitclient import AccountLimitClient + +# Initialize clients +rse_client = RSEClient() +account_limit_client = AccountLimitClient() + +# 1. Add the RSE +rse_client.add_rse(rse='RSE_NAME', rse_type='DISK') # or rse_type='TAPE' + +# 2. Add attributes +rse_client.add_rse_attribute(rse='RSE_NAME', key='lfn2pfn_algorithm', value='identity') +#rse_client.add_rse_attribute(rse='RSE_NAME', key='attribute_name', value='attribute_value') + +# 3. Add protocol +params = { + 'scheme': 'https', + 'hostname': 'hostname', + 'port': 443, + 'prefix': '/path', + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + 'lan': {'read': 1, 'write': 1, 'delete': 1}, + 'wan': {'read': 1, 'write': 1, 'delete': 1, + 'third_party_copy_read': 1, 'third_party_copy_write': 1} + } +} +rse_client.add_protocol('RSE_NAME', params) + +# 4. Set account limits +account_limit_client.set_account_limit('account_name', 'RSE_NAME', bytes_limit) +``` + +--- + +## RSE Configuration Examples + +### FILE/POSIX RSEs + +:::warning +POSIX RSEs use the `file://` protocol and require direct filesystem access. They work when the Rucio server and clients can access the same filesystem path (e.g., via a shared mount). They are suitable for testing, single-node deployments, or when all components share a filesystem. +::: + +**Using CLI:** + +```bash +# Add the RSE, named POSIX_RSE +rucio rse add POSIX_RSE + +# Set backend type +rucio rse attribute add --key backend_type --value POSIX POSIX_RSE + +# Configure storage usage monitoring +rucio rse attribute add --key storage_usage_tool \ + --value 'rucio.rse.protocols.posix.Default.getSpace' POSIX_RSE +# Add protocol +rucio rse protocol add --host localhost \ + --scheme file --prefix '/data/posix-rse-1' \ + --impl 'rucio.rse.protocols.posix.Default' \ + --domain-json '{"lan": {"read": 1, "write": 1, "delete": 1}, "wan": {"read": 1, "write": 1, "delete": 1}}' POSIX_RSE + +# Set account limits +rucio account limit add --rse POSIX_RSE --bytes infinity root +``` + +--- + +### WebDAV RSEs + +WebDAV RSEs provide remote access where external clients need to upload and download files. + +#### Rucio Configuration + +**Using CLI:** + +```bash +RUCIO_RSE_NAME="WEBDAV_RSE" +HOST="webdav.example.com" +PREFIX="/webdav" + +# Add RSE +rucio rse add $RUCIO_RSE_NAME + +# Set RSE type if needed (defaults to DISK) +# rucio rse update --rse-type DISK $RUCIO_RSE_NAME + +# Set attributes +rucio rse attribute add --key lfn2pfn_algorithm --value identity $RUCIO_RSE_NAME +rucio rse attribute add --key fts --value https://fts.example.com:8446 $RUCIO_RSE_NAME +rucio rse attribute add --key greedyDeletion --value true $RUCIO_RSE_NAME + +# Update settings +rucio rse update --verify-checksum false $RUCIO_RSE_NAME + +# Add davs protocol +rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ + --scheme davs --prefix $PREFIX --port 8443 \ + --impl rucio.rse.protocols.gfal.Default \ + --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' + +# Set account limits +rucio account limit add --rse $RUCIO_RSE_NAME --bytes infinity root +``` + +**Using Python Client API:** + +```python +#!/usr/bin/env python +from rucio.client.rseclient import RSEClient + +# Initialize client +rse_client = RSEClient() + +# Add RSE (defaults to DISK type) +rse_client.add_rse('WEBDAV_RSE') + +# Set attributes +rse_client.add_rse_attribute(rse='WEBDAV_RSE', key='lfn2pfn_algorithm', value='identity') +rse_client.add_rse_attribute(rse='WEBDAV_RSE', key='fts', value='https://fts.example.com:8446') +rse_client.add_rse_attribute(rse='WEBDAV_RSE', key='greedyDeletion', value=True) + +# Update settings +rse_client.update_rse(rse='WEBDAV_RSE', parameters={'verify_checksum': False}) + +# Protocol parameters +params = { + 'scheme': 'davs', + 'prefix': '/webdav', + 'hostname': 'webdav.example.com', + 'port': 8443, + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + "lan": { + "read": 1, + "write": 1, + "delete": 1, + "third_party_copy_read": 1, + "third_party_copy_write": 1 + }, + "wan": { + "read": 1, + "write": 1, + "delete": 1, + "third_party_copy_read": 1, + "third_party_copy_write": 1 + } + } +} + +# Setup protocol +rse_client.add_protocol('WEBDAV_RSE', params) +``` + +--- + +### Disk RSEs + +Disk RSEs provide fast access to active data and are typically backed by systems like EOS. + +**Using CLI:** + +```bash +RUCIO_RSE_NAME="EXAMPLE_DISK_RSE" +HOST="storage.example.com" +PREFIX="/storage/rucio" # If using EOS, note that the path is //eos/your-path + +# Create RSE +rucio rse add $RUCIO_RSE_NAME + +# Set RSE type and attributes +rucio rse update $RUCIO_RSE_NAME --key rse_type --value DISK +rucio rse attribute add $RUCIO_RSE_NAME --key lfn2pfn_algorithm --value identity +rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts.example.com:8446 + +# Add HTTPS protocol (primary for transfers) +rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ + --scheme https \ + --prefix $PREFIX \ + --port 8444 \ + --impl rucio.rse.protocols.gfal.Default \ + --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' + +# Add ROOT protocol (alternative access method) +rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ + --scheme root \ + --prefix $PREFIX \ + --port 1094 \ + --impl rucio.rse.protocols.gfal.Default \ + --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' + +# Set account limits +rucio account limit add root --rse $RUCIO_RSE_NAME --bytes infinity +``` + +**Using Python Client API:** + +```python +#!/usr/bin/env python +from rucio.client.rseclient import RSEClient + +# Initialize client +rse_client = RSEClient() + +# Add RSE (defaults to DISK type) +rse_client.add_rse('EXAMPLE_DISK_RSE') + +# Set attributes +rse_client.add_rse_attribute(rse='EXAMPLE_DISK_RSE', key='lfn2pfn_algorithm', value='identity') +rse_client.add_rse_attribute(rse='EXAMPLE_DISK_RSE', key='fts', value='https://fts.example.com:8446') + +# Add HTTPS protocol +https_params = { + 'scheme': 'https', + 'hostname': 'storage.example.com', + 'port': 8444, + 'prefix': '/storage/rucio', + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + 'lan': {'read': 1, 'write': 1, 'delete': 1}, + 'wan': {'read': 1, 'write': 1, 'delete': 1, + 'third_party_copy_read': 1, 'third_party_copy_write': 1} + } +} +rse_client.add_protocol('EXAMPLE_DISK_RSE', https_params) + +# Add ROOT protocol +root_params = { + 'scheme': 'root', + 'hostname': 'storage.example.com', + 'port': 1094, + 'prefix': '/storage/rucio', + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + 'lan': {'read': 1, 'write': 1, 'delete': 1}, + 'wan': {'read': 1, 'write': 1, 'delete': 1, + 'third_party_copy_read': 1, 'third_party_copy_write': 1} + } +} +rse_client.add_protocol('EXAMPLE_DISK_RSE', root_params) +``` + +:::info +- Both `https` and `root` protocols are defined for flexibility +- The `identity` algorithm means the scope is appended to the prefix path +- Multiple deployments can share the same prefix if they use different scopes +::: + +--- + +### Tape RSEs + +Tape RSEs are used for long-term archival storage with higher latency but larger capacity. + +**Using CLI:** + +```bash +RUCIO_RSE_NAME="EXAMPLE_TAPE_RSE" +HOST="tape-storage.example.com" +PREFIX="/tape/archive" # If using EOS, note that the path is //eos/your-path + +# Create RSE +rucio rse add $RUCIO_RSE_NAME + +# Set RSE type and attributes +rucio rse update $RUCIO_RSE_NAME --key rse_type --value TAPE +rucio rse attribute add $RUCIO_RSE_NAME --key lfn2pfn_algorithm --value identity +rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts.example.com:8446 +rucio rse attribute add $RUCIO_RSE_NAME --key archive_timeout --value 86400 + +# Add HTTPS protocol only (root protocol can cause issues with tape) +rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ + --scheme https \ + --prefix $PREFIX \ + --port 8444 \ + --impl rucio.rse.protocols.gfal.Default \ + --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' + +# Set account limits +rucio account limit add root --rse $RUCIO_RSE_NAME --bytes infinity +``` + +**Using Python Client API:** + +```python +#!/usr/bin/env python +from rucio.client.rseclient import RSEClient + +# Initialize client +rse_client = RSEClient() + +# Add RSE with TAPE type +rse_client.add_rse('EXAMPLE_TAPE_RSE', rse_type='TAPE') + +# Set attributes +rse_client.add_rse_attribute(rse='EXAMPLE_TAPE_RSE', key='lfn2pfn_algorithm', value='identity') +rse_client.add_rse_attribute(rse='EXAMPLE_TAPE_RSE', key='fts', value='https://fts.example.com:8446') +rse_client.add_rse_attribute(rse='EXAMPLE_TAPE_RSE', key='archive_timeout', value=86400) + +# Add HTTPS protocol +params = { + 'scheme': 'https', + 'hostname': 'tape-storage.example.com', + 'port': 8444, + 'prefix': '/tape/archive', + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + 'lan': {'read': 1, 'write': 1, 'delete': 1}, + 'wan': {'read': 1, 'write': 1, 'delete': 1, + 'third_party_copy_read': 1, 'third_party_copy_write': 1} + } +} +rse_client.add_protocol('EXAMPLE_TAPE_RSE', params) +``` + +**Key Differences from Disk RSEs:** +- `rse_type` is set to `TAPE` at creation time +- `archive_timeout` attribute specifies maximum time for file staging (86400 = 24 hours) +- Typically only `https` protocol is defined (avoid `root` protocol for tape) + + + +--- + +### Setting RSE Attributes + +```bash +# Using rucio CLI +rucio rse attribute add RSE_NAME --key attribute_name --value attribute_value + +# Using Python Client API +from rucio.client.rseclient import RSEClient +rse_client = RSEClient() +rse_client.add_rse_attribute(rse='RSE_NAME', key='attribute_name', value='attribute_value') +``` + +--- + +## Protocols + +Protocols define how clients access data on an RSE. Multiple protocols can be defined for the same RSE. +See the [RSE Protocols Documentation](https://rucio.cern.ch/documentation/html/transfer_protocols/posix.html) for a full list of parameters. + +### Protocol Priority + +The priority values in the `domain-json` determine which protocol operations are enabled: +- Use `1` to enable an operation +- Use `0` to disable an operation +- Omitting an operation key indicates it's not supported + +**Example:** For a read-only RSE, set `"write": 0, "delete": 0` in the domains configuration. + +### Domain Configuration + +```json +{ + "lan": { + "read": 1, + "write": 1, + "delete": 1 + }, + "wan": { + "read": 1, + "write": 1, + "delete": 1, + "third_party_copy_read": 1, + "third_party_copy_write": 1 + } +} +``` + +- `lan`: Local Area Network (same site) +- `wan`: Wide Area Network (remote sites) +- `third_party_copy`: Direct transfers between RSEs via FTS + +--- + +## Account Limits + +Account limits control how much space an account can use on an RSE. + +### Setting Limits + +```bash +# Set unlimited quota for root account +rucio account limit add root --rse RSE_NAME --bytes infinity + +# Set specific quota (in bytes) +rucio account limit add user_account --rse RSE_NAME --bytes 1TB +``` + +--- + +## Non-Deterministic RSEs + +Non-deterministic RSEs don't follow the standard deterministic logical-to-physical file naming convention. This is most common for tape systems where physical filenames are generated based on file metadata, dataset membership, or other factors beyond just scope and name. + +### Key Differences from Deterministic RSEs + +**Deterministic RSEs** (typical disk systems): +- Use `lfn2pfn_algorithm` **setting** to derive physical paths from scope and name alone +- Support both direct uploads and replications +- Physical file path can be computed on-demand from the logical file name +- Example: File `scope:name` → `/storage/rucio/scope/name` + +**Non-Deterministic RSEs** (typical tape systems): +- Use `naming_convention` **attribute** to generate physical filenames +- The algorithm may use file metadata, dataset membership, creation time, etc. +- Only support replications (not direct uploads from clients) +- Physical file paths must be registered explicitly when files are replicated +- Example: File might be stored as `/tape/2026/02/03/archive_12345.tar` + +### Setup Example + +```bash +RUCIO_RSE_NAME="EXPERIMENT_TAPE_RSE" + +# Add as non-deterministic RSE +rucio rse add --non-deterministic $RUCIO_RSE_NAME + +# Set type and attributes +rucio rse update --key rse_type --value TAPE $RUCIO_RSE_NAME +rucio rse attribute add --key naming_convention --value experiment_tape $RUCIO_RSE_NAME +``` + +**When to Use:** +- Tape systems with internal file organization requiring metadata-driven naming +- Storage systems that don't support directory structures +- Any storage backend where the physical filename cannot be deterministically computed from LFN alone + +:::warning +For non-deterministic RSEs: +1. Physical file paths must be registered explicitly during replication +2. Files cannot be uploaded directly by clients (only via replication) +3. The `naming_convention` attribute must reference a `non_deterministic_pfn` algorithm in your [policy package](policy_packages/policy_packages_overview) +4. Do NOT set `lfn2pfn_algorithm` setting on non-deterministic RSEs +::: + +--- + +## Best Practices + +### 1. RSE Distance Configuration +RSE distance must be properly configured for transfers to work between RSEs. + +:::danger[Critical] +Please ensure to set distances in both directions if transfers are bidirectional. + +See the [**related storage documentation**](../started/concepts/rucio_storage_element/#distances-between-rses) for more details. +::: + +```bash +# Set distance between RSEs (lower value = closer) +rucio rse distance add --distance 1 SOURCE_RSE DEST_RSE +# Set reverse distance if needed +rucio rse distance add --distance 1 DEST_RSE SOURCE_RSE +``` + +### 2. Protocol Selection + +- **Disk RSEs:** Define both `https` and `root` protocols for flexibility +- **Tape RSEs:** Use only `https` protocol to avoid timeout issues +- **Testing:** Use `file://` protocol for local development only + +### 3. Prefix Planning + +When using the `identity` lfn2pfn algorithm, the scope is appended to the prefix: +- Use generic prefixes like `/storage/rucio` or `/data/experiment` +- Each deployment can use the same prefix with different scopes +- Example: Files with scope `mydata` will be under `/storage/rucio/mydata/` + +### 4. Security + +- **Certificates:** Keep host certificates up to date +- **Access Control:** Set appropriate account limits +- **SELinux:** Ensure proper context for web-accessible directories + +### 5. Monitoring + +- Set `storage_usage_tool` for RSE space monitoring +- Configure FTS endpoint for transfer tracking +- Enable logging for protocol operations + +### 6. Tape-Specific + +- Set reasonable `archive_timeout` values (24 hours recommended). This controls how long FTS waits for file staging to complete +- Use staging-aware clients for file retrieval +- Avoid `root://` protocol for tape systems +- Monitor staging queue length + +### 7. Testing + +Before deploying to production: +1. Test file upload/download +2. Verify checksum handling +3. Test third-party copy between RSEs +4. Check deletion operations +5. Validate account limits + +### 8. Common Pitfalls + +- Forgetting to set RSE distances +- Wrong file permissions on WebDAV directories +- Missing SELinux context +- Using `root://` protocol for tape RSEs +- Not setting FTS endpoint for transfers +- Insufficient archive timeout for tape staging + +--- + +## Additional Resources + +- [FTS3 Documentation](https://fts3-docs.web.cern.ch/) +- [EOS Documentation](https://eos-docs.web.cern.ch/) +- [CTA Documentation](https://eoscta.docs.cern.ch/) + +--- + +## Quick Reference Commands + +```bash +# List all RSEs +rucio rse list + +# Show RSE details +rucio rse show RSE_NAME + +# List RSE attributes +rucio rse attribute list RSE_NAME + +# List account limits +rucio account limit list ACCOUNT_NAME + +# Test file upload +rucio upload --rse RSE_NAME --scope SCOPE file.txt +```