From ecb50bb93f3234f68462023a736c22f6293ad440 Mon Sep 17 00:00:00 2001 From: Giovanni Guerrieri Date: Fri, 9 Jan 2026 14:48:39 +0100 Subject: [PATCH 1/7] Add RSE configuration guide for operators Documentation on setting up and configuring Rucio Storage Elements (RSEs) from an operator's perspective. Includes: - Overview of RSE types (POSIX, WebDAV, Disk, Tape) - Two setup methods: CLI and Python API with side-by-side examples - Configuration examples for each RSE type - WebDAV setup with Apache configuration and davs protocol - EOS disk RSE with https and root protocols - CTA tape RSE configuration with staging timeouts - RSE attributes, protocols, and account limits reference - Best practices and common pitfalls - Quick reference commands The examples use the latest `rucio` CLI commands Co-authored-by: Nikita Avdeev Co-authored-by: Luis Antonio Obis Aparicio --- docs/operator/rse_configuration.md | 672 +++++++++++++++++++++++++++++ 1 file changed, 672 insertions(+) create mode 100644 docs/operator/rse_configuration.md diff --git a/docs/operator/rse_configuration.md b/docs/operator/rse_configuration.md new file mode 100644 index 00000000000..b34145560bc --- /dev/null +++ b/docs/operator/rse_configuration.md @@ -0,0 +1,672 @@ +# RSE Configuration Guide + +This section provides an overview of Rucio Storage Elements (RSEs) from an operator's perspective, covering setup, configuration, and best practices. + +The Rucio Storage Elements (RSEs) are the fundamental storage endpoints in a Rucio deployment. They represent physical or logical storage locations where data files are stored. + +## RSE Types + +Rucio supports several types of RSEs, each suited for different use cases: + +### 1. FILE/POSIX RSEs +- Simple local filesystem access +- Limited to local machine access +- Best for testing or single-node deployments + +### 2. WebDAV RSEs +- Web-based access via HTTP/HTTPS +- Supports remote access +- Requires web server configuration + +### 3. DISK RSEs +- Standard disk-based storage +- Fast access for active data +- Examples: EOS, dCache, local filesystem + +### 4. TAPE RSEs +- Long-term archival storage +- Higher latency but larger capacity +- Examples: CTA (CERN Tape Archive) +--- + +## RSE Setup Methods + +There are two primary methods for setting up RSEs in Rucio: + +1. **Command Line Interface (CLI)**: Using the `rucio` CLI tool +2. **Python API**: Using the Rucio core Python library + +Both methods achieve the same result. Choose the method that best fits your workflow. + +### Method 1: Using the Rucio CLI + +The CLI approach is straightforward and suitable for interactive setup or shell scripts. + +**Basic workflow:** + +```bash +# 1. Add the RSE +rucio rse add RSE_NAME + +# 2. Set RSE type +rucio rse update RSE_NAME --key rse_type --value DISK # or TAPE + +# 3. Add attributes +rucio rse attribute add RSE_NAME --key attribute_name --value attribute_value + +# 4. Add protocol(s) +rucio rse protocol add --host hostname --scheme scheme --prefix /path \ + --port port --impl implementation \ + --domain-json '{"wan": {...}, "lan": {...}}' \ + RSE_NAME + +# 5. Set account limits +rucio account limit add account_name --rse RSE_NAME --bytes quota +``` + +### Method 2: Using the Python API + +The Python API is useful for automation, complex configurations, or when integrating with other Python tools. + +**Basic workflow:** + +```python +#!/usr/bin/env python +from rucio.core.rse import add_rse, add_protocol, get_rse_id, add_rse_attribute +from rucio.core.account import set_account_limit + +# 1. Add the RSE +add_rse('RSE_NAME') +rse_id = get_rse_id('RSE_NAME') + +# 2. Add attributes +add_rse_attribute(rse_id, key='rse_type', value='DISK') +add_rse_attribute(rse_id, key='attribute_name', value='attribute_value') + +# 3. Add protocol +params = { + 'scheme': 'https', + 'hostname': 'hostname', + 'port': 443, + 'prefix': '/path', + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + 'lan': {'read': 1, 'write': 1, 'delete': 1}, + 'wan': {'read': 1, 'write': 1, 'delete': 1, + 'third_party_copy_read': 1, 'third_party_copy_write': 1} + } +} +add_protocol(rse_id, params) + +# 4. Set account limits +set_account_limit('account_name', rse_id, bytes_limit) +``` + +--- + +## RSE Configuration Examples + +### FILE/POSIX RSEs + +:::warning +POSIX RSEs are the simplest type but cannot be accessed from remote machines. They are suitable for testing or single-node deployments only. +::: + +**Using CLI:** + +```bash +# Add the RSE +rucio rse add POSIX_RSE + +# Set backend type +rucio rse attribute add POSIX_RSE --key backend_type --value POSIX + +# Configure storage usage monitoring +rucio rse attribute add POSIX_RSE --key storage_usage_tool \ + --value 'rucio.rse.protocols.posix.Default.getSpace' + +# Add protocol +rucio rse protocol add --host localhost \ + --scheme file \ + --prefix '/data/posix-rse-1' \ + --impl 'rucio.rse.protocols.posix.Default' \ + --domain-json '{"lan": {"read": 1, "write": 1, "delete": 1}, "wan": {"read": 1, "write": 1, "delete": 1}}' \ + POSIX_RSE + +# Set account limits +rucio account limit add root --rse POSIX_RSE --bytes infinity +``` + +--- + +### WebDAV RSEs + +WebDAV RSEs provide remote access where external clients need to upload and download files. + +#### Apache Configuration + +WebDAV RSEs require an Apache web server with WebDAV and SSL modules enabled. + +**File:** `/etc/httpd/conf.d/rse.conf` + +```apache +DAVLockDB /var/lib/dav/lockdb + + + + ServerAdmin admin@hostname + ServerName hostname + ServerAlias hostname + DocumentRoot /path/to/rse + + SSLEngine on + SSLCertificateFile /etc/grid-security/hostcert.pem + SSLCertificateKeyFile /etc/grid-security/hostkey.pem + SSLCACertificateFile /etc/grid-security/ca.pem + + Alias /webdav /path/to/rse + + + DAV On + AllowOverride None + Options None + SSLVerifyClient require + SSLVerifyDepth 2 + SSLOptions +StdEnvVars + SSLCACertificateFile /etc/grid-security/ca.pem + + # For x509 authentication (recommended) + Require valid-user + + # For open access (NOT RECOMMENDED for production) + # Require all granted + + + + +``` + +**File System Permissions:** + +Apache requires correct ownership and SELinux context: + +```bash +# Set ownership +chown -R apache:apache /path/to/rse +chmod -R 755 /path/to/rse + +# Set SELinux context +semanage fcontext -a -t httpd_sys_rw_content_t "/path/to/rse(/.*)?" +restorecon -Rv /path/to/rse +``` + +:::note +If `semanage` is not available, install it with: +```bash +dnf install policycoreutils-python-utils +``` +::: + +#### Rucio Configuration + +**Using CLI:** + +```bash +RUCIO_RSE_NAME="WEBDAV_RSE" +HOST="webdav.example.com" +PREFIX="/webdav" + +# Add RSE +rucio rse add $RUCIO_RSE_NAME + +# Set attributes +rucio rse update $RUCIO_RSE_NAME --key rse_type --value DISK +rucio rse attribute add $RUCIO_RSE_NAME --key istape --value False +rucio rse attribute add $RUCIO_RSE_NAME --key verify_checksum --value False +rucio rse attribute add $RUCIO_RSE_NAME --key greedyDeletion --value True + +# Add davs protocol +rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ + --scheme davs \ + --prefix $PREFIX \ + --port 443 \ + --impl rucio.rse.protocols.gfal.Default \ + --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' + +# Set account limits +rucio account limit add root --rse $RUCIO_RSE_NAME --bytes infinity +``` + +**Using Python API:** + +```python +#!/usr/bin/env python +from rucio.core.rse import add_rse, add_protocol, get_rse_id, add_rse_attribute + +# Protocol parameters +params = { + 'scheme': 'davs', + 'prefix': '/webdav', + 'hostname': 'webdav.example.com', + 'port': 443, + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + "lan": { + "read": 1, + "write": 1, + "delete": 1, + "third_party_copy_read": 1, + "third_party_copy_write": 1 + }, + "wan": { + "read": 1, + "write": 1, + "delete": 1, + "third_party_copy_read": 1, + "third_party_copy_write": 1 + } + } +} + +# Add RSE +add_rse('WEBDAV_RSE') +rse_id = get_rse_id('WEBDAV_RSE') + +# Setup protocol +add_protocol(rse_id, params) + +# Set attributes +add_rse_attribute(rse_id, key='istape', value='False') +add_rse_attribute(rse_id, key='verify_checksum', value='False') +add_rse_attribute(rse_id, key='greedyDeletion', value='True') +``` + +--- + +### Disk RSEs + +Disk RSEs provide fast access to active data and are typically backed by systems like EOS. + +**Using CLI:** + +```bash +RUCIO_RSE_NAME="EXAMPLE_DISK_RSE" +HOST="eospublic.cern.ch" +PREFIX="//eos/workspace/e/experiment/tmp" + +# Create RSE +rucio rse add $RUCIO_RSE_NAME + +# Set RSE type and attributes +rucio rse update $RUCIO_RSE_NAME --key rse_type --value DISK +rucio rse attribute add $RUCIO_RSE_NAME --key lfn2pfn_algorithm --value identity +rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts3-pilot.cern.ch:8446 + +# Add HTTPS protocol (primary for transfers) +rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ + --scheme https \ + --prefix $PREFIX \ + --port 8444 \ + --impl rucio.rse.protocols.gfal.Default \ + --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' + +# Add ROOT protocol (alternative access method) +rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ + --scheme root \ + --prefix $PREFIX \ + --port 1094 \ + --impl rucio.rse.protocols.gfal.Default \ + --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' + +# Set account limits +rucio account limit add root --rse $RUCIO_RSE_NAME --bytes infinity +``` + +**Using Python API:** + +```python +#!/usr/bin/env python +from rucio.core.rse import add_rse, add_protocol, get_rse_id, add_rse_attribute + +# Add RSE +add_rse('EXAMPLE_DISK_RSE') +rse_id = get_rse_id('EXAMPLE_DISK_RSE') + +# Set attributes +add_rse_attribute(rse_id, key='rse_type', value='DISK') +add_rse_attribute(rse_id, key='lfn2pfn_algorithm', value='identity') +add_rse_attribute(rse_id, key='fts', value='https://fts3-pilot.cern.ch:8446') + +# Add HTTPS protocol +https_params = { + 'scheme': 'https', + 'hostname': 'eospublic.cern.ch', + 'port': 8444, + 'prefix': '//eos/workspace/e/experiment/tmp', + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + 'lan': {'read': 1, 'write': 1, 'delete': 1}, + 'wan': {'read': 1, 'write': 1, 'delete': 1, + 'third_party_copy_read': 1, 'third_party_copy_write': 1} + } +} +add_protocol(rse_id, https_params) + +# Add ROOT protocol +root_params = { + 'scheme': 'root', + 'hostname': 'eospublic.cern.ch', + 'port': 1094, + 'prefix': '//eos/workspace/e/experiment/tmp', + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + 'lan': {'read': 1, 'write': 1, 'delete': 1}, + 'wan': {'read': 1, 'write': 1, 'delete': 1, + 'third_party_copy_read': 1, 'third_party_copy_write': 1} + } +} +add_protocol(rse_id, root_params) +``` + +:::info +- Both `https` and `root` protocols are defined for flexibility +- The `identity` algorithm means the scope is appended to the prefix path +- Multiple deployments can share the same prefix if they use different scopes +::: + +--- + +### Tape RSEs + +Tape RSEs are used for long-term archival storage with higher latency but larger capacity. + +**Using CLI:** + +```bash +RUCIO_RSE_NAME="EXAMPLE_TAPE_RSE" +HOST="eosctaatlaspps.cern.ch" +PREFIX="//eos/ctaatlaspps/archivetest/experiment/tmp" + +# Create RSE +rucio rse add $RUCIO_RSE_NAME + +# Set RSE type and attributes +rucio rse update $RUCIO_RSE_NAME --key rse_type --value TAPE +rucio rse attribute add $RUCIO_RSE_NAME --key lfn2pfn_algorithm --value identity +rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts3-pilot.cern.ch:8446 +rucio rse attribute add $RUCIO_RSE_NAME --key archive_timeout --value 86400 + +# Add HTTPS protocol only (root protocol can cause issues with tape) +rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ + --scheme https \ + --prefix $PREFIX \ + --port 8444 \ + --impl rucio.rse.protocols.gfal.Default \ + --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' + +# Set account limits +rucio account limit add root --rse $RUCIO_RSE_NAME --bytes infinity +``` + +**Using Python API:** + +```python +#!/usr/bin/env python +from rucio.core.rse import add_rse, add_protocol, get_rse_id, add_rse_attribute + +# Add RSE +add_rse('EXAMPLE_TAPE_RSE') +rse_id = get_rse_id('EXAMPLE_TAPE_RSE') + +# Set attributes +add_rse_attribute(rse_id, key='rse_type', value='TAPE') +add_rse_attribute(rse_id, key='lfn2pfn_algorithm', value='identity') +add_rse_attribute(rse_id, key='fts', value='https://fts3-pilot.cern.ch:8446') +add_rse_attribute(rse_id, key='archive_timeout', value='86400') + +# Add HTTPS protocol +params = { + 'scheme': 'https', + 'hostname': 'eosctaatlaspps.cern.ch', + 'port': 8444, + 'prefix': '//eos/ctaatlaspps/archivetest/experiment/tmp', + 'impl': 'rucio.rse.protocols.gfal.Default', + 'domains': { + 'lan': {'read': 1, 'write': 1, 'delete': 1}, + 'wan': {'read': 1, 'write': 1, 'delete': 1, + 'third_party_copy_read': 1, 'third_party_copy_write': 1} + } +} +add_protocol(rse_id, params) +``` + +**Key Differences from Disk RSEs:** +- `rse_type` is set to `TAPE` +- `archive_timeout` attribute specifies maximum time for file staging (86400 = 24 hours) +- Typically only `https` protocol is defined (avoid `root` protocol for tape) + +**Testing Environment:** +At CERN, the path `//eos/ctaatlaspps/archivetest/experiment/tmp` can be used for tape testing. This is a temporary space where data is not expected to be kept long-term. + +--- + +## RSE Attributes + +RSE attributes control behavior and capabilities. Common attributes include: + +| Attribute | Description | Example Values | +|-----------|-------------|----------------| +| `rse_type` | Type of storage | `DISK`, `TAPE` | +| `lfn2pfn_algorithm` | Logical to physical name mapping | `identity`, `hash` | +| `fts` | FTS (File Transfer Service) endpoint | `https://fts3-pilot.cern.ch:8446` | +| `istape` | Is this a tape system? | `True`, `False` | +| `verify_checksum` | Enable checksum verification | `True`, `False` | +| `greedyDeletion` | Delete files immediately when removed | `True`, `False` | +| `archive_timeout` | Timeout for tape staging (seconds) | `86400` (24 hours) | +| `naming_convention` | Custom naming scheme | Custom algorithm name | + +### Setting Attributes + +```bash +# Using rucio CLI +rucio rse attribute add RSE_NAME --key attribute_name --value attribute_value + +# Using Python API +from rucio.core.rse import add_rse_attribute, get_rse_id +rse_id = get_rse_id('RSE_NAME') +add_rse_attribute(rse_id, key='attribute_name', value='attribute_value') +``` + +--- + +## Protocols + +Protocols define how clients access data on an RSE. Multiple protocols can be defined for the same RSE. + +### Common Protocol Schemes + +- `file://` - Local filesystem (POSIX) +- `https://` - HTTP with SSL/TLS +- `davs://` - WebDAV with SSL/TLS +- `root://` - ROOT protocol (XRootD) +- `srm://` - Storage Resource Manager +- `s3://` - S3 compatible storage; for more information, refer to the [Rucio S3 Protocol Documentation](s3_rse_config/#how-to-setup-an-s3-rse). +- `gridftp://` - GridFTP protocol + +### Protocol Priority + +The priority values in the `domain-json` determine which protocol is preferred: +- Higher numbers indicate higher priority +- Priority `1` is standard for active protocols +- Priority `0` or omitted disables the protocol + +### Domain Configuration + +```json +{ + "lan": { + "read": 1, + "write": 1, + "delete": 1 + }, + "wan": { + "read": 1, + "write": 1, + "delete": 1, + "third_party_copy_read": 1, + "third_party_copy_write": 1 + } +} +``` + +- `lan`: Local Area Network (same site) +- `wan`: Wide Area Network (remote sites) +- `third_party_copy`: Direct transfers between RSEs via FTS + +--- + +## Account Limits + +Account limits control how much space an account can use on an RSE. + +### Setting Limits + +```bash +# Set unlimited quota for root account +rucio account limit add root --rse RSE_NAME --bytes infinity + +# Set specific quota (in bytes) +rucio account limit add user_account --rse RSE_NAME --bytes 1099511627776 # 1 TB +``` + +### Common Quota Values + +```bash +1 TB = 1099511627776 bytes +1 PB = 1125899906842624 bytes +infinity = unlimited +``` + +--- + +## Non-Deterministic RSEs + +Non-deterministic RSEs don't follow the standard logical-to-physical naming convention. This is common for tape systems where files may be stored with system-generated names. + +### Setup Example + +```bash +RUCIO_RSE_NAME="ESCAPE_TAPE_RSE" + +# Add as non-deterministic +rucio rse add --non-deterministic $RUCIO_RSE_NAME + +# Set type and attributes +rucio rse update $RUCIO_RSE_NAME --key rse_type --value TAPE +rucio rse attribute add $RUCIO_RSE_NAME --key naming_convention --value escape_tape +``` + +**When to Use:** +- Tape systems with internal file organization +- Storage systems that don't support directory structures +- Systems requiring custom naming schemes + +--- + +## Best Practices + +### 1. RSE Distance Configuration + +:::danger[Critical] +RSE distance must be properly configured for transfers to work between RSEs. +::: + +```bash +# Set distance between RSEs (lower value = closer) +rucio rse distance add --distance 1 SOURCE_RSE DEST_RSE +``` + +### 2. Protocol Selection + +- **Disk RSEs:** Define both `https` and `root` protocols for flexibility +- **Tape RSEs:** Use only `https` protocol to avoid timeout issues +- **Testing:** Use `file://` protocol for local development only + +### 3. Prefix Planning + +When using the `identity` lfn2pfn algorithm, the scope is appended to the prefix: +- Use generic prefixes like `//eos/workspace/e/experiment/tmp` +- Each deployment can use the same prefix with different scopes +- Example: Files with scope `ams` will be under `//eos/ams/` + +### 4. Security + +- **Certificates:** Keep host certificates up to date +- **Access Control:** Set appropriate account limits +- **SELinux:** Ensure proper context for web-accessible directories + +### 5. Monitoring + +- Set `storage_usage_tool` for RSE space monitoring +- Configure FTS endpoint for transfer tracking +- Enable logging for protocol operations + +### 6. Tape-Specific + +- Set reasonable `archive_timeout` values (24 hours recommended) +- Use staging-aware clients +- Avoid `root://` protocol for tape systems +- Monitor staging queue length + +### 7. Testing + +Before deploying to production: +1. Test file upload/download +2. Verify checksum handling +3. Test third-party copy between RSEs +4. Check deletion operations +5. Validate account limits + +### 8. Common Pitfalls + +- Forgetting to set RSE distances +- Wrong file permissions on WebDAV directories +- Missing SELinux context +- Using `root://` protocol for tape RSEs +- Not setting FTS endpoint for transfers +- Insufficient archive timeout for tape staging + +--- + +## Additional Resources + +- [FTS3 Documentation](https://fts3-docs.web.cern.ch/) +- [EOS Documentation](https://eos-docs.web.cern.ch/) +- [CTA Documentation](https://eoscta.docs.cern.ch/) + +--- + +## Quick Reference Commands + +```bash +# List all RSEs +rucio rse list + +# Show RSE details +rucio rse info RSE_NAME + +# List RSE attributes +rucio rse attribute list RSE_NAME + +# List RSE protocols +rucio rse protocol list RSE_NAME + +# Check RSE usage +rucio rse usage RSE_NAME + +# List account limits +rucio account limit list ACCOUNT_NAME + +# Test file upload +rucio upload --rse RSE_NAME --scope SCOPE file.txt +``` From de5fc2d8b9f1bb840de1fd7f492761e50da558af Mon Sep 17 00:00:00 2001 From: Giovanni Guerrieri Date: Sat, 10 Jan 2026 09:15:17 +0100 Subject: [PATCH 2/7] fix: pre commit format --- docs/operator/rse_configuration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/operator/rse_configuration.md b/docs/operator/rse_configuration.md index b34145560bc..8145890748b 100644 --- a/docs/operator/rse_configuration.md +++ b/docs/operator/rse_configuration.md @@ -174,10 +174,10 @@ DAVLockDB /var/lib/dav/lockdb SSLVerifyDepth 2 SSLOptions +StdEnvVars SSLCACertificateFile /etc/grid-security/ca.pem - + # For x509 authentication (recommended) Require valid-user - + # For open access (NOT RECOMMENDED for production) # Require all granted From 22717730cc144cadb1fbf759682059ca796be018 Mon Sep 17 00:00:00 2001 From: Giovanni Guerrieri Date: Mon, 12 Jan 2026 15:58:05 +0100 Subject: [PATCH 3/7] fix: links to attributes and protocols, and python API using client instead of core Used RSEClient class for rse operations, and AccountLimitClient for account. Additionally added a paragraph about configuration concepts. --- docs/operator/rse_configuration.md | 186 ++++++++++++++++------------- 1 file changed, 104 insertions(+), 82 deletions(-) diff --git a/docs/operator/rse_configuration.md b/docs/operator/rse_configuration.md index 8145890748b..31273a876e1 100644 --- a/docs/operator/rse_configuration.md +++ b/docs/operator/rse_configuration.md @@ -27,6 +27,43 @@ Rucio supports several types of RSEs, each suited for different use cases: - Long-term archival storage - Higher latency but larger capacity - Examples: CTA (CERN Tape Archive) + +## RSE Configuration Concepts +Before diving into the setup methods, it's important to understand the main concepts that define RSE configuration: + +### RSE Attributes + +RSE attributes are key-value pairs that control the behavior and capabilities of an RSE. They define how Rucio interacts with the storage system. + +**Common attributes include:** +- Storage characteristics (type, capacity, availability) +- File naming and organization conventions +- Transfer and checksum policies +- Integration endpoints (FTS, monitoring) + +An exhaustive list of RSE attributes can be found in the [RSE attributes page](configuration_parameters/#rse-attributes). + +### Protocols + +Protocols define the methods clients use to access data on an RSE. Each protocol specifies: +- **Scheme**: The access protocol (e.g., `https`, `root`, `davs`) +- **Hostname/Port**: Network endpoint for the storage system +- **Prefix**: Base path for file storage +- **Implementation**: Python class handling protocol operations +- **Domains**: Separate configurations for local (LAN) and remote (WAN) access +- **Operations**: Supported operations (read, write, delete, third-party copy) + +Multiple protocols can be configured for a single RSE, allowing different access methods for different use cases. + +### Account Limits + +Account limits (also called quotas) control how much storage space a Rucio account can consume on a specific RSE. Limits are: +- Set per account per RSE +- Specified in bytes (or as `infinity` for unlimited) +- Enforced by Rucio's replication rules and placement decisions + +Without appropriate limits, accounts cannot create replicas on an RSE. + --- ## RSE Setup Methods @@ -64,24 +101,27 @@ rucio rse protocol add --host hostname --scheme scheme --prefix /path \ rucio account limit add account_name --rse RSE_NAME --bytes quota ``` -### Method 2: Using the Python API +### Method 2: Using the Python Client API -The Python API is useful for automation, complex configurations, or when integrating with other Python tools. +The Python Client API is the recommended approach for automation, complex configurations, or when integrating with other Python tools. **Basic workflow:** ```python #!/usr/bin/env python -from rucio.core.rse import add_rse, add_protocol, get_rse_id, add_rse_attribute -from rucio.core.account import set_account_limit +from rucio.client.rseclient import RSEClient +from rucio.client.accountlimitclient import AccountLimitClient + +# Initialize clients +rse_client = RSEClient() +account_limit_client = AccountLimitClient() # 1. Add the RSE -add_rse('RSE_NAME') -rse_id = get_rse_id('RSE_NAME') +rse_client.add_rse('RSE_NAME') # 2. Add attributes -add_rse_attribute(rse_id, key='rse_type', value='DISK') -add_rse_attribute(rse_id, key='attribute_name', value='attribute_value') +rse_client.add_rse_attribute('RSE_NAME', 'rse_type', 'DISK') +rse_client.add_rse_attribute('RSE_NAME', 'attribute_name', 'attribute_value') # 3. Add protocol params = { @@ -96,10 +136,10 @@ params = { 'third_party_copy_read': 1, 'third_party_copy_write': 1} } } -add_protocol(rse_id, params) +rse_client.add_protocol('RSE_NAME', params) # 4. Set account limits -set_account_limit('account_name', rse_id, bytes_limit) +account_limit_client.set_account_limit('account_name', 'RSE_NAME', bytes_limit) ``` --- @@ -237,11 +277,14 @@ rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ rucio account limit add root --rse $RUCIO_RSE_NAME --bytes infinity ``` -**Using Python API:** +**Using Python Client API:** ```python #!/usr/bin/env python -from rucio.core.rse import add_rse, add_protocol, get_rse_id, add_rse_attribute +from rucio.client.rseclient import RSEClient + +# Initialize client +rse_client = RSEClient() # Protocol parameters params = { @@ -269,16 +312,15 @@ params = { } # Add RSE -add_rse('WEBDAV_RSE') -rse_id = get_rse_id('WEBDAV_RSE') +rse_client.add_rse('WEBDAV_RSE') # Setup protocol -add_protocol(rse_id, params) +rse_client.add_protocol('WEBDAV_RSE', params) # Set attributes -add_rse_attribute(rse_id, key='istape', value='False') -add_rse_attribute(rse_id, key='verify_checksum', value='False') -add_rse_attribute(rse_id, key='greedyDeletion', value='True') +rse_client.add_rse_attribute('WEBDAV_RSE', 'istape', 'False') +rse_client.add_rse_attribute('WEBDAV_RSE', 'verify_checksum', 'False') +rse_client.add_rse_attribute('WEBDAV_RSE', 'greedyDeletion', 'True') ``` --- @@ -291,8 +333,8 @@ Disk RSEs provide fast access to active data and are typically backed by systems ```bash RUCIO_RSE_NAME="EXAMPLE_DISK_RSE" -HOST="eospublic.cern.ch" -PREFIX="//eos/workspace/e/experiment/tmp" +HOST="storage.example.com" +PREFIX="/storage/rucio" # If using EOS, note that the path is //eos/your-path # Create RSE rucio rse add $RUCIO_RSE_NAME @@ -300,7 +342,7 @@ rucio rse add $RUCIO_RSE_NAME # Set RSE type and attributes rucio rse update $RUCIO_RSE_NAME --key rse_type --value DISK rucio rse attribute add $RUCIO_RSE_NAME --key lfn2pfn_algorithm --value identity -rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts3-pilot.cern.ch:8446 +rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts.example.com:8446 # Add HTTPS protocol (primary for transfers) rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ @@ -322,27 +364,29 @@ rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ rucio account limit add root --rse $RUCIO_RSE_NAME --bytes infinity ``` -**Using Python API:** +**Using Python Client API:** ```python #!/usr/bin/env python -from rucio.core.rse import add_rse, add_protocol, get_rse_id, add_rse_attribute +from rucio.client.rseclient import RSEClient + +# Initialize client +rse_client = RSEClient() # Add RSE -add_rse('EXAMPLE_DISK_RSE') -rse_id = get_rse_id('EXAMPLE_DISK_RSE') +rse_client.add_rse('EXAMPLE_DISK_RSE') # Set attributes -add_rse_attribute(rse_id, key='rse_type', value='DISK') -add_rse_attribute(rse_id, key='lfn2pfn_algorithm', value='identity') -add_rse_attribute(rse_id, key='fts', value='https://fts3-pilot.cern.ch:8446') +rse_client.add_rse_attribute('EXAMPLE_DISK_RSE', 'rse_type', 'DISK') +rse_client.add_rse_attribute('EXAMPLE_DISK_RSE', 'lfn2pfn_algorithm', 'identity') +rse_client.add_rse_attribute('EXAMPLE_DISK_RSE', 'fts', 'https://fts.example.com:8446') # Add HTTPS protocol https_params = { 'scheme': 'https', - 'hostname': 'eospublic.cern.ch', + 'hostname': 'storage.example.com', 'port': 8444, - 'prefix': '//eos/workspace/e/experiment/tmp', + 'prefix': '/storage/rucio', 'impl': 'rucio.rse.protocols.gfal.Default', 'domains': { 'lan': {'read': 1, 'write': 1, 'delete': 1}, @@ -350,14 +394,14 @@ https_params = { 'third_party_copy_read': 1, 'third_party_copy_write': 1} } } -add_protocol(rse_id, https_params) +rse_client.add_protocol('EXAMPLE_DISK_RSE', https_params) # Add ROOT protocol root_params = { 'scheme': 'root', - 'hostname': 'eospublic.cern.ch', + 'hostname': 'storage.example.com', 'port': 1094, - 'prefix': '//eos/workspace/e/experiment/tmp', + 'prefix': '/storage/rucio', 'impl': 'rucio.rse.protocols.gfal.Default', 'domains': { 'lan': {'read': 1, 'write': 1, 'delete': 1}, @@ -365,7 +409,7 @@ root_params = { 'third_party_copy_read': 1, 'third_party_copy_write': 1} } } -add_protocol(rse_id, root_params) +rse_client.add_protocol('EXAMPLE_DISK_RSE', root_params) ``` :::info @@ -384,16 +428,17 @@ Tape RSEs are used for long-term archival storage with higher latency but larger ```bash RUCIO_RSE_NAME="EXAMPLE_TAPE_RSE" -HOST="eosctaatlaspps.cern.ch" -PREFIX="//eos/ctaatlaspps/archivetest/experiment/tmp" +HOST="tape-storage.example.com" +PREFIX="/tape/archive" # If using EOS, note that the path is //eos/your-path # Create RSE rucio rse add $RUCIO_RSE_NAME # Set RSE type and attributes rucio rse update $RUCIO_RSE_NAME --key rse_type --value TAPE +rucio rse attribute add $RUCIO_RSE_NAME --key istape --value True rucio rse attribute add $RUCIO_RSE_NAME --key lfn2pfn_algorithm --value identity -rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts3-pilot.cern.ch:8446 +rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts.example.com:8446 rucio rse attribute add $RUCIO_RSE_NAME --key archive_timeout --value 86400 # Add HTTPS protocol only (root protocol can cause issues with tape) @@ -408,28 +453,30 @@ rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ rucio account limit add root --rse $RUCIO_RSE_NAME --bytes infinity ``` -**Using Python API:** +**Using Python Client API:** ```python #!/usr/bin/env python -from rucio.core.rse import add_rse, add_protocol, get_rse_id, add_rse_attribute +from rucio.client.rseclient import RSEClient + +# Initialize client +rse_client = RSEClient() # Add RSE -add_rse('EXAMPLE_TAPE_RSE') -rse_id = get_rse_id('EXAMPLE_TAPE_RSE') +rse_client.add_rse('EXAMPLE_TAPE_RSE') # Set attributes -add_rse_attribute(rse_id, key='rse_type', value='TAPE') -add_rse_attribute(rse_id, key='lfn2pfn_algorithm', value='identity') -add_rse_attribute(rse_id, key='fts', value='https://fts3-pilot.cern.ch:8446') -add_rse_attribute(rse_id, key='archive_timeout', value='86400') +rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'rse_type', 'TAPE') +rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'lfn2pfn_algorithm', 'identity') +rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'fts', 'https://fts.example.com:8446') +rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'archive_timeout', '86400') # Add HTTPS protocol params = { 'scheme': 'https', - 'hostname': 'eosctaatlaspps.cern.ch', + 'hostname': 'tape-storage.example.com', 'port': 8444, - 'prefix': '//eos/ctaatlaspps/archivetest/experiment/tmp', + 'prefix': '/tape/archive', 'impl': 'rucio.rse.protocols.gfal.Default', 'domains': { 'lan': {'read': 1, 'write': 1, 'delete': 1}, @@ -437,7 +484,7 @@ params = { 'third_party_copy_read': 1, 'third_party_copy_write': 1} } } -add_protocol(rse_id, params) +rse_client.add_protocol('EXAMPLE_TAPE_RSE', params) ``` **Key Differences from Disk RSEs:** @@ -445,36 +492,20 @@ add_protocol(rse_id, params) - `archive_timeout` attribute specifies maximum time for file staging (86400 = 24 hours) - Typically only `https` protocol is defined (avoid `root` protocol for tape) -**Testing Environment:** -At CERN, the path `//eos/ctaatlaspps/archivetest/experiment/tmp` can be used for tape testing. This is a temporary space where data is not expected to be kept long-term. ---- -## RSE Attributes - -RSE attributes control behavior and capabilities. Common attributes include: - -| Attribute | Description | Example Values | -|-----------|-------------|----------------| -| `rse_type` | Type of storage | `DISK`, `TAPE` | -| `lfn2pfn_algorithm` | Logical to physical name mapping | `identity`, `hash` | -| `fts` | FTS (File Transfer Service) endpoint | `https://fts3-pilot.cern.ch:8446` | -| `istape` | Is this a tape system? | `True`, `False` | -| `verify_checksum` | Enable checksum verification | `True`, `False` | -| `greedyDeletion` | Delete files immediately when removed | `True`, `False` | -| `archive_timeout` | Timeout for tape staging (seconds) | `86400` (24 hours) | -| `naming_convention` | Custom naming scheme | Custom algorithm name | +--- -### Setting Attributes +### Setting RSE Attributes ```bash # Using rucio CLI rucio rse attribute add RSE_NAME --key attribute_name --value attribute_value -# Using Python API -from rucio.core.rse import add_rse_attribute, get_rse_id -rse_id = get_rse_id('RSE_NAME') -add_rse_attribute(rse_id, key='attribute_name', value='attribute_value') +# Using Python Client API +from rucio.client.rseclient import RSEClient +rse_client = RSEClient() +rse_client.add_rse_attribute('RSE_NAME', 'attribute_name', 'attribute_value') ``` --- @@ -482,16 +513,7 @@ add_rse_attribute(rse_id, key='attribute_name', value='attribute_value') ## Protocols Protocols define how clients access data on an RSE. Multiple protocols can be defined for the same RSE. - -### Common Protocol Schemes - -- `file://` - Local filesystem (POSIX) -- `https://` - HTTP with SSL/TLS -- `davs://` - WebDAV with SSL/TLS -- `root://` - ROOT protocol (XRootD) -- `srm://` - Storage Resource Manager -- `s3://` - S3 compatible storage; for more information, refer to the [Rucio S3 Protocol Documentation](s3_rse_config/#how-to-setup-an-s3-rse). -- `gridftp://` - GridFTP protocol +See the [RSE Protocols Documentation](https://rucio.cern.ch/documentation/html/transfer_protocols/posix.html) for a full list of parameters. ### Protocol Priority @@ -595,9 +617,9 @@ rucio rse distance add --distance 1 SOURCE_RSE DEST_RSE ### 3. Prefix Planning When using the `identity` lfn2pfn algorithm, the scope is appended to the prefix: -- Use generic prefixes like `//eos/workspace/e/experiment/tmp` +- Use generic prefixes like `/storage/rucio` or `/data/experiment` - Each deployment can use the same prefix with different scopes -- Example: Files with scope `ams` will be under `//eos/ams/` +- Example: Files with scope `mydata` will be under `/storage/rucio/mydata/` ### 4. Security From f80b901ee85bf11c2d383fb720afe0fa136ad869 Mon Sep 17 00:00:00 2001 From: Giovanni Guerrieri Date: Thu, 15 Jan 2026 15:23:37 +0100 Subject: [PATCH 4/7] fix: elaborate on istape attribute Removed istape from RSE config guide, as it's not needed by Rucio and it can be replaced by rse_type. Additionally, added a clearer description of istape in the attributes page. --- docs/operator/configuration_parameters.md | 2 +- docs/operator/rse_configuration.md | 44 ++++++++--------------- 2 files changed, 15 insertions(+), 31 deletions(-) diff --git a/docs/operator/configuration_parameters.md b/docs/operator/configuration_parameters.md index 5ec5f34e4a2..4426ad2d5e8 100644 --- a/docs/operator/configuration_parameters.md +++ b/docs/operator/configuration_parameters.md @@ -545,7 +545,7 @@ and how to set them up [here](operator/configuration.md#creating-new-rses). - **hop_penalty**: Integer. Usage cost of this RSE as an intermediate in [multihop transfers](operator_transfers/transfers_overview.md). Overrides the global `transfers/hop_penalty` configuration for this particular RSE. Requires `available_for_multihop` attribute is True on the RSE. No default. - **is_object_store**: Boolean. Control the auditor daemon's behavior. Instead of dumping all files, list them by date. Default: `False`. -- **istape**: Boolean. Default: `False`. +- **istape**: Boolean. Default: `False`. Historically used within RSE expressions. Not needed by Rucio itself. Refer to `rse_type` instead. - **lfn2pfn_algorithm**: String. Name of the algorithm to be used for generating paths to files on the storage. Must be defined in the configured policy package. Default: `default`. - **mock**: Boolean. Default: `False`. - **multihop_tombstone_delay**: Integer. Delay before a multihop transfer intermediate rule is to be deleted. Defined in seconds. Default: `7200`. diff --git a/docs/operator/rse_configuration.md b/docs/operator/rse_configuration.md index 31273a876e1..c236a01eb04 100644 --- a/docs/operator/rse_configuration.md +++ b/docs/operator/rse_configuration.md @@ -4,30 +4,6 @@ This section provides an overview of Rucio Storage Elements (RSEs) from an opera The Rucio Storage Elements (RSEs) are the fundamental storage endpoints in a Rucio deployment. They represent physical or logical storage locations where data files are stored. -## RSE Types - -Rucio supports several types of RSEs, each suited for different use cases: - -### 1. FILE/POSIX RSEs -- Simple local filesystem access -- Limited to local machine access -- Best for testing or single-node deployments - -### 2. WebDAV RSEs -- Web-based access via HTTP/HTTPS -- Supports remote access -- Requires web server configuration - -### 3. DISK RSEs -- Standard disk-based storage -- Fast access for active data -- Examples: EOS, dCache, local filesystem - -### 4. TAPE RSEs -- Long-term archival storage -- Higher latency but larger capacity -- Examples: CTA (CERN Tape Archive) - ## RSE Configuration Concepts Before diving into the setup methods, it's important to understand the main concepts that define RSE configuration: @@ -41,12 +17,19 @@ RSE attributes are key-value pairs that control the behavior and capabilities of - Transfer and checksum policies - Integration endpoints (FTS, monitoring) +:::danger[Mandatory attributes] +To ensure proper operation of Rucio with the RSE, these attributes must be set: +1. `rse_type`: Defines whether the RSE is of type `DISK` or `TAPE`. +2. `fts`: URL of the FTS (File Transfer Service) endpoint used for managing transfers. +3. `lfn2pfn_algorithm`: Algorithm for mapping logical file names (LFNs) to physical file names (PFNs). +::: + An exhaustive list of RSE attributes can be found in the [RSE attributes page](configuration_parameters/#rse-attributes). ### Protocols Protocols define the methods clients use to access data on an RSE. Each protocol specifies: -- **Scheme**: The access protocol (e.g., `https`, `root`, `davs`) +- **Scheme**: The access protocol (e.g., `https`, `root`, `davs`, see [RSE Protocols Documentation](https://rucio.cern.ch/documentation/html/transfer_protocols/posix.html) for more information) - **Hostname/Port**: Network endpoint for the storage system - **Prefix**: Base path for file storage - **Implementation**: Python class handling protocol operations @@ -261,7 +244,8 @@ rucio rse add $RUCIO_RSE_NAME # Set attributes rucio rse update $RUCIO_RSE_NAME --key rse_type --value DISK -rucio rse attribute add $RUCIO_RSE_NAME --key istape --value False +rucio rse attribute add $RUCIO_RSE_NAME --key lfn2pfn_algorithm --value identity +rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts.example.com:8446 rucio rse attribute add $RUCIO_RSE_NAME --key verify_checksum --value False rucio rse attribute add $RUCIO_RSE_NAME --key greedyDeletion --value True @@ -269,7 +253,7 @@ rucio rse attribute add $RUCIO_RSE_NAME --key greedyDeletion --value True rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ --scheme davs \ --prefix $PREFIX \ - --port 443 \ + --port 8443 \ --impl rucio.rse.protocols.gfal.Default \ --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' @@ -291,7 +275,7 @@ params = { 'scheme': 'davs', 'prefix': '/webdav', 'hostname': 'webdav.example.com', - 'port': 443, + 'port': 8443, 'impl': 'rucio.rse.protocols.gfal.Default', 'domains': { "lan": { @@ -318,7 +302,8 @@ rse_client.add_rse('WEBDAV_RSE') rse_client.add_protocol('WEBDAV_RSE', params) # Set attributes -rse_client.add_rse_attribute('WEBDAV_RSE', 'istape', 'False') +rse_client.add_rse_attribute('WEBDAV_RSE', 'lfn2pfn_algorithm', 'identity') +rse_client.add_rse_attribute('WEBDAV_RSE', 'fts', 'https://fts.example.com:8446') rse_client.add_rse_attribute('WEBDAV_RSE', 'verify_checksum', 'False') rse_client.add_rse_attribute('WEBDAV_RSE', 'greedyDeletion', 'True') ``` @@ -436,7 +421,6 @@ rucio rse add $RUCIO_RSE_NAME # Set RSE type and attributes rucio rse update $RUCIO_RSE_NAME --key rse_type --value TAPE -rucio rse attribute add $RUCIO_RSE_NAME --key istape --value True rucio rse attribute add $RUCIO_RSE_NAME --key lfn2pfn_algorithm --value identity rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts.example.com:8446 rucio rse attribute add $RUCIO_RSE_NAME --key archive_timeout --value 86400 From 040ee732b99342818b6b65528c6be748b02eeda9 Mon Sep 17 00:00:00 2001 From: Giovanni Guerrieri Date: Tue, 20 Jan 2026 20:31:18 +0100 Subject: [PATCH 5/7] fix: remove apache webdav config and add bidirectional distance between RSEs --- docs/operator/rse_configuration.md | 71 +++--------------------------- 1 file changed, 6 insertions(+), 65 deletions(-) diff --git a/docs/operator/rse_configuration.md b/docs/operator/rse_configuration.md index c236a01eb04..df7d6ef4d16 100644 --- a/docs/operator/rse_configuration.md +++ b/docs/operator/rse_configuration.md @@ -166,70 +166,6 @@ rucio account limit add root --rse POSIX_RSE --bytes infinity WebDAV RSEs provide remote access where external clients need to upload and download files. -#### Apache Configuration - -WebDAV RSEs require an Apache web server with WebDAV and SSL modules enabled. - -**File:** `/etc/httpd/conf.d/rse.conf` - -```apache -DAVLockDB /var/lib/dav/lockdb - - - - ServerAdmin admin@hostname - ServerName hostname - ServerAlias hostname - DocumentRoot /path/to/rse - - SSLEngine on - SSLCertificateFile /etc/grid-security/hostcert.pem - SSLCertificateKeyFile /etc/grid-security/hostkey.pem - SSLCACertificateFile /etc/grid-security/ca.pem - - Alias /webdav /path/to/rse - - - DAV On - AllowOverride None - Options None - SSLVerifyClient require - SSLVerifyDepth 2 - SSLOptions +StdEnvVars - SSLCACertificateFile /etc/grid-security/ca.pem - - # For x509 authentication (recommended) - Require valid-user - - # For open access (NOT RECOMMENDED for production) - # Require all granted - - - - -``` - -**File System Permissions:** - -Apache requires correct ownership and SELinux context: - -```bash -# Set ownership -chown -R apache:apache /path/to/rse -chmod -R 755 /path/to/rse - -# Set SELinux context -semanage fcontext -a -t httpd_sys_rw_content_t "/path/to/rse(/.*)?" -restorecon -Rv /path/to/rse -``` - -:::note -If `semanage` is not available, install it with: -```bash -dnf install policycoreutils-python-utils -``` -::: - #### Rucio Configuration **Using CLI:** @@ -582,14 +518,19 @@ rucio rse attribute add $RUCIO_RSE_NAME --key naming_convention --value escape_t ## Best Practices ### 1. RSE Distance Configuration +RSE distance must be properly configured for transfers to work between RSEs. :::danger[Critical] -RSE distance must be properly configured for transfers to work between RSEs. +Please ensure to set distances in both directions if transfers are bidirectional. + +See the [**related storage documentation**](../started/concepts/rucio_storage_element/#distances-between-rses) for more details. ::: ```bash # Set distance between RSEs (lower value = closer) rucio rse distance add --distance 1 SOURCE_RSE DEST_RSE +# Set reverse distance if needed +rucio rse distance add --distance 1 DEST_RSE SOURCE_RSE ``` ### 2. Protocol Selection From a5a0b217e6d4e2a756c84e03529899c648966ee0 Mon Sep 17 00:00:00 2001 From: Giovanni Guerrieri Date: Tue, 3 Feb 2026 14:50:39 +0100 Subject: [PATCH 6/7] fix/chore: RSE configuration documentation improvements This commit consolidates multiple documentation corrections and clarifications: - Corrected RSE settings vs attributes distinction - Fixed TypedDict field listings to match actual implementation - Removed non-existent protocol priority fields (priority_lan, priority_wan) - Clarified that lfn2pfn_algorithm is an RSE attribute, not a creation parameter - Restored geographic fields (city, country_name, latitude, longitude, region_code, time_zone) as valid RSE settings that can be set via gateway API - Clarified deterministic vs non-deterministic RSE behavior - lfn2pfn_algorithm: for deterministic RSEs (disk), computes paths from scope+name only - naming_convention: for non-deterministic RSEs (tape), uses metadata/timestamps - Added detailed comparison table explaining the differences - Updated all examples to reflect correct usage patterns - Fixed Python API examples** - Removed incorrect lfn2pfn_algorithm parameter from add_rse() calls - Shows correct attribute-based configuration via add_rse_attribute() - Updated workflow examples to match actual client implementation - CLI fixes - Minor fixes to commands structure --- docs/operator/configuration_parameters.md | 62 ++++---- docs/operator/rse_configuration.md | 181 ++++++++++++---------- 2 files changed, 136 insertions(+), 107 deletions(-) diff --git a/docs/operator/configuration_parameters.md b/docs/operator/configuration_parameters.md index 4426ad2d5e8..83f37c25c77 100644 --- a/docs/operator/configuration_parameters.md +++ b/docs/operator/configuration_parameters.md @@ -495,43 +495,53 @@ a setting already specified in the Configuration File. - ***VO***: _(Optional)_ Internal short VO name. No default. ## RSE settings -The RSE settings are set separately using `rucio.RSEClient.update_rse` or `rucio rse update`, and specifies RSE configuration used by the Rucio instance. +The RSE settings are set separately using `rucio.RSEClient.update_rse` or `rucio rse update`, and specify RSE configuration used by the Rucio instance. Mutable settings are `deterministic`, `rse_type`, `staging_area`, `volatile`, `qos_class`, `availability_delete`, `availability_read`, `availability_write`, `city`, `country_name`, `latitude`, `longitude`, `region_code`, and `time_zone`. +Geographic fields (`city`, `country_name`, `latitude`, `longitude`, `region_code`, `time_zone`) can also be set at RSE creation time via API parameters, though they are stored and returned as part of the RSESettingsDict structure. Read more about RSEs [here](started/concepts/rucio_storage_element.md) and how to set them up [here](operator/configuration.md#creating-new-rses). - **availability_delete**: Boolean. Control if this RSE allows deletions by the Reaper daemon using any protocol. Default: `True`. - **availability_read**: Boolean. Control if this RSE allows reads using any protocol. Default: `True`. - **availability_write**: Boolean. Control if this RSE allows writes using any protocol. Default: `True`. -- **credentials**: Dictionary[String, Any]: Specify token credentials used for accessing this RSE if it is in a commercial cloud. No default. -- **delete_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for deletions. Default: `1`. -- **deterministic**: Boolean: Controls if the RSE is allowed to generate paths based solely on the DID (scope:filename). More info about non-deterministic RSEs [here](started/concepts/replica_workflow.md#replica-paths-on-storage). Default: `True`. -- **domain**: List[String]: Specifies the locations this RSE may be accessed by clients from. Cannot be changed. -- **id**: String: Identification string of the RSE. Cannot be changed. -- **lfn2pfn_algorithm**: String: Name of the algorithm in the configured policy package to be used for creating replica paths. Cannot be changed. If no lfn2pfn_algorithm attribute is set, then the setting defaults to lfn2pfn_algorithm_default in the configured policy package. Default: `default`. -- **qos_class**: String: No functionality in modern Rucio. No default. -- **read_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for reads. Default: `1`. -- **rse**: String: The name of the Rucio Storage Element as given at creation time. Cannot be changed. -- **rse_type**: String: Specify `DISK` or `TAPE` for control of [QoS](operator/qos_rse_config.md). Default: `DISK`. -- **sign_url**: Optional[str]. Enable cloud support for this storage element. No default. -- **staging_area**: Boolean.Specify if this RSE is a disk buffer to a tertiary storage backend, subject to additional constraints (specifically a lifetime for rules on this RSE must be defined). Default: `False`. -- **third_party_copy_read_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for TPC reads. Default `1`. -- **third_party_copy_write_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for TPC writes. Default `1`. -- **verify_checksum**: Boolean: Specifies if the RSE has support for checksum verification. Default: `True`. +- **city**: Optional[String]. City where the RSE is located. Used for geographic RSE selection. No default. +- **continent**: Optional[String]. Continent where the RSE is located. No default. +- **country_name**: Optional[String]. Country where the RSE is located. Used for geographic RSE selection. No default. +- **credentials**: Optional[Dictionary[String, Any]]. Specify token credentials used for accessing this RSE if it is in a commercial cloud. No default. +- **deterministic**: Boolean. Controls if the RSE is allowed to generate paths based solely on the DID (scope:filename). More info about non-deterministic RSEs [here](started/concepts/replica_workflow.md#replica-paths-on-storage). Default: `True`. +- **domain**: List[String]. Specifies the locations this RSE may be accessed by clients from. Cannot be changed. +- **id**: String. Identification string of the RSE. Cannot be changed. +- **latitude**: Optional[Float]. Latitude coordinate of the RSE location. Used for geographic calculations. No default. +- **lfn2pfn_algorithm**: String. Name of the algorithm in the configured policy package to be used for creating replica paths. This field is populated from the RSE attribute of the same name. If the attribute is not set, it defaults to `lfn2pfn_algorithm_default` from the configured policy package. Default: `default`. +- **longitude**: Optional[Float]. Longitude coordinate of the RSE location. Used for geographic calculations. No default. +- **protocols**: List[RSEProtocolDict]. List of protocol configurations for accessing this RSE. Cannot be changed directly; use protocol-specific methods. +- **qos_class**: Optional[String]. No functionality in modern Rucio. No default. +- **region_code**: Optional[String]. Region code where the RSE is located (e.g., state or province). No default. +- **rse**: String. The name of the Rucio Storage Element as given at creation time. Cannot be changed. +- **rse_type**: String. Specify `DISK` or `TAPE` for control of [QoS](operator/qos_rse_config.md). Default: `DISK`. +- **sign_url**: Optional[String]. Enable cloud support for this storage element. No default. +- **staging_area**: Boolean. Specify if this RSE is a disk buffer to a tertiary storage backend, subject to additional constraints (specifically a lifetime for rules on this RSE must be defined). Default: `False`. +- **time_zone**: Optional[String]. Time zone of the RSE location (e.g., "Europe/Zurich"). No default. +- **verify_checksum**: Boolean. Specifies if the RSE has support for checksum verification. Default: `True`. - **volatile**: Boolean. Specifies if the RSE is cache storage. Subject to volatile RSE restrictions detailed [here](https://rucio.github.io/documentation/operator/qos_rse_config). Default: `False`. -- **write_protocol**: Integer: Cannot be changed. Controls matching of protocol priorities for writes. Default: `1`. + +The following fields are compatibility fields that may be present in the settings dictionary: +- **protocol**: RSEProtocolDict. A single protocol configuration (for backward compatibility). +- **prefix**: String. Protocol-specific prefix path. +- **scheme**: String. Protocol scheme (e.g., `https`, `gsiftp`). +- **hostname**: String. Hostname of the storage endpoint. ## RSE attributes -The RSE Attributes are set separately using `rucio.RSEClient.add_rse_attribute` or `rucio rse attribute add`. -and only contains information about the specific RSE's for the Rucio instance. +The RSE Attributes are set separately using `rucio.RSEClient.add_rse_attribute` or `rucio rse attribute add`, +and only contain information about the specific RSEs for the Rucio instance. Read more about RSEs [here](started/concepts/rucio_storage_element.md) and how to set them up [here](operator/configuration.md#creating-new-rses). - **associated_sites**: String. Separated by commas. Used for chaining of subscriptions so that transfers to one RSE will also be mirrored to associated_sites. No default. -- **archive_timeout**: Integer: Only used for transfers with a tape destination. Controls the number of seconds the FTS3 transfer manager will wait for the tape archival of the file to go `FAILED` or `FINISHED`. No default. -- **auto_approve_bytes**: String: Upper limit for the size in bytes of a DID for which rules will be automatically approved. Example: `500GB`. No default. -- **auto_approve_files**: Integer: Upper limit for the number of files covered by a rule which will be automatically approved. No default. -- **available_for_multihop**: Boolean. If True, allow to use this RSE as an intermediate hop in a multi-hop transfer. Default: `False`. +- **archive_timeout**: Integer. Only used for transfers with a tape destination. Controls the number of seconds the FTS3 transfer manager will wait for the tape archival of the file to go `FAILED` or `FINISHED`. No default. +- **auto_approve_bytes**: String. Upper limit for the size in bytes of a DID for which rules will be automatically approved. Example: `500GB`. No default. +- **auto_approve_files**: Integer. Upper limit for the number of files covered by a rule which will be automatically approved. No default. +- **available_for_multihop**: Boolean. If True, allow to use this RSE as an intermediate hop in a multi-hop transfer. Default: `False`. - **block_manual_approval**: Boolean. Disable manual rule approval for this RSE. Default: `False`. - **bittorrent_tracker_addr**: String. Used to configure the URL of the bittorrent tracker API when using the torrent transfer manager. No Default. - **checksum_key**: String. Used to specify an alternate RSE attribute to search for supported checksums beyond those with global support (ADLER32, MD5). Default: `supported_checksums`. @@ -541,15 +551,15 @@ and how to set them up [here](operator/configuration.md#creating-new-rses). - **fts**: String. Specify the REST API URL of the FTS3 transfer manager. No default. - **greedyDeletion**: Boolean. Allow files without a rule locking them to be deleted by a Reaper Daemon. Default behavior only marks a file for deletion when there is no space on an RSE for a new required file. Default: `False`. - **group_by_rse_attribute**: String. Control the RSE attribute (such as `country`) which transfer source RSEs will be grouped by when determining an appropriate transfer source. Default: `UNKNOWN`. +- **lfn2pfn_algorithm**: String. Name of the algorithm to be used for generating paths to files on **deterministic RSEs**. Must be defined in the configured policy package. If not set, defaults to `lfn2pfn_algorithm_default` from the `[policy]` section of the config file. Common values: `identity`, `hash`. Default: `default`. Note: This attribute is also included in the RSE settings dictionary when protocols are retrieved. - **globus_endpoint_id**: String. Specify the REST API URL of the Globus transfer manager. No default. - **hop_penalty**: Integer. Usage cost of this RSE as an intermediate in [multihop transfers](operator_transfers/transfers_overview.md). Overrides the global `transfers/hop_penalty` configuration for this particular RSE. Requires `available_for_multihop` attribute is True on the RSE. No default. - **is_object_store**: Boolean. Control the auditor daemon's behavior. Instead of dumping all files, list them by date. Default: `False`. -- **istape**: Boolean. Default: `False`. Historically used within RSE expressions. Not needed by Rucio itself. Refer to `rse_type` instead. -- **lfn2pfn_algorithm**: String. Name of the algorithm to be used for generating paths to files on the storage. Must be defined in the configured policy package. Default: `default`. +- **istape**: Boolean. Default: `False`. Historically used within RSE expressions. Not needed by Rucio itself. Refer to `rse_type` setting instead. - **mock**: Boolean. Default: `False`. - **multihop_tombstone_delay**: Integer. Delay before a multihop transfer intermediate rule is to be deleted. Defined in seconds. Default: `7200`. -- **naming_convention**: String. Name of the algorithm in the configured policy package which is to be used to validate DIDs on this RSE. Default: `None`. +- **naming_convention**: String. Name of the `non_deterministic_pfn` algorithm in the configured policy package used to generate physical filenames for **non-deterministic RSEs** (typically tape systems). Unlike `lfn2pfn_algorithm` (used for deterministic RSEs), this algorithm can use file metadata, dataset membership, and other contextual information to construct filenames. Only applies to replications (not direct uploads). Default: `None`. - **oidc_support**: Boolean. Specifies that the RSE supports OIDC authentication for FTS3 transfers. Default: `False`. - **overwrite_when_only_on_disk**: Boolean. On a `TAPE` RSE, controls if a file can be overwritten. A file may only be overwritten if it has not yet been written to the tape backend. Default: `False`. - **overwrite**: Boolean. Controls if a file can be overwritten on the RSE. Default: `True` for `rse_type: DISK`. `False` for `rse_type: TAPE`. diff --git a/docs/operator/rse_configuration.md b/docs/operator/rse_configuration.md index df7d6ef4d16..c00646faaed 100644 --- a/docs/operator/rse_configuration.md +++ b/docs/operator/rse_configuration.md @@ -7,24 +7,32 @@ The Rucio Storage Elements (RSEs) are the fundamental storage endpoints in a Ruc ## RSE Configuration Concepts Before diving into the setup methods, it's important to understand the main concepts that define RSE configuration: -### RSE Attributes +### RSE Settings vs Attributes -RSE attributes are key-value pairs that control the behavior and capabilities of an RSE. They define how Rucio interacts with the storage system. +Rucio distinguishes between **RSE settings** (properties of the RSE record itself) and **RSE attributes** (key-value metadata). -**Common attributes include:** -- Storage characteristics (type, capacity, availability) -- File naming and organization conventions -- Transfer and checksum policies -- Integration endpoints (FTS, monitoring) +**RSE Settings** (configured via `rucio rse update` or `RSEClient.update_rse()`): +- `rse_type`: Defines whether the RSE is `DISK` or `TAPE` (defaults to `DISK`) +- `verify_checksum`: Boolean controlling checksum verification +- `deterministic`: Boolean controlling whether RSE uses deterministic file paths +- Geographic fields: `city`, `country_name`, `latitude`, `longitude`, `region_code`, `time_zone` -:::danger[Mandatory attributes] -To ensure proper operation of Rucio with the RSE, these attributes must be set: -1. `rse_type`: Defines whether the RSE is of type `DISK` or `TAPE`. -2. `fts`: URL of the FTS (File Transfer Service) endpoint used for managing transfers. -3. `lfn2pfn_algorithm`: Algorithm for mapping logical file names (LFNs) to physical file names (PFNs). +More can be found in the [RSE Settings configuration page](configuration_parameters/#rse-settings). + +**RSE Attributes** (configured via `rucio rse attribute add` or `RSEClient.add_rse_attribute()`): +- `lfn2pfn_algorithm`: Algorithm name for generating physical paths on **deterministic RSEs** (defaults to `lfn2pfn_algorithm_default` from policy config if not set) +- `naming_convention`: Algorithm name for generating physical file names on **non-deterministic RSEs** (typically tape systems) +- `fts`: FTS endpoint URL (required only if using FTS for third-party transfers) +- `archive_timeout`: Timeout for tape archival completion +- Storage characteristics, transfer policies, monitoring endpoints + +:::danger[Important Configuration Notes] +1. **RSE Type**: Set via `rucio rse update --rse RSE_NAME --rse-type TAPE` (not as an attribute). Defaults to `DISK` if omitted. +2. **FTS Endpoint**: Only required if your deployment uses FTS for transfers. Set as an attribute: `rucio rse attribute add --rse RSE_NAME --key fts --value https://fts.example.com:8446` +3. **LFN2PFN Algorithm**: Must be set as an RSE attribute during initial setup. This is effectively immutable and should be chosen carefully based on whether your RSE is deterministic. ::: -An exhaustive list of RSE attributes can be found in the [RSE attributes page](configuration_parameters/#rse-attributes). +An exhaustive list of RSE attributes can be found in the [configuration parameters page](configuration_parameters/#rse-settings). ### Protocols @@ -100,11 +108,11 @@ rse_client = RSEClient() account_limit_client = AccountLimitClient() # 1. Add the RSE -rse_client.add_rse('RSE_NAME') +rse_client.add_rse('RSE_NAME', rse_type='DISK') # or rse_type='TAPE' # 2. Add attributes -rse_client.add_rse_attribute('RSE_NAME', 'rse_type', 'DISK') -rse_client.add_rse_attribute('RSE_NAME', 'attribute_name', 'attribute_value') +rse_client.add_rse_attribute('RSE_NAME', 'lfn2pfn_algorithm', 'identity') +#rse_client.add_rse_attribute('RSE_NAME', 'attribute_name', 'attribute_value') # 3. Add protocol params = { @@ -132,7 +140,7 @@ account_limit_client.set_account_limit('account_name', 'RSE_NAME', bytes_limit) ### FILE/POSIX RSEs :::warning -POSIX RSEs are the simplest type but cannot be accessed from remote machines. They are suitable for testing or single-node deployments only. +POSIX RSEs use the `file://` protocol and require direct filesystem access. They work when the Rucio server and clients can access the same filesystem path (e.g., via a shared mount). They are suitable for testing, single-node deployments, or when all components share a filesystem. ::: **Using CLI:** @@ -142,22 +150,19 @@ POSIX RSEs are the simplest type but cannot be accessed from remote machines. Th rucio rse add POSIX_RSE # Set backend type -rucio rse attribute add POSIX_RSE --key backend_type --value POSIX +rucio rse attribute add --key backend_type --value POSIX POSIX_RSE # Configure storage usage monitoring -rucio rse attribute add POSIX_RSE --key storage_usage_tool \ - --value 'rucio.rse.protocols.posix.Default.getSpace' - +rucio rse attribute add --key storage_usage_tool \ + --value 'rucio.rse.protocols.posix.Default.getSpace' POSIX_RSE # Add protocol rucio rse protocol add --host localhost \ - --scheme file \ - --prefix '/data/posix-rse-1' \ + --scheme file --prefix '/data/posix-rse-1' \ --impl 'rucio.rse.protocols.posix.Default' \ - --domain-json '{"lan": {"read": 1, "write": 1, "delete": 1}, "wan": {"read": 1, "write": 1, "delete": 1}}' \ - POSIX_RSE + --domain-json '{"lan": {"read": 1, "write": 1, "delete": 1}, "wan": {"read": 1, "write": 1, "delete": 1}}' POSIX_RSE # Set account limits -rucio account limit add root --rse POSIX_RSE --bytes infinity +rucio account limit add --rse POSIX_RSE --bytes infinity root ``` --- @@ -178,23 +183,25 @@ PREFIX="/webdav" # Add RSE rucio rse add $RUCIO_RSE_NAME +# Set RSE type if needed (defaults to DISK) +# rucio rse update --rse-type DISK $RUCIO_RSE_NAME + # Set attributes -rucio rse update $RUCIO_RSE_NAME --key rse_type --value DISK -rucio rse attribute add $RUCIO_RSE_NAME --key lfn2pfn_algorithm --value identity -rucio rse attribute add $RUCIO_RSE_NAME --key fts --value https://fts.example.com:8446 -rucio rse attribute add $RUCIO_RSE_NAME --key verify_checksum --value False -rucio rse attribute add $RUCIO_RSE_NAME --key greedyDeletion --value True +rucio rse attribute add --key lfn2pfn_algorithm --value identity $RUCIO_RSE_NAME +rucio rse attribute add --key fts --value https://fts.example.com:8446 $RUCIO_RSE_NAME +rucio rse attribute add --key greedyDeletion --value true $RUCIO_RSE_NAME + +# Update settings +rucio rse update --verify-checksum false $RUCIO_RSE_NAME # Add davs protocol rucio rse protocol add --host $HOST $RUCIO_RSE_NAME \ - --scheme davs \ - --prefix $PREFIX \ - --port 8443 \ + --scheme davs --prefix $PREFIX --port 8443 \ --impl rucio.rse.protocols.gfal.Default \ --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' # Set account limits -rucio account limit add root --rse $RUCIO_RSE_NAME --bytes infinity +rucio account limit add --rse $RUCIO_RSE_NAME --bytes infinity root ``` **Using Python Client API:** @@ -206,6 +213,17 @@ from rucio.client.rseclient import RSEClient # Initialize client rse_client = RSEClient() +# Add RSE (defaults to DISK type) +rse_client.add_rse('WEBDAV_RSE') + +# Set attributes +rse_client.add_rse_attribute('WEBDAV_RSE', 'lfn2pfn_algorithm', 'identity') +rse_client.add_rse_attribute('WEBDAV_RSE', 'fts', 'https://fts.example.com:8446') +rse_client.add_rse_attribute('WEBDAV_RSE', 'greedyDeletion', True) + +# Update settings +rse_client.update_rse('WEBDAV_RSE', {'verify_checksum': False}) + # Protocol parameters params = { 'scheme': 'davs', @@ -231,17 +249,8 @@ params = { } } -# Add RSE -rse_client.add_rse('WEBDAV_RSE') - # Setup protocol rse_client.add_protocol('WEBDAV_RSE', params) - -# Set attributes -rse_client.add_rse_attribute('WEBDAV_RSE', 'lfn2pfn_algorithm', 'identity') -rse_client.add_rse_attribute('WEBDAV_RSE', 'fts', 'https://fts.example.com:8446') -rse_client.add_rse_attribute('WEBDAV_RSE', 'verify_checksum', 'False') -rse_client.add_rse_attribute('WEBDAV_RSE', 'greedyDeletion', 'True') ``` --- @@ -294,11 +303,10 @@ from rucio.client.rseclient import RSEClient # Initialize client rse_client = RSEClient() -# Add RSE +# Add RSE (defaults to DISK type) rse_client.add_rse('EXAMPLE_DISK_RSE') # Set attributes -rse_client.add_rse_attribute('EXAMPLE_DISK_RSE', 'rse_type', 'DISK') rse_client.add_rse_attribute('EXAMPLE_DISK_RSE', 'lfn2pfn_algorithm', 'identity') rse_client.add_rse_attribute('EXAMPLE_DISK_RSE', 'fts', 'https://fts.example.com:8446') @@ -382,14 +390,13 @@ from rucio.client.rseclient import RSEClient # Initialize client rse_client = RSEClient() -# Add RSE -rse_client.add_rse('EXAMPLE_TAPE_RSE') +# Add RSE with TAPE type +rse_client.add_rse('EXAMPLE_TAPE_RSE', rse_type='TAPE') # Set attributes -rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'rse_type', 'TAPE') rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'lfn2pfn_algorithm', 'identity') rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'fts', 'https://fts.example.com:8446') -rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'archive_timeout', '86400') +rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'archive_timeout', 86400) # Add HTTPS protocol params = { @@ -408,8 +415,8 @@ rse_client.add_protocol('EXAMPLE_TAPE_RSE', params) ``` **Key Differences from Disk RSEs:** -- `rse_type` is set to `TAPE` -- `archive_timeout` attribute specifies maximum time for file staging (86400 = 24 hours) +- `rse_type` is set to `TAPE` at creation time +- `archive_timeout` attribute controls maximum time for tape archival operations (86400 = 24 hours) - Typically only `https` protocol is defined (avoid `root` protocol for tape) @@ -437,10 +444,12 @@ See the [RSE Protocols Documentation](https://rucio.cern.ch/documentation/html/t ### Protocol Priority -The priority values in the `domain-json` determine which protocol is preferred: -- Higher numbers indicate higher priority -- Priority `1` is standard for active protocols -- Priority `0` or omitted disables the protocol +The priority values in the `domain-json` determine which protocol operations are enabled: +- Use `1` to enable an operation +- Use `0` to disable an operation +- Omitting an operation key indicates it's not supported + +**Example:** For a read-only RSE, set `"write": 0, "delete": 0` in the domains configuration. ### Domain Configuration @@ -478,40 +487,56 @@ Account limits control how much space an account can use on an RSE. rucio account limit add root --rse RSE_NAME --bytes infinity # Set specific quota (in bytes) -rucio account limit add user_account --rse RSE_NAME --bytes 1099511627776 # 1 TB -``` - -### Common Quota Values - -```bash -1 TB = 1099511627776 bytes -1 PB = 1125899906842624 bytes -infinity = unlimited +rucio account limit add user_account --rse RSE_NAME --bytes 1TB ``` --- ## Non-Deterministic RSEs -Non-deterministic RSEs don't follow the standard logical-to-physical naming convention. This is common for tape systems where files may be stored with system-generated names. +Non-deterministic RSEs don't follow the standard deterministic logical-to-physical file naming convention. This is most common for tape systems where physical filenames are generated based on file metadata, dataset membership, or other factors beyond just scope and name. + +### Key Differences from Deterministic RSEs + +**Deterministic RSEs** (typical disk systems): +- Use `lfn2pfn_algorithm` **setting** to derive physical paths from scope and name alone +- Support both direct uploads and replications +- Physical file path can be computed on-demand from the logical file name +- Example: File `scope:name` → `/storage/rucio/scope/name` + +**Non-Deterministic RSEs** (typical tape systems): +- Use `naming_convention` **attribute** to generate physical filenames +- The algorithm may use file metadata, dataset membership, creation time, etc. +- Only support replications (not direct uploads from clients) +- Physical file paths must be registered explicitly when files are replicated +- Example: File might be stored as `/tape/2026/02/03/archive_12345.tar` ### Setup Example ```bash -RUCIO_RSE_NAME="ESCAPE_TAPE_RSE" +RUCIO_RSE_NAME="EXPERIMENT_TAPE_RSE" -# Add as non-deterministic +# Add as non-deterministic RSE rucio rse add --non-deterministic $RUCIO_RSE_NAME # Set type and attributes -rucio rse update $RUCIO_RSE_NAME --key rse_type --value TAPE -rucio rse attribute add $RUCIO_RSE_NAME --key naming_convention --value escape_tape +rucio rse update --key rse_type --value TAPE $RUCIO_RSE_NAME +rucio rse attribute add --key naming_convention --value experiment_tape $RUCIO_RSE_NAME ``` **When to Use:** -- Tape systems with internal file organization +- Tape systems with internal file organization requiring metadata-driven naming - Storage systems that don't support directory structures -- Systems requiring custom naming schemes +- Systems where physical file names must be generated using business logic beyond scope/name +- Any storage backend where the physical filename cannot be deterministically computed from LFN alone + +:::warning +For non-deterministic RSEs: +1. Physical file paths must be registered explicitly during replication +2. Files cannot be uploaded directly by clients (only via replication) +3. The `naming_convention` attribute must reference a `non_deterministic_pfn` algorithm in your policy package +4. Do NOT set `lfn2pfn_algorithm` setting on non-deterministic RSEs +::: --- @@ -560,8 +585,8 @@ When using the `identity` lfn2pfn algorithm, the scope is appended to the prefix ### 6. Tape-Specific -- Set reasonable `archive_timeout` values (24 hours recommended) -- Use staging-aware clients +- Set reasonable `archive_timeout` values (24 hours recommended). This controls how long FTS waits for tape archival operations to complete +- Use staging-aware clients for file retrieval - Avoid `root://` protocol for tape systems - Monitor staging queue length @@ -600,17 +625,11 @@ Before deploying to production: rucio rse list # Show RSE details -rucio rse info RSE_NAME +rucio rse show RSE_NAME # List RSE attributes rucio rse attribute list RSE_NAME -# List RSE protocols -rucio rse protocol list RSE_NAME - -# Check RSE usage -rucio rse usage RSE_NAME - # List account limits rucio account limit list ACCOUNT_NAME From 60b9905b2d80c1f07998e386953eafbe68f81c1f Mon Sep 17 00:00:00 2001 From: Giovanni Guerrieri Date: Wed, 4 Feb 2026 08:46:36 +0100 Subject: [PATCH 7/7] fix: typos and minor fixes --- docs/operator/configuration_parameters.md | 3 +- docs/operator/rse_configuration.md | 39 +++++++++++------------ 2 files changed, 20 insertions(+), 22 deletions(-) diff --git a/docs/operator/configuration_parameters.md b/docs/operator/configuration_parameters.md index 83f37c25c77..31b1c076c4d 100644 --- a/docs/operator/configuration_parameters.md +++ b/docs/operator/configuration_parameters.md @@ -497,7 +497,7 @@ a setting already specified in the Configuration File. ## RSE settings The RSE settings are set separately using `rucio.RSEClient.update_rse` or `rucio rse update`, and specify RSE configuration used by the Rucio instance. Mutable settings are `deterministic`, `rse_type`, `staging_area`, `volatile`, `qos_class`, `availability_delete`, `availability_read`, `availability_write`, `city`, `country_name`, `latitude`, `longitude`, `region_code`, and `time_zone`. -Geographic fields (`city`, `country_name`, `latitude`, `longitude`, `region_code`, `time_zone`) can also be set at RSE creation time via API parameters, though they are stored and returned as part of the RSESettingsDict structure. +Geographic fields (`city`, `country_name`, `latitude`, `longitude`, `region_code`, `time_zone`) can also be set at RSE creation time via API parameters. Read more about RSEs [here](started/concepts/rucio_storage_element.md) and how to set them up [here](operator/configuration.md#creating-new-rses). @@ -512,7 +512,6 @@ and how to set them up [here](operator/configuration.md#creating-new-rses). - **domain**: List[String]. Specifies the locations this RSE may be accessed by clients from. Cannot be changed. - **id**: String. Identification string of the RSE. Cannot be changed. - **latitude**: Optional[Float]. Latitude coordinate of the RSE location. Used for geographic calculations. No default. -- **lfn2pfn_algorithm**: String. Name of the algorithm in the configured policy package to be used for creating replica paths. This field is populated from the RSE attribute of the same name. If the attribute is not set, it defaults to `lfn2pfn_algorithm_default` from the configured policy package. Default: `default`. - **longitude**: Optional[Float]. Longitude coordinate of the RSE location. Used for geographic calculations. No default. - **protocols**: List[RSEProtocolDict]. List of protocol configurations for accessing this RSE. Cannot be changed directly; use protocol-specific methods. - **qos_class**: Optional[String]. No functionality in modern Rucio. No default. diff --git a/docs/operator/rse_configuration.md b/docs/operator/rse_configuration.md index c00646faaed..f20807a086e 100644 --- a/docs/operator/rse_configuration.md +++ b/docs/operator/rse_configuration.md @@ -9,7 +9,7 @@ Before diving into the setup methods, it's important to understand the main conc ### RSE Settings vs Attributes -Rucio distinguishes between **RSE settings** (properties of the RSE record itself) and **RSE attributes** (key-value metadata). +Rucio distinguishes between **RSE settings** (properties of the RSE itself) and **RSE attributes** (key-value metadata). **RSE Settings** (configured via `rucio rse update` or `RSEClient.update_rse()`): - `rse_type`: Defines whether the RSE is `DISK` or `TAPE` (defaults to `DISK`) @@ -32,7 +32,7 @@ More can be found in the [RSE Settings configuration page](configuration_paramet 3. **LFN2PFN Algorithm**: Must be set as an RSE attribute during initial setup. This is effectively immutable and should be chosen carefully based on whether your RSE is deterministic. ::: -An exhaustive list of RSE attributes can be found in the [configuration parameters page](configuration_parameters/#rse-settings). +An exhaustive list of RSE attributes can be found in the [configuration parameters page](configuration_parameters/#rse-attributes). ### Protocols @@ -108,11 +108,11 @@ rse_client = RSEClient() account_limit_client = AccountLimitClient() # 1. Add the RSE -rse_client.add_rse('RSE_NAME', rse_type='DISK') # or rse_type='TAPE' +rse_client.add_rse(rse='RSE_NAME', rse_type='DISK') # or rse_type='TAPE' # 2. Add attributes -rse_client.add_rse_attribute('RSE_NAME', 'lfn2pfn_algorithm', 'identity') -#rse_client.add_rse_attribute('RSE_NAME', 'attribute_name', 'attribute_value') +rse_client.add_rse_attribute(rse='RSE_NAME', key='lfn2pfn_algorithm', value='identity') +#rse_client.add_rse_attribute(rse='RSE_NAME', key='attribute_name', value='attribute_value') # 3. Add protocol params = { @@ -146,7 +146,7 @@ POSIX RSEs use the `file://` protocol and require direct filesystem access. They **Using CLI:** ```bash -# Add the RSE +# Add the RSE, named POSIX_RSE rucio rse add POSIX_RSE # Set backend type @@ -217,12 +217,12 @@ rse_client = RSEClient() rse_client.add_rse('WEBDAV_RSE') # Set attributes -rse_client.add_rse_attribute('WEBDAV_RSE', 'lfn2pfn_algorithm', 'identity') -rse_client.add_rse_attribute('WEBDAV_RSE', 'fts', 'https://fts.example.com:8446') -rse_client.add_rse_attribute('WEBDAV_RSE', 'greedyDeletion', True) +rse_client.add_rse_attribute(rse='WEBDAV_RSE', key='lfn2pfn_algorithm', value='identity') +rse_client.add_rse_attribute(rse='WEBDAV_RSE', key='fts', value='https://fts.example.com:8446') +rse_client.add_rse_attribute(rse='WEBDAV_RSE', key='greedyDeletion', value=True) # Update settings -rse_client.update_rse('WEBDAV_RSE', {'verify_checksum': False}) +rse_client.update_rse(rse='WEBDAV_RSE', parameters={'verify_checksum': False}) # Protocol parameters params = { @@ -307,8 +307,8 @@ rse_client = RSEClient() rse_client.add_rse('EXAMPLE_DISK_RSE') # Set attributes -rse_client.add_rse_attribute('EXAMPLE_DISK_RSE', 'lfn2pfn_algorithm', 'identity') -rse_client.add_rse_attribute('EXAMPLE_DISK_RSE', 'fts', 'https://fts.example.com:8446') +rse_client.add_rse_attribute(rse='EXAMPLE_DISK_RSE', key='lfn2pfn_algorithm', value='identity') +rse_client.add_rse_attribute(rse='EXAMPLE_DISK_RSE', key='fts', value='https://fts.example.com:8446') # Add HTTPS protocol https_params = { @@ -394,9 +394,9 @@ rse_client = RSEClient() rse_client.add_rse('EXAMPLE_TAPE_RSE', rse_type='TAPE') # Set attributes -rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'lfn2pfn_algorithm', 'identity') -rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'fts', 'https://fts.example.com:8446') -rse_client.add_rse_attribute('EXAMPLE_TAPE_RSE', 'archive_timeout', 86400) +rse_client.add_rse_attribute(rse='EXAMPLE_TAPE_RSE', key='lfn2pfn_algorithm', value='identity') +rse_client.add_rse_attribute(rse='EXAMPLE_TAPE_RSE', key='fts', value='https://fts.example.com:8446') +rse_client.add_rse_attribute(rse='EXAMPLE_TAPE_RSE', key='archive_timeout', value=86400) # Add HTTPS protocol params = { @@ -416,7 +416,7 @@ rse_client.add_protocol('EXAMPLE_TAPE_RSE', params) **Key Differences from Disk RSEs:** - `rse_type` is set to `TAPE` at creation time -- `archive_timeout` attribute controls maximum time for tape archival operations (86400 = 24 hours) +- `archive_timeout` attribute specifies maximum time for file staging (86400 = 24 hours) - Typically only `https` protocol is defined (avoid `root` protocol for tape) @@ -432,7 +432,7 @@ rucio rse attribute add RSE_NAME --key attribute_name --value attribute_value # Using Python Client API from rucio.client.rseclient import RSEClient rse_client = RSEClient() -rse_client.add_rse_attribute('RSE_NAME', 'attribute_name', 'attribute_value') +rse_client.add_rse_attribute(rse='RSE_NAME', key='attribute_name', value='attribute_value') ``` --- @@ -527,14 +527,13 @@ rucio rse attribute add --key naming_convention --value experiment_tape $RUCIO_R **When to Use:** - Tape systems with internal file organization requiring metadata-driven naming - Storage systems that don't support directory structures -- Systems where physical file names must be generated using business logic beyond scope/name - Any storage backend where the physical filename cannot be deterministically computed from LFN alone :::warning For non-deterministic RSEs: 1. Physical file paths must be registered explicitly during replication 2. Files cannot be uploaded directly by clients (only via replication) -3. The `naming_convention` attribute must reference a `non_deterministic_pfn` algorithm in your policy package +3. The `naming_convention` attribute must reference a `non_deterministic_pfn` algorithm in your [policy package](policy_packages/policy_packages_overview) 4. Do NOT set `lfn2pfn_algorithm` setting on non-deterministic RSEs ::: @@ -585,7 +584,7 @@ When using the `identity` lfn2pfn algorithm, the scope is appended to the prefix ### 6. Tape-Specific -- Set reasonable `archive_timeout` values (24 hours recommended). This controls how long FTS waits for tape archival operations to complete +- Set reasonable `archive_timeout` values (24 hours recommended). This controls how long FTS waits for file staging to complete - Use staging-aware clients for file retrieval - Avoid `root://` protocol for tape systems - Monitor staging queue length