Added documentation for additional CLI commands #2007
Conversation
jhiemstrawisc
left a comment
jhiemstrawisc
left a comment
There was a problem hiding this comment.
One extra comment that applies to each of the new sections - Can you provide a real example using our new canonical object: pelican://osg-htc.org/pelicanplatform/test/hello-world.txt or the pelican://osg-htc.org/pelicanplatform/test collection where appropriate?
| To use the pelican client to list objects in a collection, use Pelican's `object ls` sub-command | ||
|
|
||
| ```bash | ||
| pelican object ls pelican://<federation-url></namespace-prefix></path/to/collection> |
There was a problem hiding this comment.
There are additional flags for this sub command we should document:
Flags:
-C, --collection-only List collections only
--collections-url string URL to use for collection listing, overriding the director's response
-j, --json Print results in JSON format
-l, --long Include extended information
-O, --object-only List objects only
|
|
||
| Pelican also has an `object sync` command which will sync objects from a source object or collections to a destination object or collection. If the objects already exist in the destination and are the same size or larger than the source objects of the same name, they will not be overwritten, but the objects which don't exist will be added. | ||
|
|
||
| > **Warning**: In the case where an object in the destination is smaller than an object with the same name in the source, the sync command assumes that a partial sync was interrupted and completes it by appending the necessary bytes. If it issn't the same file, but has the same name, this can result in the mashing of two files together. |
There was a problem hiding this comment.
How can I user tell this is the case before they sync things and create potentially bad data? Can you also mention that this is a side-effect of "object immutability"?
|
|
||
| > **Warning**: In the case where an object in the destination is smaller than an object with the same name in the source, the sync command assumes that a partial sync was interrupted and completes it by appending the necessary bytes. If it issn't the same file, but has the same name, this can result in the mashing of two files together. | ||
| ### Sync Objects from a Collection to a Local Directory |
There was a problem hiding this comment.
Few comments:
- Can you provide some discussion of or a link to how this can be done with tokens?
- I think
syncfrom remote to local also requires thatlistingsis enabled, doesn't it? Can you explain any assumptions/requirements on Origin/prefix capabilities and how the user can verify those? - I can't remember all the implementation details, but are there any other assumptions to document about any importance of trailing
/s in anything? I know you had a big chart to explain all the operationssynchad to perform. Maybe that table is even worth including here?
There was a problem hiding this comment.
I think the table is bit too mountain dewy. It was very much meant for internal use more than anything else. But I will try to clarify and odd edge cases or trailing '/' issues.
3 participants
Added descriptions of CLI commands for
ls,stat, andsync