Serve paths that are in store, but don't extist upstream?

[sg2002]

Hello.

Should ncps be able to serve paths that exist locally, but do not exist in the upstream cache? It's not clearly answered in the README and my testing says "no".

I run ncps on the same machine I use as a builder. For the upstream paths everything works correctly. But, anything built on the machine is not available through ncps. Thus I have to configure the client to also use the same machine that runs ncps as an ssh substituter(or run another binary cache server on it). Which is not that big of a deal, but is a bit redundant. Thus I think the optional ability to serve the downstream substitutes(whatever currently exists in the local store) does not seem that unreasonable?

At the very least, can you please explicitly cover this question in the README.

Thanks in advance.

[kalbasit]

Hi @sg2002, ncps does not currently handle that out of the box, however, you can achieve what you want by deploying Harmonia on the same box as ncps and using it as an upstream cache. Does that solve your problem?

[sg2002]

Yes, I'm aware of Harmonia being another option that would work in tandem with ncps. I'm just thinking that having to run two different binary cache servers to effectively serve the same store is a bit inelegant. Not a big deal by any means, but still.

[john-craig]

Does anyone have a working example of ncps in front of a harmonia upstream with this configuration?

[kalbasit]

@john-craig you can probably look at the NixOS test for that: https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/ncps.nix

[MagicRB]

If i were to run ncps in front of harmonia, would stoee apth duplication occur? My usecase is having CI/CD and a pull through cache in my home. Then if my laptop pulls from ncps, it would cause the store paths from ci/cd to get copied into ncps if i understand things correctly. Is it possible to tell ncps to not cache a specific upstream?

With the current SSD prices im concerned aboht storage space

[kalbasit]

@MagicRB ncps is designed as a pull-through cache so not storing cache from one or more upstream kind of goes against what it's designed for.

If you're concerned about space, why don't you set the cache.max-size option to limit the size of the store and let the LRU kick the old/unused paths?

I do get the your use-case though and it would be nice if ncps does support not caching a store but I'm not sure how large of a change this would be. Does the max-lru work for you or do you need me to do some assessment for a no-cache pull-through feature?

[MagicRB]

@kalbasit thanks for the speedy reply. Please don't waste any actual dev time on this hypothetical :) i havent deployed anything yet, im still in the planning phase. Ill try with the cache.max-size approach and see where it gets me. I'll report back with the result.

[kalbasit]

@kalbasit thanks for the speedy reply. Please don't waste any actual dev time on this hypothetical :) i havent deployed anything yet, im still in the planning phase. Ill try with the cache.max-size approach and see where it gets me. I'll report back with the result.

I appreciate your care. Absolutely, please do! If you validate the use-case, please open a separate issue requesting specifically a no cache passthrough for an upstream (or another way of doing it) since this issue is requesting to replace Harmonia as opposed to what you're requesting.