Feature-gate heavy DataLoader deps (arrow, parquet, hf-hub, foyer)

[darinkishore]

Problem

DataLoader pulls in heavy transitive dependencies:

• arrow (56.1.0)
• parquet (56.1.0)
• hf-hub (0.4.3)
• rayon (1.10.0)
• foyer (0.20.0)
• tempfile (3.23.0)

These are for data loading (CSV, Parquet, HuggingFace datasets) and caching — tangential to the core module/optimizer/signature architecture. Users who only need the typed prediction pipeline pay the full compile-time and binary-size cost.

Proposal

Feature-gate behind optional features:

[features]
default = ["dataloader"]
dataloader = ["dep:arrow", "dep:parquet", "dep:hf-hub", "dep:rayon"]
cache = ["dep:foyer", "dep:tempfile"]

data/example.rs and data/prediction.rs stay in core (used by optimizers/cache).
data/dataloader.rs, data/serialize.rs, data/utils.rs move behind dataloader feature.
utils/cache.rs moves behind cache feature.

Impact

• Faster compiles for the common case
• Smaller binary for embedding/deployment scenarios
• No behavioral change for default feature set

[darinkishore]

Codebase audit — dependency weight

Current Cargo.toml unconditional deps that should be gated:

Dep	Version	Used by	Proposed feature
arrow	56.1.0	DataLoader parquet reading	dataloader
parquet	56.1.0	DataLoader parquet reading	dataloader
hf-hub	0.4.3	DataLoader HuggingFace dataset download	dataloader
rayon	1.10.0	serialize.rs parallel JSONL, DataLoader	dataloader
foyer	0.20.0	ResponseCache hybrid disk cache	cache
tempfile	3.23.0	ResponseCache temp dir for foyer	cache
csv	1.3.1	DataLoader CSV loading	dataloader
kdam	0.6.3	Progress bars in DataLoader	dataloader
reqwest	0.12.23	URL downloads in DataLoader	dataloader

Files to gate:

• data/dataloader.rs → #[cfg(feature = "dataloader")]
• data/serialize.rs → #[cfg(feature = "dataloader")] (uses rayon, operates on untyped Example)
• data/utils.rs → #[cfg(feature = "dataloader")] (URL detection helper)
• utils/cache.rs → #[cfg(feature = "cache")]
• Corresponding pub use in data/mod.rs and lib.rs → conditional

What stays unconditional:

• data/example.rs — used by cache and optimizers internally
• data/prediction.rs — used by cache
• serde, serde_json, tokio, anyhow, bamltype, facet, rig-core, tracing — core deps

Proposed Cargo.toml:

[features]
default = ["dataloader", "cache"]
dataloader = ["dep:arrow", "dep:parquet", "dep:hf-hub", "dep:rayon", "dep:csv", "dep:kdam", "dep:reqwest"]
cache = ["dep:foyer", "dep:tempfile"]

Straightforward refactor, no behavioral changes for default builds.