guidellm image doesn't work with a non-root user #600
Description
Bug Description
oc runi ghcr.io/vllm-project/guidellm:v0.5.3
When running as a non-root user (on OpenShift), I get this error:
(app-root) sh-5.2$ /opt/app-root/bin/guidellm benchmark run --target http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/llm-d-project/llama-llm-d/v1 --profile sweep --max-seconds=30 --processor=RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic --data=prompt_tokens=256,output_tokens=128
⚠ Note: the following environment variables are set and **may** affect configuration
GUIDELLM_OUTPUT_DIR
✔ OpenAIHTTPBackend backend validated with model llama-3-1-8b-instruct-fp8
{'target': 'http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/llm-d-project/llama-llm-d', 'model': None, 'timeout': 60.0, 'http2': True, 'follow_redirects':
True, 'verify': False, 'openai_paths': {'health': 'health', 'models': 'v1/models', 'text_completions': 'v1/completions', 'chat_completions': 'v1/chat/completions', 'audio_transcriptions':
'v1/audio/transcriptions', 'audio_translations': 'v1/audio/translations'}, 'validate_backend': {'method': 'GET', 'url':
'http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/llm-d-project/llama-llm-d/health'}}
✔ Processor resolved
Using processor 'RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic'
Traceback (most recent call last):
...
ValueError: Data deserialization failed, likely because the input doesn't match any of the input formats. See the 15 error(s) that occurred while attempting to deserialize the data prompt_tokens=256,output_tokens=128:
...
- Deserializer 'synthetic_text': (OSError) PermissionError at /home/guidellm/.cache when downloading RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic. Check cache directory permissions. Common causes: 1) another user is downloading the same model (please wait); 2) a previous download was canceled and the lock file needs manual removal.
...
The Containerfile most likely needs this command:
RUN chgrp -R 0 "$HOME" && chmod -R g=u "$HOME"
Expected Behavior
The guidellm image works with a non-root user
Steps to Reproduce
- Run guidellm on OpenShift with a non root user, and no volume mounted on
/home/guidellm
Operating System
OpenShift
Python Version
Python 3.13.11
GuideLLM Version
ghcr.io/vllm-project/guidellm:v0.5.3
Installation Method
Official Container Image (Docker)
Installation Details
No response
Error Messages or Stack Traces
- Deserializer 'synthetic_text': (OSError) PermissionError at /home/guidellm/.cache when downloading RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic. Check cache directory permissions. Common causes: 1) another user is downloading the same model (please wait); 2) a previous download was canceled and the lock file needs manual removal.Additional Context
No response