-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
Script that causes the problem: $(PROJECT_ROOT)/Method-Level-Summary/SparseCoder/run.sh
Dockerfile:
FROM pytorch/pytorch:2.0.0-cuda11.7-cudnn8-runtime
WORKDIR /app
COPY ./requirements.txt .
RUN apt update && \
apt-get install git -y && \
pip install -r requirements.txt
##############################################################################
# Installation/Basic Utilities
##############################################################################
RUN apt-get update && \
apt-get install -y --no-install-recommends \
software-properties-common build-essential autotools-dev \
nfs-common pdsh \
cmake g++ gcc \
curl wget vim tmux emacs less unzip screen nano git \
htop iftop iotop ca-certificates openssh-client openssh-server \
rsync iputils-ping net-tools sudo \
llvm-9-dev
requirements.txt
aiohttp==3.8.4
aiosignal==1.3.1
async-timeout==4.0.2
attrs==22.2.0
beautifulsoup4==4.11.2
bs4==0.0.1
certifi==2022.12.7
charset-normalizer==3.0.1
clearml==1.9.2
filelock==3.9.0
frozenlist==1.3.3
fsspec==2023.1.0
furl==2.1.3
huggingface-hub==0.12.1
idna==3.4
joblib==1.2.0
jsonschema==4.17.3
lightning-utilities==0.7.1
multidict==6.0.4
numpy==1.24.2
orderedmultidict==1.0.1
packaging==23.0
pandas==1.5.3
pathlib2==2.3.7.post1
Pillow==9.4.0
psutil==5.9.4
PyJWT==2.4.0
pyparsing==3.0.9
pyrsistent==0.19.3
python-dateutil==2.8.2
pytorch-lightning==1.9.3
pytz==2022.7.1
PyYAML==6.0
regex==2022.10.31
requests==2.28.2
scikit-learn==1.2.1
scipy==1.10.1
six==1.16.0
soupsieve==2.4
threadpoolctl==3.1.0
tokenizers==0.13.2
torch==2.0.0
torchmetrics==0.11.3
tqdm==4.64.1
transformers==4.28.1
typing_extensions==4.5.0
urllib3==1.26.14
yarl==1.8.2
boto3==1.26.94
datasketch
typer
fairscale
nltk
openai
peft==0.2.0
deepspeed==0.8.3
pygments
datasets==2.11.0
bitsandbytes==0.38.1
sentencepiece==0.1.97
notebook==6.5.4
jupyterlab==3.6.3
Error:
Traceback (most recent call last):
File "/home/modzyuba1/SparseCoder/Method-Level-Summary/SparseCoder/run.py", line 44, in <module>
from lsg_converter import LSGConverter
File "/opt/conda/lib/python3.10/site-packages/lsg_converter/__init__.py", line 1, in <module>
from .converter import LSGConverter
File "/opt/conda/lib/python3.10/site-packages/lsg_converter/converter.py", line 6, in <module>
from .bart.convert_bart_checkpoint import *
File "/opt/conda/lib/python3.10/site-packages/lsg_converter/bart/convert_bart_checkpoint.py", line 1, in <module>
from .modeling_lsg_bart import *
File "/opt/conda/lib/python3.10/site-packages/lsg_converter/bart/modeling_lsg_bart.py", line 4, in <module>
from transformers.modeling_attn_mask_utils import _prepare_4d_attention_mask, _prepare_4d_causal_attention_mask
ModuleNotFoundError: No module named 'transformers.modeling_attn_mask_utils'
OS version: Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-144-generic x86_64)
Steps to reproduce the error:
- Build an image, run container
- Pip install transformers ==4.37.1
- Then run Python in interactive mode:
python
>>> import transformers
>>> transformers.__version__
>>> 4.37.1
- Run
$(PROJECT_ROOT)/Method-Level-Summary/SparseCoder/run.shin the docker container and get the error - Do step 3 again
>>> 4.32.0 - Do step 2 again
transformers==4.37.1 successfully installed - Do step 3 again
>>> 4.32.0
Expected behavior:
This code should work, because missing modules are presented in transformers 4.37.1. https://github.com/huggingface/transformers/blob/v4.37.1/src/transformers/modeling_attn_mask_utils.py
The installed version of transformers is 4.37.1. After your code runs, Python packages become broken.
The transformers version does not change after the second pip install, but should
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels