get_image fails unpredictably #29
Description
I've newly installed imgstore and have unpredictable failures of get_image. Any ideas on how to fix it? I'm guessing there is an issue with my installation. I followed the instructions (opencv < 4.9) but additionally needed to use numpy < 2 (and tried using python 3.10 before figuring out the numpy issue). I initially thought that the post-install tests of imgstore looked fine (except for failed write tests, but I only need to read) but then noticed two other failed tests which might be relevant, entered as a comment below.
The problem: get_image works fine in a loop where I sequentially process each frame in a for loop, but fails for some frame numbers when called later for the frames of interest. Example below from debugging attempts, for a frame where it fails sometimes and works sometimes.
The code I'm using worked fine on my old PC, but that was so old that it was python 2. So I suspect it's an issue with my install, but can't figure out what needs to be changed.
My installed packages:
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
asttokens 3.0.0 pypi_0 pypi
bzip2 1.0.8 h5eee18b_6
ca-certificates 2025.7.15 h06a4308_0
contourpy 1.3.2 pypi_0 pypi
cycler 0.12.1 pypi_0 pypi
decorator 5.2.1 pypi_0 pypi
exceptiongroup 1.3.0 pypi_0 pypi
executing 2.2.0 pypi_0 pypi
expat 2.7.1 h6a678d5_0
fonttools 4.59.0 pypi_0 pypi
imgstore 0.3.7 pypi_0 pypi
iniconfig 2.1.0 pypi_0 pypi
ipython 8.37.0 pypi_0 pypi
jedi 0.19.2 pypi_0 pypi
kiwisolver 1.4.9 pypi_0 pypi
ld_impl_linux-64 2.40 h12ee557_0
libffi 3.4.4 h6a678d5_1
libgcc-ng 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libstdcxx-ng 11.2.0 h1234567_1
libuuid 1.41.5 h5eee18b_0
libxcb 1.17.0 h9b100fa_0
matplotlib 3.10.5 pypi_0 pypi
matplotlib-inline 0.1.7 pypi_0 pypi
ncurses 6.5 h7934f7d_0
numpy 1.26.4 pypi_0 pypi
opencv-python 4.8.1.78 pypi_0 pypi
openssl 3.0.17 h5eee18b_0
packaging 25.0 pypi_0 pypi
pandas 2.3.1 pypi_0 pypi
parso 0.8.4 pypi_0 pypi
pexpect 4.9.0 pypi_0 pypi
pillow 11.3.0 pypi_0 pypi
pip 25.1 pyhc872135_2
pluggy 1.6.0 pypi_0 pypi
prompt-toolkit 3.0.51 pypi_0 pypi
pthread-stubs 0.3 h0ce48e5_1
ptyprocess 0.7.0 pypi_0 pypi
pure-eval 0.2.3 pypi_0 pypi
pygments 2.19.2 pypi_0 pypi
pyparsing 3.2.3 pypi_0 pypi
pytest 7.4.4 pypi_0 pypi
python 3.10.18 h1a3bd86_0
python-dateutil 2.9.0.post0 pypi_0 pypi
pytz 2025.2 pypi_0 pypi
pyyaml 6.0.2 pypi_0 pypi
readline 8.3 hc2a1206_0
setuptools 78.1.1 py310h06a4308_0
six 1.17.0 pypi_0 pypi
sqlite 3.50.2 hb25bd0a_1
stack-data 0.6.3 pypi_0 pypi
tk 8.6.14 h993c535_1
tomli 2.2.1 pypi_0 pypi
traitlets 5.14.3 pypi_0 pypi
typing-extensions 4.14.1 pypi_0 pypi
tzdata 2025.2 pypi_0 pypi
tzlocal 5.3.1 pypi_0 pypi
wcwidth 0.2.13 pypi_0 pypi
wheel 0.45.1 py310h06a4308_0
xorg-libx11 1.8.12 h9b100fa_1
xorg-libxau 1.0.12 h9b100fa_0
xorg-libxdmcp 1.1.5 h9b100fa_0
xorg-xorgproto 2024.1 h5eee18b_1
xz 5.6.4 h5eee18b_1
zlib 1.2.13 h5eee18b_1
Example of the error (from debugging in ipython; fn is the frame_number output of get_frame_metadata():
In [247]: store.get_image(fn[266])
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[247], line 1
----> 1 store.get_image(fn[266])
File ~/miniconda3/envs/imgstore_py_3_10/lib/python3.10/site-packages/imgstore/stores.py:716, in _ImgStore.get_image(self, frame_number, exact_only, frame_index)
714 return self._get_image_by_frame_index(frame_index)
715 else:
--> 716 return self._get_image_by_frame_number(frame_number, exact_only=exact_only)
File ~/miniconda3/envs/imgstore_py_3_10/lib/python3.10/site-packages/imgstore/stores.py:696, in _ImgStore._get_image_by_frame_number(self, frame_number, exact_only)
693 if chunk_n == -1:
694 raise ValueError('frame #%s not found in any chunk' % frame_number)
--> 696 return self._get_image(chunk_n, frame_idx)
File ~/miniconda3/envs/imgstore_py_3_10/lib/python3.10/site-packages/imgstore/stores.py:648, in _ImgStore._get_image(self, chunk_n, frame_idx)
645 self._load_chunk(chunk_n)
647 # ensure the read works before setting frame_number
--> 648 _img, (_frame_number, _frame_timestamp) = self._load_image(frame_idx)
649 img = self._decode_image(_img)
650 self._chunk_current_frame_idx = frame_idx
File ~/miniconda3/envs/imgstore_py_3_10/lib/python3.10/site-packages/imgstore/stores.py:1140, in VideoImgStore._load_image(self, idx)
1137 _, _img = self._cap.read()
1138 if self._color:
1139 # almost certainly no-op as opencv usually returns color frames....
-> 1140 img = ensure_color(_img)
1141 else:
1142 img = ensure_grayscale(_img)
File ~/miniconda3/envs/imgstore_py_3_10/lib/python3.10/site-packages/imgstore/util.py:48, in ensure_color(img, ensure_copy)
47 def ensure_color(img, ensure_copy=False):
---> 48 return _cvt_color(img, cv2.COLOR_GRAY2BGR, ensure_copy=ensure_copy)
File ~/miniconda3/envs/imgstore_py_3_10/lib/python3.10/site-packages/imgstore/util.py:27, in _cvt_color(img, code, ensure_copy)
25 def _cvt_color(img, code, ensure_copy=True):
26 # protect against empty last dimensions
---> 27 _is_color = (img.shape[-1] == 3) & (img.ndim == 3)
29 if code == cv2.COLOR_GRAY2BGR:
30 if _is_color:
AttributeError: 'NoneType' object has no attribute 'shape'
And then it works fine for that frame if I grab the one before it first.
In [248]: store.get_image(fn[265])
Out[248]:
(array([[[ 3, 3, 3],
[ 3, 3, 3],
[ 3, 3, 3],
...,
[27, 34, 20],
[28, 35, 21],
[28, 35, 21]],
[[ 3, 3, 3],
[ 3, 3, 3],
[ 3, 3, 3],
...,
[27, 34, 20],
[28, 35, 21],
[28, 35, 21]],
[[ 3, 3, 3],
[ 3, 3, 3],
[ 3, 3, 3],
...,
[27, 34, 20],
[28, 35, 21],
[28, 35, 21]],
...,
[[ 6, 5, 0],
[ 6, 5, 0],
[ 6, 5, 0],
...,
[ 1, 5, 0],
[ 1, 5, 0],
[ 1, 5, 0]],
[[ 6, 5, 0],
[ 6, 5, 0],
[ 6, 5, 0],
...,
[ 1, 5, 0],
[ 1, 5, 0],
[ 1, 5, 0]],
[[ 6, 5, 0],
[ 6, 5, 0],
[ 6, 5, 0],
...,
[ 1, 5, 0],
[ 1, 5, 0],
[ 1, 5, 0]]], dtype=uint8),
(266, 1749119598.7516336))
In [249]: store.get_image(fn[266])
Out[249]:
(array([[[ 3, 3, 3],
[ 3, 3, 3],
[ 3, 3, 3],
...,
[25, 34, 20],
[26, 35, 21],
[26, 35, 21]],
[[ 3, 3, 3],
[ 3, 3, 3],
[ 3, 3, 3],
...,
[25, 34, 20],
[26, 35, 21],
[26, 35, 21]],
[[ 3, 3, 3],
[ 3, 3, 3],
[ 3, 3, 3],
...,
[25, 34, 20],
[26, 35, 21],
[26, 35, 21]],
...,
[[ 8, 10, 3],
[ 6, 8, 1],
[ 4, 6, 0],
...,
[ 1, 5, 0],
[ 1, 5, 0],
[ 1, 5, 0]],
[[ 8, 10, 3],
[ 6, 8, 1],
[ 4, 6, 0],
...,
[ 1, 5, 0],
[ 1, 5, 0],
[ 1, 5, 0]],
[[ 8, 10, 3],
[ 6, 8, 1],
[ 4, 6, 0],
...,
[ 1, 5, 0],
[ 1, 5, 0],
[ 1, 5, 0]]], dtype=uint8),
(267, 1749119598.768245))
Very grateful for any suggestions!
Clio