Trouble reading Pickle file & the wayout

[kumar-ab]

While reading/loading the pickle files (for Kyle Orderbook problems), I was facing errors like:

UnpicklingError: the STRING opcode argument must be quoted

ValueError: could not convert string to float

After trying to debug that for a long time I finally got the solution. Apparently, I was getting the error because the files were saved in dos encoding which needed to be converted to unix encoding before the jupyter notebook could read the data.

The following code may be used to save the files in the required encoding/format:

---

original = "D:\Github\BootCamp2019\Econ\Wk5_Asset\data\THO"
destination = "D:\Github\BootCamp2019\ProblemSets\Econ\Data\THO"

content = ''
outsize = 0
with open(original, 'rb') as infile:
content = infile.read()
with open(destination, 'wb') as output:
for line in content.splitlines():
outsize += len(line) + 1
output.write(line + str.encode('\n'))

print("Done. Saved %s bytes." % (len(content)-outsize))

---

Here "original" is the filename with path of the original file & "destination" is the filename with path of the converted file.

I hope this would help those who are facing similar problems.