Which OCR model/configuration is used for Chinese handwriting recognition?

[enjiushuozhexie]

Hello InkSight Team,
First of all, thank you for sharing this incredible project! The ability to convert offline handwriting to digital ink is truly impressive.
I'm currently experimenting with the handwriting segmentation part of the pipeline, specifically for pages containing multi-language text. I've been testing with an image sample that includes Chinese, English, and French handwritten text (as shown below).

I've noticed that when using the doctr option for segmentation, the ocr_predictor(pretrained=True) is called. This default predictor works wonderfully for segmenting the English and French text, but it seems to ignore the Chinese characters entirely, resulting in no bounding boxes for them.
My understanding is that the default doctr pretrained model is primarily trained on Latin scripts, which would explain this behavior.

Could you please provide some guidance on what OCR engine, model architecture (det_arch, rec_arch), or specific pretrained weights you recommend or used internally to successfully segment pages that include handwritten Chinese characters?
For example, is there a specific rec_arch from the doctr model zoo that you found works best, or do you recommend a different OCR tool/API altogether for this task?
Any advice would be greatly appreciated. Thank you for your time and for this great contribution to the community!

[CharlieLeee]

Hi @enjiushuozhexie thank you for the interests in our work and your kind words. We share the details of the OCR engine in our paper: https://arxiv.org/pdf/2402.05804 and we used the publicly available (but not free) version of the Google Cloud Vision API: https://cloud.google.com/vision/docs/handwriting.

As you observed, Doctr does not replicate the behaviour of this API; however, it’s free, so we included it in our repository for demonstration. Please feel free to use whatever works best, and we’d be happy to take pull requests for this.