Treatement of alphanumeric fields

[dalbo]

I note the history of first expanding what was allowed in the Customer Reference Number field, and then moving it back to being strictly alphanumeric only.

I do not have a long history with the bai spec, but it doesn't seem to me that it is terribly well defined. But I note a few things that would indicate to me that alphanumeric could be intended to mean more than strictly a-z0-9.

• Specifically mentioning that alphanumeric fields must not contain , or /. This would be completely redundant.
• The Text field, which is designated alphanumeric, specifically calls out that is can contain , and / (just not as the first character). And it certainly not seem helpful to have a free text field like this that can't include space or some other word separator.

I note that this parser does allow a looser definition of alphanumeric for the Text field, but not for Bank Reference Number or Customer Reference Number.

The parser otherwise seems excellent, but the strict validation on some of the reference fields makes it unusable for us in its current state. We will likely make a private fork and change the validation to match our needs, unless you would be willing to revert to looser validation again?

[bayprogrammer]

Hi @dalbo; thank you for the feedback. I agree that the specification leaves much to be desired.

After having dealt with the data actually generated by banks who read the specs less carefully (or else disagree on how to interpret its various ambiguities) I would write things differently if given the opportunity (namely to allow various parsing details to be overridden by the library user in various ways).

Sadly that opportunity has not presented itself to me. The problem with expanding the allowed bytes in these "alphanumeric" fields is the question of where to draw the line. What should be included? Anything which isn't a comma or slash? I'm not really sure how to resolve that problem cleanly.

As you found from the history of this project I did go down the road of expanding the allowed characters for a while. But I had to keep expanding and expanding it well beyond anything resembling a letter of the alphabet or a number.

I finally gave up and started pushing back against the banks for generating clearly deviant data.

[dalbo]

Thanks for the response.

I had thought it would be a great feature to be able to configure how strictly to perform validation. But as you say, that takes time and effort, and there's not a huge market for bai parsing!

My personal opinion when it comes to implementing specifications is that when generating (or validating) data, adhere to the narrowest interpretation, but when reading data try to be as loose as possible, so long as it doesn't make the data ambiguous.

Every use case is going to be very different, but for our use case, we don't care what values are in the alphanumeric fields, as long as we can escape them for sql and render them safely to html. And it is on us to ensure we can do that with any data that we handle. But we're not producing further bai or other files to a specification, or passing the values on to anyone else who has strict requirements about the content. So for our use case, treating alphanumeric as anything that isn't , or / would be fine.

No promises, but if you'd be open to it, I'll consider if I could come up with a PR for the user being able to configure the strictness of validation.

[bayprogrammer]

My personal opinion when it comes to implementing specifications is that when generating (or validating) data, adhere to the narrowest interpretation, but when reading data try to be as loose as possible, so long as it doesn't make the data ambiguous.

That sounds like a very good policy and advice I wish I could go back in time and give myself. 😅

No promises, but if you'd be open to it, I'll consider if I could come up with a PR for the user being able to configure the strictness of validation.

I think having such an option would make this library more beneficial to everyone for sure. At the very least, worst-case, I think we could link to your fork in our README if for whatever reason we weren't in a position to merge your changes outright (if you do decide to take a crack at it).

Definitely let me know if you have any questions I can help you with if and when you proceed. 🤜🤛