Expand 4cat to other languages (Finnish)

[anitabraida]

These are the changes proposed in this pull request:

Wordlists added:

1. Finnish hatespeech wordlist (no offensiveness scores available)
2. Finnish wordlist with common vocabulary

Processors changed:

1. lexical_filter. Added Finnish hatespeech, introduced choice for the user to select the language. If the language selected is a highly inflected one (in this case, Finnish), it gives the user the option to choose between exact word match (present previously) and subword match (added in this pull request).
2. wikipedia_network. Added the option to find Wikipedia links in a Finnish language dataset.
3. neologisms. Added the Finnish wordlist with common vocabulary. The user has now the option to choose either English (default) or Finnish as the language. Pipeline changed accordingly.
4. tokenise. Added Finnish wordlist with common vocabulary for the filter. Added a lemmatization option for Finnish (uses SpaCy).
5. similar_words (presets). Included additional information.

In the setup, SpaCy is included again to allow for lemmatization on Finnish text.

[matnel]

@ErikBorra @stijn-uva Would be lovely if you could check and merge this soonish! :) Please let me know if there are any issues with the code, happy to patch this PR.

[dale-wahl]

@sal-uva you removed spacy last year. I cannot recall or find what the specific issue was and only vaguely remember some dependency problems. Do you recall?

[sal-uva]

@sal-uva you removed spacy last year. I cannot recall or find what the specific issue was and only vaguely remember some dependency problems. Do you recall?

There were some dependency problems (iirc it didn't like our version of NumPy) and because of its resource-heavy nature was only suited for small-scale datasets for (among others) entity recognition, which can now be done much better with LLMs.

But if it's handy for Finnish and it doesn't cause any dependency errors for you, all good to add back in!

[matnel]

I think we did not encounter issues, but happy to test more if that helps - any hints on what to test out?

[dale-wahl]

I think the lexical_filter needs a change to be clearer to users, but otherwise this looks good.

[dale-wahl]

I do not like the language option as it does not do anything at the moment yet implies it does to the user.

I see that the Finnish list is regex already compared to the English lists. I understand that is why you have added the match-subword option as the regex is designed to handle those particular inflections. The problem is that we allow users to select multiple lexicons as well as add an optional custom list. And right now, the options are all combining to run on all lexicons. This need a rework. I see now that the as_regex would run on the English lists as well though it should not.

Perhaps language could to allow two separate option lists to be shown (finn/eng). And it be made clearer what options are relevant to the custom word list vs the pre-made ones. Loading the lexicons could then be made clearer.

[dale-wahl]

Not exactly true. You could use "^=lemm_" and start the eng/finn keys with lemm_ for example. But this is a poorly documented feature.