Made a colab to show how we can use MamayLM with GLiNER model to solve the PII detection task for Ukrainian.

In test cases, only one sample GLiNER could not successfully recognize, but after English translation - it does.

So, the algorithm is simple:

- Translate Ukrainian sentence to English
- Load GLiNER model that is zero-shot NER model
- Pass translated sentences and detect PII entities

Instead of MamayLM, we can use MADLAD models or other NMT models.

Colab: https://colab.research.google.com/drive/1P3jBiixz0Mv31NF6cc6K3wG5HtOW_eL9?usp=sharing

#ai #pii #nlp #security

Only one sample was not processed correctly with knowledgator/gliner-x-large-v0.5

But the model is more efficient than previous solution with MamayLM + GLiNER

Colab: https://colab.research.google.com/drive/1tP2gNfvgzscaGKQNkQ0AqBQGSKW6ZUis?usp=sharing

#ai #nlp #pii #security

Also, a test with the translation model. This hybrid solution should give better results.

Colab: https://colab.research.google.com/drive/1fiOSthyg2BI08x4EKNgZOdGC4cD9DQKe?usp=sharing

#ai #pii #security #nlp