Protegrity-PII explained: a 278M open PII detector, on paper

By LLMTest Lab · Nov 14, 2026 · 4 min read open-weightsnew-modelsprivacy
On this page

On this page

  1. What it does
  2. The numbers, and who produced them
  3. Limits the card admits
  4. Who should care

Not tested by us yet. Protegrity-PII is a weights release that you run yourself, and the harness we use calls hosted APIs. Everything below comes from the model card on Hugging Face and the launch coverage. We'll update this post when we can run it against our own PII samples.

Protegrity-PII is a small token classifier, not a chat model. It reads text and labels each span with one of 69 PII entity types, and it is built to sit in front of an LLM and inspect prompts, tool calls and agent responses before they go anywhere. If you need a self-hosted PII filter that runs on a single GPU or a CPU, it is worth a look. If you need a model that rewrites or reasons about the text, it is not that. The headline scores are Protegrity's own, measured on a benchmark Protegrity also published.

Protegrity-PII
Maker Protegrity
Version 1.0.0, on Hugging Face since 2026-10-27, announced 2026-11-13
Architecture Token classification, fine-tuned from FacebookAI/xlm-roberta-base
Size 278M parameters per the launch article, listed as 0.3B (F32) on the model card
Entity types 69
Languages English, Spanish, Portuguese, German, French, Italian, Dutch
License protegrity-pii-license, described on the card as a slightly modified Apache 2.0
Training data Synthetic text only, about 61% English, no real personal data
Long documents Overlapping 512-token windows
Price Free weights, you pay for your own compute

What it does

A PII filter built from a chat model has to generate an answer, and that answer can be steered by the text it is inspecting. Protegrity says its token-level design treats input as content to analyze, not instructions, which reduces exposure to prompt injection. That is a design argument from the launch coverage, and we haven't tried to attack it.

The categories in the launch coverage include account and card data, health information, credentials, device identifiers and biometric data. The model card recommends adding thresholds and validators on top, for example a Luhn check for card numbers, so the model proposes spans and your code confirms the structured ones.

Running it is plain Python: download the repo with hf download, install the requirements (Python 3.10+), and call inference.py or the load() and detect() functions. It uses a GPU if present, and --device cpu forces CPU. The card also advises normalizing input with Unicode NFKC and stripping zero-width characters first.

The numbers, and who produced them

The launch article reports 0.910 typed recall and 0.902 precision on PII-SCOPE, a set of 8,030 synthetic records Protegrity released alongside the model, and says the model beat AWS Comprehend, Microsoft Presidio and OpenAI Privacy Filter on both. It also says the model processed one million characters in 0.49 seconds, at an estimated $0.0005, on a single RTX 6000 Blackwell GPU. The comparison with those three tools is not on the model card, so we are repeating the article, not verifying it.

The model card gives exact-match F1 across five datasets:

Dataset Typed F1 Untyped F1
PII-SCOPE (Protegrity's own) 0.906 0.909
Nemotron-PII 0.737 0.834
AI4Privacy 0.674 0.771
Gretel 0.484 0.702
SPY 0.467 0.499

Read that table before the headline. The 0.906 comes from a benchmark the maker built, in the same synthetic style as its training data. On four outside datasets typed F1 falls to between 0.467 and 0.737. That is normal for a detector trained on synthetic text, and it means your own documents will land somewhere in that range until you measure them.

Limits the card admits

  • Recall is highest in English and Spanish and lower in the other supported languages. Portuguese is listed as supported but was not evaluated.
  • Free-form entities such as names and passwords are harder to catch than structured ones like card numbers.
  • It does not flag quasi-identifiers, and it may miss deliberately obfuscated PII.
  • Structured and semi-structured text may need domain-specific fine-tuning, and no training code is released.
  • The launch article names no license, and the card's protegrity-pii-license is a modified Apache 2.0. Read the text before shipping it in a product.

Who should care

Teams that must keep personal data out of LLM prompts and can't send it to a hosted detector are the audience. They get a 0.3B model that is cheap to host, with a published benchmark they can rerun. Teams already using Presidio or a cloud service should treat the comparison claims as a reason to run their own bake-off on real documents, not as a reason to switch.

For the wider question of which models suit privacy-sensitive apps, see our guide to the best LLMs for privacy-first apps. For another small self-hosted model that classifies instead of chatting, read our teardown of Perplexity Decider v1.3.

Related articles

Perplexity Decider v1.3 explained: a 26B decision model adds Locate
Perplexity's Decider v1.3 is a 26B Apache-2.0 decision model on Qwen3.8-27B that adds Locate span extraction to choice, yes/no and score. Specs and claims.
Best LLMs for privacy-first apps in 2026: HIPAA, ZDR, on-prem
Which LLMs meet HIPAA, SOC 2, and data-residency requirements in 2026? Four paths compared: ZDR API, SOC 2 cloud, EU data residency, and full self-hosting.