Using Generative AI for Content Redaction
Overview
Redaction of content is an important step in Data Engineering to remove PII/ Sensitive data before the data is further processed in the pipeline. There are many code libraries to achieve this.
This code achieves it using Generative AI. When redacted the content is replaced with text of format [Redacted <Data Type>]. Example: An email ravi@contoso.com in the content would be replaced with [Redacted Email]. The type of content that was tested includes Email, Names of Individuals and Companies, Phone Numbers, Server Names, IP Addresses, Physical Addresses, Zip Codes and Passwords.
Code is available in Github at GenAI-Content-Redaction-OpenAI
Solution approach:
- Chat Completions API of the Azure OpenAI was used.
- A System message was provided to the prompt.
- Also, one Few shot example was used in the prompt.
Example:
Input:
From: Adele Vance adele.vance@fabrikam.com
Sent: Wednesday, August 16, 2023 7:27:40 PM
To: Grady Archie grady.archie@m365supportgroup.com
Cc: Joni Sherman joni.sherman@m365supportgroup.com
Subject: Re: Microsoft 365 Deployement for fabrikam.com
Hi Grady,
I have shared our godaddy details in a separate email. Please check.
Thank you,
Adele
Output:
From: [Redacted Name] <[Redacted Email]>
Sent: Wednesday, August 16, 2023 7:27:40 PM
To: [Redacted Name] <[Redacted Email]>
Cc: [Redacted Name] <[Redacted Email]>
Subject: Re: Microsoft 365 Deployement for [Redacted Company Name]
Hi [Redacted Name],
I have shared our [Redacted Company Name] details in a separate email. Please check.
Thank you,
[Redacted Name]
Code
Detailed instructions to setup and execute the code are provided in the Readme. Important pieces of it are explained here.
System message for the Prompt
The system message conveys to the LLM
- The role of the assistant
- What to replace sensitive data with
- Instructs to do redaction even if the probability of a particular text being sensitive is low
- Provides an example of a Server name, assuming the LLM wasn’t aware.
systemMessage = {"role":"system",
"content":"""You are to help redact PII and Sensitive data from the content the user provides.
The nature of the content is mostly an email or multiple emails in an email thread.
Sensitive data is any PII or any data that can help in identifying an entity.
It includes, but not limited to email, names of individuals, companies, phone numbers, ip addresses, physical addresses, web site urls, server names, zip codes.
To redact replace PII/ Sensitive words with [Redacted <type of data>]. When you redact email it should be replaced with [Redacted Email] and so on for each type of data redacted.
Do redaction even if you have atleast 0.25 probability of something being sensitive.
You already know to recognize an email, name of an individual, name of a company, phone number, ip address, physical address, zip code, web site url.
If you dont know to recognize a server name, an example is \"MA1PR01MB4163.INDPRD01.PROD.OUTLOOK.COM\". """}
fewshotTrainingMessage1prompt = {"role":"user","content":"From:ravi@zelarsoft.com"}
fewshotTrainingMessage1completion = {"role":"assistant","content":"From:[Redacted Email]"}
input_text = """Hi Suresh,
please note my alternate email ravi@hotmail.com
"""
prompt_message = {"role":"user", "content": input_text}
messages = [systemMessage, fewshotTrainingMessage1Prompt, fewshotTrainingMessage1Completion, prompt_message]Azure OpenAI API
This code uses Azure OpenAI API Sdk ≥ 1.0. Creates the client using Azure OpenAI endpoint, key and api version, then calls the completion API.
from openai import AzureOpenAI
client = AzureOpenAI(
azure_endpoint=azure_oai_endpoint,
api_key=azure_oai_key,
api_version="2023-07-01-preview")
completion = client.chat.completions.create(
model = azure_oai_model,
temperature = 0.5,
max_tokens = 800,
messages = messages
)
print(completion.model_dump_json(indent=2))And the output is:
{
"id": "chatcmpl-8L5i2luJS9euGujDzO2tocd1vRtlR",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"content": "Hi Suresh,\nplease note alternate email [Redacted Email]",
"role": "assistant",
"function_call": null,
"tool_calls": null
},
"content_filter_results": {
"hate": {
"filtered": false,
"severity": "safe"
},
"self_harm": {
"filtered": false,
"severity": "safe"
},
"sexual": {
"filtered": false,
"severity": "safe"
},
"violence": {
"filtered": false,
"severity": "safe"
}
}
}
],
"created": 1700037526,
"model": "gpt-35-turbo",
"object": "chat.completion",
"system_fingerprint": null,
"usage": {
"completion_tokens": 14,
"prompt_tokens": 284,
"total_tokens": 298
},
"prompt_filter_results": [
{
"prompt_index": 0,
"content_filter_results": {
"hate": {
"filtered": false,
"severity": "safe"
},
"self_harm": {
"filtered": false,
"severity": "safe"
},
"sexual": {
"filtered": false,
"severity": "safe"
},
"violence": {
"filtered": false,
"severity": "safe"
}
}
}
]
}Understanding the output:
- The redacted content is at choices:message:content property.
- gpt-35-turbo has a token limit of 4000. The number of tokens for the prompt, completion is mentioned in the usage section of the json
- The results of content filtering for unsafe content is specified in the prompt_filter_results section of the json
Possible issues:
The LLM processed only part of the input text:
Multiple reasons can exist, some of them are:
- LLM exceeded the Max Tokens to be used for the output specified in the max_tokens param to the completions api. This can be deciphered by looking at the value of usage:completion_tokens in the output json. If this value is same or closer to the MaxTokens, then this could be the reason.
- Some content in the input is unsafe. The LLM applies content filters. This can be understood from the prompt_filter_results of the json output.
Not all sensitive data was identified:
Try improving the System Message and adding few more Few Shot Prompts
