June 30, 2026

Responding to DSRs When Personal Data Is Used in AI

Data subject rights requests (DSRRs or DSRs) can become more difficult to assess and fulfill as organizations adopt AI across everyday business functions. An employee may request access to personal data after AI was used to summarize performance reviews or support workforce analytics. A customer may request deletion after their support tickets, chat history, or account information were used to test or improve an AI tool. A job applicant may ask how their information was used in an AI-assisted screening process.

These requests can be harder to answer than traditional DSRs. In many cases, organizations can search for personal data in customer databases, employee files, support platforms, or document repositories. When AI is involved, the organization may also need to consider whether personal data was used in prompts, uploaded files, outputs, logs, evaluation datasets, fine-tuning records, embeddings or vector databases, model-improvement records, or, in some cases, the model itself. AI systems may require a broader review because personal data may have been used to develop, test, operate, monitor, fine-tune, or improve the system.

This article explains why DSRs become more complex when personal data is used in AI systems, how rights such as access, correction, deletion, objection, and automated decision-making may apply, and why emerging technical methods such as machine unlearning are relevant but not a complete answer.

Why DSRs Become More Difficult in AI Systems

Many privacy laws give individuals rights in relation to their personal data or personal information, although the scope of those rights, the applicable exceptions, and the response procedures vary significantly by jurisdiction. Under the EU’s General Data Protection Regulation, for example, individuals may have the right to:

be informed about how their personal data is used
obtain confirmation of processing and access to their personal data
rectification of inaccurate personal data
erasure, also known as the right to be forgotten
restrict certain processing
data portability
object to certain processing

Individuals may also have rights and safeguards relating to certain solely automated decisions, including profiling, where the decision produces legal or similarly significant effects.

Other privacy laws, including the California Consumer Privacy Act (CCPA) and similar comprehensive U.S. state privacy laws, also give individuals rights to access, correct, and delete their personal data, opt out of certain processing, limit certain uses of their personal information, or receive information about certain automated decision-making or profiling activities. For example, California’s 2026 CCPA regulations require businesses to respond to requests to know, delete, correct, while also introducing specific rights related to automated decision-making technology (“ADMT”), including the right to request access to information about how ADMT is used, the right to opt out of certain uses of ADMT (such as profiling for significant decisions), and obligations for businesses to conduct and document risk assessments for high-risk ADMT uses. Although the rights, exceptions, and response timelines differ across jurisdictions, organizations still need to be able to explain what personal data they hold, how it is being used, and what actions need to be taken in response to a request.

In an AI context, those questions can be harder to answer. For a standard DSR, organizations often focus on locating personal data, reviewing it, applying any exceptions, and responding within the applicable response period. When AI is involved, the same process can require a broader review because the data may have been used at different stages of the AI lifecycle. That does not mean every AI-related DSR requires a model-level investigation. The analysis depends on the system architecture, the organization’s role, the purposes of processing, retention practices, vendor terms, and whether the personal data can still be identified, extracted, inferred, or linked to the requester.

Where to Look When a DSR Involves AI

When a DSR involves an AI system, the organization needs to first identify the type of request being made, such as access, correction, deletion, objection, or restriction, and then determine where the relevant personal data may exist within the AI lifecycle. This assessment should not be limited to ordinary systems of record, such as customer databases, employee files, support tools, or CRM systems. Depending on how the AI system is designed and used, personal data may also appear in prompts, uploaded files, retained outputs, decision records, training datasets, fine-tuning datasets, evaluation datasets, embeddings or vector stores, model-improvement logs, or, in some cases, the model itself.

A broader mapping exercise is necessary because AI-related processing may involve several stages. The European Data Protection Board’s ChatGPT Taskforce report describes several stages of personal data processing in large language models , including training data collection, pre-processing, training, prompts, outputs, and the use of prompts for further training. As a result, an AI-related DSR may require the organization to review not only the primary system in which the individual’s data was collected or stored, but also the downstream AI systems and related technical environments where the data may have been used, retained, transformed, or derived.

Transformation or incorporation into an AI system may affect how a right can be fulfilled, but it does not remove the obligation to assess the request and provide a reasoned response.

The relevant locations are not legally or technically equivalent. Data in a system of record, prompt history, or an uploaded file may be directly retrievable and linked to an individual. Data in retained outputs, decision records, or evaluation datasets may require contextual analysis to determine whether it relates to the individual and whether it remains identifiable. Data in embeddings, vector stores, training datasets, or model-improvement logs may be harder to locate, isolate, or modify. Data incorporated into a model may raise still more complex questions about whether the model contains personal data, whether the individual can be identified or singled out by means reasonably likely to be used, and what technical measures are feasible.

Organizations should therefore avoid assuming that data used in or derived from AI systems is anonymous merely because it has been transformed, tokenized, embedded, aggregated, or incorporated into a model. The European Data Protection Board has emphasized that AI models trained on personal data cannot automatically be treated as anonymous, and that anonymization must be assessed on a case-by-case basis. Similarly, the EDPB Support Pool of Experts Programme report on the effective implementation of data subjects’ rights recognizes that, where AI systems are developed using personal data, the implementation of rights such as rectification and erasure may require AI-specific technical measures, including data curation, provenance, deletion of training input data, and measures to address the influence of specific data points in the trained model.

In practice, controllers should assess whether the relevant data remains personal data, whether it can be linked to the individual using means reasonably likely to be used, and what response measures are legally required and technically feasible. The appropriate response may differ depending on where the data is located and how it is stored. For some data, the organization may be able to retrieve, disclose, correct, or delete the relevant record directly. For other data, the appropriate measure may be to suppress specific outputs, exclude data from future training or fine-tuning, apply filtering or access controls, update decision records, or document why a requested action is not technically possible or not legally required in the specific circumstances. Transformation or incorporation into an AI system may affect how a right can be fulfilled, but it does not remove the obligation to assess the request and provide a reasoned response.

Training Data

Personal data may be included in datasets used to train, test, validate, or evaluate an AI system. These datasets may include, for example, customer records, employee information, support tickets, transaction history, public-facing content, or other information relating to identifiable individuals. If a person later submits a subject access request, deletion request, or correction request, the organization may need to determine whether that person’s data was included in one or more of those datasets.

As a practical first step, organizations should identify the relevant source datasets and determine whether they can be searched, linked back to source systems, or mapped to particular individuals. This may require reviewing dataset inventories, data provenance records, data lineage information, feature documentation, and records showing whether the data came from internal systems, vendors, public sources, or user interactions.

Organizations should not assume that training data falls outside data protection law merely because it has been transformed, summarized, stripped of obvious identifiers, or otherwise pre-processed. The key question is whether the requester can still be identified directly or indirectly, including by being singled out from the dataset, either from the training data alone or in combination with other information available to the organization.

For DSR handling, this means the organization should assess whether it can identify the requester in the relevant training data directly or indirectly. If it can, it should consider the request in the ordinary way, subject to applicable limitations and exemptions. If it cannot identify the requester directly or indirectly, it should document the basis for that conclusion and consider whether any additional information provided by the requester would enable identification.

Prompts and Uploaded Content

Personal data may also enter an AI system during use, not only during development. Users may submit names, account details, employee information, customer records, financial information, health information, legal information, or other personal data in prompts or uploaded files. Where these prompts or files are stored, linked to a user account, reviewed by personnel, used for troubleshooting, or reused for model improvement, they may need to be considered when responding to a DSR.

In practice, organizations should understand what happens to prompts and uploaded content after submission. This means checking whether prompts, files, chat transcripts, attachments, and related metadata are stored; whether they are linked to user IDs, account IDs, employee accounts, device identifiers, or session IDs; how long they are retained; who can access them; and whether they are shared with vendors or subprocessors. Organizations should also distinguish between content used only to generate the immediate response and content reused for secondary purposes, such as abuse monitoring, quality review, analytics, model evaluation, fine-tuning, or training.

Where an AI tool is provided by a third party, the organization should not assume that prompt or uploaded-content data is unavailable. It should check the contract, product settings, administrative controls, and provider support process to determine whether the provider can search, export, delete, suppress, or otherwise assist with DSRs. If the organization acts as controller and the provider acts as processor, the arrangement should require the processor to assist with rights requests. If the AI service is operated under a joint-controller or independent-controller structure, the organization should identify which party is responsible for responding to which parts of the request.

Outputs and Predictions

Personal data may also appear in AI-generated outputs. An AI system may generate a summary, recommendation, classification, prediction, or response that relates to an identifiable person. If that output is stored, shared, added to a profile, or used to support a decision, it may need to be reviewed as part of a DSR.

For DSR handling, organizations should map where outputs go after generation. This includes checking whether outputs are displayed only transiently, saved in chat history, copied into a case file, added to a customer or employee profile, incorporated into a risk score, sent to another system, or used to support a decision. The ICO notes that, once deployed, AI outputs are typically stored in an individual’s profile and used to take some action about them. Where those outputs constitute personal data, they will generally be subject to individual rights, unless an exemption or limitation applies.

The risk is higher when an output is inaccurate or treated as factual. The ChatGPT Taskforce report notes that LLMs generate coherent and context-relevant text that appears human-like despite being biased or made up, and that users may treat those outputs as factually accurate. The EDPB Support Pool of Experts report notes that generative AI systems may “hallucinate” and generate factually incorrect content that could reveal personal data about people. For correction requests, this means organizations may therefore need to look beyond the original source record and consider whether inaccurate AI-generated information was retained, shared, added to a profile, or used in a decision-making process.

Logs, Retention, and Model Improvement

Prompts, uploaded files, outputs, feedback, audit logs, quality-control records, troubleshooting records, and any other interaction records may be retained after an AI interaction. These records may be used to improve the system, monitor performance, fine-tune models, detect misuse, investigate issues, or develop future systems. If those records contain personal data, they may also need to be considered when responding to a DSR.

This is why AI-related DSR management should include retention practices. Organizations should know whether prompts and outputs are stored, how long they are kept, who can access them, whether they are shared with other parties, and whether they are used for model improvement, evaluation, fine-tuning, or further training.

The Model Itself

In some cases, the analysis may go beyond the source data, prompts, outputs, or logs. Personal data may sometimes be contained in or inferable from the model itself. The ICO explains that this may happen by design, where certain models contain key examples from the training data in their internal logic, or by accident, where a model leaks personal data and allows elements of training data to be recovered or inferred from the model’s behavior.

This is where DSRs can become especially difficult. If a request involves access, correction, or erasure of personal data contained in or inferred from the model, the organization should involve technical personnel to assess what is technically feasible. In some cases, the organization may need to consider retraining, machine unlearning, additional safeguards, or whether it can explain why the data cannot be identified in the model. Relevant questions include what type of model is used, whether the model may contain training examples, whether personal data can be extracted or inferred, whether the requester’s data can be identified or isolated, and whether retraining, redeployment, model deletion, machine unlearning, output suppression, or other safeguards are possible.

How Key Data Subject Rights Can Apply in AI Systems

Once an organization has identified where personal data may exist in the AI lifecycle, the next step is to assess what the particular request requires. The response will differ depending on whether the individual is asking for access, correction, deletion, objection, restriction, or information about automated decision-making. The same AI system may raise different issues depending on the right being exercised, the role of the organization, the applicable law, and whether the relevant data can be identified, retrieved, corrected, erased, restricted, or explained.

Access Requests

In general, an access request may require the organization to provide more than a copy of information from a customer database, HR file, or account record. If personal data was used in connection with an AI system or other processing activities, the organization may need to explain whether it processes the individual’s personal data, the categories of personal data involved, the purposes of processing, the recipients or categories of recipients, applicable retention periods, and other information required under the relevant law.

The practical challenge is not that the right of access changes, but that the organization may need enough internal visibility to answer accurately. For example, the organization may need to determine whether the requester’s data was used as an input, appeared in a stored output, was included in a retained interaction record, or was used in connection with a decision about the individual. If the request involves training data that has been transformed or stripped of obvious identifiers, the organization should assess whether the requester can still be identified directly or indirectly, including by being singled out from the data.

Where AI is used for solely automated decision-making with legal or similarly significant effects (for example, if AI is used in loan approvals, job applications, credit scoring, fraud detection, or similar contexts), access requests may also require information about the logic involved, as well as the significance and envisaged consequences of the processing for the individual.

Correction Requests

Correction requests can be more complex where inaccurate personal data appears in an AI-generated output, prediction, score, summary, classification, or decision record. The organization should first determine whether the challenged information is personal data and whether it is inaccurate in the relevant context.

This distinction matters. A factual statement about an individual may be inaccurate and require correction. A prediction, score, or classification may not be “inaccurate” simply because the individual disagrees with it or because it expresses a probability rather than a factual assertion. The ICO notes that predictions are not inaccurate if they are intended as prediction scores rather than statements of fact. However, the organization should still assess whether the input data was wrong, whether the output was presented misleadingly as fact, whether the result was used to make or support a decision, and whether any profile, record, explanation, or downstream system should be corrected, annotated, restricted, or updated.

In practice, correcting the original source record may not be enough if inaccurate AI-generated information was retained elsewhere or used to make a decision. Organizations should check whether the incorrect output was saved, shared, copied into another system, added to a profile, or relied on by human reviewers. Where the same error may recur, the organization should also consider whether additional safeguards are needed, such as correcting source data, updating decision records, suppressing a particular output, adjusting instructions or retrieval sources, or excluding inaccurate data from future use.

Deletion Requests

Deletion requests require a careful distinction between deleting personal data from source systems, deleting copies or logs, and addressing any continued use or influence of that data in AI systems. In many cases, the organization may be able to delete records from ordinary systems, prompt histories, uploaded files, logs, evaluation datasets, or future training datasets. The harder question is whether the data has already affected a trained model.

The ICO notes that erasing personal data from training data does not necessarily require erasing every machine learning model based on that data, unless the model itself contains the data or can be used to infer it. However, the organization should still assess whether the model contains or can reveal personal data, whether further technical measures are possible, and whether the data should be excluded from future training, fine-tuning, or evaluation.

Where model-level action is required, the organization may need technical input. Potential measures may include retraining the model without the relevant data, deleting and redeploying the model, applying machine unlearning techniques, suppressing certain outputs, or documenting why the requested action is not technically possible or not legally required in the circumstances. Machine unlearning refers to technical methods that aim to remove or reduce the influence of specific data points from a trained model. However, it should not be treated as a complete answer to every deletion request involving AI. Depending on the method used, unlearning may reduce the influence of data in the model, support retraining efficiencies, or suppress certain outputs, but it may also involve technical limitations, uncertainty, cost, and trade-offs. Organizations should understand what a proposed method can and cannot do before relying on it as part of a DSR response.

Automated Decision-Making and Human Review

If the AI system is used to make or support decisions about individuals, the organization should assess whether the request involves solely automated decision-making, including profiling, with legal or similarly significant effects. If so, additional rights and safeguards may apply. Under the GDPR, these include requirements to provide meaningful information about the logic involved, as well as the significance and envisaged consequences of the processing, and safeguards such as the ability to obtain human intervention, express a point of view, contest the decision, and obtain an explanation of the decision logic.

Organizations should not assume that adding a human reviewer automatically takes a decision outside automated decision-making rules. Human involvement must be meaningful and not a token gesture. Reviewers should have the authority and competence to challenge or change the decision, should consider all relevant data, and should not merely rubber-stamp the AI system’s output.

For DSR handling, this means the organization should be prepared to explain whether the AI system made the decision or merely supported a human decision-maker, what information was used, what role the output played, whether a human review occurred, and whether the individual can contest the outcome. Organizations should also keep records of automated decisions, human interventions, challenges, and any changes made as a result. The ICO notes that these records support both accountability and ongoing monitoring, including whether the system needs to be amended where decisions are regularly changed after individuals exercise their rights.

Objection and Restriction Requests

Objection and restriction requests may require the organization to stop or limit certain uses of personal data in an AI system, rather than delete the data outright. For example, an individual may object to their data being used for model improvement, profiling, analytics, or future training, or may request restriction while the accuracy or lawfulness of the processing is assessed.

In practice, organizations should identify whether they can suppress the relevant data from future use, exclude it from future training or fine-tuning, restrict access to retained outputs or logs, pause use of a disputed decision record, or apply flags that prevent the data from being reused for particular purposes. The feasibility of these measures will depend on the system design, retention practices, vendor capabilities, and whether the relevant data can be linked to the requester. These requests are another reason organizations should maintain clear data provenance, retention, and model-management records.

Practical Steps for Responding to DSRs Involving AI

Organizations should treat AI-related DSRs as both a privacy issue and an operational issue. The response process should help the organization identify the relevant AI system, determine where the requester’s personal data may have been used or retained, involve the right internal or external stakeholders, and decide what action is legally required and technically feasible.

A practical response process should include the following steps:

Identify the AI system and use case. Determine which AI system is involved, how it is used, whether it is internally developed or provided by a third party, and whether the organization uses it for development, testing, deployment, monitoring, model improvement, decision support, or automated decision-making.
Map the request to the relevant part of the AI lifecycle: Determine whether the request concerns source data, training or evaluation data, prompts or uploaded content, outputs, decision records, logs, model-improvement records, or personal data that may be contained in or inferred from the model itself. This helps narrow the review and avoids treating every AI-related DSR by requiring a full model-level investigation.
Assess identifiability and retrievability: Consider whether the requester’s personal data can be identified directly or indirectly, including where data has been transformed, pre-processed, or stripped of obvious identifiers. If the organization can identify the relevant data, it should assess the request as usual, subject to applicable limitations and exemptions. If it cannot identify the data, it should document that conclusion and consider whether additional information from the requester would enable identification.
Review retained outputs and decision records: Do not focus only on the original source record. If an AI-generated output was stored, shared, added to a profile, copied into another system, or used to support a decision, it may need to be reviewed. This is especially important for correction requests, because inaccurate AI-generated information may affect the individual more directly than an error in training data.
Involve technical teams where model-level issues arise. If the request may involve personal data contained in or inferable from a model, the organization should involve appropriate technical personnel. Relevant questions include whether the model may contain training examples, whether personal data can be extracted or inferred, whether the requester’s data can be isolated, and whether measures such as retraining, redeployment, model deletion, machine unlearning, output suppression, or other safeguards are possible.
Check third-party AI providers and contracts. If a DSR involves an outsourced AI service, the organization should determine what personal data the provider processes and what support is available. A vendor’s technical limitations do not, by themselves, remove the organization’s obligations as controller. Contracts and operating procedures should require appropriate assistance with access, correction, deletion, restriction, objection, and other rights requests where the provider processes relevant personal data.
Consider whether automated decision-making rights are triggered. If the AI system is used to make or support decisions about individuals, assess whether the request involves solely automated decision-making, including profiling, with legal or similarly significant effects. If so, the response may need to address meaningful information about the logic involved, the significance and consequences of the processing, and safeguards such as human intervention, the ability to express a point of view, and the ability to contest the decision.
Maintain records that make future responses possible. Organizations should keep clear records of which AI systems are used, where they are deployed, what data they rely on, what vendors are involved, whether prompts or outputs are retained, whether data is used for model improvement, and when models are updated, retrained, or retired. Good records make it easier to determine whether a DSR relates to source data, prompts, outputs, logs, training data, or the model itself, and what response is realistically available.

AI can make DSRs harder to assess, but difficulty alone should not be treated as a reason to dismiss a request. In some cases, an organization may have grounds to refuse to act on a request or charge a fee if the request is unfounded or excessive. However, that assessment should be based on the facts of the request itself and the applicable law, not simply on the fact that personal data is harder to locate, review, or address in an AI context.

Looking Ahead

As organizations use AI systems more widely, DSR handling will increasingly depend on traceability, documentation, vendor oversight, and technical review. Organizations need to understand how personal data enters AI systems, how it is used, whether it is retained, whether it is reused for model improvement, and what action can realistically be taken when an individual exercises their rights.

Organizations that want a stronger DSR process should focus on traceability, documentation, vendor oversight, and technical review. They need to understand how personal data enters AI systems, how it is used, whether it is retained, and what action can realistically be taken when an individual exercises their rights.

A strong DSR process should therefore be built into AI governance from the start. Privacy, legal, security, procurement, product, engineering, and vendor-management teams should work together to ensure that AI systems are documented, retention practices are understood, vendor obligations are clear, and technical teams can support requests when needed.

If you need support with DSRs involving AI, VeraSafe can help assess your current process, identify gaps, and recommend practical next steps. Book a free consultation today.