Accountability in Deepfakes

The fact that Grok (developed by xAI) has faced almost no real consequence from the recent slew of deepfake NCII (non-consensual intimate image) distribution is worrisome and reveals the challenges in regulating entrenched tech and social media systems.

Personally, I don’t see any reasonable case for deepfake or similar technology being readily available to the general public. Just as how things like payment services are licensable and regulated, such services can be licensable for specific purposes and be subject to stringent terms. This can specifically target the unpaid and unconsented use of personal features of individuals such as voice and images.

A recent article from the Singapore Academy of Law Journal discusses how existing legal frameworks feature in the world of digital replicas. The following can be distilled:

1.   Existing Intellectual Property (IP) laws offer limited protection in this area, since copyrights are not ascribed to an individual’s voice, style of singing, physical likeness, etc. This is unsurprising since IP laws were never intended to deal with situations such as this. Instead, IP laws have always and continue to focus on the intellectual expressions and innovations, not the personage of an individual.

2.   The Personal Data Protection Act (PDPA) may apply insofar as an individual’s voice and image are considered personal data protected by the PDPA. However, if these are publicly available, then no consent is required for collection and use of data. As pointed out by the authors however, it is possible that a “reproduction” of the personal data may still attract separate protections that may not have been available for the original publicly available source. Again however, it appears that the personal data protection regime is ill-suited to deal with the specific act of creating deepfakes of an individual.

3. The Online Criminal Harms Act (OCHA) potentially covers acts of a similar nature by empowering authorities to issue stop communication directions and service restriction directions, and criminalizing non-compliance with the same. However, it remain It however remains to be seen whether companies like xAI with platforms facilitating such acts can be held to account with sufficient force and urgency to deter and prevent these from occurring in the first place. Further, once an image is produced, it can be easily reproduced on various platforms and media, and any stop communication direction may be simply be too late to remedy the damage that can occur to an individual.

The ill-suitedness of existing legal frameworks becomes evident in situations involving masses of public users referring to photographs already available online, who now have the capacity to create volumes of deepfakes instantaneously, without a second thought. Would xAI be treated as having been the “collector”, “user” or “discloser” of personal data in such a situation? And in the context of the OCHA, could xAI avail itself to the “reasonable excuse” defence by stating it had taken all such measures as could reasonably taken after the fact?

As above, my view is that offering a service capable of creating a hyper-realistic deepfake of an individual should either be outlawed or be a licensable where a use case exists. For example, there have been therapeutic applications of deepfake voices / images to assist with the processing of grief (although the efficacy and propriety of such modalities remains a question of contention). If a professional intends to use such service, it should be licensable, and be subject to strict regulation.

The fact that the world will lose the "joy" of creating harmless but funny deepfakes is a small price to pay considering the real psychological harm that deepfakes can inflict on their victims.

This entry is adapted from a LinkedIn post I had crafted earlier this year when Grok had gone out of control and allowed users to seamlessly post deepfakes directly on X (i.e. twitter) without any consequence. While the matter appears to have been fixed, it does not change the fact that such tools remain easily accessible to the public at little to no costs.

Interestingly, it appears that the Penal Code was recently amended in August 2026 to include a new offence under Section 292B of distributing, publicly exhibiting or circulating to 10 or more individuals on an online location an "obscene object". This would arguably cover the situation of deepfakes being used for the purposes of generating intimate images of persons without their consent, and appears to be consistent with Singapore's gradual shift in regulatory approach towards the maintenance of a safe online environment and the prevention of it from being used to perpetrate harms.

The review from of the existing legal framework: Commercialisation of Digital Replicas of Individual Primer on the Intellectual Property and Personal Data Protection Laws in Singapore [2026] SAL Prac 5, accessible at: https://journalsonline.academypublishing.org.sg/Portals/0/EFirst/Html/3776_[2026]%20SAL%20Prac%205_Commercialisation%20of%20Digital%20Replicas%20(22%20Jan%202026).htm