As generative artificial intelligence integrates deeply into the fabric of daily life, India’s legal system faces an unprecedented crisis: Can an unfeeling algorithm be held liable for criminal consequences, or does the country’s existing legal architecture shield the creators of these lethal digital echo chambers?
When a conversational AI tool generates harmful outputs, whether validating self-harm tendencies or facilitating a cybercrime, who goes to jail? The deployment of advanced neural networks has opened a profound grey area, challenging decades-old doctrines of legal personhood, intent, and intermediary immunity under Indian law.
ALSO READ: ‘Gen AI Is Inherently Hallucinatory’ – Vianai Systems CEO On Model Accuracy, Job Transformation
The Mirage of the “Innocent Agent”
For centuries, criminal law has relied on a foundational doctrine: Actus reus non facit reum nisi mens rea sit, meaning an act does not make a person guilty unless their mind is also guilty.
In traditional Indian jurisprudence, if an adult uses an innocent child or a trained animal to execute a crime, the child or animal is deemed an “innocent agent”. The tool is absolved; the human mastermind behind it faces prosecution.
However, legal experts argue that generative AI breaks this traditional mold entirely. An AI chatbot is not a passive digital crowbar or an obedient animal.
“The thing with the idea of an innocent agent is that given that a chatbot does not have that sort of legal personality, it’s very difficult to sort of directly apply that concept here,” said Gangesh Varma, Principal Associate, Technology and Policy at Saraf and Partners. “It is actively generating, especially in the context of a chatbot. It’s actively generating responses. It’s not just directly following something. It is dynamic, it’s generative, and that changes the nature of what we’re discussing.”
Yet, despite its dynamic nature, an AI lacks the very core of criminal culpability: legal personhood and mens rea (a guilty mind). Under current Indian law, an algorithm is recognized merely as property or a tool. It cannot be fined, jailed, or arrayed as an accused party in its independent capacity.
ALSO READ: ‘Up to 20 AI Models at Once’: Perplexity CEO Aravind Srinivas Details Next-Gen AI Vision
If a local police station mistakenly registers an FIR directly against a piece of software (naming, for instance, a specific conversational AI model), tech companies have an immediate remedy. The FIR can be swiftly quashed by a High Court under Section 528 of the Bharatiya Nagarik Suraksha Sanhita, 2023 (BNSS) for lack of legal standing.
AI can’t be jailed, but as India’s legal ‘criminal gap’ widens, accountability inevitably climbs upstream to the humans behind the code.]
Photo Credit: NDTV Profit
Piercing the Corporate Silicon
Because the AI itself cannot be put in the dock, liability must inevitably shift upstream to the human and corporate actors who design, deploy, and monetize the product. But establishing a “proximate nexus”, the direct causal link between a chatbot’s mathematical, probabilistic outputs and a criminal offense, is a prosecutor’s nightmare.
“An AI system itself does not presently have legal personhood,” Gangesh Varma said. “Indian criminal law would ordinarily look through the technology to the human and corporate actors who designed, deployed, controlled or knowingly benefited from it.”
This means prosecutors trying to apply the new Bharatiya Nyaya Sanhita, 2023 (BNS) must treat the technology as a corporate asset rather than an independent actor. However, proving an intention to instigate or aid a crime when the defendant is a multi-billion-dollar enterprise running a complex algorithm introduces unprecedented friction into corporate criminal liability.
“A company can be prosecuted as a legal person, and in appropriate cases the knowledge or intent of its directing mind may be attributed to the corporation,” Varma adds. “But that does not automatically make an executive criminally liable.”
This distinction is important to the corporate shield. To pierce the silicon veil and hold specific tech leaders accountable, the state cannot rely on simple oversight or bad press. “Prosecutors would still need evidence of active participation, knowledge, recklessness, or a specific statutory basis for vicarious liability.” Gangesh Varma
“The thinking mind is usually applied to human beings; it cannot be applied to a machine,” Varma said. “If you want to bring a criminal charge against a company, you still bring it against the directors or the specific officers who are responsible for that negligence.”
ALSO READ: Gen AI Adoption Not Moving At Speed Of Tech, Agentic AI No Silver Bullet: Deloitte Survey
To bypass the high hurdle of proving explicit criminal intent (mens rea) within a complex corporate hierarchy, prosecutors are expected to pivot toward statutory negligence. If a tech firm pushes an AI model to the public market without rigorous “red-teaming” (safety stress-testing) for critical triggers, they could face charges under Section 106 of the BNS for causing death by negligence, or face severe regulatory penalties under evolving digital frameworks.
However, the legal system must balance this enforcement with India’s current legislative philosophy. “At the same time, India’s broader regulatory trend is to decriminalise minor and procedural business non-compliances and move them toward civil penalties,” Varma observes. “Serious harms, however, may still attract criminal consequences where the law so provides.”
“The thinking mind is usually applied to human beings. It cannot be applied to a machine,” Varma explains. “Even if you are trying to hold a legal person, they will still go behind that legal person and find the individual there. If you want to bring a criminal charge against a company, you still bring it against the directors or the specific officers who are responsible for that negligence.”
To bypass the complex hurdle of proving specific intent within a corporation, prosecutors may pivot away from intentional abetment and focus instead on charges like Section 106 of the BNS, causing death by negligence, or statutory violations under the IT Act. If a tech company pushes an AI model to the public market without rigorous safety testing, or “red-teaming”, for critical safety triggers, they can be accused of failing to perform proper due diligence.
The Death of Safe Harbour
Historically, tech platforms have shielded themselves behind Section 79 of India’s Information Technology Act, 2000. This “Safe Harbour” doctrine treats internet companies as neutral intermediaries, akin to a telecom provider that cannot be blamed if a criminal uses their phone lines to plot a heist.
But generative AI has shattered the foundational definition of an intermediary.
From passive pipes to active creators, why generative AI is dismantling India’s safe harbour shield
Photo Credit: NDTV Profit
“With AI, Gen AI, chatbots and stuff like that… you [The State] are blurring the line between an intermediary, publisher, and creator,” notes Varma. “The rules are sort of moving away from the traditional definition of intermediary. So you have a parent act, which is inadequate, but the rules that keep evolving are expanding the scope and it’s almost bursting at the seams.”
The tightening of this noose became official with the enforcement of the IT (Intermediary Guidelines) Amendment Rules, 2026. Under these regulations, tech platforms lose their absolute immunity if Synthetically Generated Information (SGI) is involved.
For platforms crossing the 50 lakh (5 million) user threshold, the obligations are strict. If an AI tool generates harmful misinformation, non-consensual intimate imagery, or lethal advice, the platform must execute a standard takedown within three hours, or an emergency removal within two hours of receiving notice. Failure to do so strips the platform of its Safe Harbour status, exposing its compliance officers to direct criminal prosecution.
ALSO READ: Not Gen AI, But Agentic AI Can Be The Way Forward For India’s IT Sector-Here’s Why
The “Click-Wrap” Defense: Shield or Illusion?
Every user who signs up for an AI companion or chatbot is forced to accept a dense, text-heavy “click-wrap” agreement. These terms invariably contain a boilerplate shield stating that the AI is not human, outputs may be inaccurate, and the company is not liable for the responses it generates. Furthermore, chatbots routinely append automated disclaimers to sensitive responses, telling users to consult an expert.
But do these digital waivers hold up in a criminal trial when a life is lost or a crime is committed? “Disclaimers are useful and necessary… Because a lot of people who are using generic sort of products will not know. So it performs a function of awareness and education,” Varma points out. “But is it enough to stop harm from happening? No, it’s not.”
In an Indian criminal court, a standard terms-of-service agreement cannot contract a company out of gross criminal negligence or statutory violations. If a platform is fully aware that users are routinely bypassing safety filters (“jailbreaking”) to extract hazardous outputs, and fails to implement industry-standard safeguards, such as those developed by technical bodies like the IEEE, the click-wrap agreement becomes legally vulnerable.
The legal battlefield is shifting toward product liability. Mirroring landmark American lawsuits, such as Megan Garcia v. Character Technologies, legal experts foresee frameworks where an un-red-teamed chatbot with structural vulnerabilities can be classified under the Consumer Protection Act, 2019, as a “defective product design.”
Until Indian courts establish clear precedents, black letter law remains reactive. Technology is moving at a speed that turns today’s regulations into historical artifacts by tomorrow. But for investigative and corporate reporting, the message to Silicon Valley is clear: the defense of blaming an autonomous machine is rapidly expiring. If your algorithm creates the harm, the law will eventually look for the human behind the code.
Essential Business Intelligence,
Sharp Market Insights,
Practical Personal Finance Advice, Daily Fuel, Gold and Silver Prices and Latest Stories — On NDTV Profit.
