The financial sector in India is currently undergoing massive changes as a result of exponential increase in the usage of ‘Artificial Intelligence’ (AI) and ‘Machine Learning’ (ML). The ‘regulated entities’ (i.e. entities regulated by the Securities and Exchange Board of India (“SEBI”)) are increasingly relying on technologies to function such as large language models, chatbots and generative artificial intelligence to improve investor experience, increase efficiency and to ensure compliance.


In view of the same, SEBI, vide its multiple circulars[1] mandated reporting of AI and ML tools (“AI and ML Circulars”). SEBI has also released a Consultation Paper[2] proposing amendments for putting onus on the regulated entities for usage of artificial intelligence tools.


AI and ML through the lens of SEBI

The scope of the AI and ML Circulars was aimed at having a reporting mechanism to understand the trend on adoption of AI and ML in the securities market ecosystem. The reporting would cover ‘any set of applications/ software/programs/executable/systems (computer systems)’:


  1. offered to the investors to facilitate investing and trading;
  2. used to disseminate investment strategies and advice;
  3. used for compliance, operations or activities; or
  4. used in public product offerings or for compliance purposes.


As per the AI and ML Circulars, applications and systems belonging to the following categories are deemed to be AI and ML technology:


  1. Natural Language Processing (NLP), sentiment analysis or text mining systems that gather intelligence from unstructured data;
  2. Any systems that use a number of nodes (physical or software simulated nodes) mimicking natural neural networks of any scale;
  3. Machine learning through supervised, unsupervised learning or a combination of both;
  4. A system that uses statistical heuristics method;
  5. A system that uses a feedback mechanism to improve its parameters and bases its subsequent execution steps on these parameters; and
  6. A system that does knowledge representation and maintains a knowledge base.


The Consultation Paper describes AI as executable programmes in machines and computers that can learn, reason and act in ways that would normally require human intelligence.


Proposed amendments in the Consultation Paper:


Cardinal Principles:

In the Consultation Paper, SEBI has recognized the growing role of AI/ML systems/tools in market analysis, stock selection, investment strategies and building an investment portfolio. The central idea of proposing the amendments is to ensure a balance between the use of AI and ML tools by regulated entities and the protection of investors with the usage of such tools. The Consultation Paper seeks to make the regulated entities using AI/ML systems more accountable and has proposed that any regulated entity using AI and ML tools (either designed by themselves or by a third party) for conducting activities in the securities market, regardless of the scope and size of the adoption shall be responsible for the following:


  1. ‘privacy, security and integrity’ of investors and stakeholders’ data in the processes involved including data maintained by the entity in a fiduciary capacity;
  2. output from the use of AI tools and techniques is relied upon or dealt with; and
  3. compliance with applicable laws in force.


The Consultation Paper has thus proposed moving beyond the reporting requirements under the current AI and ML Circulars and making regulated entities responsible and liable for the usage of AI tools[3]. In this regard, SEBI has proposed amendments to the Securities and Exchange Board of India (Intermediaries) Regulations, 2008 (“Intermediaries Regulations”), the Securities Contracts (Regulations) (Stock Exchanges and clearing corporation) Regulations, 2018, and the Securities and Exchange Board of India (Depositories and Participants) Regulations, 2018 to give effect to the above principles.


Key Takeaways:


  • Scope of accountability:


The Consultation Paper holds regulated entities accountable for “output” generated by AI and ML tools. However, the scope of such accountability is not clear and raises the question of whether SEBI is seeking to hold regulated entities responsible for matters which would otherwise have been qualified by ‘risk factors’. The Consultation Paper also does not address scenarios where the outputs are significantly influenced by third party data sources or models.

 

Many AI systems deployed by regulated entities provide a probabilistic output based on the likelihood of various events (eg. risk scores), leaving the ultimate decision to the end user. In such a scenario, a definitive output is not provided by the regulating entities deploying AI and ML tools.

 

  • Risk Assessment of AI/ML tools:

 

The regulated entities will be required to ensure thorough due diligence before deploying the AI tools sourced from third parties, especially where the regulated entities do not have full control on working of AI and ML tools. As a risk mitigation measure, regulated entities planning to roll out AI tools will need to have back-to-back arrangements with such third-party providers of AI tools as risk mitigation.

 

  • Data Privacy:

 

The Consultation Paper has emphasized on the privacy of investors and stakeholders data. This regulated entities will have the same role and responsibility as the ‘Data Fiduciaries’ under the Digital Data Protection Act, 2023.


While the proposals and amendments suggested in the Consultation Paper are yet to be given effect, these form a growing series of proposals to regulate AI. The RBI has recently constituted a committee to develop FREE-AI in the Financial Sector. This committee has been mandated to assess AI adoption in financial services globally and in India, review global regulatory approaches, and identify potential AI-related risks. Similarly, MEITY has issued a consultation paper proposing to stream-line AI regulations and have a ‘whole of government approach’. All these attempts are trending towards India having a comprehensive framework for regulating AI on the lines of the European Union. It must however be ensured that these regulations do not hinder the growth and use of AI through onerous regulations.


[1] Stock Brokers and Depository Participants (vide Circular No. SEBI/HO/MIRSD/DOS2/CIR/P/2019/10 dated January 4, 2019); Market Infrastructure Institutions (vide Circular No. SEBI/HO/MRD/DOP1/CIR/P/2019/24 dated January, 31, 2019); and Mutual Funds, Asset Management Companies, Trustee Companies, Board of Trustees of Mutual Funds (vide Circular No. SEBI/HO/IMD/DF5/CIR/P/2019/63 dated May 9, 2019).

[2] SEBI Consultation Paper available here: https://www.sebi.gov.in/reports-and-statistics/reports/nov-2024/proposed-amendments-with-respect-to-assigning-responsibility-for-the-use-of-artificial-intelligence-tools-by-market-infrastructure-institutions-registered-intermediaries-and-other-persons-regulated-b-_88470.html

[3] Under changes proposed, “AI Tools” has been defined as follows “AI Tools may include any application or software programme or executable system or a combination thereof, offered by the person regulated by the Board, to investors/stakeholders or used internally by it to facilitate investing and trading or to disseminate investment strategies and advice, carry out its activities including compliance requirements and the same is portrayed as part of the public product offering or under usage for compliance or management or other business purposes”. It is to be noted that it is not an exhaustive definition.