AI and Procurement – A Practical Approach

The Ada Lovelace Institute is an independent research institute who use a range of research methodologies to explore data and AI, including emerging technologies and the impact of AI.

The Institute has recently published a discussion paper under their ‘Society, Justice & Public Services’ research domain on whether the public sector is equipped to procure technology and AI in the public interest. The review is aimed primarily at local government in England and the information available to employees on what to consider when procuring these technologies.

Key sections of the report have been summarised in this article with our conclusions and potential next steps to assist local authorities when procuring AI to ensure compliance with Procurement Legislation and the Public Sector Equality Duty.

Key themes emerging from the paper

• Procurement legislation and the Public Sector Equality Duty requires local authorities to procure goods and services in accordance with fairness and equality. However, there is no clear guidance as to how this can be achieved whilst complying with the above.
• The Institute recognised that there were inconsistencies between the requirements of legislation such as the General Data Protection Regulations and the Public Sector Equality Duty in that they did not align with each other and did not allow room for engaging suppliers in relation to social impacts of technologies, requesting access to underlying data for testing, or supplier accountability.
• Definitions of key terms relating to fairness, transparency and societal benefit were confusing and used interchangeably but had varying meanings in the context of various documents.
• Most AI solutions are private sector solutions which don’t account for public sector use balanced against equality requirements and the ability to make financial savings with automated processes where local authorities are struggling financially. The imbalance of expertise within underfunded local authorities and private companies does not allow for fair negotiations to procure effective AI solutions. 

Practical steps for improving procurement of AI solutions

The paper provides practical solutions for improving procurement processes. These initial findings will be further developed following further research with public sector and private sector stakeholders:  

• Reviewing and streamlining Government guidance on procurement of AI and data-driven systems.
• Gaining consensus on definitions, leveraging existing data ethics frameworks and Government AI regulatory principles to clarify and consolidate relevant terminology.
• Improving governance, including the planned roll-out of the Algorithmic Transparency Recording Standard and implementing the Government’s AI regulatory principles.
• Piloting an Algorithmic Impact Assessment Standard for local government to use when procuring AI and data-driven systems.
• Setting out metrics for success at procurement stage that technologies can be assessed against post deployment.
• Clarifying when and how to engage with the public and experts in this process.
• Supporting local government to upskill teams to ensure effective AI use and auditing.
• Enabling transparency mechanisms so local government teams and suppliers have clarity and coherence on what transparency means for them, and procurers are equipped to engage with suppliers.
• Defining responsibilities across the AI procurement process, including between public- and private-sector actors.

Our Views

These steps and recommendations aim to equip local government procurement teams with the tools and knowledge needed to make informed decisions that lead to positive social impact when procuring AI and data-driven systems.

It’s important to ensure that the procurement process is transparent, fair, and aligned with the public interest. Whilst changes will be incremental, positive steps can be beneficial for all parties and allow for a better procurement experience and more effective solutions