Singapore launches Project Moonshot – a generative Artificial Intelligence testing toolkit to address LLM safety and security challenges

At ATxAI, a signature conference on Day 2 of Asia Tech x Singapore (ATxSG) 2024, Singapore’s Minister for Communications and Information, Mrs Josephine Teo, launched AI Verify Project Moonshot (“Moonshot”), an easy-to-use testing toolkit designed to address security and safety challenges often associated with the use of large language models (LLMs), the cornerstone of many AI-driven solutions. It is one of the world’s first open-sourced tools to bring red-teaming, benchmarking, and baseline testing together in an easy-to-use platform – testament to Singapore’s commitment to harnessing the power of the global open-source community in addressing AI risks.

An open beta, Project Moonshot aims to provide intuitive results of the quality and safety of a model or application in an easily understood manner, even for a non-technical user. It was developed through working with partners such DataRobot, IBM, Singtel, and Temasek to ensure that the tool is useful and aligned with industry needs.

"IBM is pleased to be a design partner and contributor to AI Verify Moonshot. The provision of this new tool is significant as it aims to help developers and data scientists test their LLM applications against a baseline of risks, thereby accelerating the adoption of AI. We look forward to working closely with IMDA to develop appropriate open standards through our contributions." - Anup Kumar, Distinguished Engineer, CTO Data & AI, Head Client Engineering Asia Pacific, IBM

“DataRobot is thrilled to partner with IMDA to make LLM evaluation more accessible. The integration of Project Moonshot into the DataRobot AI Platform allows AI builders to confidently and responsibly scale generative AI within their organisations.” - Jay Schuren, Chief Customer Officer, DataRobot.

Project Moonshot is also part of an important move towards global testing standards. Two of the leading AI testing organisations – AI Verify Foundation and MLCommons – have come together to build a common safety benchmark suite.

"We are proud that leading AI testing organisations AI Verify Foundation and MLCommons are coming together to support common testing benchmarks for AI safety, building on innovative efforts like Project Moonshot. To further the collaboration, AI Verify Foundation and ML Commons have signed a memorandum of intent (MOI) on 29th May, intending to positively impact AI safety by providing model and application developers a globally accepted approach of safety testing for generative AI." - Peter Mattson, MLCommons President and co-chair AI Safety working group.

AI Verify Foundation celebrates first anniversary

The AI Verify Foundation (AIVF) celebrates its first anniversary at ATxSG. Launched in June 2023, AIVF aims to harness the power of collective expertise in enabling responsible use of AI. The foundation has since doubled its membership to over 120, with Amazon Web Services (AWS) and Dell joining as new premier members. AIVF has expanded the remit from AI testing tooling to the development of trust enhancing AI safety products such as the Model AI Governance Framework for Generative AI.

At the AI Verify Foundation 1st year anniversary celebration held yesterday, Dr Ong Chen Hui, Chair, Governing Committee, AI Verify Foundation, shared that key feedback from users were the need for AI Verify to align with emerging standards and cater for sector specific testing. In response to this feedback, she announced that AI Verify Foundation has completed the mapping of AI Verify with ISO 42001 and would be integrating AI Verify with MAS' Veritas toolkit.

Championing Women in Tech in Southeast Asia

Minister Teo also spoke on a panel at Women and Youth in Tech, where she unveiled findings from an IMDA and Boston Consulting Group (BCG) joint report titled Closing Tech’s GenderGap in Southeast Asia. The report surveyed over 4,000 individuals across the region including Singapore, Indonesia, Malaysia, the Philippines, Thailand, and Vietnam. According to the report, 83% of women in the tech sector are satisfied with their jobs and 70% believe that their company has initiatives to improve the recruitment, retention, and advancement of women – an increase from 49% in 2020. It also found that with 40% of women in technology, Singapore remains at the forefront of promoting gender diversity and inclusion in the industry within Southeast Asia. However, while there have been greater strides to support women in the sector across the region, more can be done to improve women representation in technical roles and leadership.

Minister Teo reiterated Singapore’s commitment to nurture and support women in tech with initiatives such as the SG Women in Tech (SGWIT) movement, the Singapore 100 Women in Tech list, which identifies and recognises mentors and role models to help women in their leadership journey; and TechSkills Accelerator (TeSA), which enables professionals to acquire the relevant in-demand skills, including helping women return more easily to technical roles after a career break. Such initiatives have made a significant impact as Singapore continues to lead the way in the representation of women in tech in Southeast Asia.

Other key highlights at ATxSG include:

Start of Global Consultation on Digital Inclusion

The United Nations Development Programme (UNDP), in collaboration with IMDA, convened a high-level roundtable at the ATxSummit and launched a 100-day Global Consultation and Digital Inclusion Playbook. The Playbook features IMDA’s Digital Skills for Life (DSL) framework¹, which underpins the national Digital for Life (DfL) movement, among other global case-studies. With Singapore as the starting point, the roundtable kickstarted the consultation process with a productive dialogue among senior government officials from various countries, as well as representatives from the international organisations, private sector and civil society

IMDA partners Google to bring Gemini Academy’s GenAI content and resources to the community

In support of the national DfL movement, Google has rolled out the Gemini Academy, a skilling programme designed to educate learners on the fundamentals of AI. Originally designed for teachers, Singapore will be the first country where the Academy’s content is extended beyond the classrooms to the members of public. IMDA will take reference from Gemini Academy’s curated GenAI content, in our update of the DSL resources to include GenAI content. GenAI content can be found on the DfL portal, which also features resources for learning DSL. IMDA will also work with Google’s Gemini Academy on GenAI workshops at NLB's libraries. The public can register for these workshops later in 2024 via NLB’s Eventbrite page.

For details and the latest agenda on ATxSG, please visit: www.asiatechxsg.com.