Late last year, Microsoft announced the launch of Azure OpenAI Service, a fully managed, enterprise-focused product designed to give businesses access to AI lab OpenAI’s technologies with added governance features. Initially invite-only as a part of Azure Cognitive Services, the service allows access to OpenAI’s API through the Azure platform for applications like language translation and text autocompletion.
Now, coinciding with Build and fueled by new innovations from OpenAI, Microsoft is expanding Azure OpenAI Service to support additional use cases. The service is in limited access preview, meaning customers who want to use it can apply for pay-as-you-go access. Previously, Azure OpenAI Service was available by invitation only.
“We want to help customers move from the experimentation stage to using this technology as a core part of their business to really grow,” Eric Boyd, corporate vice president of AI platform at Microsoft, said in a phone interview with TechCrunch.
Fully managed models
Built by OpenAI, the models powering Azure OpenAI Service, including GPT-3, can be customized to perform tasks, from translating natural language into code to generating answers to questions. GPT-3 has been publicly available since mid-2020 through OpenAI’s API. But Azure OpenAI Service adds corporate-tailored layers on top of the models that the API doesn’t, including greater scaling capacity, private networking and access management.
For example, companies can use Azure OpenAI Service to run models in particular geographic regions for compliance reasons or centrally manage API endpoints and use customer-supplied keys for encryption. Azure OpenAI Service also ostensibly makes billing more convenient for existing Azure customers by charging for model usage under a single bill, versus separately under the OpenAI API.
While customers could previously fine-tune the models in Azure OpenAI Service using examples from their own data, the fine-tuning capabilities are expanding with the updates to the service, Boyd said. “You can take the standard OpenAI and [get] greater efficiency and effectiveness on real results,” he added. “We’re bringing the full array of models [from OpenAI] … so that companies don’t have to start from scratch to get really accurate results.”
The improvements are in part thanks to InstructGPT, a family of GPT-3-based models recently developed by OpenAI that are less likely to generate problematic text while more closely aligning with a user’s intent. Owing to the way that AI systems are developed, previous versions of models developed by OpenAI were found to produce toxic text, like placing words like “naughty” or “sucked” near female pronouns and “Islam” near words like “terrorism.” While InstructGPT isn’t perfect, its performance on internal tests at OpenAI was strong enough for the company to make it the default language-generating model in the OpenAI API.
Another new collection of models in Azure OpenAI Service, called embedding models, are turned to perform well on tasks like text similarity, text search and code search. Text similarity captures the semantic similarity between pieces of text, while text and code search find information in files to satisfy a set of criteria.
Meanwhile, New Zealand-based rural supplies cooperative Farmlands tapped Azure OpenAI Service to summarize customer interactions, classifying them as “neutral,” “negative” or “positive” and extracting keywords. Farmlands is additionally testing a chatbot and AI-generated product descriptions, both powered by Azure OpenAI Service, for its e-commerce store.
Beyond enhanced fine-tuning and new models, Azure OpenAI Service now offers access to Codex, which can generate code given a natural language prompt. (Codex launched in the OpenAI API last August.) Codex can be used for programming tasks including transpilation, explaining code and refactoring code in languages spanning Python, JavaScript, Go, Perl, PHP, Ruby, Swift and TypeScript.
Boyd said that he’s seen examples of customers using Codex to talk directly to the Minecraft API, among other applications.
Lastly, Azure OpenAI Service has a new “responsible AI” system designed to filter out content related to sex, violence, hate and self-harm. The system attempts to detect patterns of abuse or harm by a user of a model, which, when spotted, prompts a dedicated Microsoft team to work with customers to investigate and block the abuse if needed. The team is also responsible for updating the content filters as new forms of hate speech and slurs come into use.
“The new responsible AI system in Azure OpenAI Services includes automatic content filtering to deliver higher-quality content for customers. For APIs where generation of harmful content is a concern, it is on by default. Customers cannot opt out,” a Microsoft spokesperson clarified via email. “The content filtering is aligned with Microsoft’s content policy that governs use of Azure OpenAI Service. The content filters block content in four categories today: sexual/adult, violence, hate and self-harm. Each of these filters is based on a machine learning model developed specifically for that topic. To help ensure Azure OpenAI Service is used for intended purposes, the responsible AI system also checks for content that may be related to misuse of the system such as disinformation generation. Those filters are developed based on information Bing Threat Intelligence and the resulting alerts are sent to a team of human reviewers to investigate.”
Limited access
Azure OpenAI Service uptake has been strong, according to Boyd, with unnamed Fortune 500 companies using the service. When asked about wider availability, Boyd said that Azure OpenAI Service will remain in limited access for the foreseeable future.
Only companies that plan to implement “well-defined” use cases can sign up for the service, Boyd said, and they need to demonstrate that they’re going to use the models in a “reliable” manner. Microsoft is providing user experience design guidelines and patterns and a “transparency note” describing the limits, intended uses and characteristics of Azure OpenAI Service.
“The design guidelines are based on our HAX Toolkit but are tailored to give more specific guidance for generative AI applications,” the spokesperson continued. “The transparency note for Azure OpenAI Service will be published soon. It will be similar to other transparency notes for Azure Cognitive Services, found here. Transparency notes are part of a broader effort at Microsoft to put our AI principles into practice.”
Peering into the future, Boyd said that Microsoft will continue to work closely with OpenAI to develop and figure out ways to commercialize the lab’s technologies. Microsoft maintains a close relationship with OpenAI, having invested $1 billion in the lab in 2020 and exclusively licensed GPT-3. Microsoft maintains an “AI supercomputer” in Azure that OpenAI uses to train its models, and the company has integrated GPT-3 and Codex into several of its services, including Power Apps and GitHub’s Copilot.
“We definitely talk to OpenAI, and OpenAI is in the loop with customers, as well,” Boyd said.
Comment