But they found that an alternative approach, called Bayesian model merging, offered similar performance with advantages in reproducibility and iteration speed. With the Applied Modeling and Data Science (AMDS) group’s grammar induction tool, that whole process takes seconds.ĪMDS research scientists Ge Yu and Chris Hench and language engineer Zac Smith experimented with a neural network that learned to produce grammars from golden utterances. Given a list of, say, 50 golden utterances, a computational linguist could probably generate a representative grammar in a day, and it could be operationalized by the end of the following day. The question mark indicates that the some_more variable is optional. Below the rules of the grammar is a diagram of a computational system (a finite-state transducer, or FST) that implements them.Ī toy grammar, which can model requests to play pop or rock music, with or without the modifiers “some” or “more”, and a diagram of the resulting finite-state transducer. A grammar is a set of rewrite rules for varying basic template sentences through word insertions, deletions, and substitutions.īelow is a very simple grammar, which models requests to play either pop or rock music, with or without the modifiers “more” and “some”. Grammars have been a tool in Alexa’s NLU toolkit since well before the first Echo device shipped. Guided resampling concentrates on optimizing the volume and distribution of sentence types, to maximize the accuracy of the resulting NLU models. The other tool, guided resampling, generates new sentences by recombining words and phrases from examples in the available data.
#Finite state automata toy grammar series#
From those patterns, it produces a series of rewrite expressions that can generate thousands of new, similar sentences. One of the tools, which uses a technique called grammar induction, analyzes a handful of golden utterances to learn general syntactic and semantic patterns. The new bootstrapping tools, from Alexa AI’s Applied Modeling and Data Science group, treat the available sample utterances as templates and generate new data by combining and varying those templates. Alexa feature teams will propose some canonical examples of customer requests in the new language, which we refer to as “golden utterances” training data from existing locales can be translated by machine translation systems crowd workers may be recruited to generate sample texts and some data may come from Cleo, an Alexa skill that allows multilingual customers to help train new-language models by responding to voice prompts with open-form utterances.Įven when data from all these sources is available, however, it’s sometimes not enough to train a reliable NLU model. When a new-language version of Alexa is under development, training data for its NLU systems is scarce. But interpreting that text - determining what the customer wants Alexa to do - is the job of Alexa’s natural-language-understanding (NLU) systems. These three locales were the first to benefit from two new in-house tools, developed by the Alexa AI team, that produce higher-quality synthetic data more efficiently.Įach new locale has its own speech recognition model, which converts an acoustic speech signal into text. At a high level, the solution is to use synthetic data.
#Finite state automata toy grammar how to#
Like all new-language launches, these addressed the problem of how to bootstrap the machine learning models that interpret customer requests, without the ability to learn from customer interactions. In the past few weeks, Amazon announced versions of Alexa in three new languages: Hindi, U.S.
0 kommentar(er)
