June 23, 2022

Improving NLU Training over Linked Data with Placeholder Concepts SpringerLink

Software development

It only provides a feature that the intent classifier will use to learn patterns for intent classification. To make it easier to use your intents, give them names that relate to what the user wants to accomplish with that intent, keep them in lowercase, and avoid spaces and special characters. The format of the generated file is the second allowed format that is described in the JSON format section.

This paper presents an approach to generate training datasets for the NLU component from Linked Data resources. We analyze how differently designed training datasets can impact the performance of the NLU component. Whereby, the training datasets differ mainly by varying values for the injection into fixed sentence patterns. As a core contribution, we introduce and evaluate the performance of different placeholder concepts. Our results show that a trained model with placeholder concepts is capable of handling dynamic Linked Data without retraining the NLU component. Thus, our approach also contributes to the robustness of the NLU component.

Rasa NLU - Understanding Training Data

One possibility is to have a developer who is not involved in maintaining the training data review test set data annotations. When analyzing NLU results, don't cherry pick individual failing utterances from your validation sets (you can't look at any utterances from your test sets, so there should be no opportunity for cherry picking). No NLU model is perfect, so it will always https://tomatdvor.ru/sovety-dlja-cvetnika/1409-chem-podkormit-mnogoletnie-cvety-osenju-sovety-dlja-cvetnika.html be possible to find individual utterances for which the model predicts the wrong interpretation. However, individual failing utterances are not statistically significant, and therefore can't be used to draw (negative) conclusions about the overall accuracy of the model. Overall accuracy must always be judged on entire test sets that are constructed according to best practices.

• If the user input does not correspond to any of the learned intent labels, the model will still match it to one of them [16].
• For example for our check_order_status intent, it would be frustrating to input all the days of the year, so you just use a built in date entity type.
• The name of the lookup table is subject to the same constraints as the name of a regex feature.
• The database contains all entity values that users might use in their utterances.
• If all potential entity values that an NLU shall be able to extract are known in advance it is best to use them all for training.

The embeddings model is based on the StarSpace model developed by Facebook [24]. During training, the embeddings classifier learns its own embeddings for each of the words in the training dataset, thereby taking into account domain-specific uses of words [15]. The created feature vectors are enriched by an additional three dimensions using the intent_featurizer_ngrams. Again, the three most common n-grams in the training data are determined and the three added dimensions are used to indicate whether a given token includes one of these n-grams.

Define intents and entities that are semantically distinct

The / symbol is reserved as a delimiter to separate retrieval intents from response text identifiers. This is the case for the origin and destination slot names in the previous example, which have the same slot type city. The problem of annotation errors is addressed in the next best practice below. In conversations you will also see sentences where people combine or modify entities using logical modifiers—and, or, or not.