Get data from an expression with contextual entities

In this tutorial, you will learn how to save valuable information that was mentioned in intents, using contextual entities. Entities are important pieces of information that are extracted from an expression. There are four types of entities. You can find more information about them here:

In this tutorial we will focus on one entity type: contextual entities. Contextual entities use machine learning to identify entities in sentences by learning which type of word your entity is, where it's placed in the sentence, and what the specific context is.

You want to store these contextual entities in a separate variable so you can re-use them later on. In the next tutorial, you will learn how you can ask explicitly for missing information.

Let's create a new intent that tells us the user wants to book a train ticket. A few different expressions can be:

  • I want a train ticket

  • I need a ticket

  • Can I book a train ticket here?

But what happens when the user says something like:

  • I want a train ticket to Amsterdam

  • I need to go to Antwerp tomorrow

  • Can I book a train ticket to Brussels please?

These expressions contain valuable information. We want to make sure we capture that information, in this case the destination and time, and save it as entities.

Creating contextual entities

Not all users will immediately mention their destination, so let's make sure we train our intent without those specific entities as well:

  • Go to NLP > Intents

  • Add a new intent called book train ticket

  • Add some simple expressions, like:

    • I want a train ticket

    • I need a ticket

    • Can I book a train ticket, please?

If you have trouble doing this, please read the previous tutorial.

Next, it's time to add an entity.

  • Click on + Add expression

  • Select the book train ticket intent

  • Enter an expression that contains an entity, for example:

I want to book a ticket from Brussels to Paris

  • Select Brussels in this sentence.

  • Click on the '+ entity' icon in the bottom right of the expression box to create a new entity for 'Brussels'

  • Brussels is the location the user wants to depart from, so we will name this entity origin

  • Type origin in the Create new entity field and click on 'Create new entity' to confirm

  • Brussels will be added to the list of possible values for the @origin variable

  • Do the same thing for Paris as a 'destination' entity

  • Add some other values to the 'origin' and 'destination' entities in the expression field. These will be saved for all future expressions.

  • Add more expressions that contain the entities origin and destination

We recommends adding at least 30 expressions per entity, to guarantee the quality of the entity detection.

  • Some ideas for expressions:

    • Can I book a train from Cologne to Brussels?

    • I need to be in Rotterdam

    • I need a train to London

    • I want to travel to Lyon

    • I want to buy a ticket from Moscow to Vladivostok

    • I need a ticket from New York to Baltimore


  • Tip: when typing a new expression, use the '@' to choose from an existing entity to add it to the sentence.

  • Make sure you retrain the NLP model by clicking the Update NLP button.

We have built a way to create a lot of expressions really fast: the expression generator. Follow the instructions on this page to get started with that tool.

Testing contextual entities

After we have retrained our model, let's see if its good enough to recognise the destination entity.

  • Go to NLP >Test to open the testing console

  • Write 'I would like to go to Brussels from Amsterdam' as the expression to be tested

  • Click on Test

You'll see that the entity gets recognized with a 89.72% confidence. The results will be different based on your training set. If the entity is not recognized correctly, you can add it here as a training expression immediately.

Make sure you retrain the model before testing newly added expressions.

Using variables in messages

When a user says something containing an entity, and the entity is successfully detected, our tool will automatically store the entity as a variable for that specific user.

To reuse the variable later on in the conversation, you can put it in between curly brackets like this: {variable_name}. When writing this message to the users, we will automatically substitute {variable_name} with the value of the variable. If the variable is empty, an empty string will be shown.

At the moment, when you test your bot, the user is stuck after giving the information about the ticket:

We can change that by adding a new Bot message:

  • In Bot dialogs, create a bot dialog of the type 'Bot message' book train ticket and link the book train ticket intent to it in the NLP tab.

  • Add a new text message with the text "So you want to go to {destination}, I can help you with that!"

Variables can be used everywhere throughout the platform, for example in API calls, list templates and button labels.

Not only entities can be variables. Later on you will learn how to use variables to build flows.

Testing entities in the emulator

Now that we have linked everything, we are ready to test if everything is configured correctly by using the emulator.

  • Open the emulator (aka Test your bot)

  • Enter "I want to go to Amsterdam" and click on submit

  • Open the debugger

In the tab 'NLP Result' you can now see if the entity was extracted correctly.

All entities are automatically stored in the variables list. In the 'Debugger' tab, you can find a list of all known variables under 'User Session'. Since the entity was recognized in the previous message, this contains the variable 'destination'.

If your entity is recognised by the NLP but doesn't show up in the User Session, it did not pass the threshold. By default the threshold is set to 80%. You can lower it in the NLP tab, but better still is to add more expressions to your bot so that the model becomes more robust and will work better in the future.

Multiple entities

You can add as many entities as you want to one expression. Sometimes it makes sense to prepare for a lot of entities and add an expression like

  • I want to go from Antwerp to Brussels tomorrow at 9am in first class

with the following entities:

  • origin: Antwerp

  • destination: Brussels

  • departure-date: tomorrow

  • departure-time: 9am

  • class: first

Keep in mind that NLP techniques are probabilistic in nature. When you try to capture five expressions in one sentence, it might not be able to recognise all of them correctly. As a general rule of thumb, you can start to expect reasonable results for one entity when the NLP was given at least 30 expression to learn from.

Additional suggestions for expressions

  • I need to be in Paris next Thursday

  • I need to be in New York on Friday

  • I want to go to Brussels on Monday

  • Friday I want to go from Antwerp to Amsterdam

  • I want to travel in second class from Ghent to Brussel on Friday

  • I want to travel in first class from Antwerp to Aalst on Thursday

  • I like to book a first class ticket from Aalst to Brussels at nine o'clock

  • Tomorrow I want to go from Antwerp to Brussels on the train from 9:00 in first class

Missing entities

Testing your entities

Update the book train ticket message to display the entities:

So I have a request for a train ticket; {origin} to {destination} on {departure-date}, {departure-time}, {class} class.

Now retrain your NLP model and test your bot

Uh oh, this isn't really what we expected. As you can see, the departure date and time is not set (your result may be different depending on the expressions you used). So, what's the problem? Lets have a look at the NLP Results:

origin, destination and departure-date were found correctly, but only origin and destination have a confidence score above 80%. So departure-date was not processed and put into a variable. More expressions can help you fix this problem!

Not every user will input all the entities you need. In the next tutorial, you will learn how to check if a user has already provided certain information, and ask for what's missing.