Am I obliged to wait the complete result of the Speech Recognition?

The answer is no, in the last tutorial about Speech Recognition in Windows 10 , we have used the Speech Recognition like a ‘black box’ in the program we made, that is we speak and we wait for the complete result.

If you want to see the words you have spoken in real time, there is two interesting events that you can use for this.

Indeed, in the SpeechRecognizer object, we can find events for HypothesisGenerated and StateChanged, which we can attach to monitor our process.

Good news, after reading this article you’ll be able to:

  • Get each word detected by the speech recognizer engine instead of waiting the complete result
  • Get the state of the speech recognizer object (idle, capturing, sound started, speech detected, speech ended)

But first, we are going to retrieve the code we have done in the previous article:

And we are going to add these lines in the constructor:

Note that the line for the ResultGenerated event is missing, because we don’t need it here.

Then, we add the attached methods to the events in the class:

At the end, if you have followed the steps correctly the final code you should have is:

And now let us try this and see the result:

hypthesis

As you can notice in the picture above, the Visual Studio’s Output window is showing the results gradually ! The states and the words !

That’s it for this tutorial, if you have any questions or remarks, please do not hesitate 😀

How to perform a Speech Recognition in UWP and Windows 10?

To perform a speech recognition in your application, you could use the Bing Speech API from Microsoft Cognitive Services.

In Windows 10, you don’t need to download any SDK since the operating system has already the libraries to perform this task.

Thanks to the Speech API of Microsoft, we can either convert a text to speech or a speech to a text.

For now, let us focus on how to make a speech recognition.

I will talk in another article about how to make a text to speech action in your application.

Therefore, in this tutorial, I will explain to you how:

  • To create a SpeechRecognizer object which you can find in the namespace Windows.Media.Recognition
  • To start listening a person speech
  • To get a result from the detected speech in a text format

At the end, you will be able to:

  • know the bases to start a simple speech recognition.
  • add a speech to text functionality in your a UWP project.

But before we start, just for you to have a better view, you can find below the steps we are going to follow together:

  • Create a new UWP project in Visual Studio
  • Create a Speech Recognition Object
  • Call the constraints compiler
  • Start the speech recognition
  • Test & Run

1 – Create a new UWP project in Visual Studio

Open your Visual Studio and click on File -> New -> Project and then search for the template “Blanc App (Universal Windows)” like the picture below.

Capture

Just give a name to the project and click on Ok.

2 – Create a Speech Recognition Object

The object that can make a speech recognition is called SpeechRecognizer.

To use it, open the file “MainPage.xaml.cs“ and add the namespace “Windows.Media.Recogntion” with this line:

Next, in the code, add a private attribute with the type of SpeechRecognizer:

Then, in the constructor method of the MainPage class, instantiate the object like below:

At the end, you should have something like this:

3 – Call the constraints compiler

Yeah, it’s good to have been able to instantiate a SpeechRecognizer object, but we have to teach our object what it can recognize before using it. Otherwise, it will throw an Exception.

Indeed, you can add a list of constraints to our object so that it can only recognize a group of words chosen by you. But for this tutorial and in order to make it simple, we will limit ourselves with just calling the method to compile the constraints without adding nothing. This will alow us to use the default behavior.

In fact, if you don’t add a new constraint then, by default, your application will be able to recognize any words spoken by the user.

In order to do that, the method you have to call is CompileConstraintsAsync:

After that, your object will be able to understand any words spoken by the users.

Yeah, but where to add it ?

Indeed, I didn’t tell you where, but for now, just create a new method in the code named StartListeningAsync() and add your line into it just like this:

4 – Start the speech recognition

The SpeechRecognizer object has two methods to trigger the action that can listen a person speech.

The first one is called by using the method RecognizeAsync(), the latter is able to listen in a limited time and when it cannot hear anything, it stop.

We can call it like this:

After the execution, « result » will contain the result of the speech recognition (if there is any).

The second method is called by using a property named ContinuousRecognitionSession and inside of this one, you can use the method called StartAsync() to launch the action.

In this tutorial, I will focus on the property ContinuousRecognitionSession and show you how to retrieve a result from this latter.

So first of all, if you look inside the object ContinuousRecognitionSession you can find an event called ResultGenerated.

Therefore, we are now going to subscribe a method to this event.

To do so, go to the MainPage construtor method and add this line:

Next, create a method with the name of ContinuousRecognitionSession_ResultGenerated like below:

This method will show us the result of the speech recognition in the ouput console.

If you have followed correctly the instructions, you should have something like this:

Remember of the CompileConstraintsAsync() method in StartListeningAsync(). After this one, we are going to call the StartAsync() method to launch the speech recognition like shown below:

The program is now ready to run !

5 – Test & Run

Finally, we can test our application and check if it is working !

Capture3

That’s it for now, I just hope this will help you to start using the speech recognition engine in your application.

Nevertheless, they are still some details that I didn’t talk about yet. But don’t worry, I will try to add more information in the future if I have the time.

If you have any questions, do not hesitate to ask me 🙂

What to do next ?

How to access your Raspberry PI from the Internet ?

They are many ways to access a Raspberry PI from a remote computer. But today, I would like to present you a way to do it.

All you need is a basic knowledge about computer network.

To follow this tutorial, you just need:

  • Obviously, to have an internet access and a modem (maybe provided by your ISP)
  • A raspberry PI connected to your local network
  • A computer to test if all your settings work fine.

Maybe you don’t know yet how to connect locally to the Raspberry PI. So I will start to explain how to do it. Then, I will present you how to connect your Raspberry PI from the internet.

Nevertheless, if you’re already familliar with a local connection. Just skip the first step and jump to the next one.

Step 1. How to connect locally ?

To begin with, we’ll use XRDP to perform a local remote connection between a Windows client and our Raspberry PI. (Feel free to use an another way to do it)

First, open your terminal in your Raspbian operating system and enter this command line:

If you don’t install it, the program will not be able to work.

And then use the Remote Desktop Connection software from your Windows client to perform a connection to your Raspberry PI:

Remote Desktop Connection - Computer

Use the LAN IP address of your Raspberry PI and then enter « Connect ».

If everything went right, you should have this screen:

ex2

Enter the username and the password. By default, Raspbian has the user « pi » installed and the default password is « raspberry ».

After that, you’ll be able to have access to the desktop of your Raspbian OS.

ex3

Step 2. How to connect from the internet ?

Imagine you want to access your Raspberry from your work place ? It’ll be interesting if you could access it from your public IP address, isn’t it ?

All you need to do is to make your Raspbian accessible to the internet.

Therefore, open Google Chrome (or something else if you prefer) and enter the ip address of your modem. Type in 192.168.1.1 for instance in the navigation bar.

Normally, you’ll be able to have access to the web interface of your box. For instance, I have something like this:

ex4

After you logged in, you have to configure a port-forwarding for your Raspberry PI. Of course, the menu is different for each ISP.

But you should find your way even if I show you an example with proximus.

ex5

You’ll notice that I have an overview of all the devices connected to the network and those who were.

Next, I select my Raspberry PI :

ex6

After clicking on it, this window appears:

ex7

You can notice here that I have mapped two port number :

  • one for an HTTP server (the default port for this is 80).
  • and an other to enable connection with the remote desktop connection software.

    If you want to create a custom one, you’ll have to follow these steps :

  1. Choose an existing service or just create a custom one.
  2. Choose the protocol (UDP, TCP or Any)
  3. Enter the port number that you are forwarding in the boxes External Port, and Internal Port.
  4. In the Internal Host box, enter the LAN IP address of the device you are forwarding this port to.
  5. Click Ok and close the window

That’s it ! You can now connect with your public IP address instead of your local address just like we did before.

If you are wondering where you can find your IP address, just look there : www.myipaddress.com

Hope this helped !