Build SPFX (SharePoint Framework) Bot Using Azure Bot Framework SDKV4 With Luis And QnA Maker

The zure Bot Framework SDKV4 can be used with LUIS models and QnA Maker knowledge bases. It helps to determine which LUIS model or QnA Maker knowledge base best matches the user input.

 

SPFX (SharePoint Framework) is the recommended and only option to deploy Azure bots into SharePoint Online Modern Sites.

 

In this article, I am going to extend my previous article with SPFX (SharePoint Framework).

https://manojmittalblogs.blogspot.com/2020/04/conversation-bot-using-luis-and-qna.html

SPFX (SharePoint Framework) interacts with Azure Bot Services via directline.

 

Let's get started with implementation steps. 

 

Step 1

 

Navigate to Deployed Azure Bot and Add Direct Line as Channel & get Secret Key

1. Select Channel and add a featured channel option; i.e. Direct Line, Ms Teams and Cortona. Select Direct Line to add.
2. As you select direct line as channel, it will be added.
3. Click edit to get the secret key. This key will be used in SPFX webpart to post requests to bot via direct line api & secret key. 

Step 2 - Create SFPX Project

1. Open Command Prompt or Visual Studio Code and Navigate or create a blank folder; i.e. ChatBot, and use CD command to navigate into folder.
2. Type yo@Microsoft/SharePoint generator to create a framework.
3. Type Solution Name: spfx-chat-bot
4. Select SharePoint Online
5. Select option to place a file as "Current Folder"
6. Place the WebPart Name & Description as ChatBot
7. Select framework as React to start.

Step 2 - Install NPM Package

 

Install the below defined npm packages to SPFX solution. 

1. npm install botframework-directlinejs  
2. npm install botframework-webchat  

Step 3

 

Create State Class for Modal Dialog

 

State will hold variable to switch the modal dialog box.

1. export interface IChatBotState {    
2.     showModal: boolean;    
3.     isDraggable: boolean;    
4.     directline: any;    
5.     styleSetOptions: any;    
6. }     

Step 4 - Update Properties Interface

 

Properties have variables which help to set background color, font etc. 

1. import { IWebPartContext } from '@microsoft/sp-webpart-base';  
2. export interface IChatbotProps {  
3.   description: string;  
4.   directLineToken: string;  
5.   bubbleBackground: string;  
6.   bubbleTextColor: string;  
7.   bubbleFromUserBackground: string;  
8.   bubbleFromUserTextColor: string;  
9.   backgroundColor: string;  
10.   botAvatarImage: string;  
11.   botAvatarInitials: string;  
12.   userAvatarImage: string;  
13.   userAvatarInitials: string;  
14.   hideUploadButton: boolean;  
15.   sendBoxBackground: string;  
16.   sendBoxTextColor: string;  
17.   context: IWebPartContext;  
18. }  

Step 5 - React Component Implementation

 

Define construcutor, which will help to initilaize the defined properties and set the state variable.

1. constructor(props) {  
2.   super(props);  
3.    
4.   this.state = {  
5.     showModal: false,  
6.     isDraggable: true,  
7.     directline: null,  
8.     styleSetOptions: styleOptions  
9.   };  
10.   this._showModal = this._showModal.bind(this);  
11.   this._closeModal = this._closeModal.bind(this);  
12. }  

Show Modal and Close Modal used to set toggle variable.

1. private _showModal = (): void => {  
2.    this.setState({ showModal: true });  
3.    this.fetchToken();  
4.  };  
5.   
6.  private _closeModal = (): void => {  
7.    this.setState({ showModal: false });  
8.  };  

Initiate the post request to directline .

1. async fetchToken() {  
2.    var myToken ='<<--Secret Key-->>';  
3.    const myHeaders = new Headers()  
4.    myHeaders.append('Authorization', 'Bearer ' + myToken)  
5.    const res = await fetch(  
6.      'https://directline.botframework.com/v3/directline/tokens/generate',  
7.      {  
8.        method: 'POST',  
9.        headers: myHeaders  
10.      }  
11.    )  
12.    const { token } = await res.json();  
13.    console.log(token);  
14.    this.setState({  
15.      directline: createDirectLine({ token })  
16.    });  
17.    this.state.directline.postActivity({  
18.      from: { id: "serId", name: "USER_NAME" },  
19.      name: "requestWelcomeDialog",  
20.      type: "event",  
21.      value: "token"  
22.    }).subscribe(  
23.      id => console.log(`Posted activity, assigned ID ${id}`),  
24.      error => console.log(`Error posting activity ${error}`)  
25.    );  
26.   
27.  }  

Use component did mount to call this method.

1. async componentDidMount() {  
2.     this.fetchToken();  
3.   }  

Render Method

 

On Icon Click call Modal Dialog and which will invoke ReactWebChat Tag with direct line state variable,

1. <div className={ styles.chatbot }>  
2.         <DefaultButton style={{ border: 'none', textDecoration: 'none', background: 'none', outline: 'none' }} onClick={this._showModal}>  
3.        <img src="https://data.blob.core.windows.net/boticon/ChatBotIcon.JPG" alt="Chatbot" height="100px" width="100px" />  
4.      </DefaultButton>  
5.      <Modal  
6.        titleAriaId={this._titleId}  
7.        subtitleAriaId={this._subtitleId}  
8.        isOpen={this.state.showModal}  
9.        onDismiss={this._closeModal}  
10.        isBlocking={true}  
11.        containerClassName={contentStyles.container}  
12.        dragOptions={this.state.isDraggable ? this._dragOptions : undefined}  
13.      >  
14.        <div className={contentStyles.header}>  
15.          <span id={this._titleId}>Virtual Assistant</span>  
16.          <IconButton  
17.            styles={iconButtonStyles}  
18.            iconProps={{ iconName: 'Cancel' }}  
19.            ariaLabel="Close popup modal"  
20.            onClick={this._closeModal as any}  
21.          />  
22.        </div>  
23.        <div id={this._subtitleId} className={contentStyles.body}>  
24.          <ReactWebChat directLine={this.state.directline} styleOptions={this.state.styleSetOptions} />  
25.        </div>  
26.      </Modal>  
27.      </div>  

Step 6 - Test locally

 

To run your solution type "Gulp Serve" in the terminal window. It will prompt the new window with local host workbench url " https://localhost:4321/temp/workbench.html"

Click icon to get pop up. Start typing the question as defined into Luis and QnA Maker and results start appearing without any delay.

The previous article can be found from these links.

• Learn and Build Azure Bot With Bot Framework SDK V4
• Build QnAMaker Azure Bot with Bot Framework SDK V4
• Design Rich Card with Bot Framework SDK V4
• State Management with Bot Framework SDK V4
• Store Conversation to CosmosDB with Bot Framework SDK V4
• Build Conversation Bot using Luis and QnA Maker

I hope you have enjoyed and learned something new in this article. Thanks for reading and stay tuned for the next article.

Conversation Bot Using Luis And QnA Maker With Azure Bot Framework SDKV4

Azure botframework SDKV4 can be used with LUIS models and QnA Maker knowledge bases. It helps to determine which LUIS model or QnA Maker knowledge base best matches the user input.

In this tutorial, you will learn the following.

1. Build SDKV4 Bot Framework Project 
2. Build Luis App
3. Build QnA Maker
4. Build and test your bot with Emulator.

Let's understand the information flow.

Azure Bot framework (SDKV4) interacts with waterfall dialog model. Waterfall dialog model has three basic parameters; i.e.:

1. Intro Step Async -> Used to initalize the luis configuration.
2. Act Step Async -> Used to gather all potiential information and process the request based on defined intent.
3. Final StepAsync -> Used to restart or call the mail dialog again. 

On Memebr Added Async is called as and when a member is added and is used to greet the members.

OnMessageActivityAsync is called for each user input received. This module finds the top scoring user intent and is used to process the request based on defined intent or QnA Maker.

Step 1 - Create a VS Project using BotFramework V4 (Echo Bot)

Select echo bot and click next to select the location and finsh. 

Step 2 - Visual Studio Project Strucutre

VS project strucutre looks like this,

Installing packages

Prior to running this app for the first time ensure that several NuGet packages are installed,

1. Microsoft.Bot.Builder
2. Microsoft.Bot.Builder.AI.Luis
3. Microsoft.Bot.Builder.AI.QnA

Step 3 - Create Luis Recognizer Class

Luis recognizer class implements IRecognizer interface. This method will help to recognize the intent and entity.

In LusRecognizer.cs, the information contained within configuration file appsettings.json is used to connect your bot to the Luis. The constructors use the values you provided to connect to these services. 

1. private readonly LuisRecognizer _recognizer;  
2.   
3.         public LusRecognizer(IConfiguration configuration)  
4.         {  
5.             var luisIsConfigured = !string.IsNullOrEmpty(configuration["LuisAppId"]) && !string.IsNullOrEmpty(configuration["LuisAPIKey"]) && !string.IsNullOrEmpty(configuration["LuisAPIHostName"]);  
6.             if (luisIsConfigured)  
7.             {  
8.                 var luisApplication = new LuisApplication(  
9.                     configuration["LuisAppId"],  
10.                     configuration["LuisAPIKey"],  
11.                     "https://" + configuration["LuisAPIHostName"]);  
12.   
13.                 var recognizerOptions = new LuisRecognizerOptionsV3(luisApplication)  
14.                 {  
15.                     PredictionOptions = new Microsoft.Bot.Builder.AI.LuisV3.LuisPredictionOptions  
16.                     {  
17.                         IncludeInstanceData = true,  
18.                     }  
19.                 };  
20.   
21.                 _recognizer = new LuisRecognizer(recognizerOptions);  
22.             }  
23.         }  

Step 4 - Create Main Dialog class

Main Dialog class implements the ComponentDialog. This class method will be responsible to define the waterfall model and is based on defined intent which helps to process the query.

Initalize the constructor

1. private readonly LusRecognizer _luisRecognizer;  
2.        public QnAMaker qnAMakerObject { get; private set; }  
3.   
4.               // Dependency injection uses this constructor to instantiate MainDialog  
5.        public MainDialog(LusRecognizer luisRecognizer, QnAMakerEndpoint endpoint)  
6.            : base(nameof(MainDialog))  
7.        {  
8.            _luisRecognizer = luisRecognizer;  
9.            qnAMakerObject = new QnAMaker(endpoint);  
10.   
11.            AddDialog(new TextPrompt(nameof(TextPrompt)));  
12.            AddDialog(new WaterfallDialog(nameof(WaterfallDialog), new WaterfallStep[]  
13.            {  
14.                  
15.                IntroStepAsync,  
16.                ActStepAsync,  
17.                FinalStepAsync,  
18.            }));  
19.   
20.            // The initial child Dialog to run.  
21.            InitialDialogId = nameof(WaterfallDialog);  
22.        }  

Initilaize the Intro Step Asyc Method -> It will help to intialize or create the Luis Object,

1. private async Task<DialogTurnResult> IntroStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)  
2.        {  
3.            if (!_luisRecognizer.IsConfigured)  
4.            {  
5.                //await stepContext.Context.SendActivityAsync( MessageFactory.Text("NOTE: LUIS is not configured. To enable all capabilities, add 'LuisAppId', 'LuisAPIKey' and 'LuisAPIHostName' to the appsettings.json file.", inputHint: InputHints.IgnoringInput), cancellationToken);  
6.   
7.                return await stepContext.NextAsync(null, cancellationToken);  
8.            }  
9.            
10.            return await stepContext.PromptAsync(nameof(TextPrompt), new PromptOptions { }, cancellationToken);  
11.        }  

Initilaize the Act Step Asyc Method -> It will help to gather all potienial information to process the request based on defined intent.

When the model produces a result, it indicates which service can most appropriately process the utterance. The code in this bot routes the request to the corresponding service, and then summarizes the response from the called service. Depending on the intent returned, this code uses the returned intent to route to the correct LUIS model or QnA service.

1. private async Task<DialogTurnResult> ActStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)  
2.         {  
3.             // Call LUIS and gather any potential booking details. (Note the TurnContext has the response to the prompt.)  
4.             var luisResult = await _luisRecognizer.RecognizeAsync<CognitiveModel>(stepContext.Context, cancellationToken);  
5.   
6.             switch (luisResult.TopIntent().intent)  
7.             {  
8.                 case CognitiveModel.Intent.LeaveRequest:  
9.                     //Do processing.  
10.                     await stepContext.Context.SendActivityAsync(MessageFactory.Text("Luis Intent Called i.e. Leave Request"), cancellationToken);  
11.                     break;  
12.   
13.                 default:  
14.                     var results = await qnAMakerObject.GetAnswersAsync(stepContext.Context);  
15.                     if (results.Length > 0)  
16.                     {  
17.                         var answer = results.First().Answer;  
18.                         await stepContext.Context.SendActivityAsync(MessageFactory.Text(answer), cancellationToken);  
19.                     }  
20.                     else  
21.                     {  
22.                         Activity reply = ((Activity)stepContext.Context.Activity).CreateReply();  
23.                         reply.Text = $"😢 **Sorry!!! I found nothing** \n\n Please try to rephrase your query.";  
24.                         await stepContext.Context.SendActivityAsync(reply);  
25.   
26.                     }  
27.                     break;  
28.             }  
29.   
30.             return await stepContext.NextAsync(null, cancellationToken);  
31.         }  

Initilaize the Final Step Asyc Method -> It will help to re-initiate the dialog,

1. private async Task<DialogTurnResult> FinalStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)  
2.        {  
3.            
4.            return await stepContext.ReplaceDialogAsync(InitialDialogId, null, cancellationToken);  
5.        }  

Step 5 - Manually update your appsettings.json file

Once all of your service apps are created, the information for each needs to be added into your 'appsettings.json' file.

For each of the entities shown below, add the values you recorded earlier in these instructions,

appsettings.json

1. "MicrosoftAppId": "",  
2. "MicrosoftAppPassword": "",  
3.   
4. "QnAKnowledgebaseId": "<knowledge-base-id>",  
5. "QnAEndpointKey": "<qna-maker-resource-key>",  
6. "QnAEndpointHostName": "<your-hostname>",  
7.   
8. "LuisAppId": "<app-id->",  
9. "LuisAPIKey": "<your-luis-endpoint-key>",  
10. "LuisAPIHostName": "<your--app-region>",  

Step 6 - Create Luis App

Log into the Luis web portal "https://www.luis.ai" Under the My apps section, select the create new app. The following Dialog Box will appear: 

Step 7 - Define Luis Model

1. Select the build button to define intent and entity
2. Select Intent
3. Enter phrases; i.e., "apply 2 days sick leave"
4. Select Entities
5. Add Pre-defined entitiy i.e. number , Date & time and custom entity i.e. Leave Type.
6. Train the Model
7. Publish the Model

Choose the 'production' environment and then select the Publish button.

Once your new LUIS app has been published, select the MANAGE Tab. From the 'Application Information' page, record the values Application ID as "app-id-for-app" and Display name as "name-of-app". From the 'Key and Endpoints' page, record the values Authoring Key as "your-luis-authoring-key" and Region as "your-region". These values will later be used within your 'appsetting.json' file.

Step 8 - Create QnA Maker knowledge base

The first step to setting up a QnA Maker knowledge base is to set up a QnA Maker service in Azure. To do that, follow the step-by-step instructions found here.

Once your QnA Maker Service has been created in Azure, you need to record the Cognitive Services Key 1 provided for your QnA Maker service. This will be used as <azure-qna-service-key1> when adding the QnA Maker app to your dispatch application.

Now sign in to the QnAMaker web portal. https://qnamaker.ai

1. Your Azure AD account.
2. Your Azure subscription name.
3. The name you created for your QnA Maker service. (If your Azure QnA service does not initially appear in this pull down list, try refreshing the page.)
4. Create QnA
5. Provide a name for your QnA Maker knowledge base. For this example use the name 'sample-qna'.
6. Select the option + Add File, navigate to the CognitiveModel folder of your sample code, and select the file 'QnAMaker.tsv'. There is an additional selection to add a Chit-chat personality to your knowledge base but our example does not include this option.
7. Create QnA
8. Select Create your knowledge base.
9. Once the knowledge base is created from your uploaded file, select Save and train and when finished, select the PUBLISH Tab and publish your app.
10. Once your QnA Maker app is published, select the SETTINGS Tab, and scroll down to 'Deployment details'. Record the following values from the Postman Sample HTTP request.

Step9 - Add Luis & QnaMaker Keys, ID and Endpoint to VS solution 

Service authoring keys

The authoring key is only used for creating and editing the models. You need an ID and key for each of the two LUIS apps and the QnA Maker app.

App	Location of information
LUIS App ID	In the LUIS portal, for each LUIS app, in the Manage section, select Keys and Endpoint settings to find the keys associated with each app. If you are following this tutorial, the endpoint key is the same key as the <your-luis-authoring-key>. The authoring key allows for 1000 endpoint hits then expires.
QnA Maker App ID	In the QnA Maker portal, for the knowledge base, in the Manage settings, use the key value shows in the Postman settings for the Authorization header, without the text of EndpointKey.

Update these values into Appsetting.json file.

Step 10 - Test your bot

1. Using your development environment, start the sample code. Note the localhost address shown in the address bar of the browser window opened by your App: "https://localhost:<Port_Number>".
2. Open your Bot Framework Emulator, then select Create a new bot configuration. A .bot file enables you to use the Inspector in the bot emulator to see the JSON returned from LUIS and QnA Maker.
3. In the New bot configuration dialog box, enter your bot name, and your endpoint URL, such as http://localhost:3978/api/messages. Save the file at the root of your bot sample code project.
4. Select the bot name in the My Bots list to access your running bot

Output

First,  the bot prompts with a greeting message.

Second, the user asks a query, the system defines the intent -> it goes to Luis and responds.

Third, the user asks a non-intent question -> it goes to qna maker and responds.

The previous article can be found from these links.

• Learn and Build Azure Bot With Bot Framework SDK V4
• Build QnAMaker Azure Bot with Bot Framework SDK V4
• Design Rich Card with Bot Framework SDK V4
• State Management with Bot Framework SDK V4
• Store Conversation to CosmosDB with Bot Framework SDK V4

I hope you have enjoyed and learned something new in this article. Thanks for reading and stay tuned for the next article.