What's the correct flow to handle image inputs in a dialog that uses LUIS?

问题

I'm putting together a bot in c# that accepts image inputs from the user as well as text input. I'm using LUIS as the AI framework to determine intents in a dialog pattern. However, it seems like both types of input can't coexist: LUIS and attachments. I would like to know if there is a recommended pattern for this scenario. Please help! :|

回答1:

In the MessageController, you can get the image/attachments values

activity.Attachments

 await Conversation.SendAsync(activity, () => new RootLuisDialog();

LuisDialog will handle the text messages,apart from text it will consider all other thing as null parameter. But,

The Prompts.attachment() method will ask the user to upload a file attachment like an image or video. The users response will be returned as an IPromptAttachmentResult.

Here is the reference link.

回答2:

You can filter out messages that have attachments in the MessageController.

You can check for attachments using

activity.Attachments == null

回答3:

You can also handle it within the LUIS dialog itself.

Here is an example using PromptDialog.Attachment:

[LuisIntent("SomeIntent")]
    public async Task SomeIntent(IDialogContext context, LuisResult result)
    {

        PromptDialog.Attachment(
               context: context,
               resume: ResumeAfterExpenseUpload,
               prompt: "I see you are trying to add an expense. Please upload a picture of your expense and I will try to perform character recognition (OCR) on it.",
               retry: "Sorry, I didn't understand that. Please try again."
           );
     }

    private async Task ResumeAfterExpenseUpload(IDialogContext context, IAwaitable<IEnumerable<Attachment>> result)
    {
        var attachment = await result as IEnumerable<Attachment>;
        var attachmentList = attachment.GetEnumerator();
        if (attachmentList.MoveNext())
        {
            //output link to the uploaded file
            await context.PostAsync(attachmentList.Current.ContentUrl); 

        }
        else
        {
            await context.PostAsync("Sorry, no attachments found!");
        }

    }

回答4:

So I found out a better pattern, though is along the lines to Praveen's answer.

In the MesssageController you want to check for attachments activity.Attachments == null but in addition, you have to create what is called a RootDialog and send all conversations there from which you can forward conversations to other Dialogs.

In my case, I'm forwarding the messages that I want LUIS to handle to a dialog class that inherit LUIS as a service. Other messages, such as attachments are sent to another dialog class to be handled.

Another way to get an attachment and handle it inside your dialog code is by using the PromptAttachment call as a catcher for user's input of the attachment:

 var dialog = new PromptDialog.PromptAttachment(message.ToString(), "Sorry, I didn't get the receipt. Try again please.", 2);
        context.Call(dialog, AddImageToReceiptRecord);

Cheers! :)

来源：https://stackoverflow.com/questions/41712617/whats-the-correct-flow-to-handle-image-inputs-in-a-dialog-that-uses-luis

标签

botframework

luis