Sentiment Analysis in Power BI in Five Minutes

Sentiment Analysis,

You may have heard of it. It's a bit of a Pandora's box; we will change that today.

Businesses have started to tap into more qualitative data sources like customer reviews, survey responses, and social media posts. Sentiment analysis, or opinion mining, is one of the ways to extract insights from such data that comes in the form of natural language - which can be helpful, especially if you have a lot of written responses about your company or product.

In this post, we'll walk you through performing sentiment analysis in Power BI using Azure Cognitive Services.‍

‍What You'll Need

• An Azure account with an active subscription. You can kick off a new account with $200 free credit here.
• Power BI Desktop
• IMDB movie reviews data with a column "Review". Dataset here.

A note on extracting data from TZ files. The process differs between operating systems, so search yours for the best way. Also, I found this dataset to be huge, so I cut the extraction short to reduce the number of rows in the dataset. Remember - especially if paying for Azure - that the more rows you have, the more it will cost you in Azure.

Step 1: Create a Text Analytics resource in Azure

First, we need to create a Text Analytics resource on Azure, which will provide us with an API key and an endpoint URL we can use to access Azure Cognitive Services.

1. Sign in to the Azure portal.
2. Click on "Create a resource", search for "Text Analytics", and then create a new resource.
3. Once your resource deployment is complete, navigate to it and click "Keys and Endpoint". Here, note down Key1 and the Endpoint URL. We'll use these later.

‍

Step 2: Preparing Data in Power BI

Now, let's load our IMDB reviews data into Power BI.

1. Open Power BI Desktop and select "Get Data".
2. Choose your data source (this will depend on where your IMDB movie reviews data is stored).
3. Load your data into Power BI.

Once your data is loaded, you should see a table with a column of reviews; if it's not already named "Review", name it accordingly.

‍

Step 3: Connecting Power BI to Azure Cognitive Services

The next step is to connect Power BI to Azure Cognitive Services.

1. Click on "Transform Data" to open Power Query.
2. In Power Query, click on "Advanced Editor".
3. Here, create a new function:‍

Close the Advanced Editor and name this new function "GetSentimentScore".

Step 4: Adding Sentiment Scores to Your Data

Now, let's add sentiment scores to our data.

1. Still in Power Query, select your table and click on "Add Column" > "Invoke Custom Function".
2. Name the new column whatever you like (I used SentimentScore)
3. Choose the function "GetSentimentScore" and use the "Review" column as the argument.
4. Click OK. Power Query will call the Azure Text Analytics API for each row and add the sentiment scores to a new column.

Click "Close & Apply" to apply the changes.

Step 5: Visualising Sentiment Scores

You're now ready to create a visual with the "Review" and "SentimentScore" columns.

To create a simple table:

1. Click on the table icon in the Visualizations pane.
2. Drag the "Review" and "SentimentScore" fields to the "Values" area.

You now have a table that shows the sentiment score (positive, neutral, negative) for each movie review.

‍

Final Notes

Azure Cognitive Services is a powerful suite of AI services that brings machine learning capabilities into the hands of developers without the need to know a heap about what's going on under the hood.

However, it does come with its limitations. Things like restrictions on data input size, limits on API call rates, and sometimes the 'black box' nature of its models can occasionally make troubleshooting challenging - especially for junior developers.

Despite these constraints, you're not entirely at the mercy of the pre-defined models.

There are several strategies you can employ to make the most out of Azure Cognitive Services:

Batching: You can batch your requests to work within API call limits, especially when analysing large amounts of data.

Combining Services: You can leverage the strengths of various Cognitive Services to create more complex applications. For example, use the Text Analytics API for initial processing and feed its output into a Custom Vision model.

Using Custom Models: For more control over your machine learning models, consider using Azure's custom services like Custom Vision, Custom Neural Voice, or Language Understanding Intelligent Service (LUIS), which allow you to train your own models using your data.

Retraining: Most Azure Cognitive Services offer the ability to retrain models on new data, helping you to adapt pre-built models to specific tasks.

Fine-tuning: For the best results, you often need to fine-tune your approach, experiment with different settings and parameters, and iterate on your models based on the results you get.

While Azure Cognitive Services may not offer the same level of granular control as building your own machine-learning models from scratch, the trade-off is often worth it. You get access to sophisticated machine learning models that Microsoft continually improves, and you can use them with just a few lines of code.

My PBIX file here: https://drive.google.com/file/d/1aYyP88KfE0BZjU9s7Y2tkEMHcyrf9RF-/view?usp=drive_link