- Blog
- 06.24.2024
- Data Fundamentals, Product
Sentiment Analysis in Amazon Redshift with Mistral 7B Instruct using Amazon Bedrock
Sentiment analysis is a powerful technique that enables organizations to gain valuable insights from unstructured data sources, such as product reviews, social media posts, and customer feedback. This article will explore various methods for performing sentiment analysis in Redshift using Mistral 7B Instruct through Amazon Bedrock, starting with a Python-based approach.
We will also introduce Mistral 7B Instruct, a state-of-the-art language model developed by Anthropic, and its capabilities in the realm of sentiment analysis.
What is Sentiment Analysis?
Sentiment Analysis is a crucial aspect of Natural Language Processing (NLP) that translates unstructured text into quantifiable sentiment scores, shedding light on the emotional tone present in textual data. Large Language Models (LLMs), such as Mistral 7B Instruct, are adept at performing Sentiment Analysis due to their intricate architectures and pre-training on diverse text corpora. These models leverage deep learning techniques to grasp context, irony, and nuanced expressions of sentiment, providing reliable and precise sentiment scores.
To harness the full potential of Sentiment Analysis using LLMs, data preparation must be meticulous and reliable. Data engineers play a pivotal role here, ensuring that raw text data is meticulously preprocessed, cleaned, and formatted for effective ingestion by the LLM. This involves implementing pipelines that seamlessly bridge the gap between databases and the LLM, ensuring robust data flows, low latency, and optimal resource utilization. Effective interface management between datasets and LLMs ensures high-quality, actionable sentiment insights.
Business examples of Sentiment Analysis:
- Brand Monitoring: Analyze social media conversations to gauge public sentiment towards a company's products or services, enabling proactive reputation management.
- Customer Service: Automatically categorize customer feedback and prioritize negative sentiments, allowing for timely resolution of issues and improved customer experience.
- Market Research: Gain insights into consumer preferences and opinions about competitors' offerings, informing product development and marketing strategies.
What is Mistral 7B Instruct?
The Mistral 7B Instruct is a large language model developed by Mistral AI, a leading AI company based in Paris, France. It is a transformer-based model with 7 billion parameters, making it one of the larger language models available. The model is trained on a diverse dataset, including books, websites, and other text sources, enabling it to generate human-like text in English and French.
Technically, Mistral 7B Instruct uses a multi-turn conversation capability, allowing it to engage in extended dialogues and maintain context over multiple interactions. It also includes a "memory" feature that allows it to recall and use information from previous conversations.
Pros:
- Multi-turn conversation capability
- Large dataset for diverse knowledge base
- Ability to maintain context over multiple interactions
- "Memory" feature for recalling and using previous information
Cons:
- May generate incorrect or incomplete responses due to its size and complexity
- May require fine-tuning for specific use cases
- Limited to English and French languages
Ideal use cases:
- Customer service chatbots
- Language translation and localization
- Content generation for websites and blogs
- Personalized marketing and advertising
- Educational applications for language learning and tutoring.
How to perform Sentiment Analysis in Redshift with Mistral 7B Instruct using Python with the Amazon Bedrock SDK
Prerequisites for the boto3 Amazon Bedrock Python SDK
Start by installing the prerequisite libraries:
python3 -m pip install psycopg2-binary boto3
Afterwards load your source data into Redshift.
Python boto3 for Mistral 7B Instruct
The example below involves product reviews, and assumes that the source data has been loaded into a table named "stg_sample_reviews" with four columns: id (the primary key), stars, product and review.
The Python script is shown below. Note it is good practice to handle credentials more securely than shown in this simple example. You might choose to use a secret management service instead of environment variables or hardcoding.
Also please note that handling large amounts of data using fetchall() can be inefficient, and may result in memory issues. For large datasets, you should use a cursor to fetch rows incrementally using the fetchmany() method instead.
If you are working in a schema other than "public" you will need the -c connection option to specify the object search path. Set your RS_OPTIONS environment variable to "-c search_path=yourSchemaName,public" replacing the schema name with your own. The newly created table will be added to this named schema.
import os
import psycopg2
import re
import logging
import json
import boto3
import botocore
from botocore.exceptions import ClientError
logger = logging.getLogger("demo")
# Use the Amazon Bedrock Converse API
def analyze_sentiment(text):
abc = boto3.client(service_name="bedrock-runtime", region_name="us-east-1")
model_id = "mistral.mistral-7b-instruct-v0:2"
prompt = f"""Analyze the given text from an online review and determine the sentiment score. Return a single number between 1 and 5, with 1 being the most negative sentiment and 5 being the most positive sentiment. No further explanation or justification is required.
Text: {text}
Respond with a single number only. Do not include any notes, justification, explanation or confidence level, just the number.
"""
response = abc.converse(modelId = model_id,
messages = [{"role": "user", "content": [{"text": prompt}]}],
inferenceConfig = {"temperature": 0.3},
additionalModelRequestFields = {"top_k": 200}
)
return re.sub(r'[\s\.].*', '', response['output']['message']['content'][0]['text'].strip())
# Database connection parameters
db_params = {
'dbname': os.environ["RS_DBNAME"],
'user': os.environ["RS_USER"],
'password': os.environ["RS_PASSWORD"],
'host': os.environ["RS_HOST"],
'port': os.environ["RS_PORT"],
'options': os.environ["RS_OPTIONS"]
}
# Create a connection to the Redshift database
try:
conn = psycopg2.connect(**db_params)
except Exception as e:
print(f"Unable to connect to the database: {e}")
exit(1)
conn.autocommit = True
# Create a cursor object
cur = conn.cursor()
# Fetch all the rows from the source table
query = f'SELECT "id", "review" FROM "stg_sample_reviews"'
try:
# Create the table to hold the results, if it does not already exist
cur.execute('''CREATE TABLE IF NOT EXISTS "stg_sample_reviews_genai"
("id" INT NOT NULL, "ai_score" VARCHAR(1024) NOT NULL)''')
cur.execute(f'DELETE FROM "stg_sample_reviews_genai"')
# Execute the query
cur.execute(query)
# Fetch and process each row
for row in cur.fetchall():
ai_score = analyze_sentiment(row)
cur.execute(f'INSERT INTO "stg_sample_reviews_genai" ("id", "ai_score") VALUES ({row[0]}, {ai_score})')
except Exception as e:
print(f"SQL error: {e}")
finally:
cur.close()
conn.close()
After running the above script, you should find a new table has been created, which contains the AI-generated review score for every input record. Join this table to the original on the common id column to compare the AI-generated sentiment scores against the original star review.
The LLM was asked to score between 1 and 5, so you may choose to classify the scores more broadly as follows:
- 4 or 5 - Positive
- 3 - Neutral
- 1 or 2 - Negative
Sentiment Analysis in Redshift using Matillion to run Mistral 7B Instruct via Amazon Bedrock
In the Matillion Data Productivity Cloud, orchestration pipelines like the one shown in the screenshot below can:
- Directly extract and load data, or call other pipelines to do so (as shown)
- Invoke Mistral 7B Instruct, with a nominated prompt, against all rows from a nominated table
Sentiment Analysis in Redshift using Matillion
Data pipelines such as this manage all the connectivity and plumbing between the Redshift source and target tables, and the LLM.
This allows you to focus on the overall design and architecture, and the data analysis. To compare the AI-generated sentiment scores against the original star review, use a transformation pipeline like the one in the next screenshot.
Checking the results of Sentiment Analysis in Redshift using Matillion
The data sample shows two of the records. In one case the LLM's decision matches the original sentiment identically, but in the other record the ratings differ slightly. This is an example of the subjective nature of sentiment analysis.
Summary
Matillion is a data pipeline platform that allows data teams to build and manage pipelines efficiently for AI and analytics applications at scale. It offers a code-optional UI with pre-built components, as well as support for coding in SQL, Python, and DBT.
Matillion integrates with cloud providers, customer data platforms, large language models, and more. It enables collaboration through Git integration and provides AI-generated documentation.
Features include no-code connectors, custom REST API connectors, parameterization with variables, data lineage tracking, and pushdown ELT processing. Matillion also introduces AI capabilities like generative AI prompting, vector store connectivity, reverse ETL for insights, and a natural language pipeline builder.
For more examples of Matillion's AI components in action, check out our library of AI Videos and Demos.
To try Matillion yourself, using your own data, sign up for a free trial.
If you are already a Matillion user or trial customer, you can download the sentiment analysis example shown in the screenshots earlier, and run it on your own platform.
Featured Resources
Big Data London 2025: Key Takeaways and Maia Highlights
There’s no doubt about it – Maia dominated at Big Data London. Over the two-day event, word spread quickly about Maia’s ...
BlogSay Hello to Ask Matillion, Your New AI Assistant for Product Answers
We’re excited to introduce a powerful new addition to the Matillion experience: Ask Matillion.
BlogRethinking Data Pipeline Pricing
Discover how value-based data pipeline pricing improves ROI, controls costs, and scales data processing without billing surprises.
Share: