deeptools Video 2: Using Cursor to build a market research script
Science & Technology
Introduction
In this video, I walk you through the process of using Cursor along with the Perplexity API to conduct market research. The focus is on creating a script that queries Perplexity for real-time information on various companies.
Setting Up the Environment
To kick things off, we begin by launching our coding environment and selecting some coding music to set the mood. Next, we clear out previous files and set up an empty requirements.txt file. Here, we plan to query the Perplexity API.
Following that, we choose to load the API key. Instead of extracting it from the environment variables, we decide to read it from a YAML configuration file structured with a root object as Keys and a sub-object perplexity that holds our API key as a string.
We then create a file named API Keys.yaml—taking care to add it to our .gitignore to prevent accidental exposure of sensitive information.
Debugging and Querying Perplexity
Upon running our Python script for the first time, we encounter a few errors, mainly due to a missing module for YAML handling. After we install the required packages, we set a breakpoint to debug the program. However, initially, we face a "file not found" error for our API Keys.yaml, which prompts us to adjust the file path configuration.
Once the file is located properly, we run the script and receive a 'bad request' error from the Perplexity API. After consulting the API documentation, we realize that we need to include the authorization bearer token in our request. TEDAPI details are examined thoroughly to ensure we select a valid model for querying.
After some tweaks and testing, we successfully make a valid query to the Perplexity API for a company’s market share data.
Utilizing OpenAI for Query Generation
As the next step, we integrate OpenAI to generate a list of queries based on the input company name, allowing for a more comprehensive market research routine. Essential to this is reading the API key from the YAML file.
We run the script with sample inputs and gather the output. Our queries get generated successfully, and we aim to save these findings into a CSV format for easy output management.
Transitioning to Markdown Output
Noticing that the data output is formatted in Markdown style, we adjust our code to save the results in Markdown rather than CSV. By refining our original code, we manage to output the results in a user-friendly format.
We also implement code to include the current date in our queries, ensuring that we're pulling the most recent data regarding the companies.
Experimenting with Additional Companies
With our script working well for Tesla, we attempt to run it for another company, X (formerly Twitter). Despite not getting comprehensive results, we realize that additional prompts may refine our output further.
To summarize, this entire project demonstrates how to efficiently use Cursor in conjunction with the OpenAI API and the Perplexity API to generate real-time market research insight. With the right configurations, it could potentially be automated for weekly updates on market trends and news regarding specific companies.
Conclusion
In this video, we successfully navigate through integrating various APIs while building a real-time market research script. Future tasks include enhancing our queries, splitting the codebase into modules, and potentially hosting it on a web server for broader accessibility.
Keywords
- Cursor
- Perplexity API
- OpenAI API
- Market research
- YAML configuration
- Python script
- API keys
- Documentation
- Markdown output
- Automation
FAQ
Q: In what programming environment was the script developed?
A: The script was developed using Cursor, which assists in coding and debugging in Python.
Q: How does the script handle sensitive information like API keys?
A: API keys are stored in a YAML file that is added to the .gitignore to prevent it from being uploaded to version control.
Q: What was the main goal of the script?
A: The main goal was to automate the process of retrieving market research data about specific companies using the Perplexity API.
Q: Can the queries generated by OpenAI be refined?
A: Yes, the queries can be adjusted and refined based on feedback and results for better accuracy.
Q: What formats can the output be saved in?
A: Initially the output was designed to be saved in CSV format, but adjustments were made to output in Markdown format due to its compatibility with the data received from the Perplexity API.