AskFinancials

• 

  Home page

• 

  Dashboard w/o insights

• 

  Q&A with 10K Report

• 

  Dashboard w/ insights

Inspiration

What inspired is the lack of equity in investing. Many individuals who come from a non-finance background would like to invest, but they do not have the time nor the financial expertise to read through a company's financial reports, which often exceed 100 pages. Our team relates strongly to this problem: we come from a computer background and do not have the time to conduct hours of financial research.

What it does

AskFinancials allows you to upload the 10K report of a company and uses the Gemini model to read through it. The platform then presents a beautiful dashboard that summarizes important information from the 10k and depicts real-time stock data from Yahoo Finance’s API. The second page of the website allows users to ask questions and chat with the 10K report of the company to solidify their understanding of the company.

How we built it

The front end is built through React, and our back end is supported by Express, and Flask. We use Gemini’s 1.5 API to summarize a company’s 10K report, and input the extracted information into Yahoo’s Finance API i.e. the Ticker symbol. This allows us to provide real-time financial metrics and utilize historical data to build a dynamic graph of the stock price for the past 20 days.

Challenges we ran into

The Yahoo Finance API is hosted on Flask for convenience, but it was difficult to convert into the JSON format that the Recharts module on Node uses. We returned a JSONify object through Flask, and then prompted Gemini to convert the JSON object into a format that works with Rechart. Essentially, we asked it to return a filtered JSON object that did not include undesired financial metrics that were returned by Yahoo Finance.

We also struggling with embeddings. Since we needed to convert our PDF to text to generate embeddings, we had to parse our PDF. Due to the format of our project, we weren’t able to work with LangChain’s PDFLoaders (apparently there are dependency issues when you try using WebPDFLoader with a Create React App project), which meant that we ended up having to use a completely separate compatible PDF parser to extract the text and generate embeddings from that.

This one was avoidable, yet still a valid challenge we faced. We were also using the wrong Gemini model for the longest time, which meant that generating embeddings for a 150-page document kept spitting out errors. The way we managed to work around it was to introduce chunking with overlap (similar to LangChain’s RecursiveCharacterTextSplitter). Until we figured out, about 11 hours before the deadline, that we could just switch our model to Gemini Pro 1.5 which had 1M tokens and could easily handle the PDF file without chunking.

Accomplishments that we're proud of

We are very proud of our application of Gemini, as we use its full extended token capabilities to read through 150+ pages of financial content and transform data formats. We are proud of creating a user-friendly dashboard and showcasing information in a concise yet informative manner.

What we learned

We learned a lot about leveraging multi-modal LLMs, which will surely continue to improve, in unique situations. Apart from that, we learned a lot about prompting, how tokens work, function-calling, working with embeddings, connecting APIs from different sources, etc.

What's next for AskFinancials

We want to fine-tune the model to return text that can be copy/pasted into Excel and to specialize in helping with financial modeling. Hopefully, we can manage to reduce the latency of queries by adding a database or fine-tuning the model to only return relevant information, Balancing the balance sheet, for example, is quite tedious and error-prone, so leveraging the LLM to help would be invaluable. Gemini has the capabilities: I added mistakes to a previously balanced balance sheet, and Gemini was able to identify the issues.

Built With

• express.js
• flask
• gemini
• node.js
• react
• yahoo-finance