ChatGPT falls short of replacing human financial data analysts - The limitations of AI
Despite the growing adoption of Artificial Intelligence (AI), it is still far from replacing the average financial data analyst. According to McKinsey’s The State of AI in 2022 report, up to 60% of organisations use AI in at least one business area. However, only 20% of them, including FinTechs, use AI technologies in a core business process or at scale. High costs and low precision are the main reasons wider adoption of AI is still lacking.
ChatGPT, the OpenAI chatbot, is limited in its purpose and design even after the introduction of GPT-4 architecture. ChatGPT can summarise large amounts of textual information and offer generalised insights or examples which might be helpful for data professionals. This includes advising on KPIs, solving common coding issues, and writing SQL codes or mathematical formulas. However, the chatbot does not take into account changing circumstances that surround a particular company or financial data it is asked to process because it has a limited context window.
Julius Černiauskas, the CEO of Oxylabs, said “GPT-4 has its merits, being a generative AI model learning from specific data, building on it and offering new content, but it is not generic AI. Based on this architecture, ChatGPT mainly processes textual and, to some extent, visual information, delivering textual outputs. However, one can’t upload an Excel with thousands or millions of financial data points to ChatGPT and expect it to analyse the information. It cannot collect data directly or interact with company dashboards or data systems and is not designed for accurate and comprehensive business data analysis”.
According to Černiauskas, almost every AI system today is built on Machine Learning (ML) technology, and the main limitation of any ML model is its complete dependency on the training data. ChatGPT doesn’t process real-time data from the internet, functioning on a massive but limited dataset that must be constantly updated. As such, it can miss new data or not process it well and suffer from biases and human errors. The latest model can still suffer from hallucinating facts and does not learn from experience, as per the OpenAI technical report.
Chatting with ChatGPT might be absorbingly real, but so are the limitations of the virtual brains and their potential to fully take over data collection and analytics. This might change, but current generic and generative AI models have very low precision in narrow use cases. Financial organisations may solve the problem by using specific techniques, but they can be incredibly data-greedy, with organisations rarely having enough datasets to achieve near-human cognition and accuracy.
