The AI Efficiency Paradox

ChatGPT Workplace Challenges And An Alternative

Perry C. Douglas November 21, 2024

I recently read an article titled The AI Productivity Paradox: Why Aren’t More Workers Using ChatGPT? The Author Julia Winn says that: Despite the transformative potential of tools like ChatGPT, most knowledge workers I’ve spoken to don’t use it at all. Those who do primarily stick to basic tasks like summarization. Only a little over 5% of ChatGPT’s user base pays for plus — a small fraction of potential professional users — suggesting a scarcity of power users.

The assent of generative AI into the future of work conversation has been nothing less than spectacular; the technology conversation has shifted to everything AI and the revolutionary opportunities it’s supposed to bring to the enterprise. In April of 2023, Goldman Sachs predicted then a $7 trillion increase in global GDP attributed to generative AI and lift productivity growth by 1.5 percentage points over 10 years. But in June 2024, less than a year later, typical of Wall Street firms, GS changed its tune dramatically.

The evidence on the ground no longer supported the hype-driven grandiose predictions. Now, Goldman Sachs June 2024 Global Macro Research report Gen AI: Too much spend, too little benefit? — laid out the harsh reality of how many artificial intelligence efforts are not worth the time or the money for their expected returns.

Jim Covello, Head of Global Equity Research, states, “AI technology is exceptionally expensive, and to justify those costs, the technology must be able to solve complex problems, which it isn’t designed to do.”

Apple AI scientists published a paper showing even the most advanced Large Language Model (LLM) AIs lack basic reasoning. That trivial changes to prompts and complex problems reveal that “reasoning” by LLMs is unreliable. They miss a formal understanding of concepts needed for consistent, reliable generations.

[W]e investigate the fragility of mathematical reasoning in these models and demonstrate that their performance significantly deteriorates as the number of clauses in a question increases. We hypothesize that this decline is due to the fact that current LLMs are not capable of genuine logical reasoning; instead, they attempt to replicate the reasoning steps observed in their training data. 

— Apple Research Paper

Covello says that “truly life-changing inventions like the internet enabled low-cost solutions to disrupt high-cost solutions even in its infancy, unlike costly AI tech today.”

For OpenAI, costs don’t decrease with scaling, expenses outpace revenue. Common sense tells you that this is unsustainable. The herd mentality has always been problematic.

At the height of the dot-com bubble, the entire internet economy generated $1.5T in revenue (adjusted to 2024 dollars). Today, generative AI companies produce less than $10B in revenue while planning infrastructure investments that could exceed $1T. 

— Foundation Capital

So, despite the unprecedented launch of ChatGPT in the fall of 2022 and the continued excessive hype, the march to the “AI revolution,” about AI replacing humans and adding incredible efficiencies to the workplace, has been hit by a dose of reality.

With the rise of OpenAI’s ChatGPT, generative AI…large language models (LLMs) have been touted as potentially being more intelligent than humans one day, i.e., the fantasy of artificial general intelligence (AGI) — continues to be repeated like doctrines of a new religion.

The AI promoters have done a fantastic job of fooling people into believing the AI hype, and hundreds of billions continue to be spent on brute force adoption efforts and tactics to tell us what they think is good for us.

Nevertheless, common sense resistance remains despite those brute force efforts from the big tech aristocracy.

Most executives today are inundated with everything AI…buzzwords about deploying ChatGPT in the workplace say it will lead to great efficiencies and economic value creation for organizations overnight. However, this has not come to pass, and evidence shows that most organizations today aren’t even getting started with ChatGPT in the workplace.

This has been the stubborn reality for ChatGPT, for generative AI in general. Once you get past the basic fascinations of text generation, there is little else of really useful practical business application that can change the game. It is great for marketing copy and code generation, but the reality is that core things in business operations still take real and tactical domain knowledge and real people with experience. Generative AI has not proven more cost-effective and meaningful better than people at doing the work.

Workplace Adoption Challenges

Some obvious challenges for AI adoption are the LLMs that run AI; they cannot convey context, which is paramount for authentic communication and understanding in the physical world. Next, AI can’t ever be held accountable, so it is difficult for someone to trust their livelihood to something without accountability or responsibility. This is not often discussed about AI deployment in the workforce — the psychological human impact around job security and risk.

A Harvard Business Review (HBR) Change Management article titled Digital Transformation Is Not About Technology makes the important point that companies are pouring millions into digital transformation initiatives — but a high percentage of those fail to pay off. That’s because companies are putting the cart before the horse, focusing on a specific technology (“we need a machine-learning strategy!”) rather than doing the hard work of fitting the change first into the overall business strategy.

When it comes to employee adoption, the fear of being replaced comes to mind. “When employees perceive that digital transformation could threaten their jobs, they may consciously or unconsciously resist the changes.” If the digital transformation turns out ineffective, management will eventually abandon the effort, and their jobs will be saved (or so the thinking goes). Whether this thinking is right or wrong, it is critical for leaders to recognize those fears and to emphasize that technology selection or the digital transformation process is an opportunity for employees to upgrade their expertise to suit the marketplace of the future.

Nothing has changed about humans and technology automation and adaptation. “If you were one of those people who learned how to work with computers, you had a very good career. This is a similar turning point: as long as you embrace the technology, you will benefit from it,” says Andrew Blyton, VP and CIO of DuPont Water &Protection. The challenge for ChatGPT and others, however, is whether they can become as easy to use and useful enough as the computer.

The report points out that successful digital transformations seek to match the employees’ unique contributions to the organization and then connect those strengths to components of the digital transformation process — which they will then take charge of. The brute force approach of bringing technology and expecting employees to adapt usually fails.

In one of the case studies in the HBR article (the CenturyLink case study), the sales team had been considering adopting artificial intelligence to increase their productivity. Yet, selecting a specific AI solution and how it should be deployed remained an open question. Ultimately, the team customized an AI tool to optimize each salesperson’s effort by suggesting which customers to call and when to make those calls. The customized tool also contained a gamification component, which made the selling process more interesting. The observations showed that empowering people with more targeted or specific technology applications that are task-driven created employee buy-in and an entrepreneurship culture. It made selling more fun, which translated into increased customer satisfaction — and a 10% increase in sales, according to the HBR article.

The key takeaway here is that whether it’s efficiency gains or bold growth objectives, the organization’s leaders must first figure out the business strategy before investing in anything. ChatGPT in itself is not a transformative solution.

Informed leaders who aim to enhance organizational performance through digital technologies often have a specific tool in mind. “Our organization needs a machine learning strategy,” perhaps, but digital transformation should be guided by the broader business strategy. “There is no single technology that will deliver “speed” or “innovation” as such. The best combination of tools for a given organization will vary from one vision to another.”

Often, one of the mistakes organizations make is that they don’t "leverage their insiders," their employees, instead, they (those that can afford it) “frequently bring in an army of outside consultants who tend to apply one-size-fits-all solutions in the name of best practices.” However, what has been proven more effective “in transforming our respective organizations is to rely instead on insiders who have intimate knowledge about what works and what doesn’t in their daily operations.”

OpenAI’s ChatGPT model has demonstrated proficiency in answering questions and summarizing vast volumes of text, generating valid computer code, and even writing original prose and music. However, that’s not transformation or growth in any sense. Growth requires first improving customer satisfaction and intimacy, and then efforts must be preceded by a diagnostic and strategy phase to ensure the aspirations match real-world market realities. Writing crafty emails and getting nice summaries doesn’t get you there.

For example, Microsoft investing billions in OpenAI have forced it to develop a chatbot that, so far, has proven unable to get meaningfully sustainable market traction. Microsoft Copilot, the new chatbot “assistant,” will “fundamentally change the way we work,” says Microsoft, by summarizing meetings, creating PowerPoint presentations, drafting emails, and generating charts and graphs in Excel. However, Microsoft cautions users that the technology can still be “usefully wrong.” In other words, hallucinations. Things sounding plausible but incorrect or nonsensical answers are barriers against Copilot being adopted in the workplace.

Who needs that added stress? Workers have enough stress in their daily jobs, and owners want to avoid risk to their ongoing concerns. Reliability and trust remain fundamental when deciding whether or not to use a given product. ChatGPT has not achieved that crucial trust factor yet, and there is no credible evidence that it will.

Copilot is struggling to be adopted; you hear reports of CEOs moving away from it for one reason or another. Some comments are “it just adds more work,” “can’t really use it,” and “a fifth grader can do better presentations.” If people don’t trust the product, they won’t use it. So, it is hard to see how LLM-based products will sustainably penetrate the workplace market in any human-centric way. It is counterintuitive to ask people to create more work for themselves before getting the efficiencies the products claims to have. Creating more problems to solve is not the best productivity strategy.

Instead of training AI models to do everything, it might make more sense to train them to do one or a few key things exceptionally — specialization. However, LLMs-based products continue to cast the quintessential jack-of-all-trades and masters-of-none complex. So far, humans have been smart enough to know what’s good or not good for them. So unless ChatGPT can demonstrate authentic value beyond helping with emails and summarization, adoption will be elusive.

LLMs are data hogs — requiring more and more data, but it never seems enough to scale these models to perform at acceptable levels for business confidence. Further, that need for ever-increasing data makes these models incredibly expensive to run, putting them out of the reach of most enterprises — the majority of the businesses in the world are small and medium businesses (SMEs). Data needs to be cheap and accessible for them.

Furthermore, most of the data an organization might need can easily be found online through a simple Google search: scientific journals, research reports and articles, news and events, discoveries, blogs, etc., you can even get published reports from big consulting firms. Therefore, it is difficult for an intelligent manager or owner to pay for things they may be able to get for free.

The problem of explainability also arises. LLMs can’t explain themselves in context to the physical world because they don’t know the real world, just their machine language. People seek good explanations with references or a trail of sources of information; background and context are critical for authentic knowledge. Humans also learn through experience-based storytelling embedded in explanations, but AI can’t achieve such necessary nuances.

And if one fears machine hallucination, they will be forced to double-check constantly, this does not provide comfort nor build trust, and trust is the foundation for building good relationships, both humans and machines.

Finally, if you are in a competitive business environment or industry, you better think twice before authorizing any third-party language model in the workplace. If that data is breached and somehow makes its way into the public domain or the hands of competitors, you are the one responsible, not the LLM.

Competitive advantages are also critical in business; think for a minute: if everyone is using similar large language models, relatively speaking, everyone has access to the same information, and so too, are effectively employing the same strategies. You become part of the ignorant herd.

Realistic Real-world Application

Winn says, “Ultimately, the question executives need to ask isn’t “How can we use AI to do things faster? Or can this feature be built with AI? but rather, How can we use AI to create more value?” These questions are being answered by workers and owners daily, with the choices made about the technology they choose to use. So, if people decide not to use AI, it doesn’t matter what the AI promoters say; ChatGPT’s sustainable adoption objectives in the workplace just won’t get past the skeptical inertia of resistance.

“A thousand years of history and contemporary evidence make one thing clear: progress depends on the choices we make about technology. New ways of organizing production and communication can either serve the narrow interests of an elite or become the foundation for widespread prosperity.”

— Power and Progress: Our 1000-year Struggle Over Technology & Prosperity; Authors: MIT Professors Daron Acemoglu & Simon Johnson.

Professor Acemoglu estimates that only a quarter of AI-exposed tasks will be cost-effective to automate within the next 10 years, implying that AI will impact less than 5% of all tasks. And he doesn’t take much comfort from history that shows technologies improving and becoming less costly overtime, arguing that AI model advances likely won’t occur nearlyas quickly — or be nearly as impressive — as many believe. He forecasts AI will increase US productivity by only 0.5% and GDP growth by only 0.9% cumulatively over the next decade.

Every human invention should be celebrated, he says, and generative AI is a true human invention. However, “too much optimism and hype may lead to the premature use of technologies that are not yet ready for prime time. Creating bottlenecks and other problems for firms that no longer have the flexibility and trouble-shooting capabilities that human capital provides.”

“That’s why it’s important to resist the hype and take a somewhat cautious approach, which may include better regulatory tools, as AI technologies continue to evolve,” he concludes.

Winn says she recently “leveraged large language models (LLMs) — the technology behind tools like ChatGPT — to tackle a complex data structuring and analysis task that would have traditionally taken a cross-functional team of data analysts and content designers a month or more.” This is all nice, but how practical is it? Most SMEs don’t have the human resources personnel, financial resources, and capacity to do this. So, it doesn’t fit with business reality.

She provides another example:

Here’s what I accomplished in one day using Google AI Studio:

1. Transformed thousands of rows of unstructured data into a structured, labeled dataset.

2. Used the AI to identify key user groups within this newly structured data.

3. Based on these patterns, developed a new taxonomy that can power a better, more personalized end-user experience.

Does the average SME have the time, relatively skilled staff, and capacity to do similar? AI may sound easy to use, but there is still a learning curve there, no matter how small. Doing what Winn does requires investment in hiring and training, capacity building and organizational culture shifts. “I spent hours crafting precise prompts, providing feedback (like an intern, but with more direct language), and redirecting the AI when it veered off course,” she then added, “I was compressing a month’s worth of work into a day.” Realistically, some level of technical skill training is required that the average SME usually don’t have.

AI being adopted in the workplace hinges on employees having time and resources and managers and owners providing the leadership to support integration. Staff dedicating full days to understanding and working with AI is a luxury. Adaptation requires creating authentic demand.

Winn eventually admits to the AI paradox:

“Most people don’t have time to figure out how they can save time.”

With intensified workloads, multiple new mandates to fulfill, marketing and sales, operations, and customer service. The paradox is that chatbots might be useful, however, adding complexity and more work, more stress and anxieties, and perceived risk is something people often shy away from.

Winn says, “Chances are, your company already has a few AI enthusiasts — hidden gems who’ve taken the initiative to explore LLMs in their work. “LLM whisperers” could be anyone: an engineer, a marketer, a data scientist.” Then she says, “Once you’ve identified these internal experts, invite them to conduct one or two-hour-long “AI audits,” reviewing your team’s current workflows and identifying areas for improvement.” And create “starter prompts for specific use cases, share their AI workflows, and give tips on how to troubleshoot and evaluate going forward.” 

This is wishful thinking. It is highly unrealistic and unlikely for the vast majority of organizations in the world.

If LLMs really do present undeniable value, then adaptation can't be denied; free markets work well, and the workplace market will figure things out if they see the real competitive advantages to be had. The following is a helpful illustration of what AI has to do to solve for widespread adoption:

For AI to be long-term and sustainably adopted in the workplace, it must demonstrate its unique intrinsic value proposition, the sweet spot, hitting the core dimensions of value and ease of implementation. The red dots above indicate the relevant sweet spots that would make LLMs in the workplace more likely to be achieved over time — it remains a show-me game, however, not a hype game.

I found the following on Reddit, which I believe sums up the ChatGPT productivity paradox nicely.

I’m confused. I play around with ChatGPT to make little silly stories or what have you. It’s novel, entertaining, and interesting. But I genuinely cannot think of ways it could be useful for me. So tell me what I’m not seeing. I’m no longer in school, and using it for work is a non-starter due to what I do.

I feel comfortable expressing myself in writing, and I care too much about how I come across to send a letter or email written by AI without thoroughly checking it and fixing the writing style, so I feel like it wouldn’t save that much time.

And I feel like once I have researched something (say if I have to pull information from different sources), most of the work is done and putting it in writing is not such a big deal.

I could see how it could be helpful if I had to come up with some text, any text, on a subject I don’t know anything about, and I didn’t mind editing it for accuracy and style. In other words, using it to brainstorm. But this hasn’t come up yet in a genuinely useful way.

AI Must Adjust To Humanity to be Truly Adopted

Technology advancement is not about being superior to humans, what’s the sense in that? It’s about technology aligning with human progress, with humanity. Human intelligence is multidimensional; it involves our passions, bold aspirations, ideas and ambitions, suffering, morality, and happiness, forming our experiences, and our experiences develop our general intelligence.

Humans have a unique place in the physical world, and the technology we make must serve us best. We are intelligent because of our physical bodies — our biology separates us from machines — our five senses, sexual reproduction, DNA, natural selection capacity, etc., all represent natural human states and lived experiences. So authentic intelligence requires context, and context requires consciousness; AI can’t be conscious, so it can’t replicate human intelligence. So, instead of wasting good brain cells trying to use AI to duplicate and surpass human intelligence, we should focus on language models that can augment human intelligence optimally for human progress and prosperity.

Models that focus on authentic understanding and insight generation for thoughtful progression through criteria setting, research optimization, insight generation, and usefulness rankings to identify the most relevant information to a topic.

This requires proper models for learning: identify all the entities and structural data, the relationship between pairs for downstream decisioning and construct models of reality to test the idea against reality — separating the signals from the noise. Context is everything, so models must look for more evidence of real-world events to substantiate and objectively build ideas forward or eliminate them. Information is best understood in context, with all the other information around it — the insight engine is utilized to analyze any claim, fact, or assertion, identify any supporting evidence or contradictions, and return these to the user for optimal contextualization of information.

We must focus on the “Why” because you can’t figure out a strategy unless you can clearly state what winning looks like and, most importantly, what winning looks like in context to your situation or domain. Breaking things down into different mental and physical constructs allows us to create ways to identify the optimal playing field — strengthening our realistic assessment process and capacity for winning.

The FLMs Alternative

Therefore, focused language models (FLMs) use data science, advanced analytics, and specialized cognitive IP to test and confirm ideas to reality — a six-step process (IP) that assists users in discovering what they don’t know — new insights to build highly executable strategies.

FLMs are better, highly focused, reliable, practical, easier to manage and significantly less expensive than training large language models from scratch. “The largest LLMs are, by dint of their size, tainted by false information online. If you’re doing something in a more focused domain, you can avoid all the random junk and unwanted information from the web.” According to Matei Zaharia, cofounder and chief technology officer at Databricks and associate professor of computer science at the University of California, Berkeley.

“Companies cannot simply produce their own versions of these extremely large models. The scale and costs put this kind of computational work beyond the reach of all but the largest organizations: OpenAI reportedly used 10,000 GPUs to train ChatGPT. At the current moment, building large-scale models is an endeavour for only the best-resourced technology firms,” says MIT CIO Perspectives on generative AI. Smaller models, however, provide a viable alternative, the report continues to say.

And “I believe we’re going to move away from ‘I need half a trillion parameter models’ [to more focused ones because the reduction in complexity comes from narrowing the focus] from an all-purpose model that claims to know all knowledge.” To a more very high-quality focused one that is only concerned with your business.

“Thankfully, smaller does not mean weaker. Generative models have been fine-tuned for domains requiring less data, as evidenced through models like BERT—for biomedical content (BioBERT), legal content (Legal-BERT), and French text (the delightfully named CamemBERT),” says Carbin.

FLMs enable companies to cheaply build and customize their own tools and platforms to democratize access to generative AI for strategy development. Smaller open-source models, like Meta’s LLaMA, prove FLMs could rival the performance of large models and allow practitioners to innovate, share, and collaborate. One team built an LLM using the weights from LLaMA at a cost of less than $600, compared to the $100 million involved in training GPT-4.

“Much of this technology can be within the reach of many more organizations,” says Carbin. The FLM is the alternative solution because it’s not just the OpenAIs and the Googles and the Microsofts of the world that can play in this game.

It’s an easy-to-use/do-it-yourself solution that doesn’t require any special training or skills and is accessible to all at a fraction of the cost of hiring consultants, specialists, or advisors.

We want to empower business users to craft their own dashboards and drive their own insights from data to build the bold growth strategies of the 21st century! We want ideas about AI to start coming from the workforce, and that can’t happen if big tech LLM dominate. We want to generate the start of a more self-service and entrepreneurial era within organizations.

FLMs are engineered to leverage AI for human-centric applications; it doesn’t pursue developing AI-based solutions to be competitive, surpassing or replacing human intelligence. It takes an augmenting approach powered by applied intelligence | ai IP: a six-step (6ai) process which utilizes generative AI as an amplifying tool used practically, purposefully, and responsibly to augment human intelligence capacity, capabilities, and ingenuity.

The overriding philosophical approach of 6ai is rooted in the Socratic method of inquiry, the questioning approach popularized by the ancient Greek philosopher Socrates. This involves carefully crafted questions to probe the logical consistency of a participant’s beliefs and assumptions against reality, leading to a deeper understanding and meaning of the topic.

The process facilitates critical thinking and insight generation by probing, clarifying, and exploring. Challenging assumptions and beliefs while encouraging multiple perspectives for review to find the best insights for building a specific strategy.

6ai leverages the computational power of generative AI, purposefully and responsibly, to augment the open-ended inquiry approach with enormous speed, accuracy and relevant consistency. The ultimate goal is to apply methods, not to provide quick and not-so-thoughtful definitive answers but to guide the participant toward a deeper, more nuanced and robust understanding — redefining how strategy is crafted in the age of AI.

By focusing on challenging assumptions and exposing gaps, the 6ai process brings a higher dimensional level of information retrieval, augmentation and integration, supported by prioritization and contextualization, FLMs provide a stark contrast to the one-dimensional, quick-to-draw, closed-ended approach taken by hyped-up LLMs.

The applied intelligence process underpins FLMs and provides a disciplined strategy development framework for knowledge acquisition and learning to build strategies that can be effectively applied to real-world applications for enterprise transformations and growth.

6ai’s unique value proposition is straightforward: Every successful strategy must be underpinned by real, in-depth knowledge of its industry, sector, or domain. Accordingly, the six-step process identifies the whitespaces of market opportunity.

Our strategy canvas allows anyone to design anywhere from high-level multi-billion-dollar corporate strategies to personal growth strategies at an infinitesimal fraction of the cost. Without the need for specialized skills, technical tools, and expensive consultants who just tell you what you already know.