Collaborate or Compete - How AI Weaknesses Enhance Human Roles in Tech
How to embrace the unmatched human touch in development

by Anthony Capone, Rahul Bagga, and Patrick Hanck

Artificial Intelligence (AI) tools have opened new frontiers of efficiency and innovation in the rapidly evolving field of software development. As a result, there is a lot of fear that AI technologies will replace humans in many industries, particularly technical and development jobs. McKinsey states that “by 2030, activities that account for up to 30 percent of hours currently worked across the US economy could be automated—a trend accelerated by generative AI (1).” The hype may be valid in the long term, but today’s development teams should not worry about their jobs, and leaders should not look to squeeze headcount by replacing humans with AI. Understanding AI’s current capabilities and weaknesses is crucial before deciding what roles it can effectively play on successful teams.
Human developers are essential for context, creativity, and ethical judgment; areas where AI falls short.
So far, AI's contribution to software development is primarily as a tool developers use to enhance their productivity. Tools like Github Copilot in Software Development assist in complex problem-solving by providing suggestions and insights, but the final judgment and decision-making rest with human developers. AI is incapable of good quality independent development, human intervention is necessary for essential coding tasks like comprehensive debugging and future-proofing (2).
AI falls short in code clarity where human experience shines
Research indicates that developers spend approximately 70% of their time comprehending existing code, while actual coding takes up only about 5% (3). In the realm of software development, understanding and modifying code is often more critical than creating it from scratch. Therefore, the clarity and readability of code is paramount.

AI-generated code, while efficient in solving problems and providing explanatory comments, tends towards verbosity and often lacks the nuanced understanding of human developers. According to a study by GitClear, there is unnecessary code being generated by AI and used by developers. Developers using tools like Copilot are “being inundated with suggestions for added code, but never suggestions for updating, moving, or deleting code (4).” This study explains how “code suggestion is not optimized by the same incentives as code maintainers.” All the extra and ill-understood code culminates in a future headache for the code maintenance team and time wasted overall.
AI has limited contextual and strategic understanding of real-world business scenarios
AI cannot fully comprehend the business and functional contexts around a piece of code; training datasets do not contain exhaustive detail or nuance. What constitutes a successful transaction, a priority customer, or a market opportunity can vary drastically between industries and individual organizations. AI, in its current form, cannot distinguish these subtle differences.
“This is because, while neural networks are efficient at interpolating between data points they’ve seen during training, they’re terrible at dealing with situations not covered by their training data. Humans, on the other hand, are good at extrapolating their knowledge and experience to previously unseen situations because they build abstract representations (5).”
Every business operates within a unique ecosystem characterized by its own set of guidelines, processes, customer relationships, and market dynamics. These nuances are often deeply ingrained in the company's culture and operational procedures. Understanding them requires more than technical skills. Contextual awareness and business acumen are required.
For example, integrating AI tools with existing legacy systems is a common challenge in software development. These systems are often deeply ingrained in an organization’s operational history and intertwined in business processes. Customizing AI solutions to work harmoniously with these systems requires more than technical expertise; it requires a deep understanding of the existing architecture, business logic, and processes it’s designed to support. Full understanding is impossible for AI but is second nature to good developers and development teams.
Creativity is an irreplaceable human element
AI cannot replicate the unique and complex nature of the human creative process. Creativity often involves subjective and personal elements based on experience, emotion, and individual perspective. Only a human can be in the right place, physically or mentally, to explore unconventional paths or find chance discoveries.
“AI cannot support all skills essential for idea development. Various studies have shown that inspiring peers at conferences, learning new skills or simply unwinding, reflecting and daydreaming are activities associated with idea generation. AI cannot replicate such real-life experiences or personal interactions (6).”
The premise that generative AI creates new content or ideas is misleading. In reality, the models take data they have been trained on and generate derivative content. A new mix of old content to create something else. This something else may fit the requested prompt, but often, it creates something weird.
Human interaction is required to root out this weirdness and validate the AI generation is correct and appropriate for what was requested. AI can assist and augment the human creative process but cannot replace our ability to judge relevance.
“Looking forward, the most successful ideas likely won’t come from bright thinkers alone but from those best at mindfully steering intelligent machines while remaining firmly in the driver’s seat (6).”
AI is prone to hidden bias
All AI models inherit the biases present in the dataset on which they are trained. Much human effort has gone into correcting this problem in commercially available LLMs like ChatGPT. Without intervention, biased training will perpetuate itself in AI output. For example, biased data in hiring or lending practices could result in discriminatory AI decisions.
"AI systems are only as good as the data we put into them. Unfortunately, if this data reflects historical biases and prejudices, AI can exacerbate them, or even make them appear to be scientifically justified (7).”
What is less obvious is that AI bias is usually entirely hidden from the user. AI tools produce results that may look objective but are biased in one way or another. It is easier to infer, understand, or account for the biases of a human team member.

Have AI tools in your toolbox and know when to use them
Despite these weaknesses, if your development team is not using AI tools, they are not as efficient as possible. Tasks like automated document generation, natural language processing, automated testing, and code refactoring can be handled effectively by AI tools when supported by minimal human interaction. The time savings associated with use of these tools is meaningful, and it allows developers to focus on more complex tasks that aren’t appropriate for AI, like highly creative or artistic projects, solving problems in dynamic and unpredictable environments, or making decisions based on ethical considerations. As with many tools, understanding AI’s strengths and weaknesses helps ensure its impacts are additive, not destructive.

Once you’ve implemented AI tools, keep your ability to use them effectively sharp. Stay up-to-date on the latest AI advancements, and follow the AI industry news and blogs. Participate in online communities on platforms like Reddit, Stack Overflow, etc. where people discuss the latest AI advancements and tools. A plethora of professional organizations, conferences, and workshops are available to stay informed about the most recent AI releases.
AI is a useful tool that enhances human developers
The integration of AI into software development is a complement, not a replacement, for human developers. While AI tools bring automation and assistive capabilities, they fall short in inherently human areas like understanding context, customizing for specific business needs, and offering creative solutions.
As we navigate this evolving landscape, successful leaders will see AI as a tool that enhances human capabilities rather than a technology that renders human skills obsolete. The future of software development is a collaboration between AI and human intelligence, leveraging the strengths of each to drive innovation and progress.
"I think the future of global competition is about creative talent... Everyone will have access to amazing AI... Your creative talent though — that will be who you are... Invest in turning your talent into a team of explorers who can solve amazing problems using AI as the tool… that is the company that wins in the end (8)."
(2) Cloud Now Tech
(3) University of Lugano, Switzerland
(4) GitClear
(5) TechTalks
(6) World Economic Forum
(8) Salesforce