By Tom Cranstoun
Tom Cranstoun is an established expert on Adobe Experience Manager and widely recognised as The AEM Guy
I recently attended the annual CMS Experts meeting in New York, where artificial intelligence (AI) emerged as a recurring theme despite not being the event's primary focus. Influential speakers such as Sree Sreenivasan and Alan Pelz-Sharpe offered insightful viewpoints that inspired me to share my thoughts on this technology.
Sree’s talk was provocatively and timely titled: “We have no idea what we are doing with AI, but we are clearly going to do it.”. He argued that using the term "hallucinations" to describe issues with AI technology lets developers off the hook for addressing these problems. Sreenivasan also points out that in the AI industry, users are both the product and the lab, creating a more dangerous situation than social media.
Alan’s talk was “Is your Organization ready for AI?” and the brief answer is no. He explained that organisations must identify the processes that they think AI will improve, the tasks that will power those processes, have the inputs have been quality controlled, does the organisation which people and skills are required to build and operate the solution, and have decision-makers agreed on the metrics for success.
AI has become a buzzword, promising dramatic advances in various areas, including healthcare and transportation. However, it is crucial to maintain a balanced perspective and be aware of AI's advantages and disadvantages.
The AI Bubble: Déjà Vu?
The current AI investment frenzy resembles historical economic bubbles like the Dutch Tulip Mania and the dot-com bubble.
The hype surrounding AI has led to inflated expectations and valuations, with companies racing to rebrand their products as "AI-powered" and investors pouring billions of dollars into the sector.
However, as history has shown, these bubbles often end dramatically, with disappointment.
Recognising AI's Limitations
Despite the remarkable progress made in recent years, AI is still in its early stages and faces numerous challenges. AI systems can be complex, biased, and error-prone, lacking the contextual awareness that comes naturally to humans. Additionally, AI relies heavily on diverse and often unknown data, which can lead to biased results if the data is limited or skewed. One significant concern is the anthropomorphization of AI, where the technology's "hallucinations"—fabricated responses in the absence of data—are mistaken for human-like reasoning.
Choosing the Proper AI Model:
The process of testing an AI machine-learning model involves several key steps:
Define the Model's Goals:
Clearly articulate the objectives and intended use cases for the model.
Specify the desired performance metrics and the level of accuracy required.
Compare Different Models:
Research and identify various modelling techniques and algorithms that are suitable for the problem at hand.
Conduct a review to understand the strengths and weaknesses of each model.
Perform a comparative analysis based on accuracy, interpretability, computational complexity, and scalability.
Compile a Comprehensive Test Dataset:
Collect a diverse and representative dataset that covers a wide range of scenarios and conditions.
Ensure that the dataset is large enough to provide statistically significant results.
Address data imbalance issues, if present, to avoid biased evaluations.
Assess the Model's Performance Using Relevant Metrics:
Select appropriate performance metrics that align with the model's objectives.
Calculate metrics such as accuracy and precision.
Understand the trade-offs between different metrics and prioritize the most relevant to the problem.
Examine the Model's Explainability and Interpretability:
Evaluate the model's ability to provide explanations for its predictions.
Assess the model's interpretability by non-experts and consider the need for additional visualizations or explanations.
Consider Ethical Implications:
Identify potential biases or discriminatory outcomes that may arise from the model's predictions.
Ensure compliance with relevant regulations and guidelines regarding the use of AI in specific domains.
Continuously Monitor and Update the Test Data:
Regularly monitor the model's performance on the test dataset to detect any degradation or changes in accuracy.
Update the test dataset over time to reflect real-world changes and evolving conditions.
Consider deploying a continuous integration/continuous testing (CI/CT) pipeline to automate the testing and monitoring.
The Dynamic AI Landscape
The AI industry is fueled by intense competition and innovation. To remain informed and make responsible decisions regarding AI implementation, it is essential to continuously engage with and navigate this complex subject while staying aware of both the promise and pitfalls of AI.
As AI becomes more ubiquitous, addressing the risks and responsibilities associated with its development and deployment is crucial. The potential for AI to cause harm, whether through job displacement, algorithmic bias, or misuse by bad actors, cannot be ignored. AI companies and developers must be held accountable for the issues arising from their products, and using terms like "hallucinations" should not absolve them of their responsibility to address these problems.
Effective regulation and ethical guidelines are essential to ensure that AI is developed and deployed responsibly. The European Union has taken the lead in this regard, but much more needs to be done globally.
In the United States, appointing chief AI officers in every federal agency by the end of 2024 is a step in the right direction, but it is only the beginning. The AI industry must prioritize transparency, accountability, and the development of AI systems that align with human values and societal well-being.
Look beyond the AI buzz
Navigating the AI hype requires a balanced and well-informed perspective.
While AI's potential is undeniable, we must approach it with skepticism and realism. By understanding AI's limitations, selecting the suitable models, staying informed about the latest developments, and prioritizing responsible and ethical AI practices, we can harness the power of this transformative technology while mitigating its potential drawbacks.
We can ensure that AI truly benefits humanity through collaboration, regulation, and a commitment to the greater good.
Learn more about the state of AI right now
Getting back to our two guest speakers in New York City, Alan and Sree, here’s more from their side:
Alan Pelz-Sharpe: AI Readiness - “Ready for what, exactly?”
Sree Sreenivasan: Sree's Non-Scary Guide to AI