While generative AI tools such as ChatGPT have dominated the headlines in recent months, the reality is that AI has been present for a number of years.
There’s no denying that artificial Intelligence (AI) has become one of the fastest growing and largest areas of enterprise technology investment and innovation in recent years. Given there are so many practical applications for this technology, it’s no surprise that AI is supporting mainstream use cases, ranging from healthcare and life sciences to semiconductor and chip manufacturing, automotive, financial services, and beyond.
While generative AI tools such as ChatGPT have dominated the headlines in recent months, the reality is that AI has been present for a number of years. However, the latest wave of widely accessible generative AI tools is resulting in more machine generated data than ever before, and this is driving the unprecedented growth of unstructured data worldwide. In fact, IDC predicts that by 2025, the total amount of digital data created globally will rise to 175 zettabytes (from approximately 40 zettabytes in 2019). This estimate can actually be considered conservative, given the surge in AI-generated data we are seeing today.
Companies should also evaluate vendor purchasing options that can build in seamless capacity and technology upgrades for years ahead.
Patrick Smith, CTO EMEA, Pure Storage
In a somewhat perpetual cycle, greater volumes of data and the acceleration of AI means a bigger opportunity for businesses to turn this information into actionable intelligence, to innovate faster than their competitors, increase customer satisfaction, streamline operations, and ultimately become a more successful company. However, just as we refine oil into useful products such as fuel and plastics, data must also be refined before it can provide value. This is where data analytics (increasingly AI-based) comes in.
How can businesses succeed with AI projects?
In order to power AI, and AI-based data analytics, organisations need a flexible, reliable, performant, and perhaps most importantly, sustainable data storage infrastructure in place.
- Performance is key because AI relies on sending massive amounts of data into GPUs, over and over again. The faster organisations do that, the quicker and better results they get. AI resources (GPUs, data scientists) are expensive and in high-demand, so keeping them waiting on access to data can lead to a hefty bill. Just as important as feeding the GPUs, is accelerating the whole data preparation and curation workflows, helping to collect and process the data in the first place.
- Flexibility comes in as AI is easily the most rapidly evolving space in technology — tools, techniques, data-sets and use-cases are evolving every single day. As a result, it’s critical to invest in technology and infrastructure choices that are going to allow organisations to adapt to changes quickly.
- Enterprise reliability and controls are more important to organisations than ever with AI environments. These are mission critical environments, and any downtime can lead to exorbitant costs. As a result, availability and reliability are essential. Additionally, AI projects are often large sprawling projects and heavily automated. Having controls around quotas, security, and ease of management is critical.
- Last but certainly not least is one of the planet’s most pressing concerns, sustainability.
Why do businesses need to run AI sustainably?
Current estimates have data centres accounting for between 1-4% of all global energy consumption. In fact, in some countries data centre expansion has been halted because they cannot access adequate power. AI is not going anywhere, and overall, it will be an overwhelmingly positive tool for humanity, helping us automate repetitive tasks, treat diseases more effectively, and better understand our world through weather and climate patterns. However, from an environmental perspective, it only adds to energy consumption and carbon footprint concerns. In the wake of this immense challenge and opportunity, building an efficient and sustainable technology infrastructure for AI is critical to mitigating global warming and the worst impacts of climate change.
How can customers capitalise on AI in a sustainable way?
As data volumes grow and high performance becomes mainstream as a requirement for AI, sustainability concerns come to the fore. As these needs increase, so do costs in terms of power, cooling and the space to house equipment. In today’s context of soaring energy prices, this is not only an environmental issue, but an operational and financial challenge for businesses too.
Fortunately, some companies are designing and building products and delivering services that allow customers to dramatically decrease their own environmental footprints. For example, all-flash storage solutions are considerably more efficient than their spinning disk (HDD) counterparts. In some cases, all-flash solutions can help companies achieve up to 80% reduction in direct energy usage by data systems compared to competitive products. What’s more, flash storage is much better suited to running AI projects.
This is because the key to results is connecting AI models or AI powered applications to data. To do this successfully you need lots of data, this data can’t be cold, and crucially data needs to be easily accessible, across silos and applications. This simply isn’t possible with HDD based storage underpinning your operations, all-flash is needed.
To further bolster the adoption of sustainable technology choices, consider whether your organisation has a sustainability officer, someone responsible for the company’s overall carbon footprint. Involve those stakeholders at the beginning of the process to ensure no stone goes unturned on your journey to sustainable AI.
What does success look like?
Many companies are already applying these best practices to embark on their AI journeys. Meta, for example, wanted to help its AI researchers build new and better AI models that can learn from trillions of examples, work across hundreds of different languages, seamlessly analyse text, images, and video together, develop new augmented reality tools, and much more. As a result the company set out to create its AI Research SuperCluster (RSC), with the intention of being the world’s fastest AI supercomputer.
Meta chose Pure Storage as it needed a partner that could deliver robust and scalable storage capabilities to power RSC. With FlashArray and FlashBlade systems from Pure Storage, RSC has unparalleled performance to rapidly analyse both structured and unstructured data, underpinned by Pure Storage’s foundation of performance, reliability, flexibility and sustainability. RSC will pave the way toward building technologies for Meta’s next major computing platform, the metaverse, where AI-driven applications and products will play an important role.
How can you replicate this success?
To prepare for a world in which ever-growing amounts of unstructured data will be the subject of much-increased use of AI-based analytics, companies will need storage in colossal volumes that offers rapid access and is efficient in sustainability terms.
Businesses should look for vendors with a roadmap for high density flash storage capacity that can handle workloads from the most performance-hungry to those currently categorised as secondary, but which will gain in importance with the rise of constant AI processing. Companies should also evaluate vendor purchasing options that can build in seamless capacity and technology upgrades for years ahead.
Lastly, organisations should look for all-flash storage providers that can demonstrate third-party verified ESG metrics, so that AI projects can be executed without damaging the environment, and their bottom line.
