Eylem Oruc, Senior Director, Unstructured Data Solutions (UDS), META, Dell Technologies, discusses the practicalities and potential Generative AI can bring to businesses in this exclusive op-ed.
Artificial Intelligence (AI) and Generative AI (GenAI) are some of the hottest technologies of the year and organisations are racing to embrace the benefits. According to McKinsey, GenAI could add an estimated $2.6-$4.4 trillion to the global economy annually.
However, GenAI comes with (data) baggage. To build and train GenAI models, organisations need vast amounts of information. In turn, these same models also generate reams of data back into the business. So, the question each business leader must ask before embracing AI and GenAI is: are our storage solutions up to the task?
In 2024 and beyond, it is a scalable, secure, and economically sound data architecture that will set apart the organisations simply running in the AI race and those leading it.
Storage solutions for the GenAI age
For GenAI to be successfully deployed, organisations must rethink, rearchitect and optimise their storage to effectively manage GenAI’s hefty data management requirements. By doing so, organisations will avoid a potential slowdown in processes due to inadequate or improperly designed storage.
The reality is that traditional storage systems are already struggling to keep pace with the explosion of data, and as GenAI systems advance and tackle new, more complex tasks the requirements will only increase. In other words, storage platforms must be aligned with the more complex realities of unstructured data, also known as qualitative data, and the emerging needs of GenAI.
In fact, unstructured data accounts for over 90% of the data created each year – largely due to a rise in human generated data, meaning the sphere is made up of cluttered and muddled columns of analysis. Enterprises need new ways to cost-effectively store data of this scale and complexity, while still providing easy and quick access to it and protecting it against cyber criminals. Unstructured data specifically is of interest to hackers, due to its value and sheer volume.
Put simply, organisations want and expect better data movement, access, scalability, and protection. As a quick fix, many have turned to cloud-first strategies, where data is stored across multiple public cloud environments. While this provides a potential solution in the short term, in the long run organisations will be faced with escalating ingress and egress costs, security concerns and data optimisation challenges. For GenAI to truly take effect, it needs simple, easy access to data – something a cloud first strategy will struggle to provide.
Organisations should instead look to adopt a multicloud by design approach. This will help them unlock the full potential of multicloud in the short and long-term, without being constrained by siloed ecosystems of proprietary tools and services. Multicloud by design brings management consistency to storing, protecting and securing data in multicloud environments.
Investing in new storage technologies
Businesses need new, novel approaches that cater to GenAI’s specific requirements and vast, diverse data sets. Some of these cutting-edge technologies include distributed storage, data compression and data indexing.
- Distributed storage enhances the scalability and reliability of GenAI systems by housing data across multiple locations. For example, organisations can rapidly scale their storage needs across several nodes, should demand increase, as well as replicate their most critical data, allowing it to be vaulted in a separate location and easily retrieved in the event of a cyber-attack.
- Another key concern facing many organisations is cost. However, this can be addressed in part through data compression. By removing unwanted data through data compression methods, organisations can reduce their storage needs. This is achieved by more effectively analysing data and removing unnecessary information to achieve a more summarised version. This in turn reduces the amount of storage required by the organisation and consequently, saves on costs.
- Data Indexing on the other hand improves retrieval capabilities, and contributes to faster, more efficient search capabilities and training by more effectively organising the data into specific locations.
Together, these three technologies enhance performance, efficiency, and cost-savings. Three of the key priorities for business leaders looking for a painless transition to GenAI technologies.
It’s tempting to skip ahead to introducing and driving effective training and modelling, but to be successful, GenAI requires a solid storage foundation as a first step. It might not be the most exciting topic for business leaders, but the way organisations store and manage data will drive greater business value in the future.
AI and GenAI are significant enablers of competitive advantage and a way to disrupt markets. However, they must be deployed correctly – don’t jump into the AI race blind without warming up, make sure you’re in the best possible condition. There is tremendous opportunity ahead and those that do so with future-proof technology will be most competitively placed to capitalise on the benefits.
