Chris Luther (Director of Product & Pre-Sales, Americas) works with Masstech end-users every week, helping to deploy storage systems that meet and exceed their needs. He’s picked up a few ideas on how the smart storage of the future should work.
I talk with many end users of media storage on a daily basis. A recurring conversation with many of them is that storage is not keeping up. That is, it’s keeping up in its capacity, or bandwidth to cope with bigger files, or varied storage locations; but it is failing to keep up in intelligence.
The first system that took advantage of Hierarchical Storage Management (HSM) was developed by IBM in 1974; essentially, this system moved files off disk to reel-to-reel tape, based on the date the files were last accessed. And, broadly speaking, this sure sounds like what all storage management systems still do to this day.
Before all the non-tapers rush in to decry HSM as a disk-to-tape only process, remember that HSM processes apply equally in the cloud storage environment – the movement of content between storage tiers, based on its attributes, still applies. And it could be argued that the defining triggers for HSM apply even more in the cloud, as monthly costs can mount at an alarming rate for content that has been incorrectly stored in a cloud tier that is inappropriate to its current or potential value.
So the question remains: what can be done to improve this? For some years now it has been claimed that more metadata would solve the problem; but how? Until very recently, metadata has only helped humans know more about files, but unless that knowledge is put to use in the decision-making around where content should reside, we are still left with a simplistic decision. Usually the value of a piece of content is defined by where and when it was last used, and content that may be far more appropriate for re-utilization is left unused because it is harder to find than more recently created or processed video.
The smart storage of the future will create, gather and intelligently analyse metadata. It will generate and collect metadata from all available sources, and store it tightly coupled to the asset. This smart storage will learn how these median associated files relate together, and provide suggestions of similar content that may be of interest. We’ve all seen widely used internet search enhancement such as “you may also be interested in”, or “frequently purchased together”. These suggestions and the users reaction to them will help the a storage management system learn what is valuable to each and every user.
Smart storage management systems will also need very detailed analytics in order to provide the answers to: How much does this cost to keep? How much does this cost to retrieve? Or how fast can I have it? Coupled with modern AI systems, this can also provide predictive modelling of what an organisation is likely to have to spend over the coming months and years.
Of course the considerations above – and many others – form a large part of the planning and design that helps us build the roadmap for our Kumulate platform. This feedback from the those users in the trenches that I work with every week helps us plan the next generation of intelligent, self-learning and self-healing storage systems.
I would love to hear your comments, opinions, and ideas on how we can continue to improve. Please drop me a note.