As I was bouncing around the country once again, I struck up a conversation with a complete stranger sitting next to me on the plane, which is my usual modus operandi.  Without knowing what industry I work in, he brought up the term "high performance computing" within the first minute of our friendly exchange.  Come to find out, the gentleman is a defense attorney to helicopter pilots involved in crashes. 

During the boarding process, he had his phone glued to his ear as he was engaged in a serious conversation with a couple of aeronautical engineers from Harvard. The engineers were conducting structural research on using multi-dimensional modeling techniques on super computers to help him build his case in determing why a helicopter recently crashed.  It became apparent to me that supercomputers continue to proliferate in our data-driven culture, and play a role in nearly every aspect of our everyday lives. 

Scientists, engineers and generally smart people continue to leverage the power of massive and distributed processers for calculation-intensive tasks such as quantum physics problems, weather forecasting, climateresearch, molecular modeling (computing the structures and properties of chemical compounds, biological macromolecules, polymers and crystals), and physical simulations (such as simulation of airplanes or helicopters in wind tunnels, simulation of the detonation of nuclear weapons, and research into nuclear fusion). 

You might be asking, what is the significance of all of this to me, to storage and to Spectra?  The way I see it, as supercomputers become more common, more and more data will continue to be created!  It also begs a few questions: Where does all that data go and how can it be preserved?  How can it be archived in a manner that makes it searchable and useable into the foreseeable future? As I ask that seemingly rhetorical question, I feel sort of like the famous Sweathog, Arnold Horshack, in Welcome Back, Kotter with my hand raised high in the air saying, “Ooh-ooh-ooooh, pick me Mister Kotter!"  Knowing what I know, I am ecstatic about the supercomputing revolution that we are experiencing because a large majority of the data generated, according to just about any of the more educated storage analysts you talk to, is going to be on tape.  And again, knowing what I know about Spectra and our track record for growth, profitability, and more importantly innovation over the past 32 years, our name is becoming synonymous with "enterprise" tape since we have the world's most scalable, and feature-rich tape system!  Even though I just revealed my age with the reference to Welcome Back, Kotter, I couldn't be more excited about the continued growth of the HPC market and the subsequent growth of the data explosion as a direct result of HPC.  If you can’t see the HPC market being a tremendous opportunity for continued tape growth because of the inherent characteristics of the most reliable, dense and economical media type, then "up your nose with a rubber hose!"  Of course, that is a line from my favorite Sweathog, Vinnie Barbarino!  Sorry if you are offended...

Gartner last month announced the results of an enterprise infrastructure survey conducted with over 1,000 large enterprises -http://www.gartner.com/it/page.jsp?id=1460213 – and they make interesting reading. According to respondents; data growth is the biggest data centre hardware infrastructure challenge for large enterprises. Now, this in itself is probably not surprising – vendors, end-users and other industry analysts have been talking about this challenge for some time. The inescapable truth is that storage demands are growing, and the answer lies somewhere between provisioning greater capacity and making more efficient use of the resources available. What is particularly striking is that 62% of respondents reported that they will be investing in data archiving or retirement by the end of 2011.

From Spectra Logic’s perspective it is particularly encouraging to see data archiving and retirement projects cited by respondents as the most popular response to the challenge of data growth.  Many of the conversations we had recently with end-users at SNW Europe centred around this theme. Backup is still important to customers – after all, disaster recovery will always be a key capability for IT and the wider business – but archiving is moving up the agenda (and rapidly so). Not only was archiving a hot topic of conversation on the show floor at SNW Europe, but our VP of Marketing & Product Management, Molly Rector  gave a very well received presentation entitled Active Archive: Data Protection for the Modern Data Center. Archiving is clearly making the transition from ‘nice to have’ to ‘business imperative’ – (Gartner will have other far cleverer terms for this I’m sure!)

While this is great news for Spectra Logic in terms of validating our position and viewpoint, it also points to a broader trend; customers are clearly beginning to look more closely at some kind of tiering strategy and/or data categorisation. Previously archiving and backup have often wrongly been lumped together under an all-encompassing tier sitting beneath production storage. I would hazard a guess that for a lot of end-user organisations ‘tiering’ has not got much more sophisticated than using disk for production / transactional data and tape for everything else. A number of technologies and drivers are forcing organisations to reassess this approach.

We can't overlook the rise of SSD (another hot topic at SNW), in this movement - it is becoming a viable option for enteprises, but current prices suggest that IT departments will have to carefully assess what data resides on that medium. This may be kicking off a trickle effect, which starts at the top and works its way down the storage hierarchy, with customers doing much closer mapping of data to storage medium and working out the best fit in terms of cost and performance.

Customers will also be looking at what data can be moved off disk altogether, and this is where archiving – specifically active archiving – comes into play. IT departments that investigate active archives will see that this approach is much less of a trade-off in terms of accessibility and performance when compared to disk than they may think. Customers will probably be shocked at just how much data they have sitting on disk which would be much more appropriately stored within an active archive setup. The data is still online and therefore still of value to the business, but on a much more cost-effective medium.

Everything points to a more sophisticated hierarchical approach to data management. Technologies like deduplication and thin-provisioning will play their part in facing up to the challenges caused by data growth, but ultimately a more radical shape-up of storage architectures is required, with active archives a new and very distinct layer.