I recently had the honour of getting on stage at the Storage Awards 2010 to collect the gong for Green Storage Product of the Year, awarded to the T-Finity. The award win wasn’t the first for Spectra Logic over the last few months, but I’m not here to crow about our success (well, maybe just a little).

Instead, the award made me think about the whole Green IT movement and where we currently stand in the UK. The hype around Green IT has far outweighed the traction – in part because Green has been leapt upon by marketeers and the buzz-word brigade, but also because the ‘movement’ pretty much met the global economic troubles head-on as soon as it started to gain any momentum.

For vendors that started off down the Green IT messaging path there was a lot of backtracking as they focused their efforts on promoting cost-savings and TCO benefits in line with customers’ priorities. Now that economic recovery seems to be slowly creeping into view, there may be a lot of vendors coming full-circle, feeling that customers may be ready to move CSR up the IT agenda.

This posturing and positioning is what faces vendors trying to pass off any technology as IT’s answer to Captain Planet in a desperate attempt to gain a competitive edge. I’m referring to technologies such as vast disk-arrays that eat up moderately less power than a competing product while spinning away idly in the background. There is nothing Green about this – it is just a matter of being slightly less inefficient. Organisations mandated to be green will see through this spin, and instead hone in on genuinely Green technologies.

From Spectra Logic’s point of view, we like to think we can talk trees or £’s to customers – however you look at it we’ll save you both.  When we started designing the T-Finity did we set out to make the Greenest storage product on the market? No. But by creating the most efficient, scalable and dense tape library on the market we created an extremely green product by default.

So it is perhaps apt that commentators are referring to the green shoots of recovery – economic growth, after a period of such severe contraction, may well lead to a shot in the arm for Green IT. For our part, you won’t notice any difference in how we position our products – if you are a business that needs to Green your approach to IT – be it for legal or less self-serving reasons - then tape is the obvious choice for backup and archive.    

The high importance of data protection is top of mind these days – specifically in light of some high profile cases of data loss in the UK. News of some potentially impending legislation this side of the pond has again drawn attention to the issue of how companies look after customer data.

The Active Archive Alliance formation was just announced last week. I wanted to take the opportunity to tell you how the concept originated, evolved and solidified, and why Spectra Logic is participating as a founding member.

Part One or Two

One of the interesting things I have discovered since I have been talking with so many HPC customers is that the term “backup” is seldom used.  You might ask if they aren’t doing traditional backups, then why would we, a backup solutions provider,  want to talk to them. Well, first you need to fully understand the difference between backup and archive.  Archive is a word you will hear more often in the HPC and M&E environments, especially if there is data in excess of the petabyte range and large files that aren’t accessed frequently but need to be kept indefinitely. 

In this blog, which is the first of a two part series, I will provide some fundamental information that can help you differentiate backup from archive.  In the subsequent blog, part two, we will peel the covers back on the process that is different from backup and archive and similar to the traditional HSM (Hierarchical Storage Management). This information will prove to be valuable for those HPC or other data intensive customers who may claim that they don’t do backups.  Stay tuned for more on this subject later.

The differences between backup and archive:

Backup: simply refers to the creation of a copy of data and storing it somewhere for restoration in the event the original version of the data was compromised in some way.  We evangelize the concept of backups because we know, and most customers realize, that data can accidentally be deleted, corruption could occur, data loss, or even worse, a natural disaster could wipe out the entire data center.

Backup is simply safeguarding or protecting the data that is being used by duplicating that data.  This is usually done in a rotating cycle or through schedules including: daily incremental which are kept for seven days, a weekly full kept for a month, a monthly full kept for a year and a yearly full kept for seven years.  Although this process has proven effective and most of the backup applications on the market today are ideal for doing this, problems occur when you start having multiple copies of the same data consuming a lot more hardware than necessary, not to mention the associated costs of running and managing that hardware. 

With backup – think business continuity

One of the key differences when comparing backup strategies to  archiving, is the difficulty of singling out select files for long term retention.  Everything in the backup gets lumped into the large full backup at the end of the year or seven years and called an “archive”.  It may in fact be called an archive but a recovery would function more like a backup recovery, which could be very costly and time consuming.  Backup strategies are more for business continuity purposes and not necessary for long term archiving.

With archive – think long-term retention

Archive: The main difference between an archive and a backup is that an archive refers to a single collection of records or data that is designated for long-term retention.  When the data is moved from the production environment to the archive environment it is tagged or indexed by metadata that assists in quickly locating that particular file or chunk of data through a search mechanism.  This process and the sophisticated software that performs it make locating a single file much more efficient than it would be in a traditional backup.  An archive is generally found in a common file system structure and the determination of where the file is located is a function of file system.  The file system may have several different storage devices that the archived data is stored on based on a number of attributes such as size, type, last accessed, etc.  This system could be a combination of expensive disk, such as fiber channel, less expensive disk, such as SATA or SAS and tape.  The key is how the data is “structured.”  In most cases, the data may never be accessed again, but it is necessary to keep it for historical purposes, regulatory compliance or unplanned event.  The goal with creating an archive is to keep it separate from the backup rotation cycle.  It is recommended that a separate copy of the archived data be made and kept in a separate location so there are at least two copies of the final archive.

Many environments will include both backup and archive.  Through the use of sophisticated software features that are available today, customers can establish policies that determine type, size, age, last accessed, remaining disk space and other characteristics of stored data that can automate the process of deciding whether to keep the data in the backup cycle or move it to the archive pool.

These two functions can be performed within a single library in separate partitions.  The software can then provide notification of what tapes need to be exported based on the function that was performed on those tapes, backup or archive.  I have seen numbers as high as 80% indicating how much data is duplicated within a storage infrastructure because the differences between backup and archive aren’t fully understood.  At the end of the day, knowing the difference and the benefits of backup and archive technologies, when to use them and how to balance the the two functions in an environment can drastically reduce the amount of redundancy, complexity and storage operating costs.

In Part Two of this discussion, we will look at how archives that contain production data, no matter how old or infrequently accessed, can still be retrieved online using high density and high speed tape systems and secondary disk systems.  Stay tuned for my next post which will look at enduring access to data. 

Want to talk more? I’ll be in Dearborn Michigan at the IDC HPC User Forum and DICE Alliance 2010 events next week. Contact me at jimm@spectralogic.com.

Dear Ms. Meade:
My data crunching company generates 2-3 TB of data per customer, and I need to store that somehow. However, I don’t have room for a tape library. The only thing I can think to do is put the data on some hard drives using Linux-based RAID software, then put the disk in a safety deposit box. Do you have any other suggestions?”
Sincerely,
Short on Space

Dear Ms. Space:
You have money and room for terabytes of disk storage, which you will squirrel away in a pretty large safety deposit box, but not a couple of dollars and rack units for a small library? Hmmm.

In pondering a polite answer to this question, Ms. Meade called to mind something similar posted on Slashdot, and is heartened that several intelligent points were discussed in that context. Being one to always encourage others in the path of light, Ms. Meade will summarize these intelligent comments and add to them.

The Short Version, by the way, in case you are averse to reading: Buy an LTO-4 tape drive and LTO-4 tapes. Forget the disk.

The Long Version: As fond as Ms. Meade is of disk, especially Spectra nTier disk, Ms. Meade understands that disk’s greatest asset is the speed at which it retrieves data—NOT its use for secure offline data storage.

Tape is cheaper than disk, even the disk to which you are likely referring. The Tape Equation: you can buy an LTO-4 tape drive for around $1400 (and likely for less), and at $40 per tape, store 800 GB of data; with these and a little compression, you are two tapes away from serious, long-term storage.  Assuming that you have more than a handful of customers annually, this pays for itself pretty rapidly, compared to purchasing cheap (and risky) jbod.

At $100/TB per hard drive and twenty customers each with 2 TB of compressed data, annually the company must shell out $4,000 per year. If, instead, the firm purchases a tape drive and LTO media, your costs are under that in just the first year-- about $2,000 for tape, and another $1,400 for the drive. You’ve paid for the drive in one year. After that, you save $60/TB. (That translates to thousands of dollars annually.)

You may want to consider a tape library, which truly are not space- or budget-hogs. Libraries such as the 4U Spectra T50e may be worth the space and time simply in convenience. This depends a great deal on your business volumes and staffing, and Ms. Meade acknowledges constraints due to current recessionary times. However, to emphasize the point: a relatively lightweight investment such as the purchase of a small library can automate data protection—and most companies that deal in data understand that their business also mandates data-caretaking.

For those unenlightened few who say that LTO tape is not a wise choice because eventually new technologies replace older ones, please consider that LTO has been around this past decade and shows no sign of going away. No migration will be necessary for years to come, given that current generations of LTO tape technology read data on tape that is two generations back, and write one generation back. With new generations about every 3 years, and giving the mobility of today’s clientele, the lifespan of at least 6-10 years is likely sufficient for your business requirements.

Frankly, the issue truly cries out for tape, and Ms. Meade is glad to add her voice to those doing the crying out.
 

Dear Ms. Meade,
I am charged with architecting a backup system without any single points of failure. Obviously, tape is SO failure-prone that I am not including it at all. How do you think I should configure such a system?
Sincerely,
Tape is Doomed

Dear Doomed,
You are doomed if you rely solely on disk for your data backup.  A possible interpretation of your question may be “How much disk does it take to replace tape?”  The answer is “unobtainium”—that is, you can’t replace tape using disk.

Further, the very concept of single point of failure is terribly funny in a terribly dark way. Failure is inevitable, unless you plan to address human imperfection? What about acts of natural and man-made disaster that may affect the national power grid? Switch problems? What about loose screws, including any screwed-up (or self-perceived screwed over) employee?

Instead, consider asking a question that does have an answer—“How can I reliably protect data?” The answer is “disk and tape.”

Ms. Meade is a major fan of disk with RAID 6, offered in Spectra’s nTier disk. With RAID 6, up to three disks can fail without affecting data integrity. Go disk and go RAID. However, disk (even with RAID 6) can’t be considered failure-proof because it has its own Achilles’ heel (aka single point of failure): the RAID controller. You can have all the data you want on all the spinning disk you want—but if the controller fails, the brains are gone, and the bits and bytes you’ve carefully protected are toast. Whither goest the RAID controller, so goeth the data. Dead controller= permanently decomposed data. So disk alone, even with the marvels of RAID, is not enough to provide true disaster recovery and continuity of operations.

Further, please note that your information about tape as failure-prone is completely wrong. Tape is, it turns out, incredibly reliable.  With tape’s reliability increase of 700% over the last decade, multiple layers of ECC protection, and smart Spectra libraries tracking media and drive health, tape meets and beats disk in terms of reliability. If you’re worried about a single point of failure,  make sure you get two tape drives. Consider the T950 and T-Finity libraries’ global spare feature—which is an installed drive that can be directed to take over in case of a drive failure.

Ms. Meade admits that she is curious about the pointy-haired boss who directed you to create the no single point of failure unobtanium backup environment….

Dear Ms. Ahmogul,
IBM recently announced that it created a 35 TB tape that will be commercially available in five years. Since then I have read articles asking good questions, like ‘Do we need 35 TB tapes?’

Sincerely,
An Easy Mark

Dear Mr. Mark,
According to urban legend, in 1981 Bill Gates defended the (then-new) PC’s limited RAM by saying: "640K ought to be enough for anybody." (He denies having said this.)

This legend comes to mind on reading your question: “Do We Need 35 TB Tapes?” (Perhaps in thirty years you may find yourself denying having asked this question.)

To answer the 35 TB question: Why, yes, we will need 35 TB tapes. Just as we continue to require ever greater RAM than 640K, we will also need ever larger storage media, simply because of the incredible rate of data creation.

Simultaneously, the percentage of data to protect over the long-term is growing. (Please refer to the IDC chart, below.)  More data, with a greater percentage of it to be stored for a decade or more, leads to an unavoidable conclusion: we will need to store all that data someplace.

Naysayers revel in pointing out (the obvious fact) that it takes a long time to fill a 35 TB tape. Hence, these Captain Obvious types hold that no one will need or use a high-capacity tape. And, of course, it’s true that filling a 35 TB tape takes awhile. It also takes awhile to fill a 1 TB tape.

The fact that it takes awhile doesn’t alter other facts that include these:
• Huge quantities of data already occupy the digital universe and are multiplying at fantastic rates.
• A steadily higher percentage of this ever-increasing data has to be protected for the long-term.
• No viable alternative to tape for long-term, proven data storage yet exists (or is on the horizon, for that matter).

Over the coming decade, the choice is between using a few big tapes or lots of small ones.  So although you, Mr. Mark, may choose to use and manage 35 one-terabyte tapes, others will chose to use and manage a single tape.

The logic is pretty straightforward.

Kindest Regards,
Ms. Meade E. Ahmogul

Source: Gantz, John F. “The Diverse and Exploding Digital Universe,” IDC White Paper, 3/08.  “Figure 8 shows a unique view of the digital universe by the degree to which the information in it might be subject to […] legal and compliance requirements such as ediscovery, HIPAA, or Sarbanes-Oxley; or be valuable enough to expect to store for 10 years or more.”

I’ve recently returned from the 2010 Storage Visions Conference in Las Vegas and wanted to share a few observations. The highlight of my week in Vegas was the honor of accepting a product of the year award for our T-Finity product for Media & Entertainment (M&E) Storage. This is Spectra Logic’s enterprise tape library that stores up to 45PB in a single tape library, and is uniquely suited to broadcast customers with large archive and storage requirements due to high availability, ease of use and scalability. This award marks the first honor for our newest tape library.

Containers for stuff are unbelievably handy. Think how difficult it would be to bring home a dozen eggs if you didn’t have an egg carton in which to carry them? Backpacks for books make getting to and from school a lot easier – even if the books are never removed from the backpack! (There’s probably a lump of coal waiting for that kid.) What about jolly ol’ Santa’s ability to make worldwide delivery without his trusty, fur-lined red bag and sleigh? I don’t think the reindeer will be delivering toys to all the girls and boys one at a time, do you?

Spectra Logic is the only provider of TeraPacks which contain 10 LTO tapes per pack. This makes transporting, importing, and exporting tapes from your library 10 times easier than with any other competitor.   If it’s ease you want, give us a call and we’ll help make your storage admin's holiday season a little bit brighter.

On the tenth day of Christmas, my true love gave to me: