Spectra Logic’s Kevin Dudak is a contributing blogger for the Inside Big Data Blog. His most recent post has been reprinted below with permission from Rich Brueckner:

Got Big Data? 4 Things to Look for at the NAB Show

I just got an email from the organizers of the NAB Show in Las Vegas this April about my registration confirmation. I’ve always enjoyed attending this show, as it has a lot of things you don’t see at the typical IT show. How many storage shows have provisions for helicopters to fly in and be displayed?

This show draws a wide cross section of organizations, with lots of educational seminars, as well as vendor displays. What they all have in common is data. The capture side creates the raw data, editors and post-production change the data, and broadcast distributes finished data.

It got me thinking, there is so much to see and do at this show, what do I want to make sure I learn at NAB? I will specifically be looking at four things:

• 4K and Beyond– Video takes space, and HD 1080P video takes lots of space. It has become so easy to capture HD video these days with things like the Go Pro Hero Cam (they should have a cool booth in the Expo Hall), I’ve got hours of footage from bike rides, autocrosses and other events consuming lots of storage. 4K makes 1080P looks small, and there is talk about what follows that. Increases in sensor resolutions will drive bigger video files not just in media, but also in security and other applications. I want to see what is coming, as storage systems need to be ready to hold the additional data.
• Digital Workflow– I think everyone looking at Big Data issues can learn a lot here. M&E has spent several years converting to digital file-based workflow. This means lots of huge, high value files that need to be analyzed for meta data creation, modified, rendered and distributed. Every year, they get a little more efficient at the process, something we can all learn from.
• Storage – Of course, as a storage guy, I am going to be interested in any new ways to use storage. There will be everything from extreme speed storage for playout at broadcast stations to long term archival storage of digital assets. Asset management of the archives ties in here and is equally important. A lot of the problems that are being solved for long term archive today for M&E will have a broader application in the near term.
• Data Movement– How does someone move a Terabyte of data, or a Petabyte securely and with confidence? I am very interested to see how different organizations are solving the data mobility challenge. With 4K video capture becoming more mainstream, and higher resolutions on the horizon, many entertainment companies are a generation or two ahead of the rest of technology users on this front. Lessons learned here will help us all.

There is a lot of potential to learn some interesting things at NAB this year. They are facing the same challenges many Big Data industries are, but they come at it without the preconceived notions of how IT is supposed to work. I think that gives them the potential to create some interesting solutions that can benefit us all.

Spectra Logic’s Kevin Dudak is a contributing blogger for the Inside Big Data Blog. His most recent post, Big Data Software – More Than Just Analytics, has been reprinted below with permission from Rich Brueckner:

Big Data Software – More Than Just Analytics

I noticed a funny thing the other day while on the Storage Networking World website, looking at the different things on the agenda. At the top of the list is the Big Data track the first day of the show. That’s pretty predictable, given everyone seems to be talking about Big Data these days.

Surprisingly, Hadoop isn’t mentioned once in any of the Big Data track session descriptions. Some of the Big Data Track sessions are about Data Analytics, bringing Big Data to the enterprise and where to start with Big Data – in all of this I am sure Hadoop will come up, but it’s interesting it was not mentioned in the titles or descriptions.

At most other events that have a Big Data focus you see Hadoop everywhere. In fact, the feedback from some people that went to the Strata Conference was Hadoop and Big Data are inseparable. It seems that many have begun to believe that Big Data = Hadoop… but does it?

If Big Data equals Hadoop, then Big Data equals Analytics. But Big Data isn’t that simple. Processing, programming, networking and storage have some type of implication to Big Data, and I am sure we will discover many more important aspects over the next year or two.

What were once simple tasks are much more complex when dealing is massive data sets. The impact to storage systems and networks when looking at data protection alone are beyond what most organizations have considered before. With data sets that can now be larger the many disk arrays, migrating to new systems is complex and time consuming.

SNW isn’t focused on Data Scientists, but on storage managers, and while Hadoop will surely be talked about, it won’t be the focus of the day. As time goes on more disciplines will start to look at the implication Big Data plays in their part of the IT ecosystem.

Spectra Logic’s Kevin Dudak recently became a contributing blogger for the Inside Big Data  Blog. His first post Thinking About Big Data on the Eve of Spring Trade Show Season has been reprinted below with permission from Rich Brueckner:

Thinking about Big Data on the eve of the spring trade show season

The month of March brings longer days, warmer weather and the start of the spring trade show season.  There seem to be as many trade shows as there are interest and industries.  Last year, we saw a lot of people start talking about Big Data at these shows.  The trend most likely will continue, with Big Data taking a bigger share of the conversation.  

Given the years I have been in the storage industry, it should come as no surprise that I tend to look at the storage part of Big Data.  Over the last year we have heard a lot about the analytics side of Big Data.  It is exciting seeing all the amazing things we can do, and things we can learn from the massive amount of data we have at our finger tips these days.  Without a doubt, we will continue to see much of the conversation focus on leveraging our data sets with tools like Hadoop.  Sometimes, it seems we forget that Big Data is more than just the analytics; it is also about storing and managing potentially massive data sets.  2012 will see users and vendors starting to address the changes Big Data brings to storage.

The 2012 Tape Summit and the HPC Symposium kick off the season.    The second annual Tape Summit is the gathering of top manufactures in the Data Tape, including drive, library, software and media companies; as well as press, analysts and bloggers.  You don’t see tape and Big Data in the same conversation too often, but I think the tape industry will be looking to change that this year.  We will be hearing about Linear Tape File System (LTFS,) continued innovation in data management software and possibly the coming LTO6 and how all of these can have a big impact on storing lots of data.

The HPC Symposium will see presentations from some of the top organizations in the distributed high performance world.  Many of the lessons the HPC world has learned over the last 5 years will make the adoption of Big Data easier and more effective. 

I’ll be watching to see how LTFS might be a good answer to Big Data portability.  We are seeing LTFS gain traction in some verticals like Media and Entertainment already.  The question of how to move Petabytes of data, either to seed a cloud provider or just move to a different location has always been a problem.  LTFS might just provide a good answer.

Dealing with massive data sets, be it integrity checking the data or protecting it is a struggle we all face at one time or another.  We are starting to see a new crop of software vendors, some in the Active Archive Alliance, that are creating data storage environments. 

Finally, with the expected shipment of LTO6 this calendar year, we will see a doubling of native capacity on media.  There should be performance improvements as well.  Since the LTO consortium is attending Tape Summit, hopefully we will get more details on it, and how it might affect the economy of storing big data.

As March rolls on, we should start to see a lot of information coming out of events such as the HPC Symposium and the Tape Summit on not only how to analyze Big Data, but how to manage and store it when it isn’t being crunch. 

By Steve Mackey, VP Sales, Europe and Africa

From the stage of world history right down to our everyday lives, everyone loves a good ‘what-if…?’. Whether it’s imagining how different the world might be if a famous political figure had chosen a different career, a sporting icon had signed for a different team, or even imagining  how different your circumstances might be if you had just taken up a different job offer a few years ago or attended a different college.

So here’s my attempt at a little bit of revisionist history, aimed at some nay-sayers who fixate on tape’s future: What if….tape WAS dead?

First of all I think you need to really work out what tape is really used for in current storage strategies. The shift from tape as a backup medium to an archive medium changes the game here – IT departments that used tape for backup and now use disk might not take the death of tape so badly, but those with archiving needs? That might be a different story altogether.

If you are an organisation that creates a lot of data and needs to retain it medium-to-long term for compliance or for revenue-generating activities then the death of tape will hit you hard – primarily in the wallet but also in other ways.

Just think - if tape WAS dead then the first thing you would have to do is scope out enough disk to match your archiving needs.  In terms of straight up acquisition costs per GB we all know that tape is significantly cheaper, but what if your CFO takes a longer term TCO view and asks how the figures stack up over the next decade or even twenty years? Are you really going to show your CFO this report from the Clipper Group, which puts the TCO of Disk over a twelve-year period somewhere around 15 times higher than tape?

Then let’s imagine that this hasn’t been enough to put you off – next up comes the chat with the facilities manager who might be in for a nasty shock when his electricity bill comes in. Those tape drives that would have otherwise sat there silently minding their own business without troubling the power supply will need to be replaced with lots of hyperactive disks that simply can’t bear to sit still. How do you justify those power costs when the disks are being constantly powered but accessed irregularly? Those disks, according to that same report that so shocked your CFO, are going to potentially consume up to 238 times as much power. That’s not good news when companies are looking to cut overhead and power costs are rising.

The financial implications of replacing tape with disk don’t bear thinking about, but there are also practical reasons why disk simply isn’t a good fit. From a disaster recovery perspective tape has always been a popular medium, simply because the media itself is so portable. If you have a primary archive you may decide that your data needs safeguarding against some kind of catastrophe. Getting tapes offline, offsite and into some kind of secure storage is relatively straightforward.

With disk? Not so simple. The only alternative approach, remembering that tape isn’t an option, might be to use an online backup / archiving service. A couple of draw-backs here – first, certainly for large enterprises, these solutions are still considered a relatively unproven solution, not just from a reliability perspective, but also in terms of the potential regulatory implications of putting sensitive data ‘out in the cloud’.  Secondly, because of the imperative on these services to keep costs to a minimum, many of them currently use tape as a storage target for customers, but of course in this brave new tape-free world they can’t! So with these services re-architected around disk, providers would undoubtedly have to hike their prices massively, making it an even less attractive proposition.

It’s worth remembering the proponents of this myth about tape’s demise are disk-vendors and it’s funny that they happen to be the only ones who profit in this dystopian IT vision, gaining an unhealthy share of their customers’ wallets. (It’s also interesting to note that most of them use tape themselves in their own IT departments!)  That said, IT departments are already struggling to manage data growth cost-effectively, and as we move into the era of Big Data, tape becomes an even more indispensible storage medium. For end-user organisations already struggling to focus IT expenditures on innovation over simply ‘keeping the lights on’ the situation would look very bleak without tape.

A big ‘thank you’ to all of you who attended our webinar entitled “Trends in Tape: Looking beyond LTO-5 with LTO-6 and LTFS.”  We had the best attendance ever. This is clearly a topic of great interest to many of you out there.  Bob Cone hosted the call and the discussion was packed with a multitude of great information including LTO-6 and the overall LTO Roadmap.  But more importantly, Bob covered the implications of the roadmap and LTFS and how they fit into the overall storage picture. With so many technologies now available, storage hierarchies and designing and choosing the right building blocks for your environment continues to get more complex.  The presentation distilled much of the vast amount of available information on numerous storage alternatives into an easy-to-understand discussion. Tape, Disk and Solid State Disk / Flash were covered including where they fit now, and where they will fit in the future.  The session was recorded and is available at the link below:

Looking beyond LTO-5 with LTO-6 and LTFS

The webinar underscores how LTO-6 fits into the LTO Roadmap and its important performance and capacity improvements over past generations. It also points out the advantages of LTO-6, which is why you may be interested in Spectra’s LTO-6 Pre-Purchase program. 

How many times have you thought about buying a new car, computer, TV, or cell phone but when you found out a new technology was just around the corner, you waited?  Personally, I need to upgrade my iPhone and considered the current 4S, but am waiting for the iPhone 5. Like me, you hold off and limp along with the old technology, anxiously awaiting the new technology.  Well, when it comes to LTO technology, you don’t have to wait.  Spectra is offering our customers a cost-efficient path to get the latest LTO-5 tape drive technology available today, along with an LTO-6 option, where they will receive an LTO-6 drive to replace the -5, as soon as the LTO-6 is available. 

In addition, this is a great opportunity for customers currently on LTO-3 drives:  LTO-6 drives have read/write compatibility with one generation back (LTO-5) and read only with two generations back (LTO-4).  So, if you have LTO-3 drives and media and want to move to a new generation, you could upgrade to the LTO-6 Pre-Purchase option now, get LTO-5 drives now and replace ALL the LTO-3s, read and re-write the data to LTO-5 media.   Then, when LTO-6s are available through the pre-purchase program (with no additional cost), swap out the LTO-5 drives, and be able to read/write with the LTO-5 media.  Otherwise, if you wait and go straight to the LTO-6 drives, you will need some other way to migrate your LTO-3 media, as it will be unreadable with the LTO-6 drives.

Everything is handled at the time of ordering the LTO-5 drives, so when the LTO-6 is available, we would contact you and find out when you would like us to ship the new drives.  Then you just send the LTO-5 drives back.  And all this is done with no additional paperwork.  The new LTO-6 tape drive will double capacity and provide a 50% increase in performance over LTO-5.  With a larger compression history buffer, the expected compression ratio will go from 2:1 to 2.5:1, so LTO-6 will offer a compressed capacity of 8 TB and data transfer rates of up to 525 MB/second.  The sixth generation of LTO tape drives provides many positive implications for IT and business managers and we are excited to offer you our LTO-6 pre-purchase program: LTO-5 today and the ability to be one of the very first to get LTO-6 and all its advantages when it becomes generally available.
 

When Thomas Gray inked the phrase, "Ignorance is Bliss, 'tis folly to be wise," I don’t think he considered how best to manage data in our present-day data Armageddon.   If you are a data manager and you adhere to the "ignorance is bliss" school of thought, I would recommend that you refresh your resume immediately!

I have spoken with too many people who have no idea of what is to come concerning the world’s rapid and exponentially growing data.  Believe it or not, I talked to a person at the Supercomputing show in Seattle who said they are actually moving all their data to disk and neglecting the tremendous, inherent values and benefits (low cost, high capacity and performance, to name a few) of tape.  As their data doubles each year, which he said it does, the plan is to continue adding more disk... Really?  In his case, I believe he really thinks ignorance is bliss.  I offered to share with him how customers with hundreds of terabytes to hundreds of petabytes are managing data with intelligent file systems and using both tape and disk in cost efficient ways and he refused to listen because his ignorance has caused him to believe that "tape is dead".  Granted, I don’t hear this very often anymore because the HPC community, as a whole, is paving the way for a cost-effective tape-based storage concept we will discuss later, called "Active Archive".  

First, I want to address the ignorance of the individuals who have sipped the "tape is dead" Kool- Aid from certain disk vendors over the past 10 years. Growing up as a teenager in the great state of Texas, I listened to AM radio in my first pickup truck.  (Yes, all it had was an AM radio!)  Anyway, one of my favorite radio talk shows was Mr. Earl Pitts, who addressed controversial topics and would start by sharing his straightforward opinion on them by saying (insert Texas accent)"Ya know what makes me sick, you know what makes me so angry I could spit?"… or something along those lines.  (http://www.youtube.com/watch?v=4DDhrRooNp4)  Then he would talk about something that is usually contradictory to the American way since he was a patriot who was always watching out for our true, red-blooded American values.  Well, I feel sort of like Earl when someone tells me that they think that tape is of no value, which simply shows their ignorance.  I want to say “you know makes me sick, you know what makes me so angry I could spit?".....Ignorance!  He would always end his lesson on values and truth by saying “Wake up America!”  Well, when someone tells me “tape is dead”, I want to grab them, shake them and say “Wake up!”

The reality today, regarding data storage, is that it is not folly to be wise and it is not bliss to be ignorant.  Wake up Storage Admins!  I have to admit that the number of people I talk to around the country at trade shows, in meetings, etc., are awake and aware of the ever present danger of data explosion.  So, needless to say, my blood pressure stays in check and I don’t get angry as often.  I try to keep things in perspective and just assume that they simply don’t know what they don’t know. 

My job, and that of my colleagues, both at Spectra and within the tape industry overall, is to educate as many people as possible about how to reduce the cost, complexity and fear of managing exponentially growing data.  Spectra is leading the charge to create an awareness of how valuable tape can now be in the data center.  Tape is no longer used just for backup.  It was great to see so many of our HPC customers at SC11, most of whom don’t even use the terminology of “backup” any longer.  As tape continued to mature over the last 10 years by getting 700% more reliable, faster and more dense, many of our HPC customers started leveraging the benefits of tape in what we call an “Active Archive”.  In other words, they are using tape as disk.  An active archive is a combination of open system applications, varying types of disk, and tape hardware that intelligently monitors and migrates data across multiple storage devices while maintaining fast user accessibility.  Traditionally, in the backup world, one could only access tapes and the data on them through a proprietary backup application such as NetBackup, Legato, Commvault, etc.  I’m not advocating that corporations discontinue backups all together because one should always have a “second” copy of data in the event of a disaster.  However, the premise of an active archive is that all data can be online all the time. 

Obviously, when someone has hundreds of terabytes or even petabytes, it is cost prohibitive to try and keep all data online all the time in the traditional way of keeping it all on primary or secondary disk.  With an active archive file system, the data can be dynamically distributed across multiple storage platforms including disk and tape.  Policies can determine where data is at any given time and it is transparent to the end user where that might be.  They simply have a drive letter and directory with all their files as normal.  Nothing proprietary about access to their data—anytime they need it.  By extending a file system across high performing disk, capacity disk and now tape, the need for IT intervention to retrieve an archived file is minimized, if not eliminated.  This data management approach is being used by many of our HPC customers and they are benefiting tremendously by having a searchable, compliant format to store data for the total lifecycle of a file based on policies, industry regulations and laws.

I could go on about the benefits of active archive or the inherent values that are characteristic of the tape technologies of today, but I would rather provide some links to more information on both so you can continue your own research and put aside any tendencies you might have to subscribe to the “ignorance is bliss” philosophy!  Tape is here to stay and is poised to solve your storage headaches today and in the future by offering greater efficiency, better reliability and maximum performance. So wake up!  Data Armageddon: tape’s got this one.

For more information on Active Archive, go to www.activearchive.com

For more information on Spectra Logic tape systems, go to www.spectralogic.com

I also welcome your emails to jimm@spectralogic.com

November marks the second anniversary of the launch of Spectra Logic’s flagship product and the world’s highest capacity storage system: the Spectra T-Finity tape library.

As with any 2 year-old, significant growth  has occurred in a short amount of time.  Since T-Finity’s launch in November 2009, library development has included growing from:

1. 25 frames in a single library to 40 frames
2. 30,520 LTO tapes to 50,100 tapes in a single library
3. 4 libraries in a complex to 8 libraries in a complex
4. 122,000+ LTO tapes in a single complex to 400,800 LTO tapes in a complex
5. LTO tape technology exclusively to including industry leading TS1140 Technology tape drives as well
6. 183 PB in a complex to 3.6 EB in a complex using TS1140 Technology

In addition to the physical growth, the library has also added next generation servers and software to better support a variety of features and enhancements that include:

1. Data Integrity Verification to preserve data viability and integrity – particularly in archives
2. Continually increasing robotics performance
3. Expanded MLM database capacity
4. XML API

And while this growth has progressed, T-Finity has continued to provide the industry’s best:

1. Library density
2. Power efficiency
3. Library management – including built-in encryption key management
4. Product reliability

It’s been an impressive couple of years since T-Finity was introduced to the world.  Watch and see what happens during the next 2 years.  Happy Birthday, T-Finity!

We should all be Thankful as “Big Data” improves storage for everyone.

It’s the beginning of the Holiday season, with Thanksgiving travel in full swing.  I’ll be getting 10 hours of windshield time shortly, as I’m headed to see family.

As more of our customers have moved into the world of “Big Data” we have been looking at how to make storage ready for ExaScale.  ExaScale sized storage has challenges that storing a handful of Terabytes never imagined.  Spectra announced the 12^thgeneration of BlueScaleearlier this month with a lot of advancements for Big Data customers.  While “Big Data” can mean a lot of different things to different organizations, one thing that is common is the need to storage and manage huge amounts of information.  We spent hours working with our customers over the last year looking at where we could make massive storage easier to use. 

Simply booting up a multi-petabytelibrary can be time consuming.  Traditionally, a library will reinventory itself when rebooting.  This takes a few minutes on a library with 50 tapes, but will take hours on a library with 15,000 tapes.  Spectra’s BlueScale 12 operating systemwill not force a fresh inventory on reboot.  If you didn’t change any tapes, why waste all that time?  If you did open the library and change things, then you can tell the system to update the inventory, whichwill save our customers hours.

The number of components that might need code updates over the life of the library grows with data storage as well.  What would take a few minutes with a 2 drive tape library could take hours with a 120 drive library.  With BlueScale 12, updates are done in parallel, so 120 drive sleds can be updated in the time of one.   

Of course, most organizations are not rebooting their libraries or updating firmware every month.  We have continued to increase the assistance Media Lifecycle Managementgives our customers.  The analytics we evaluateon our Certified Media combined with Data Integrity Verificationon the data written automatically lets informs administrators if there is an issue.  They do not need to spend any time managing it, it just works.  BlueScale 12 adds MLMsupport for TS1140 technologytape media in Spectra T-Finitylibraries. 

These enhancements and more, like the XML interface, Carbide Clean and RAIT make managing the largest storage environments easy and reliable.  The great thing about Spectra T-Series libraries is they all run BlueScale.  OursmallerT50ecustomers get the same software updates and benefits as our largest T-Finitycustomers.    All our customers do not generate multiple petabytes of data, but they all have data that is important to their business.  Being able to bring the advances that“Big Data” drives to all our customers is something I am thankful for.

Now, back to the road. Safe travels this holiday season!

Reliability: (adv.) the extent to which an experiment, test, or measuring procedure yields the same results on repeated trials.  Dependable.  Sure.  Trustworthy.  (From our friends at Merriam-Webster). And there’s a picture of tape next to the definition.  Ok, so maybe the picture statement was a stretch, but associating the definition of reliability with tape definitely is not.

Anybody who’s been in the storage industry for more than 30 minutes has likely heard the phrase, “tape’s not reliable”.  Certain marketing machines in the technology space propagate that phrase as much as possible – occasionally with bumper stickers.  Those folks have some imagination, but generally register a bit low on the fact meter.

Here are a few things people are saying about the reliability of tape.

Bit Error Rate Favors Tape Reliability Over Disk  - Horison Information Strategies, April 2011

Summary.“Tape drives and tape media now have a higher BER and longer useful life than disk products making them better suited for the long-term data retention requirements demanded by fixed content, compliance and archive applications. For a specific amount of data transmitted, tape now has a marked reliability advantage over disk - a surprise for many.”¹

^1.“Tape: New Game.  New rules.  Tape re-architects for 21^st century data explosion.” Pg. 6. April, 2011. Horison Information Strategies

Tape More Reliable Than Disk for Long Term Storage – Curtis Preston, June 2011

Summary.

“Tape drives:

1. Write data more reliably than disk
2. Read it after they've written it to make sure they did (where disks often don't do that)
3. Have significantly less "bit rot" or "bit flip" than disk drives over time.”²

^2.“Tape more reliable than disk for long term storage.” Backup Central blog, June 2, 2011

Tape Drives 700% More Reliable Than 10 Years Ago – Debbie Beach, Sylvatica Consultants, 2009

LTO drives are specified with an impressive mean-time-between-failure rate (MTBF) of 250,000 hours at 100% duty cycle, that’s 700% more than the MTBF of tape technologies created a decade ago.³

^3.“The evolving role of tape and disk in the data center.” Pg. 7. 2009, Beech, Debbie; Sylvatica White Paper

One is an accident.  Two is a coincidence.  Three becomes a trend.  The reliability of recording data to tape for storage over the long term is hard to beat.  Could that by why tape roles on?

To learn more about Why Tape Rolls On, see parts 1 through 4 of this series discussing the Security, Green Storage, Speed and Density characteristics of tape.