Over the next three years a majority of compute workloads will migrate from  non-cloud on-premises  infrastructure to a hybrid public/private cloud ecosystem. Cloud means scale up in real-time to process data and then scale down when the work is done. No wasted compute power sitting idle.

The winners will be the companies that early on figured out how to harness the cloud infrastructure as a service for SaaS delivery and those that provide enabling technologies to help IT buyers get more value for their IT budget dollars. Sonian’s Universal Data Management cloud-powered platform is one such example of putting the cloud to good use today, solving the information governance pain point in a modern, secure, reliable and efficient manner.

Kurt Monash delivered an interesting and entertaining keynote speech touching on a wide range of subjects around databases, storage, and his thoughts on how the cloud is changing the way organizations of all sizes need to think about their data management issues.

Fascinating “big data” factoids:

• Yahoo! manages 10 petabytes
• Ebay manages 6 petabytes on 96 Greenplumb nodes
• Facebook manages 2.5 petabytes on a distributed file system
• Sonian manages 100′s of millions of objects in a cloud-powered full-text enabled archive repository

Sonian is in the forefront of 5 out of 10 Strategic Technology Trends for 2010.

Cloud computing, advanced analytics, green IT and virtualization are all poised to see tremendous growth in the next year because each of these trends ultimately helps customers “do more for less IT budget.”

1. Cloud Computing. Leverage on-demand CPU and storage to solve archival and information governance needs.

2. Advanced Analytics. The cloud is the best place to analyze your terabytes of “dark data” into actionable intelligence.

3. IT for Green. Lower electricity and cooling needs to save money and the planet.

4. Social Computing. Personal and work life as it relates to IT systems are more entwined now than ever before.

5. Virtualization for Availability. Use a “hybrid IT model” – keep some services on-premise and move ancillary to a virtualization layer in the cloud.

Records management professionals are confused about “cloud storage” and where the data physically resides. The “cloud” may imply data is scattered around the globe and un-organized. But that is a misnomer and not true. Inside each cloud, Amazon Web Services for example, is a physical data center just like any other. With very tight security and very good up-time SLA’s.

Pelz-Sharpe says about cloud and records retention: “You need to know exactly how data is disposed of, and how that can be verified.” We couldn’t agree more. In a compliance mode data retention and data destruction are equally as important. Encryption and virtual shredding in the cloud are possible and can be a better way to shred data than using on-premise SAN/WormFS systems where too many touch-points expose risk. Records management in this era of all digital information is vital, and the cloud provides the confidence and low cost to ensure companies will not skimp on their records management initiatives.

This is your backup, maintained in the event you need to recover a file or an entire server. Backups are tape libraries or backup to disk infrastructures. They are not designed for investigative search or analysis.

Archive and backup are cousins to each other, with archive providing the search, e-discovery and work flow capabilities that backup systems do not. Maybe some day in the future combined systems will solve both problems, but the state of the art today is separate systems, with backup maintained on premise, and archive hosted on cloud compute infrastructure.

Local backup and hosted archive is the “win win” scenario IT needs to save money, provide actionable intelligence, and solve the dual needs of search and data recovery.

A key finding is the volume of email data is increasing, and that 95% of the storage footprint is consumed by attachments. A great archiving solution can help with attachment management.

Thanks to Peter for sharing this interesting information.

• Average number of emails received per day: 180,000.
• Percentage of email that arrives on campus that is spam: 94%.
• Percentage of storage space taken up by email, a system designed to send brief text messages, that is actually taken up the files attached to the emails: 95%.
• Ratio of the storage required for email and attachments for just the year 2007 to that of all of the preceding 5 years together: 1 to 1.

Not only is reliable cost efficient storage a necessity, but more importantly is the need for effective search to find exactly what you are looking for in ever growing digital archives.

It’s a given today’s storage choices offer low-cost options, but these systems typically only address half the problem. Now that we have nearly infinite storage resources, how do we find the “needle in the haystack”? Better search technologies, coupled with low cost reliable storage is the key. Sonian is working on advanced techniques to make searching billions of stored objects more efficient for the user that needs quick answers and “actionable intelligence” from the information archive.