Every time that I find myself hopping from airport to airport, I love seeingthe HSBC “think different” ad campaigns. Some interesting facts I read at Heathrow recently:

• Brazilian football teams have earned over $1 billion in fees from selling players internationally.
• On average, Russian billionaires are 19 years younger than those in America.
• Indian e-tutors generate $20 million annually, teaching english to American students.
• The Halal industry is worth $3 trillion worldwide.
• Two Thirds of the Worlds Billionaires Started From Scratch

The last one is definitely my favorite. You can read more about the campaign here.

These captions put forwards a very interesting proposition. They provoke us to think differently. Who would have thought that selling audio oscillators would lay foundation for a billion dollar company (Hewlett Packard).

Disruption always starts small and the key to success is to keep your eyes open about the opportunities around you.

But my days of not caring about backup recently changed. I headed off to SNW Europe in Frankfurt with my MacBook in tow – and like most of us it’s my lifeline. On the morning I was leaving Frankfurt to return to San Francisco I met our sales team for breakfast at the hotel. As we’re talking, I accidentally knocked over a glass of water. My colleagues said there was a look of horror on my face as the water moved in slow motion across my keyboard and seeped under every key. I quickly hit “send” on my draft email and proceeded to soak up as much water as I could with my napkin. I powered down just like always and took off for the airport.

14 hours later I arrived home and so did my reality. A crashed laptop, therefore no data. My laptop was replaced the day I arrived home and thanks to Druva’s inSync I was able to restore all my lost data instantly with secure browser-based access. I’m all about simplicity and ease-of-use. I was back up and working in no time thanks to inSync’s advanced WAN optimization. This made my boss very happy.

Lost critical data is way too common in remote workforces. Here at Druva, we hear stories every day. Someones laptop is lost, stolen or like me, ruined by a glass of spilled water. And yet data shows that only 39% of enterprise laptops are protected. It’s like driving without insurance or getting cancer and having no healthcare. Who would do that? Go figure.

Endpoint data is more dispersed and diverse than ever before. Some of our customers are protecting up to 1PB of data with inSync, and one of the key requests we get is to have the tools understand “what” they are backing up and how can they defuse this rapidly expanding data.

This new addition is designed to answer most of these questions with the following features :

• Data Analytics: This lets the admin gain insight into the composition of the backed up data and understand the key trends fueling the growth of data.
• Global Search: Searches the meta-data to find files with certain keywords across the users. It also helps the admin gain access to the data refereed in the results
• Reports: Helps schedule search and analytics and generate custom reports
• APIs & Tools: APIs and different export format to integrate with any existing software

Essentially this is an eDiscoveryLite for enterprise endpoints, and yet again demonstrates our commitment to enterprise endpoints, an area we feel is extremely important but lacking in tools and support.

So far, the feedback from the Data Insights beta has been great. Interestingly, some customers came back with a different use case and requested if this could be combined with our SafePoint product (data loss prevention) or with data sharing, to help the admins better understand the propagation of data and how to secure it.

You can learn more about the product, which is currently free for existing customers, by going to the Advanced Data Insights page.

Today, I could see tweets from Libya, Egypt, Yemen, all united by one thing. And could see an adult cry while typing on his iPad.

Thank you for leading the way. Your vision will continue to guide entrepreneurs like me in years to come.

R.I.P

One of the best things about startups is your ability to be nimble. Our Engineering team continues to march on and turn things around at a rapid rate, with on average a new release every 2 months, which includes new features, platform support and fixes. We regularly solicit feedback from our customers, either directly, through our Druva community or through support while moving forward with our roadmap.

You can download the latest inSync releases from here. Definitely excited for the next release of inSync, 4.2.2 which is due in a few days, while 4.3 is planned for October with more enhancements which we’re keeping under wraps right now, but stay tuned

At a previous company I worked at in the data protection space, the tagline was “Fastest Restore. Period”. My immediate question to that was, “How do we know? Have we benchmarked this?”. At Druva, when I joined I saw that our messaging said that we saved “Over 90% in storage and bandwidth”. Pretty stellar figures I thought, but can we support them? In our own testing, these were the figures we’d seen, but I wanted more concrete evidence, so we brought in ESG to run Druva inSync through it’s paces in their lab.

We worked with Tony Palmer, a Senior Analyst and Lab Engineer at ESG Labs. We gave him the background to the software, what it did, how it worked, and we pointed him to the download of the enterprise version. “Great, we’ll talk to you in a few weeks then” he said. And off he went.

Tony came back a few weeks later with the lab report. The results were in. He was actually able to recreate the “90% savings” in his working lab environment. In fact, the ESG lab team were able to achieve the following results:

• 96% savings in storage
• Deduplication resulted in total of 38.6GB disk storage for 1TB total backup size
• Deduplication ratio was 26:1
• Install & deploy in less than 20 minutes
• First backup of 5.2GB took only 12 minutes
• Server CPU peaked at only 11%
• RAM utilization maxed out at 140MB

Every environment is different, but in this “work-like” live environment, inSync passed the test. You always want to be able to back up your numbers, and now we can with this ESG Lab report. We always say at Druva “The proof of the pudding is in the eating”. So we encourage people to download inSync, try it, play with it in your environment, try to break it, and see how it works. You really won’t know until you try it.

* You can download the full ESG Report in PDF format here *

I can still remember working from a “shared” garage office when Druva was a bootstrapped company with just 7 people.

After wasting about 6 months, the “eureka” moment came in July 2008 with the launch of inSync, and since then we haven’t had the time to look back.

Today I’m delighted to announce the closing of a new “scale-up” Series B round of funding for $12M lead by Nexus Venture Partners with participation from Sequoia Capital. The official announcement and more details can be found here.

This round will help us to continue our momentum and make some strategic investments. In the recent past Druva has demonstrated technological leadership with the introduction of unique innovative products such as SafePoint, support for iOS/Android and lead the way by being the first data protection company to offer strict enterprise SLAs for data backup to the cloud. Over the next year to 18 months, our key focus will be to significantly grow sales and marketing while still continuing the spirit and culture of innovation and reinvention that has brought us success so far.

In thinking about the trip so far, being an amateur hiker, it reminds me of hiking a steep ascent. As you go higher, the view just gets better, and I’m confident that we are soon going to be at the top.

Although it did affect Netflix, foursquare and few others, thankfully Druva cloud services were completely unaffected by this. Here is a small note on how we managed to keep our promised SLA.

I think its plain ignorance or mis-planing to assume 100% availability of underlying infrastructure. Just like any hardware, the AWS infrastructure is prone to failures, but the knowledge of these potential failure points can help improve availability.

Since its a backup service, we have have divided our cloud design into 3 parts based on the availability and durability guarantees :

• Config (most available): Configuration data stored in Amazon RDS
• Meta-Data : Druva Dedupe file-system spanning across Cassandra nodes
• Data (most durable): Stored in S3

And some design changes we incorporated to avoid downtimes :

• Multi-Zone replication: Both RDS and Cassandra nodes are replicated across 3 availability zones. We use Cassandra in full-consistency mode and heavily rely on its self-healing, in case of service failures.
• Reduced Dependency on EBS: EBS is a software abstraction of an underlying SAN storage. And two independent EC2 instances may share same SAN for EBS. Given this we shifted our focus from EBS to local-storage for meta-data.
• Extra space copies in S3: We so maintain some extra redundancy on top of S3 for most referenced blocks. This essentially is to avoid the random (but less frequent) S3 time-outs and improve durability of most concurrent data.

We surely paid more for improved availability, but there are simple design changes which can help save as well. For example the 3-way replication increased our compute(EC2) cost by over 200%, but because of extra spare we could increased the data stored per instance, which was earlier restricted to maintain a good cache-vs-on-disk ratio.

Time and again multiple enterprise customers, especially those who are migrating from competing solutions, ask us about scalability of Druva inSync. Since the launch of v4.0, inSync has scaled exceptionally well, especially for large deployments. The software has succeeded where majority of competing solutions have failed or turned off deduplication.

About a week back, (on request of a large customer) we started testing one of the competing solutions. We tested the software for 1 million files of total size of 2TB, of which 48% was duplicate. Insync finished the backup in about 22 hours and the competing software is still backing up.

InSync doesn’t support any “integration” with deduplication, but the whole software was designed around the deduplication and CDP. There is NO flag to turn off dedupe and there never will be.

This article focuses on my thoughts on how Druva succeeds where majority of others fail.

Why Source Deduplication Fails to Scale for Majority Vendors ?
The biggest bottleneck for performance scalability of deduplication is the random disk IO performance. Almost all dedupe systems include a database to store the block-hash index which needs to be checked for every hash check. A server class magnetic disk usually offers a latency of 8-12ms which restricts the hash matches to about 100/sec, throttling the dedupe performance drastically.

Now, when the data set is small the entire index can reside in memory and hence the hash checks as much faster. As the index grows, the I/O congestion brings down the software’s capacity to perform inline deduplication.
Consider this: Just about 1000 users can create over 10 Billion blocks for backup. And checking them with a rate of 100/sec could take 3.21 years.

Learnings from Storage Guys
Data domain had an interesting approach. They optimized their inline dedupe performance for backup streams. Since the backup was mostly for servers with few large files and the data streams were mostly long streams of data in tar format, Data domain used a simple index read-ahead algorithm to load the relevant parts of the index before the stream blocks hashes reached the server. Since the streams changed less than 10% across two simultaneous backups, the algorithm helped deduplicate them at a very fast pace.

Solid State Disks
A simple solution to the random-I/O problem is using SSDs to store the index. Although we did tweaked/changed certain features to support SSDs but the solution wasn’t complete because of the size limitation imposed by them.

Two Step Approach for Druva: No-SQL + HyperCache
The “Data Domain approach” did not work for us as our data was much more random and coming from different sources. But on the flip side we had much more knowledge of the data formats we were backing up.
The first step towards scalability was to get rid of the inbuilt SQL database which imposed a lot of latency because of SQL query serialization and execution. We replaced PostgreSQL with Oracle no-SQL BDB as an embedded database, which improved the performance and much simpler to maintain.

The second major innovation was HyperCache – a selective in-memory cache of index. Hypercache constitutes of both a positive and a negative cache, which remembers and caches both the most probable and the least probable hashes for on-going backup. HyperCache uses an ever learning algorithm and uses different parameters like time, frequency and probability of a hash to cache it.

The Result
The result was 85% reduction in disk I/O by using 4GB of RAM for every 1TB of data stored. The reduction in IO translates to 4X better scalability, and the solution can easily scale to thousands of users with linear improvement in scalability/performance.

Use of SSDs further improves the performance by 6X. InSync core has been modified to keep only the most concurrent part of the database index on SSDs and optimize it for solid state drives.

I’ve been travelling pretty heavily over the last 2 months, and had an opportunity to meet and learn from a lot of customers. The best companion during this travel has been my iPad. Here’s my small list of productivity apps/accessories which I use the most:

• inSync iPad app – lets me access all files and all the versions backed up from my laptop
• Smart cover – makes typing emails simple
• Easy Sign app – makes signing documents easy (I realized this was the only reason I used my fax/scanner)
• Kindle app – Bought 2 books: The idea book & The upside of irrationality
• iPad VGA adapter – allows me to present directly from my iPad
• Evernote – for taking meeting notes

I’m currently a happy beta tester of the new version of the inSync app for iPad which allows offline access of folders marked as “favorites”, and this was an absolute life saver when I was travelling in Europe and didn’t want to pay heavy roaming charges

I see tablets as great access devices, which in my opinion would affect two markets the most : desktop and printer. Although I am still adjusting to carrying 1 extra device, but IMO the laptop may soon become more of an in-office/limited-mobility device.