One Month In and SDDC

This past month has been another one of those “Firehose” months. This is such an exciting time for me in my career. VMware has been treating me quite well, and I’ve been having so much fun.

My team, the Accelerate Velocity team is dedicated to promoting the full stack of Software Defined Data Center. The SDDC is the most recent iteration in an evolution of converged architectures. I remember my first experiences with converged deployments at EMC, and being so impressed by the vBlock . So well thought out and robust. Since then, a number of newer approaches have been taken toward this concept. Companies such as Simplivity, Nutanix, and of course, NetApp have evolved the concept of merging Computer/Storage/Network into one highly detailed package. These approaches have grown in both dedication to unique use-cases, and scalability.

As you may know, I previously worked with Nexenta, one of the leders in Software Defined Storage. The concept behind “Software Defined” beyond the marketecture or buzzword status is that Commodity-Off-The-Shelf (COTS) with software focused on the task at hand can be leveraged to best utilize the highest level of performance from that hardware. The key is that software makes the difference. The better the software, the better the solution.

This concept has evolved over the past couple of years to include network with spectacular growth in companies like Nicera which was acquired by VMware, and Cumulus. Again, these software applications take advantage of commodity equipment rather than necessarily the more expensive mainstays in the space while providing a software layer that’s easily as robust, and often moreso than the embedded software that comes with the native components. These can also leverage API’s and third party apps like Puppet, Salt, and Chef to automate deployment to multiple devices or even to enterprise level rollouts.

One can certainly recognize that vSphere has done much the same thing with the virtualization of operating systems within standardized x86 server class machines.

Along comes the VMware with the most impressive attempt to tie all these pieces together. This is the SDDC approach.

As we move together in this blog, I’ll discuss some of the solutions we present, deployment methodologies, recipes and various components necessary to the creation of the SDDC.

Another new beginning

I’ve been fortunate in my career for a number of reasons. I’ve had the chance to work for many fantastic companies, had the opportunity to work on some amazing technology, and have retained positive relationships with all my employers after leaving. I’ve been able to increase my network, increase my knowledge of so many different areas of great tech within the industry, and promote these to some wonderful customers. I feel truly blessed.

Sometimes, a role comes to you that is so amazing that you can do nothing but change your job. Such a circumstance has arisen for me. While I’ve not been at EarthLink long, and bear only positives for their excellent organization, I’m taking a role at VMware as a Pre-Sales Architect, Accelerate Velocity Strategist.

 This group is tasked with focusing on new technologies being released by VMware, and getting these technologies rolled in as functional implementations by some of the highest profile organizations in the world.

I can be no more excited to be rolling into this group, with its leader, Eric Ledyard ( @EricLedyard ) and some of the finest engineers I’ve had the chance to meet in my career.

 

The opportunity gives me the potential for learning and growing my skills, and to work with a team that will truly challenge me toward the excellence I’m always seeking for myself. There is no greater compliment for me than to be welcomed into such a group, to be given the opportunity to be on the bleeding edge of an organization I’ve admired, and worked with for over a decade, and to evangelize these things to a phenomenal customer base.

 

I want to say thank you, and tell the world how honored and humbled I am for this opportunity.

GeekWhisperers and I

While at Cisco Live this past week, I was fortunate to be invited to be a guest on the Geek Whisperers. This was a real highlight of my trip. While it may just have been convenient, I was very pleased to be included. I have long listened to the podcast created by Amy Lewis, John Troyer, and Matthew Brender . Respectively: @CommsNinja, @MJBrender, and @JTroyer on Twitter. Distinct presences in social media, who’ve with knowledge, intelligence and humor debated cajoled and enlightened the whole concepts of social media and community. I told Amy, actually, that I feel like listening to the podcast is like sitting around with some friends and listening to some intriguing conversation to which I wish I could add my admittedly limited value. I guess this was that opportunity.

http://geek-whisperers.com/2014/06/the-collective-noun-of-influencers-episode-50/

Episode 50 seems a bit of a landmark as well. Image

As you can see, the production methodology of this show was not at the average levels for my friends, but surprisingly, the show sound quality was really impressive.

What makes their conversations so compelling for me is that I get to hear the perspectives of marketing and other approaches to what social media can bring to you and your company in ways I honestly about which I never thought. They’ve opened my eyes to a lot of thought in this vein that I’d really never even considered. And they do it consistently in a compelling manner that never lectures, but always intrigues. They talk of Twitter, Team Building, Branding, Communities, metrics and so many other things. Their guests are generally, with the exclusion of myself, experts in the field.

So, with an amazingly professional rig, we sat and discussed many of the issues in what can be a complexly textured conversation.

I have been a member of the social media world for a long time. When I was an employee at EMC, my boss, Chad Sakac (@SakacC on twitter) said simply: If you’re not already on it, get on twitter. So I did. I hadn’t really thought about a personal/professional blog yet, but I’ve now been on Twitter for over 4 years. I’d thought of what I wanted to say, and how I wanted to be perceived on Twitter and made a number of rules for myself. These guiderails were mine, and I could preach them to others, but that’s irrelevant. What your rules are, what you choose to show about yourself, and what interests you have either inside or out side the professional career. I think that we all recognize that what we do actually does become part of the digital archive forever. But how important is that aspect to you? What you choose to publish versus what you don’t should, in my mind, be guided by that basic rule. Those who do follow me know that I’m just as likely to tweet about music, the Blackhawks, as I am virtualization and storage.

Ultimately, many of the conversations that I’ve heard about these things, in my mind, have rolled up to common sense. Some of the things that I think about are my rules on curse words, my thoughts on politics, and my thoughts on religion. I rarely have these conversations in a public sense, or with my coworkers, so why would I use a different voice on public social media? Sometimes I think about the book “Everything I ever really needed to know, I learned in Kindergarten.” I do believe in honesty, kindness and tolerance. So I would on social media as well.

I recommend The Geek Whisperers highly, not just because I like these folks, but because I respect their opinions, and intelligent approach to a potential quagmire of a subject.

 

Second Time Around

Once again, the fantastic team at VMware, led by John Mark Troyer @JTroyer and Corey Romero have compiled a comprehensive list of vExperts. I am thrilled to be included in this elite corps. The vExpert community is not a certification, but more an industry recognition of efforts. These people who are bloggers, tweeters, and have years of deep experience in this technology we all love.

The full listing and blog post by Corey is located here:

http://blogs.vmware.com/vmtn/2014/04/vexpert-2014-announcement.html#comment-9611

My first acceptance into that group was gratifying to say the least, but to be recognized for a second year in a row is truly humbling.

It comes with great appreciation and true sincerity. My commitment to you, the community of readers, and technologists who use this great tech, that I will continue to push myself forward to deliver content of meaning, and assist my customers in implementing best of breed solutions.

Quite simply, I want to thank the team, and let my co-inductees how appreciative I am to be included.

Thank you-

Matt

My take on the Eras of Paul Maritz and Pat Gelsinger at VMware

ImageImage

I was asked by TechTarget for my opinion on the comparisons of two giants of tech, and their leadership of one of my favorite companies:

This has been an interesting couple of years for VMware. With the transition from Paul Maritz to Pat Gelsinger as CEO, it’s only reasonable to look back and try to establish where each succeeded, where each failed, and what successes have shone during their respective tenures.

Maritz took over [for] Diane Greene when the company had some challenges. [VMware was] not embraced by the public, but by techies such as me who saw the potential benefits. Maritz brought the hypervisor and its capabilities to full functionality and made them rock-solid. By the time his role was determined complete, the industry surrounding virtualization had matured and many competitors had comparable hypervisor products. I remember when Maritz publicly stated the hypervisor was irrelevant and that the future was the application level. This was a bold statement for the CEO of a hypervisor company, but a true one, to be sure. I agreed with him at the time, and he’s been proven correct.

I admired Paul’s vision then and admire it now at VMware spinoff Pivotal. He’s an empire builder and a force.

Pat Gelsinger, who left Intel to join EMC, is another technology giant who has also proven his maturity and intelligence in the industry. He made an enormous decision while president of EMC to implement Intel processors in all EMC storage arrays. He standardized all storage devices on an industry processor, which ensured interoperability many years into the future. This proved to be a critical and quite prescient decision.

Upon taking over VMware, he built on Paul Maritz’s cloud vision with key acquisitions and strategic decisions. Gelsinger also renewed focus on VDI [virtual desktop infrastructure], which had been foundering for some time. A technology organization can only invest in so many technologies at one time, but this change can only help.

Gelsinger has authorized some exceptional acquisitions, namely Virsto on the infrastructure side and Desktone for end-user computing. I believe this renewed focus in not only internal growth, but also outside acquisition, shows an enormous dedication to the future.

Image  Image Image

Another New Year / New Opportunity Posting

As many of my colleagues have been posting, I too have jumped from an amazing company: Nexenta focusing on Storage and Virtualization to EarthLink, focusing on Cloud, Services and hosting. My role will be that of a Principal Solutions Architect, and I’ll be focusing in my region, primarily in the Midwestern area of the US.

I’ve been feeling lately that some of my more discrete skills, those of OpenStack, Cloud, and some of the networking pieces that surround these technologies have become less categorically a part of my portfolio, and that I’ve had to work on them privately as part of my lab work, as Nexenta is a storage company by nature. While their technology is astounding, and I’ll continue to promote them when the solution is logical, I’m now allowed to bring whatever solution makes the best sense to my customer base. This will afford me the opportunity to bring to my customer a range of technologies via an agnostic approach with an eye toward “Solutioning” significant and creative approaches to their needs that will allow us to help them in ways they may never have considered previously.

EarthLink provides more than just traditional hosting, incidentally. They started out as a basic DSL provider, who had a rock solid model. When the shakeups started happening, and all the providers began falling by the wayside, they knew that they’d have to modify their business model or they would fail as well. They did. They added talent, management skill, and acquired some phenomenal companies. One notable acquisition was CenterBeam. These folks also invented Hosted Exchange, as well as creating Active Directory alongside Microsoft. This one gives access to an advanced set of controls within the Microsoft Office 365 toolset allowing for a massive improvement on the standard set of webtools included within what Microsoft offers in the standard product.

I am thrilled to be part of this fantastic organization and this great team. And, I look forward to this new opportunity with excitement and not the smallest amount of healthy anxiety.

Stay tuned for upcoming details on some very cool offerings from this very dynamic and exciting company in future posts.

Are you afraid of changes?

Image

I’ve been in this industry for a long time. In my experience, there are many organizational styles out there. After an interesting conversation with my friend @Daverdfw, over twitter the other day, I thought that I’d write regarding my opinions therein. Following is a list of a few types:

1)            This is the way we’ve always done it, and it’s worked for us well, so why change?

2)            We don’t know any better, because we’re inexperienced in management

3)            The scattershot approach wherein management will turn it’s approach on a dime

4)            Considered, but thoughtful, in which there are new ways to approach issues, and with due concern, changes are made

Image

In the first scenario, an organization may be aware of other approaches, but because they’ve become so entrenched in “How” they’ve done things all along they’re almost afraid to make changes. There are so many examples of this old-school approach. It could be that their tech is in place, and they refuse to consider disruption in their data center. It could relate to how/whom they hire. If they’re become used to hiring in one way, or for that matter, evaluating their employees in the same way, each by the same metrics, and thereby disregarding the individual contributions or characteristics of new employee styles.

 

In a second, which is prevalent under the models of so many startups. We see this quite often in the world of storage and virtualization in which I reside, wherein the management is so new to the game, likely thinking that they could “Do it better” in their previous work experiences, that they are unwilling to truly integrate the willingness to question their skills. The danger here is that again, the change that’s required is not being made, simply because the management is unable to see that they’ve got to. They don’t know nor would they be willing to consider a different approach, as they’ve got an inability to grasp different viewpoints.

 

The third example is endemic to startups, and can be an enormous problem. In this case, a company will try something, will give it an idea a chance, but not give it an opportunity to prove the value of the plan before making yet another change. In this way, so many plans fail. Even a bad plan needs a chance to establish itself. These can be marketing, hiring, and technological plans, as well as practically any other goal. How does the engineering department plan on implementing the correct approach, when they find no ability to rely on any specific direction? Clearly, scattershot approaches can wreak havoc in any company.

 

The last one I illustrated is one in which a considered decision, weight placed on all sides, careful thought, without haste is placed into decision-making. It’s good to give your plans time to gel, and to try to determine their capacity for success before making changes. By the simple evaluation of goals and deliberate motivation, an organization may keep their customer’s needs in mind, their engineering staff on task, and their employees from guessing what they’re going to have to count on next.

 

As I’ve said in this blog previously, communication is key to the continued success of a company. If a management team is unafraid to hear conflicting viewpoints, and to hear all sides, changes in course can be managed and overall goals will be kept in target sights even while being dynamic and embracing the changes that need to be made throughout the organization and disruption can be both embraced and fears can be mitigated.

As I enter my new role at Nexenta

new-job-exit

Many people don’t realize that Nexenta offers a second product beyond its world-class open Solaris based storage platform, but we do. This product, dubbed Nexenta’s VSA for VMware Horizon View, or NV4V for short, optimizes the VMware View experience well beyond what has become its standard implementation with so many qualified features that it may just prove to be an enormous game-changer.

 

When we created this product, our goal was to fix a problem that had plagued most View implementations as they’d moved from proof of concept to either secondary test rollout of View, or even full deployment. That problem had been that the IO requirements of VDI are often so demanding that existing storage platforms already implemented in these existing storage environments either, would need to be so drastically re-architected, or completely replaced in order to accommodate for the requirements of these projects. Many of these projects simply didn’t get implemented because of the high I/O requirements. So, we undertook an approach that changed the dynamic entirely to allow for existing infrastructures to coexist with the newly-created IO loads.

 

What we did was move these deployment methods off of the storage environment, and over to each server in the vSphere/View cluster environment. It may seem revolutionary, but in much the same way that Storage vMotion caused the storage related process to alleviate the workload off of where it should never have been (the ESX server) this process leverages ZFS technology and the raw horsepower of the ESX servers to deploy desktops; from the server architecture in a much more efficient manner. In so doing, we’ve made this most significant change to the difficulties faced by the VDI deployment far more efficient and put that load where it really should have been in the first place.

 

Screen Shot 2013-10-14 at 8.29.27 AM

 

We didn’t stop there. We made a new overlay methodology into the view interface, which streamlines the process for the administrator, making their choices easier and fewer so that when a series of desktops are to be provisioned, the process to do so is so very simple that it takes just a few simple steps, and a few simple choices to do so.

 

We weren’t done, though. If your administrator attempts to deploy too many desktops for your environment, the provisioning tool will actually let them know that the environment will not be able to handle it and why. This predictive analysis is simply built into the tool, and takes away so much of the guesswork from the job of the administrator.

 

And yes, you’ve guessed it. We weren’t yet done. We included another very cool feature from our storage environment that adds some massive functionality. In this we’ve built in on-line real-time analytics into processor, network and storage insight using tools like IOMeter. And, these are easily viewable from within our dashboard at a glance.

 

And while this works best with our own storage infrastructure, another key detail is that we can work very well with existing back-end storage in play as well. So, the customer may not need to perform a rip and replace when moving on to a full deployment.

 

This is a huge product for us, and I am thrilled to be moving from my role as a dedicated storage SE to a newly created role at Nexenta to one of an SE dedicated to this product and its future iterations. Expect to see more capital on this ingenious product in the future.

 

I look forward to working with you, and to the VMware End User Computing team to deliver these solutions to you moving forward.

Storage Differentiators, and Food for thought from Duncan Epping

Duncan

Duncan Epping@DuncanYB

Duncan Epping, was voted recently as one of the top 100 “Cloud Experts on Twitter” by the Huffington Post: http://goo.gl/9aodcl and a blogger at Yellow-Bricks.com, and is an architect at VMware’s R&D department.

Well, Duncan did it again… He got me thinking with a very simple query in a tweet a few weeks ago. I happened upon it late at night. It got me thinking about all the “Storage 2.0” vendors out there, Nexenta included, and what we’re doing to manage this critical aspect of what it means to be a storage vendor, and how this will drive the market moving forward.

As an SE at Nexenta, I think about the mode of differentiation often. The worst argument at which I can ever arrive, and the one to which I choose to always go last is that of price. In fact, I choose to do this only if the customer or salesperson drives the conversation that way. Otherwise, I’ll bring it up last. While it can be a very big determining factor in the conversation regarding why an organization may choose to go with one vendor over another, it also may provide only the smallest of rationales. In all fairness, the conversation should be one of technical requirements first, and not money, for if the storage doesn’t resolve the technical requirements, then no cost savings can really make sense.

*) On to the technical decision making process. Many of these decisions are based on existing technical issues and infrastructure, and many are or will be due to future requirements. The requirements may be infrastructural, potentially a VDI project coming up, or maybe because some architecture that already is in place. For example, the previous storage may have been 8Gb fibre, and for that reason, there’s no impetus to replace that fabric with Ethernet.

So, what kind of connectivity do you need? This will lock out many pure NAS devices. Many NAS vendors have no support for FibreChannel, or maybe the desire for 16Gb FC has a role in your environment. Though this technology is emerging, there may be a need in your RFI for 16Gb, and if so, this may even lock out some traditional SAN providers. In some ways, the same issues apply in terms of Ethernet connectivity. If you’re looking for iSCSI, and desire 40Gb Ethernet, this could be an issue on many infrastructures.

*) Do you have a protocol that you require, and is that a difficult requirement? For example, is there an application that requires a specific version of SMB? Or, is your requirement an “Object Storage” file system, like Swift or Nova? These have emerged in recent years, and many storage subsystems do not support these protocols. However, it would be a wise to check the vendor’s roadmap, as your implementation on any of these above technologies may very well correspond with the storage vendor’s support of the same. There’s a great WIKI article that links to all the FileSystems currently available, and some additional specifications regarding them: http://en.wikipedia.org/wiki/Comparison_of_file_systems

*) Scalability is another issue. Many environments have the ability to scale outward, and others can scale upward. To scale upward would mean that a single infrastructure might accommodate larger and larger size environments. Thus, those environments usually can be incorporated into single environments that are more sizable. That doesn’t necessarily mean that these systems can handle filesystems that scale upwards. All it really means is that the size of the discs capacities have the ability to grow larger. In an ideal world, these would correspond, and thus the need for viable research by the purchaser.

*) Scalability outward is a different issue. In the outward vein, you, as a purchasing considerer need to understand that the overall storage size of your environment and the future growth becomes a huge issue in the consideration of this. Today, your requirements for storage could be 500Tb, but down the road, you may need upwards of 5Pb. Will your choice in vendor scale to that size? Can they federate their management layer to take care of all of these disparate devices into one distinct infrastructure? This too is a distinct consideration. Again, today, it may not be a huge issue, but as your environment is growing, your need for this may become more of an issue, so Federation as a roadmap item will be very important.

*) Perhaps one of the biggest issues is the ability to handle IO. Many older environments, or legacy storage environments have retrofitted their arrays to handle burst and sustained IO, write versus read IO thrust under different loads, and at different sizes can very possibly be the single biggest determining factor in the choosing of a storage environment. The one thing that I draw attention to in this conversation is the difference from a pure backup environment and a VDI environment. If you imagine that the former would have huge storage sizes with relatively low IO requirements, while the latter would very likely have low storage requirements, but at scale the IO requirement would very likely be hugely significant. This is not a trivial assessment.

I’ve done many presentations about the IO issues surrounding VDI. Imagine that at a 50/50 read write ratio, your POC runs 100 virtual desktops at 25 IOps per desktop. That’s a total of 2500 IOps, and only 1250 read and equal write. This can easily be accomplished on spindles. With 10kRPM discs (roughly 100IOps per disc), this is no more than 25 disc dedicated to the project. That’s not a huge investment in a POC. But scale that up to the full deployment of 10,000 VDI devices, and suddenly your environment becomes far different. At peak, that’s 25,000,000 iops, or 12.5MIOps read and the same write at peak. The question is how would a spinning disc environment handle that?
So the newer “Storage 2.0” firms have more modern file-systems that handle the IO load in different ways. My personal favorite (and a disclaimer is that I work for a firm that produces a product based on it) is ZFS. Usually, these leverage Solid State Disc, or PCIe cards with incredibly fast IO capacities to put spinning disc to shame. These have historically been cost-prohibitive, but at this point, while still more pricey than spinning disc, they’ve become quite a bit more commoditized and are in many ways an awesome form of leverage against the requirements of some of these workloads.

Then, the question is, how do these firms leverage these Solid State technologies? In some cases, the firm will build arrays that are exclusively SSD, or PCIe based. These are amazingly fast, but tend toward lower on the spatial capacities. Other firms will build architectures that have tiered storage architectures that leverage SSD for read and write, while leveraging spinning disc for the bulk storage. This tiering puts the IO where it is best utilized, and the bulk storage where it is best used. Again, the buyer should understand their needs, and ensure that the architecture must match their requirements.

**) Small plug for ZFS: The architecture ZFS employs supports both of the above environments: All SSD, as well as tiered. This flexibility and unified architecture creates a huge bonus for many customers.

*) Tunability is another important concept within storage tiering. So, sure, you’ve built your architecture as a finely designed, and well-contained environment. But, what happens when things change? You’ve decided to add an HPC database, or a VDI environment to your suite of corporate apps. Can you easily modify the architecture to accommodate for the heavily increased IO load even if you’re not necessarily increasing the requirement for space? Many of these environments may not have the ability to do this. This is a key consideration and one that I feel makes for compelling rationale when considering a vendor.

*) What about replication, or DR capacities? How difficult or costly will it be to ensure that your environment can replicate to another site? What if your storage environment at the other location is the same vendor? What if it’s a different vendor’s solution? How will you accommodate for that? What are the costs for a solution for this disc-to-disc replication? What kind of speeds or fault-tolerance methodologies can you anticipate given your bandwidth or distance limitations? Does your vendor of choice charge for their replication software? Do they give you the option to selectively choose which segments of data within the environment are replicatable? Or, for that matter, do they force you to replicate the entire volume?

*) While we’re on that subject, what is your chosen vendor’s methodology for embracing OpenStack? Will they, or for that matter, do they have support for Nova, Swift, or Cinder? Do they have this support in their roadmap and what is the timeframe? Most organizations have no plan on what their move to the cloud will actually be. For that matter if they’ll even need a cloud-storage model. But if they do, will this be important? And if it is, how will your storage vendor accommodate for this?

*) Perhaps, one of the most practical issues that a potential customer may face is that of licensing models. I find that one of my biggest pet peeves is when a vendor’s storage appliance goes “End-of-Life” and you’ve decided to replace it due to hardware being outdated, or lease expiration for example, as a customer, if you choose to replace with the same vendor’s equipment, you end up having to pay for the licensing all over again. In my opinion, this simply isn’t fair. Why is it that you can’t simply pay for fresh hardware and move your licenses over to that new hardware? Well, this is a very important issue, is it not?

Are your SSDs the weakest link or is it your file system?

My colleague Michael Letschin @mletschin wrote this post for our corporate blog, and asked me to provide a bit of color and a bit of editing. I was honored to have even a small part of this educated perspective.

Image

The latest in storage trends today rely on flash or solid state storage, which seems a great method to speed up applications, speed up virtual machines and overall, to make your system run faster; but what happens when that same fast SSD fails? Jon Toigo does a good job explaining SSD failures here – http://goo.gl/zDXd2T. The failure rate on SSD, because of the cell technology involved is of huge concern. Many companies have looked for ways to solve this: from adding in wear leveling, or cell care, or even adding capacity that is not advertised just to have cells to put data writes that are new, while deleting the old blocks in the background. This is completely dependent on the drive manufacturer to save your disk.

Now the important question – Did you get a choice when you bought your enterprise storage as to what manufacturer’s SSD were your “fast” drives? Unlikely, and, without it you wouldn’t know if your drives will be the fast rabbit that never slows down to win the race, or the one that stops on the side of the road which could easily be overtaken by the tortoise.

This is a situation in which using a ZFS based file system like Nexenta could help not only solve, but let you know exactly what you have, and how you need to manage the life span and speed of your enterprise class storage. Nexenta is based on the ZFS file system, and uses commodity drive solutions, so the first problem of not knowing what drive you have is instantly solved, because you can now use best of breed, SSD or flash, and replace them as newer technology arises.

The real secret sauce comes into play when you combine the best in class SSD protection with a file system built to optimizethe usage of the SSD, opting to use DRAM as much as possible and isolate the read and writes needed for normal usage. ZFS utilizes the hybrid storage pool for all data storage. ZFS inherently separates the read and write cache, each using its own SSD such that it should be selected specifically for the use case. SSD wear is more commonly known for write operations, in ZFS, the ZIL, or ZFS Intention Log handles this.

ImageFor ZIL drives it is recommended to use SLC (Single Layer Cell) drives or RamSSD, like ZeusRAM. SLC drives have a much lower wear rate. To see an analysis of different types of SSD look here – http://goo.gl/vE87s. Only synchronous writes are written to the ZIL, and only after they are first written to the ARC, (Adaptive Replacement Cache) or the server’s DRAM. Once data blocks are written to the ZIL a response is sent to the client, and data is asynchronously written to the spinning disk. The writes from the client are not the only SSD writes done.  When using a tiered storage methodology, blocks of data must be written to the read cache prior to being read. This is the case with ZFS and hybrid storage pools also, however the differentiator is how often the blocks are written to the L2ARC, Layer 2 Adaptive Replacement Cache. The L2ARC is normally placed on MLC or eMLC SSD drives and is the second place that the system looks to find data blocks that are commonly used after the ARC/DRAM. It is not uncommon for other files systems to use a similar approach, however they use least recently used (LRU) algorithm. This does not account for if the data blocks may be used on a frequent basis but a large data read is done, from a backup for instance, and the blocks are shifted. The algorithm used for ARC and L2ARC accounts for these blocks and maintains data blocks based on both most recently as well as most frequently used data. Specifics are found here – http://goo.gl/tIlZSv.

The way that data is moved from and to SSD with ZIL and L2ARC is impactful not just for the wear time on the SSD but also on power consumption, that is paramount in the datacenter of the future.  Using this approach allows systems to be built using all SSD footprints and minimal power, or even slower drives for the large capacity, while maintaining high level performance.

In many ways, the tortoise and hare analogy plays well in what we’ve been discussing. Leveraging the power of SSD, and the proper ones at that, will allow you to place the sheer power and lean nature of the Hare, while employing the economy and efficiency of the Tortoise. This, in a way, is the nature of ZFS. Power and economy wrapped up in one neat package. The real magic comes from the ability to tune the system either upward or down in an effort to get it to perform just the way you’d like it to. This is easily done simply by adding or removing SSD to the mix either in ZIL or L2Arc capacities.

There’s nothing inherently wrong with being a tortoise, but, to be a hare, well designed, and performing at peak efficiency, but also enduring for the entire race, really seems like the best way to go, doesn’t it?