What is Equinix? And Why Are They So Important??

Inside an Equinix data center. via Equinix

[NOTE!: This is an evolving post for my research purposes to learn more about Equinix, this backbone of the internet(s).]

Equinix is probably one of the most important companies that most people have never heard of.

Do a quick ‘man-on-the-street’ style survey asking passers-by of the Big 5 of Tech, and surely “Equinix” is not top of mind. Nonetheless, Equinix is arguably just as important as the more well-known tech titans, if not more so.

Equinix is a multinational company that runs a global network of large, industrial-scale data centers on every continent with the exceptions of Africa and Antarctica.

Equinix helps take all the necessary IT infrastructure demands of enterprises and, basically, digitizes it. Leading key players in the tech industry harness the tools & solutions provided by Equinix as the underlying foundational core of their infrastructural architecture. Kind of like a cloud for the B2B set. B2B?!? Ok, don’t worry, let’s break this all down!

What is a data center?

Data centers are these vast, huge, sometimes-sprawling buildings dedicated to holding the back-end computer systems, machinery and associated components needed to help run what we consumers know of as “the internet”. In a way these data centers are “the cloud” that we think of when we say our info is “stored out there…in the cloud”, and these data centers can use as much electricity as a small town.

Just over 2 decades old, the company Equinix was founded in 1998 by Al Avery and Jay Adelson. The idea was to create a data-centric platform that would enable even possibly competing clients to leverage a “network effect” to connect and share data traffic. In effect, by using pooled resources the network creates a better benefit for all clients. It’s mutually beneficial for all round, YAY!!!!

A publicly traded company on the Nasdaq stock exchange and a component of the S&P 500, Equinix is headquartered in Redwood City, California with 8,700 employees as of 2020 and annual revenues exceeding $5 billion for the past three years since 2018.

Equinix is currently the largest data center colocation provider in the world. Equinix owns and operates a network of over 220 IBX (International Business Exchange) data centers in 63 major metros globally, allowing for easier interconnection. IBX is a term self-designated by Equinix to “define a world-class data center” boasting “the best in physical security and the best in operations”[1]. The company has a customer base of more than 10,000 clients (B2B!) including more than 260 Fortune 500 companies(!!!), and a “six 9’s” uptime (represented as an uptime greater than six 9’s, i.e. 99.9999%).

Also, to emphasize, Equinix is a business to business company. B2B is an all-encompassing term for businesses that create products and services geared toward other businesses, and not individual retail consumers. They don’t have time for the trivial pursuits of such rabble & riff-raff.

What is cloud? What is hybrid cloud? The term “cloud” can be a tricky thing. Cloud can be thought of as many different things depending on the person and circumstances. In the corporate enterprise environment organizations have multiple deployment options, including private cloud, public cloud, on-premise or hybrid cloud. A hybrid cloud model is a mix of a company’s existing on-premise systems with private/public cloud resources and “as-a-service” packaged resources, creating a unique and tailored offering. This approach lets clients choose their preferred aspects of each various cloud resource. We as consumers have currently have a plethora of cloud services from the Big Tech companies to choose from including: Apple (iCloud), Amazon (AWS), Google (Google Cloud Platform), Microsoft (Azure), and IBM Cloud among others. The individual consumer could also go through a platform like Bluehost or GoDaddy and set up their own infrastructure by a renting a server and other backend services to cobble together their own cloud. Also, just like an individual consumer can do that, the bigger companies have had the same idea.

Generally, cloud refers to the on-demand delivery of computing resources including storage space, processing power, and bandwidth. Cloud has quickly grown in popularity with both the business set and consumers alike, especially after the tumultuous year that was 2020 and the spiking rise in remote-work demand due to Covid-19.

Heck, even State-Sponsored hacking groups are increasingly using cloud infrastructure[5]!!! (See here! This is also one of the points against data centers, but we’ll get to that part later.)

What is a Tier 1 network?

A Tier 1 network is “an IP (Internet Protocol) network that can reach every other network on the Internet solely via settlement-free interconnection”[2] (that is, they can reach the entire internet via settlement-free peering). Relax! We’re going to unpack all that.

You can think of Tier 1 networks as having a kind of “golden VIP pass” with each other, and each Tier 1 network can exchange traffic with other Tier 1 networks without paying any fees for the exchange of traffic in either direction. (This contrasts with some Tier 2 networks and all Tier 3 networks which must pay to transmit traffic on other networks.)

Universally recognized Tier 1 networks include AT&T, Verizon, T-Mobile US (formerly Sprint), Deutsche Telekom, and Orange.

NOTE that there is no authority that defines tiers of networks participating in the Internet:

It can be difficult to determine whether a network is paying for peering or transit, as these business agreements are rarely public information, or are covered under a non-disclosure agreement. The Internet peering community is roughly the set of peering coordinators present at the Internet exchange points on more than one continent. The subset representing Tier 1 networks is collectively understood in a loose sense, but not published as such.
Common definitions of Tier 2 and Tier 3 networks:
Tier 2 network: A network that peers for free with some networks, but still purchases IP transit or pays for peering to reach at least some portion of the Internet.
Tier 3 network: A network that solely purchases transit/peering from other networks to participate in the Internet.
via Wikipedia

How many of Equinix’s 220+ data centers were acquired from Tier 1 ISP’s? What advantage does that have (purchasing a data center from a Tier 1 ISP) to both parties Equinix and the Tier1 ISP?

Equinix’s purchase of 29 Verizon data center colocation facilities in April 2017, is an example of a Tier 1 acquisition. The Verizon acquisition gave Equinix an advantage by extending Equinix’s reach in key markets as well as in the government and energy sectors[3]. And Verizon benefited from being able to outsource that part of their operations which, as we’ll see, numerous industry leaders in addition to Verizon have also done.

Colocation data center is aka a multi-tenant data center. The most commonly referenced or thought of kind of data center. A facility operated by the data center provider where companies, end-users, or tenants can leaser power and space.

Also in the news, and on the other end of the Tiered spectrum, Equinix paid $161 million in 2020 to acquire GPX India and its two Tier 4 data centres in Mumbai. “GPX said its data centres host 12 telcos, over 130 ISPs, three internet exchanges, eight cloud service providers, as well as a number of content delivery networks and over-the-top content providers.” [4]

Why would companies such as Apple, Microsoft and others outsource to Equinix?

While many companies [can] build their own server farms, often located in remote areas, they use Equinix because of the fast and easy interconnectivity they can have with other providers in prime locations around the world. Equinix calls it its ecosystem. “The reason customers go with Equinix is because within Equinix’s data centers, there are lots of carriers and network providers. They go there and they pay a premium because they value being able to connect with all these different networks and service providers,” Weller said.
via Nasdaq

All the benefits of a server without having to physically host the server. All the benefits of an infrastructure without the upfront costs of physical equipment to buildout the infrastructure. Equinix offers the solutions and in scalable ways for companies. Individual consumers started flocking to “cloud” services to host them and help enable the rise of individual journalism and social media. The consumer no longer had to worry about the high barrier of upfront costs associated with the back-end stack infrastructure, outsourcing all that to focus on their main front-end projects, be it journalism or whatever. In that same way, big tech companies are finding they no longer want to or wish to deal with a physical infrastructure if they don’t have to. Data centers have shifted from being just “server farms”, to now being able to offer fully-enabled, architected and engineered hybrid digital infrastructures for companies.

While companies could choose to build and run their own data centers, the benefits of hiring Equinix to do this back-end infrastructure work has apparent appeal with numerous industry behemoths including Apple, Amazon’s AWS, Microsoft Azure, Facebook, Netflix, AT&T, T-Mobile, NASDAQ, Bloomberg, GE, eBay and Nokia, all cited as clients. A list so impressive it gives pause to think of the reliance that has been built upon Equinix. (Intentional pause added here.) Ok, let’s move on.

Data centers offer base rates of power, space, cooling and physical security. That is the base minimum and then other services are usually offered on top and can be added in an à la carte fashion. Other than power space and cooling, managed services like firewall management may be added.

In addition to providing the physical infrastructure, Equinix has data centers that are certified to meet rigorous environmental and energy management standards, another headache companies will be glad to not be hassled with. For example, the data centers in the NY metro area tout being compliant with NIST 800-53/FISMA, HIPAA, PCI-DSS, and ISO 27001.

This is another lure of the Equinix ecosphere. The company Equinix deals with the relevant regulatory, legislative and political hurdles that are always omnipresent yet extremely varying from different geographic regions. Equinix’s enterprise clients don’t have to bear that burden.

In addition, Equinix has the expert staff of engineers, technicians and customer care specialists to offer technical and logistical support 24 hours a day, 7 days a week all working to further contribute to that 99.9999% (Six 9’s!) uptime.

Not to Mention That Network Effect! The benefits and appeal of having so many networks operating out of one data center and pooling resources is mutually beneficial all around. Take this quote from Morningstar’s website (touting one of Equinix’s rival’s, Digital Realty Trust, but still) nicely summing up the power of the network effect:

Data centers that have many network providers connecting to the buildings are extremely valuable to tenants, which need to connect with networks to reach the rest of the world, and are extremely costly to replicate. Data centers that currently have dozens or hundreds of networks connecting to them are likely to remain their regions’ leaders because of the unlikelihood of having so many network providers building into new locations when an alternative already exists. These network-dense data centers are most likely to draw the cloud providers’ on-ramps, and the cloud providers then draw their customers to the same properties.
We believe the best attribute a data center can have is network density. In our view, high network density provides a huge competitive advantage because it is extremely costly and difficult to replicate. It also leads to a network effect moat source, as other network service providers and cloud providers typically need to connect with many other networks, so they are drawn to locations where many partners are present. Cloud providers in turn draw their customers, which use cloud on ramps to connect with them.[6]
via Morningstar

What is an Internet exchange point (IX or IXP)? An Internet exchange point is:

“the physical infrastructure through which Internet service providers (ISPs) and content delivery networks (CDNs) exchange Internet traffic among their networks (autonomous systems) and peer together. Typically, IXPs occupy standalone buildings with its own switches. IXPs reduce the portion of an ISP’s traffic that must be delivered via their upstream transit providers, thereby reducing the average per-bit delivery cost of their service. Furthermore, the increased number of paths available through the IXP improves routing efficiency and fault-tolerance. In addition, IXPs exhibit the characteristics of what economists call the network effect.”
via Wikipedia

What is interconnection? Why is it helpful? Data center interconnection is a model in which the assets within a multi-tenant data center are directly connected, meaning a hard-wired connection ( usually over fiber) and in a peer-to-peer fashion. An interconnect agreement is “a business contract between telecommunications organizations for the purpose of interconnecting their networks and exchanging telecommunications traffic. Interconnect agreements are found both in the public switched telephone network and the Internet.”[7].

(So this allows Equinix to be the intermediary of different companies, possibly competitors, sharing resources for the mutual benefits of all parties involved, right? Right.) The main advantages of direct interconnection are lower cost, lower latency (less lag) and greater bandwidth.

In the public switched telephone network, an interconnect agreement invariably involves settlement fees based on call source and destination, connection times and duration, when these fees do not cancel out between operators.
On the Internet, where the concept of a “call” is generally hard to define, settlement-free peering and Internet transit are common forms of interconnection. A contract for interconnection within the Internet is usually called a peering agreement.
Interconnect agreements are typically complex contractual agreements involving payment schemes and schedules, coordination of routing policies, acceptable use policies, traffic balancing requirements, technical standards, coordination of network operations, dispute resolution, etc. Legal and regulatory requirements are often an issue. For example, network operators may be forced by law to interconnect with their competitors. In the United States, the Telecommunications Act of 1996 mandated methods of interconnection and the compensation models for doing so.
via Wikipedia

Basically, interconnection is a process whereby telecom operators can handle calls and data for other (possibly even competitor) operators. This allows the end-users (the customers!) of both operators to communicate with each other, despite them having different networks. This is obviously a mutual benefit for all involved, including the underlying industry that all parties have an interest in seeing succeed.

What is Peering? And What is De-peering?

Anyone who was around during the early 2000s will recall the wireless wars and things like peak/off-peak, daytime/nighttime/weekend minutes, and the kicker of “in-network” calls where you could speak at a lower (or free) rate to contacts who had the same mobile network provider. Ah, memories!!

These “peering agreements” are deals betweens ISPs to create a direct link to route each other’s data packets rather than both paying a separate third-party network service provider for transport.

Interconnections are helpful for the purpose of exchanging traffic between the users of each network.

The pure definition of peering is settlement-free, also known as “bill-and-keep,” or “sender keeps all,” meaning that neither party pays the other in association with the exchange of traffic; instead, each derives and retains revenue from its own customers.
via Wikipedia

Public peering is accomplished across a Layer 2 (that’s the Data Link level layer) access technology, generally called a shared fabric [8].

At these locations, multiple carriers interconnect with one or more other carriers across a single physical port. Historically, public peering locations were known as network access points (NAPs). Today they are most often called exchange points or Internet exchanges (“IXP”). Many of the largest exchange points in the world can have hundreds of participants, and some span multiple buildings and colocation facilities across a city. Since public peering allows networks interested in peering to interconnect with many other networks through a single port, it is often considered to offer “less capacity” than private peering, but to a larger number of networks.
via Wikipedia

By definition, peering is the voluntary and free exchange of traffic between two networks, for mutual benefit. If one or both networks believes that there is no longer a mutual benefit, they may decide to cease peering: this is known as depeering. Some of the reasons why one network may wish to depeer another include:
A desire that the other network pay settlement, either in exchange for continued peering or for transit services.
A belief that the other network is “profiting unduly” from the no-settlement interconnection.
Concern over traffic ratios, which is related to the fair sharing of cost for the interconnection. A desire to peer with the upstream transit provider of the peered network.
Abuse of the interconnection by the other party, such as pointing default or utilizing the peer for transit.
Instability of the peered network, repeated routing leaks, lack of response to network abuse issues, etc.
The inability or unwillingness of the peered network to provision additional capacity for peering.
The belief that the peered network is unduly peering with one’s customers.
Various external political factors (including personal conflicts between individuals at each network). [16]

I think of it this way…A peer is another person on my level. We’ve all had peers growing up in school and in the workplace. Well, as it turns out, large corporate companies have peers too! And in the telecommunications world a certain clique/posse/gang/group of companies called “Tier 1” providers have full access to all avenues of these innanet’s either by way of their own networks or by “peering” with other providers they deem to be on their level.

When both companies agree and respect each others status, then things are all good. But. If one player (Player #1) decides that the other player (Player #2) isn’t holding up their end of the deal, or that they aren’t bringing as much to the table, then Player #1 may decide to “depeer” with Player #2.

This can have bad consequences for the innocent bystanders (the customers of both companies who are the ones that end up getting screwed).

What is a ‘fabric’ or a ‘data center fabric’? What is the difference between ‘fabric’ and ‘FABRIC™’? Fabric seems to be the name of the platform or solution that Equinix offers. My initial thought is that the name “Fabric” is an attempt by the company to convey the meaning of this solution being woven into the infrastructure and that pieces are stitched on and off (or in and out) as needed in an almost quilt-like nature. The underlying fabric is the base material that holds everything together. Just like how cotton is “The Fabric of Our Lives®” (seriously, check the domain name), so Equinix offers this “Fabric of Our Internets®”, if you’ll allow. (Hmmm…also…what happens if a thread becomes loose in a real-world fabric like cotton? It unravels. Hmmm…so, what happens to a virtual fabric?? Just saying.)

My inclination is that the word ‘fabric’ here can be used in multiple ways such as the generic version of the term and the trademarked version.

I did find a rather informative PDF about Data Center Fabric Fundamentals[9] while researching and this seems to be a great read. They offer the following definition:

A data center fabric is a system of switches and servers and the interconnections between them that can be represented as a fabric. Because of the tightly woven connections between nodes (all devices in a fabric are referred to as nodes), data center fabrics are often perceived as complex, but actually it is the very tightness of the weave that makes the technology inherently elegant.
A data center fabric allows for a flattened architecture in which any server node can connect to any other server node, and any switch node can connect to any server node (server refers to both compute and storage).
The flattened architecture of fabrics is key to their agility. Data center fabric architectures typically use only one or two tiers of switches as opposed to data centers that implement multi-tier data center network architectures.
In almost all data center fabric architectures traffic can be transmitted between server nodes by traversing a set number of switches, which results in extreme efficiency and low latency (a few microseconds of latency at each hop can become seconds of latency per transaction).
The data center fabric architecture model unites holistically all data center resources from processor cores to memory—servers, storage, the network, and peripherals. Its architecture is designed to handle the increase in east-west traffic while maintaining north-south traffic connectivity to end users.
via Juniper Networks

Anyway, Equinix seems to go with the latter trademarked version with their Equinix FABRIC™ offering. Equinix FABRIC™ is “an on-demand, SDN-enabled interconnection service that allows any business to connect between its own distributed infrastructure and any other company’s distributed infrastructure, including Equinix’s own Platform Equinix”[10].

Equinix’s FABRIC™ allows its customers to connect with confidence, on demand, and to everything everywhere. Or at least that’s how they tout their offerings[11].

Equinix FABRIC™ offers their clientele the ability to connect digital infrastructure and services on demand at software speed via secure, software-defined interconnection.

What is a REIT? Is Equinix a REIT? What are the tax advantages of a REIT? Are Equinix’s competitors also REITs? Aside from the tax advatages, what are other benefits of being a REIT? What does it do for them?

Equinix is a REIT. A REIT is an acronym for Real Estate Investment Trust, and it’s basically a vehicle that generates income. REITs are companies that own, operate and/or finance income-generating real estate. Established by Congress “in 1960 as an amendment to the Cigar Excise Tax Extension…The provision allows investors to buy shares in commercial real estate portfolios–something that was previously available only to wealthy individuals and through large financial intermediaries”[12]. This is real estate that exists solely for the purpose of generating income, making it possible for individual retail investors to earn dividends and benefit from real estate investments, while still shielding themselves from the burden of having to directly buy, manage or finance any properties themselves. (Ok, sidebar, but this right here reminds me of how data centers make it possible for individual companies to benefit from data center infrastructure, without having to buy, manage or finance any physical infrastructure themselves.) REITs can operate across a range of different property sectors–not just data centers but think things like apartment buildings/complexes, hotels, commercial office buildings, healthcare facilities, warehouses, self-storage, and retail centers including malls. However to qualify as REITs these real estate companies have to meet a number of requirements, including paying a minimum of 90% of its profits back to shareholders via dividend distributions. But REITs are also chock full of useful tax advantage benefits unavailable to many other investment tools for both individual investors and the REIT company itself. And even more so after a significant tax update in 2018 (more here and here). Equinix has become a REIT that is generating this income and also seems to continue to experience phenomenal growth, a winning combination that has made data-centric REITS, especially Equinix (but also rival competitors which include Digital Realty Trust, CyrusOne, CoreSite, Verison, Rackspace and CenturyLink). Digital Realty Trust, CyrusOne, Coresite is a REIT. Another interesting figure to think about is that “the 15 largest data center colocation providers in the world own about half of the market. The remaining half is extremely fragmented, which means more consolidation will follow.”[13]

As of today, the largest data center colocation provider in the world is Equinix, accounting for 11 percent of the $54 billion market, according to the latest global leaderboard by Structure Research, an analyst firm that tracks the internet infrastructure services market.
Following Equinix is Digital Realty, with 8 percent market share, and following Digital is China Telecom, with 6 percent market share.
via Data Center Knowledge

The next quote from that article venture off from Equinix but I include it because I find it extremely interesting:

It’s important to note that China Telecom is one of five Chinese companies on the leaderboard (also China Unicom, China Mobile, GDS, and 21Vianet), all of whom do business primarily in China. China’s market is so vast that these providers can stay mostly domestic (with some international presence) and still have huge share of the global market.
China’s protectionist regulatory policy makes it extremely difficult for foreign companies to compete in the country’s vast data center market, and international players’ interest in China has waned. As a result, Chinese hyperscalers’ explosive growth in recent years has driven huge growth for Chinese companies that build and operate data centers for the likes of Alibaba and Tencent.
via Data Center Knowledge

(I include that part about China as a tidbit for something to further look into.) But back to this thing about the 15 largest data center colocation providers in the world own about half of the market. The remaining half is extremely fragmented, which means more consolidation will follow. An example of one of these smaller players in this market is Flagship Solutions Group.

What does it mean to be Vendor Neutral? Are they really Vendor Neutral? To be Vendor Neutral, or Vendor Neutrality, seems like it should mean no specific preference to any certain vendor or no bias towards any certain particular vendors, or not favoring one vendor over another. Indeed, vendor neutral means a “product or specification that is not proprietary and controlled by one company. Open source software was conceived to avoid allegiance to a single vendor”[14]. PC Mag encyclopedia goes on to make note, “However, an ‘open system’ is not entirely vendor neutral as the foundation platform may be controlled by one company”. (Can Equinix really be vendor-neutral? Do they even claim to be? I think they claim to use the best possible route to create the most mutually beneficial outcome for all parties involved, no??? In effect, I guess, can their network itself be proprietary. Like essentially, it would be their “little black book of contacts” (the “network effect” they call it) that is the proprietary thing that would allow them to control the conversation and steer the ship, if you will) Vendor neutrality should provide a level playing field for all participating vendors. Vendor neutrality should ensure broad compatibility and interchangeability and interoperability of vendors and technologies. This kind of approach prevents reliance on any one particular vendor (smart!), and prevents any one vendor from controlling or gaming the system (equally smart!).

A point against data centers being in control of all this digital infrastructure is attacks against them. As threat groups start to use cloud infrastructure more themselves, using the same infrastructure as their target can make exploitation easier.

“If you find a tactic that works against Microsoft Azure, for example, they can apply that same tactic or technique to any organization that uses that same technology,” says Dennis Wilson, global director of SpiderLabs at Trustwave, a security services firm. “In the past, a company may have had this vendor for a firewall and this vendor for an [endpoint detection and response] solution, but now you have a lot of companies using the same cloud infrastructure, so now it becomes a cookie-cutter approach for the attackers to exploit companies across that infrastructure.”[5]
via Dark Reading

Questions: There seems to be the obvious of all the reliance that seems to be placed on Equinix. What if, they went down (due to a variety of reasons from malicious to nature-borne). As said earlier, Equinix handles more than HALF (260+) of the list of Fortune 500 companies(!!!). Also, CAN a virtual infrastructure really be as secure as a lone computer thats never interacted with the broader internet?? Similarly can something like what they offer say an internal infrastructure, would that really be as secure as two computers connected at the Layer 2 Data Link layer????? I have my questions and concerns, I do.

Images via Equinix

Sources:

[1] http://info.equinix.com/rs/equinixinc/images/Why_Equinix_Wins_Partner_Playbook_UAE.pdf

[2] https://en.wikipedia.org/wiki/Tier_1_network

[3] https://www.equinix.com/equinix-acquires-verizon-data-center-colocation-facilities/

[4] https://www.zdnet.com/article/equinix-parts-with-161-million-for-gpx-india/

[5] https://www.darkreading.com/threat-intelligence/state-sponsored-hacking-groups-increasingly-use-cloud-and-open-source-infrastructure/d/d-id/1339030

[6] https://www.morningstar.com/articles/982137/digital-realty-is-our-new-favorite-data-center

[7] https://en.wikipedia.org/wiki/Interconnect_agreement

[8] https://en.wikipedia.org/wiki/Peering

[9] https://www.juniper.net/documentation/en_US/learn-about/LA_DCFabricsFundamentals.pdf

[10] https://www.prnewswire.com/news-releases/equinix-to-expand-canadian-operations-with-us750-million-acquisition-of-13-bell-data-center-sites-301068269.html