The Cloud, Server Consolidation, Ephemeral Data, and Information Security: (Manageable) E-Discovery Challenges Facing CIOs and InfoSec

Today’s chief information officers and chief information security officers face challenges with a wide range of technological issues: cloud storage, server consolidation, retention of ephemeral data, data security, and privacy laws, to name a few. On top of that, electronic discovery ¹Electronic discovery, more commonly known as e-discovery, is the component of U.S.-based legal actions whereby parties (and in some cases, non-parties) are compelled to exchange information (in particular electronically stored information) relevant to the claims and defenses asserted in the lawsuit. The Federal Rules of Civil Procedure, the Federal Rules of Evidence, and state and local procedural rules have been amended to address the unique challenges posed by ESI in the context of a lawsuit. For example, in December 2006, Fed. R. Civ. P. 26(a) was amended to add “electronically stored information” as its own category of discoverable information; Rule 26(b)(2) sets up a two–tier discovery process for accessible and inaccessible data, including procedures for cost shifting on inaccessible data. Rule 26(b)(5) and Fed. R. Evid. 502 both address the increased risk of inadvertent production of privileged documents given the volume of ESI to be reviewed, and Fed. R. Civ. P. 26(f) mandates that parties meet and confer prior to the commencement of discovery in order to agree on review and production protocols. invades nearly every aspect of these issues.

Why are CIOs and CISOs asked to balance the sometimes competing needs of cost-effective technological solutions for business needs against legal preservation and collection requirements (a.k.a. e-discovery)? Simply: Because they have to.

Issues Facing CIOs and CISOs
That May Impact E-Discovery

Cloud Computing.

Today’s CIOs are storing more data in more places than ever before. In 2011, 90.9 percent of data center sites surveyed by the Association for Computer Operations Management used more storage space than they did three years ago. ²AFCOM Survey Examines Data Center, Cloud Computing Use. http://www.tmcnet.com/channels/colocation/articles/171139-afcom-survey-examines-data-center-cloud-computing-use.htm (last visited Dec. 17, 2012). During that same three-year period, 37 percent were able to reduce their staff, and 29 percent kept their staffing levels the same. ³IT Workloads Up As Data Center Technology Added Without More Staff. http://www.infoworld.com/d/data-center/it-workloads-data-center-technology-added-without-more-staff-186 (last visited Dec. 17, 2012). According to AFCOM CEO Jill Yaoz, this trend is in large part due to the development and implementation of new tools and processes that have allowed IT departments and data centers to store massive amounts of data efficiently and inexpensively. Specifically, IT departments and data centers are increasingly relying on cloud-based systems for the storage of big data.

The advent of cloud-based storage systems has had a profound impact on the way businesses collect and store their information. By offering the ability to store large amounts of data remotely, cloud storage not only enables CIOs to drastically reduce their IT budgets, but also allows IT departments the freedom to concentrate on matters more central to business growth, such as high-value activities and innovation. The number of data centers implementing cloud-based storage is only expected to increase. According to Yaoz, “[o]ur prediction is that 80 percent to 90 percent of all data centers will be adopting some form of cloud computing in the next five years.” ⁴ Id.

Yet, e-discovery concerns may require businesses to maintain comprehensive records on what data is stored in the cloud and where, precisely, that cloud exists. This information may become important in the context of e-discovery if data is located in a foreign jurisdiction with privacy laws restricting or “blocking” the transfer and/or storage of data. ⁵See, e.g., Directive 95/46/EC, of the European Parliament and of the Council of 24 October 1995 on the Protection of Individuals with Regard to the Processing of Personal Data and on the Free Movement of Such Data [hereinafter Directive 95/46], 1995 O.J. (L281) (EC), available at http://eurlex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:31995L0046:EN:HTML (prohibiting transfers of data to other countries that do not provide adequate levels of protection) (last visited Dec. 17, 2012), and Working Document 1/2009 on Pre-Trial Discovery for Cross Border Civil Litigation, at 8 (Feb. 11, 2009) [hereinafter E.U. Working Document], available at http://ec.europa.eu/justice/policies/privacy/docs/wpdocs/2009/wp158_en.pdf (noting that “[t]he mere or unsubstantiated possibility that an action may be brought before the U.S. Courts” may not be sufficient to impose preservation obligations.) (last visited Dec. 17, 2012).

In addition, some cloud service providers employ subcontractors to store data. This may be problematic for businesses ordered to produce data in a timely manner, especially if the requested data is located in more than one place with more than one subcontractor. In other words, compared to pre-cloud storage, businesses may lose control over the timing and execution of the “extraction” of data.

At a recent conference in Palo Alto, Calif., Google CIO Ben Fried addressed the importance of streamlining various cloud storage providers tapped by the same company to ensure easy and efficient integration—and to prevent what he characterized as a widespread lack of good corporate practices in the context of remote data storage. ⁶Top CIO Challenges: Cloud, BYOD, Big Data, available at http://www.cioinsight.com/c/a/Expert-Voices/Top-CIO-Challenges-Cloud-BOYD-Big-Data-490657/ (last visited Dec. 17, 2012). Bask Iyer, CIO of Juniper Networks, added that businesses must be mindful of employee use habits so as to ensure that employees utilize the company’s chosen storage service providers. ⁷Id.

Simply put, because of the ease associated with storing data in the cloud, businesses might assume the same is true for producing data from the cloud. This assumption, if untrue, might complicate a company’s attempt to comply with e-discovery demands: Data may be in a foreign location with privacy laws, manpower, and technology that impede the search, storage and extraction of data in the method or time required. Also, data for one custodian may not necessarily be grouped together on one server, and a group of custodians might be dispersed amongst many servers.

Moreover, issues can arise with respect to the notification and imposition of a company’s preservation obligations when information is stored in the cloud as opposed to on a company’s own servers.

Admissibility issues can also be triggered if the chain of custody for the various document sources is not accurately tracked and documented. In other words, the data is quite simply not all in one room; it may be in many shifting rooms, which brings us to our next topic: server consolidation.

Server Consolidation.

At the same time that CIOs are looking to reduce spending by outsourcing data management and retention to the cloud, they may also be looking to consolidate internal servers to one location, or if it is a world-wide company, to one or several “regions.” ⁸As noted above, foreign data privacy laws must be taken into consideration when implementing decisions about data storage. In the event of U.S. litigation, data stored in a foreign country will likely not be as accessible in response to U.S. discovery demands. Moreover, foreign data privacy laws may restrict the retention of certain types of data, such as personal information, and may impose restrictions on whether and how long data can be retained, making compliance with preservation obligations under U.S. case law problematic. See, e.g., the discussion of the Zubulake case, infra; Directive 95/46, supra and E.U. Working Document, supra, note 6, at 8. For a detailed analysis of the implications surrounding the imposition of U.S. litigation holds in foreign jurisdictions, see Kenneth N. Rashbaum, Matthew Knouff & Melinda Albert, “U.S. Legal Holds Across Borders: A Legal Conundrum?”, 13 N.C. JOLT 68 (Fall 2011) available at http://www.ncjolt.org/sites/default/files/Art_Rashbaum_Knouff_Albert_69_94.pdf (last visited Dec. 17, 2012).

The analysis a CIO might consider may typically include the following factors:

1. Cost savings with consolidation—will we reap them?
2. Business continuity—is there a risk of consolidating, i.e. if the servers in one location experience an interruption, how (and how long) will that impact the company? (Think power loss, storms/natural disaster, war/civil unrest.)
3. Data privacy laws—how might they be impacted by consolidation? How can and should I align disparate laws and regulations?
4. Security—how can I ensure it?

It is difficult to answer these questions in a vacuum, especially because no two companies’ needs are the same and because local counsel in differing jurisdictions are usually required for full analysis. While no U.S. court is known to have directly weighed in on the pros and cons of server consolidation, one court did recently refer to server consolidation in the context of a larger e-discovery dispute.

In the patent infringement action, Apple v. Samsung, ⁹ Apple Inc. v. Samsung Electronics, 2012 BL 188250 (N.D. Cal. July 25, 2012); Motion for Relief from Judgment granted in part by Apple Inc. v. Samsung Electronics, 2012 BL 280780 (N.D. Cal. August 21, 2012). the parties sought adverse inference jury instructions regarding spoliation of evidence. Responding to Apple’s motion, the court considered whether Samsung took adequate steps to prevent the spoliation of relevant data after it should have reasonably anticipated litigation. At issue was Samsung’s default email system, called mySingle, which automatically deletes all emails after a two-week period, unless otherwise specified by the employee user.

According to Samsung, it relied on the automated deletion feature for several business reasons. First, automated deletion can help to prevent the misappropriation of confidential business information in the event a computer is lost or stolen. Second, a two-week deletion period is less expensive to implement than a 30-day deletion period. Third, the shorter window means that confidential business information is less likely to be disclosed to a third-party via email or unauthorized access. Fourth, a two-week automated deletion period best complies with Korean privacy laws.

Apple argued that Samsung’s continued use of its bi-weekly email destruction policy without any methodology for verifying whether Samsung employees complied with Samsung’s litigation hold instructions demonstrated inadequate data retention methods resulting in spoliation of relevant evidence. The court agreed with Apple, and directed that an adverse inference instruction be used at trial against Samsung. ¹⁰ Apple Inc. v. Samsung Electronics, 2012 BL 188250, supra, at *9. Note however, that this instruction was never given because the presiding trial judge, Federal District Court Judge Lucy H. Koh, overruled Federal Magistrate Judge Paul S. Grewal’s Order, and decided instead to give the same instruction with respect to each of the parties, i.e., that they had each failed to preserve relevant evidence after the duty to preserve arose and “[w]hether this fact is important to you in reaching a verdict in this case is for you to decide.” 2012 BL 309589 (N.D. Cal. Aug. 19, 2012); Apple Inc. v. Samsung Electronics, 2012 BL 280780, supra at *20. This was a significant victory for Samsung which had argued that Apple’s duty to preserve must have arisen no later than Samsung’s. Magistrate Judge Grewal had not reached Samsung’s equal treatment argument initially because he had denied Samsung’s motion as untimely. Ultimately, however, the identical adverse inference instructions were not given at trial because “… the parties indicated at the August 20, 2012 hearing that if the Court decided to issue identical adverse inference instructions against both parties, they prefer that neither adverse inference instruction be given.” Id. at *21.

Although the opinion does not directly address the issue of server consolidation, the court does refer to the fact that all of Samsung’s email was consolidated on servers in Korea. “[Samsung’s email] contains a ‘general guideline [that] calls for all emails to be automatically deleted after the passage of two weeks.’ This functionality operates and stores email company-wide in Korea [and] has no exceptions ….” ¹¹Apple Inc. v. Samsung Electronics, 2012 BL 188250, supra, at *3. Without elaborating on its opinion of the server consolidation, the court did take the time to make this further statement: “[Samsung’s email platform] stores received and sent employee emails on company-wide servers, as opposed to dividing the servers by business unit ….” ¹²Id.

From these references, some might wonder whether the court intended to imply that a company might better manage its preservation duties by not consolidating all functionality in one location. In other words, is the court asking whether Samsung could have lifted the 14-day deletion policy for certain relevant business units and thereby complied with its preservation duties and also not had the burden of storing all emails, company-wide? ¹³Although arguably it may seem to be less burdensome to implement a hold where, as here, servers are consolidated, Samsung filed affidavits claiming that to extend “the retention policy for its employees would have cost an additional $35,983,193 per year.” Id. at *3 note 43. However, Samsung “did not estimate the cost of temporarily moving key custodians’ email accounts to unique servers that do not biweekly destroy emails, or the cost of temporarily moving …” these custodians’ accounts to Microsoft Outlook. Id. In addition to cost, data privacy laws may also affect a company’s data storage decisions. As noted above, Samsung averred that their consolidated email storage system policy was the one that “best complies with Korean privacy law.” Id. at *3.

Ephemeral Data and Data Security.

The management of cloud-based data and server consolidation is not the only consideration for a company in litigation. Ephemeral data—such as temporary internet files, frequently overwritten or automatically updated metadata (such as last-opened files), log files (such as firewall or other security related log data), RAM, and fragments of deleted files in unallocated (slack) computer space—may be possible sources of responsive data.

Two recent cases address the issue of ephemeral data in the context of e-discovery. In Nacco Materials Handling Group v. Lilly Co., ¹⁴Nacco Materials Handling Group v. Lilly Co., 278 F.R.D. 395 (W.D. Tenn. 2011). a lift truck manufacturer brought an action against a former dealer alleging that the dealer improperly accessed the manufacturer’s secure website.

The court held that the dealer had an obligation to initiate the preservation of relevant ESI—including ephemeral data such as internet browsing histories—immediately upon receiving the manufacturer’s complaint. Because the dealer failed to preserve the relevant data, the court imposed sanctions, specifically ordering the reimbursement of plaintiff’s costs for forensic examination, as well as the cost of imaging and analysis of the manufacturer’s hard drives.

In Tener v. Cremer, ¹⁵Tener v. Cremer, 931 N.Y.S.2d 552 (N.Y. App. Div. 2011). the court considered whether a non-party should be compelled to produce temporary user logs, which were deleted through the course of normal business operations. After finding that defendant posted defamatory statements using a computer owned and operated by New York University, plaintiff served the school with a subpoena seeking the names of all of the individuals who accessed the Internet on the date the statement originated.

Plaintiff argued that NYU could use special software to retrieve the data, despite NYU’s claim that the data had been overwritten numerous times and therefore could not be retrieved, at least not without undue burden or cost.

The court articulated a two-pronged inquiry to determine whether NYU would be compelled to produce the requested data. First, the court considered whether the data sought is recoverable. Second, the court engaged in a cost-benefit analysis to determine if the needs of the case warrant the data sought. Here, the court remanded the case to determine whether the data sought was accessible and whether it could be produced without undue burden or cost.

Although ephemeral data is certainly not necessary in most cases, considering the high costs associated with these forensic discovery methods, parties should consider whether ephemeral data is a source of information that might be responsive to a given litigation. ¹⁶If so, consideration should be given to admissibility issues, particularly with respect to foreign ephemeral data, such as whether such data can be authenticated and/or would fall under the Business Records exception to the hearsay rule, FRE 803(6). The Sedona Conference Commentary on ESI Admissibility notes, for example, that when a system, instead of an individual, is the creator or owner of a document or file, as when logs of certain activity are automatically created by an operating system, it may be difficult to provide foundational evidence as to how and where the data was created. The Sedona Conference Commentary on ESI Evidence and Admissibility at 11-12. (March 2008), available at https://thesedonaconference.org/publication/The%2520Sedona%2520Conference%25C2%25AE%2520Commentary%2520on%2520ESI%2520Evidence%2520%252526%2520Admissibility (last visited Dec. 17, 2012).

Another nuance in the preservation of ephemeral data is the need to sometimes preserve firewall, anti-virus, intrusion prevention and/or detection, network activity, and other logs of information that might show the path a hacker might have taken. The preservation, of course, might be for a forensic investigation or litigation needs, but may very well be different from the normal course of business.

Parties faced with this type of preservation request are often asked if they have centralized logging, and if not, how quickly the overwriting of logging can be disabled. Any CISO concerned with being prepared for investigating security incidents might want to consider how to respond to an urgent request to preserve this type of data. ¹⁷Again, as noted above, foreign data privacy laws may operate to restrict the ability to collect and preserve foreign ephemeral data, by for example, limiting that which is collected to “that which is relevant and not excessive to a particular investigation.” Directive 95/46, supra, note 6, at art. 6; E.U. Working Document, supra, note 6 at 10.

What Is the Legal Department Faced With?

As noted in the discussion of the Samsung case above, several courts have held that it is counsel’s duty to affirm that a party has complied with its applicable discovery obligations; several note that counsel’s duty does not end with the issuance of a litigation hold. ¹⁸Under Fed. R. Civ. P. 26(g), counsel of record must certify that all disclosures are correct and complete and sanctions may be imposed for non-compliance with this requirement.

Courts have also held that counsel cannot rely on a client’s assertions that all relevant information has been preserved and collected. Therefore, counsel may seek to work directly with IT personnel, many times below the rank of CIO, CISO, manager, or director, who have direct responsibility for and knowledge of relevant data sources, both on site and in the cloud.

For example, in a data security matter, counsel may wish to speak directly with the person who manages the firewall logs. Then, counsel and IT professionals might work in tandem to assist with the implementation of the litigation hold by identifying the appropriate custodians and other data sources.

Case Law.

In the seminal case on this subject, Zubulake v. UBS Warburg, ¹⁹Zubulake v. UBS Warburg LLC, 229 F.R.D. 422 (S.D.N.Y 2004) (“Zubulake V”). Judge Shira A. Scheindlin imposed sanctions on UBS for failing to guarantee that relevant data was both preserved and produced. Accordingly, she granted plaintiff’s motions for sanctions in the form of an adverse inference instruction with respect to deleted emails and the imposition of certain costs.

In reaching this conclusion, the Zubulake court discussed counsel’s role in ensuring the identification, preservation, and production of relevant information:

A party’s discovery obligations do not end with the implementation of a “litigation hold”—to the contrary, that’s only the beginning. Counsel must oversee compliance with the litigation hold, monitoring the party’s efforts to retain and produce the relevant documents. Proper communication between a party and her lawyer will ensure (1) that all relevant information (or at least all sources of relevant information) is discovered, (2) that relevant information is retained on a continuing basis, and (3) that relevant non-privileged material is produced to the opposing party. ²⁰Id. at 432.

In a recent decision, National Day Laborer Organization Network v. U.S. Immigration & Customs Enforcement Agency, ²¹ National Day Laborer Organization Network v. U.S. Immigration and Customs Enforcement Agency, 2012 BL 176257 (S.D.N.Y. July 13, 2012). Judge Scheindlin strongly admonished against the practice of “self collection,” whereby parties allow individual employees to search for relevant data and applied that caution against government agencies in the context of a Freedom of Information Act case. National Day Laborer Organization Network dealt with plaintiffs’ requests for documents under the FOIA and their assertions that defendant government entities’ searches for such information were inadequate. Although this case involves an analysis of the reasonableness of such search efforts under the FOIA, Judge Scheindlin’s opinion is applicable in the broader context of e-discovery.

Plaintiffs charged that the search efforts of the FBI and Department of Homeland Security were unreasonable. In the context of the FOIA, the court found that the scope of some of the government’s searches were adequate while some were not, noting that it is “impossible to evaluate the adequacy of an electronic search for records without knowing what search terms have been used.” ²²Id. at *1011. Disagreeing with defendants’ argument that custodians should be trusted to run effective searches of their own (i.e., “self-collect”), the court stated:

There are two answers to defendants’ question. First, custodians cannot “be trusted to run effective searches,” without providing a detailed description of those searches, because FOIA places a burden on defendants to establish that they have conducted adequate searches; FOIA permits agencies to do so by submitting affidavits that “contain reasonable specificity of detail rather than merely conclusory statements.” Defendants’ counsel recognize that, for over twenty years, courts have required that these affidavits “set [ ] forth the search terms and the type of search performed.” But, somehow, DHS, ICE, and the FBI have not gotten the message. So it bears repetition: the government will not be able to establish the adequacy of its FOIA searches if it does not record and report the search terms that it used, how it combined them, and whether it searched the full text of documents.

The second answer to defendants’ question has emerged from scholarship and case law only in recent years: most custodians cannot be “trusted” to run effective searches because designing legally sufficient electronic searches in the discovery or FOIA contexts is not part of their daily responsibilities. Searching for an answer on Google (or Westlaw or Lexis) is very different from searching for all responsive documents in the FOIA or e-discovery context. ²³Id. at *1113.

Judge Scheindlin concluded that it was unnecessary for her to evaluate the adequacy of defendants’ search terms. Instead, she ordered the parties to work cooperatively “to agree on additional search terms and protocols—and, if necessary, testing to evaluate and refine those terms. … Defendant agencies, in turn, will need to cooperate fully with plaintiffs.” ²⁴ Id. at *1315.

National Day Laborer Organization Network deals with a number of relevant current e-discovery issues, including the adequacy of keyword search terms and technology-assisted review. Importantly, from the perspective of a CIO, it also demonstrates the technological expertise required to assure the adequacy of a collection effort.

A recent case in the Eastern District of Texas is instructive on this point. In Green v. Blitz, ²⁵Green v. Blitz, 2011 BL 52006 (E.D. Tex, Mar. 1, 2011). plaintiff brought a product liability suit against a manufacturer of gasoline cans, alleging that a gas can had caused her husband’s death. Her main theory of liability was premised on the fact that the can had no flame arrester, which defendants had not incorporated and which they argued did not work anyway.

During the jury deliberations, the parties entered into a high-low settlement agreement under which the parties agreed that, notwithstanding the verdict, the case would be dismissed with prejudice and a release executed. The jury ultimately ruled in the plaintiff’s favor, but under the parties’ agreement, the amount awarded triggered a payment by the defendant on the low end of the scale.

Subsequently, counsel for plaintiff was involved in related litigation during which he learned of “extremely relevant and material” documents that had not been produced in the Green case. Plaintiff filed a motion to re-open the case and for sanctions arguing that defendant’s discovery violations were in bad faith and had led to an unfair outcome.

The court held that the defendant had committed several discovery abuses, including failing to investigate, search for, and produce relevant evidence, and never issuing a written litigation hold order. The sanctions were both steep and creative: The court ordered the defendant to pay plaintiff $250,000 in sanctions, to disclose the court’s order to plaintiffs “in every lawsuit proceeding against it,” and to disclose the order in any case in which defendant is a party for the next five years (a $500,000 sanction would be returned upon compliance).

What was the nature of the defendant’s failings in Green that warranted these extreme sanctions? According to the court, in responding to discovery, the employee “solely responsible for searching for and collecting documents relevant to the litigation,” ²⁶Id. at *94. and who also “headed up the research and investigation around flame arresters,” ²⁷Id. failed to take several steps necessary to discharge a duty to preserve and investigate responsive documents: No litigation hold was issued, key word searches were not run (a search for “flame arrester” was not run and would have garnered responsive information as it was in the subject line of emails), and the individual did not contact defendant’s IT personnel to coordinate with the search and extraction of relevant data. Making matters worse, this employee admitted that he was “about as computer literate-illiterate as they get.” ²⁸Id. at *7.

In light of the undisclosed documents, the court found that defendant’s conduct was willful and plaintiff was prejudiced. The failure to search relevant data was compounded where no coordination with IT occurred and thus preservation was not ensured. Indeed, according to the opinion, a lead member in IT had repeatedly requested that employees delete their emails during “the precise time period” that defendant “was actively defending multiple products liability suits relating to its gas cans and had a duty to preserve evidence.” ²⁹ Id. at *9.

The court also addressed defendant’s failure to halt the overwriting of back up tapes, concluding that “it will never be known how much prejudice against the plaintiff was actually caused by Blitz’s failure to preserve documents.” ³⁰Id. at *10.

While obviously an extreme factual scenario, Green should serve as an example of how valuable coordination with IT professionals can be in the context of e-discovery.

E-Discovery Tips for the IT Professional

In conclusion, the following considerations may prove useful in managing the e-discovery issues posed by big data:

1. Consider anticipating e-discovery issues in advance of litigation, taking precautions such as mapping the location and/or filepaths of relevant data (wherever located, e.g., on-site, in the cloud, etc.)
2. Consider reviewing your cloud storage vendor and subcontractor contracts and ensure that the appropriate contractual obligations are in place with respect to how and by whom stored data are to be preserved, collected, and produced.
3. Consider the above questions concerning server consolidation with appropriate counsel, along with other issues counsel may raise.
4. Consider if education among IT staff is necessary in conjunction with the legal department’s needs.
5. Plan early for preservation in countries where relevant data reside, taking into account local laws that may affect implementation of litigation holds.
6. Document data collection, review and chain of custody, particularly for data in the cloud and outside the United States, in order to lay a foundation for admissibility.