vault-solutions.com

Harnessing the Power of Information Governance

Blog

Archiving Data from Exchange Online

by Leave a Comment

Email Header with envelope and title

For many customers that currently have an email archiving system in place, such as VeritasTM Enterprise Vault, they often decide to discontinue mailbox archiving when they migrate to Exchange Online (Office 365).  One reason is that Exchange Online has mailboxes that typically come with a 100 GB size limit. 

If they still have a need for journaled copies of all emails for compliance reasons, they may decide to archive that data, but use SMTP archiving with Enterprise Vault.  They can do that on premises or use a managed solution such as bluesource’s EV247. 

But even with that, there are some use cases where they still need to archive data directly from the mailboxes in Exchange Online.   

  1. Voicemail 
  1. Calendar items 
  1. Moving away from Exchange Online Archive mailboxes 
  1. Dis-satisfaction with Office 365 eDiscovery and Compliance tools 

Let’s talk about points 1 and 2… voicemail and calendar items. 

For voicemail, let’s take a real-world case study.  We have a customer, a state government here in the US, that was using a combination of journaling with Enterprise Vault and mailbox archiving.  They migrated their mailboxes to Office 365 and switched to SMTP archiving.  They still use Enterprise Vault on premises, and Exchange Online sends all copies of messages sent and received to an SMTP address that is picked up by Enterprise Vault on premises. 

They have a regulatory need to respond to Freedom of Information Act (FOIA) requests, and as part of that requirement, they must make voice message available for request.  Their voice mail system is Cisco Unity Connection which puts copies of voice message in users’ mailboxes.  In the past, when the customer used Enterprise Vault Mailbox Archiving, these voice messages ended up in the archives so they were available for FOIA requests.  However, Enterprise Vault does not support mailbox archiving from Exchange Online, which left them in a bit of a bind when it came to the voice messages, which don’t go through the journaling process.  

To meet this demand, we’ve built the functionality into our software, Archive Accelerator, which will be shipping this month. This new functionality addresses Calendar Items as well, which also escape the journaling process and as a result aren’t found in the SMTP archives. As more and more customers come to us with the need to archive data directly from Office 365 for a variety of reasons, we’re committed to investing in enhancing this functionality and accessibility. 

If you are currently finding similar challenges in archiving to meet your needs, let us know! We’re happy to help! 

Stay safe! 

Enterprise Vault and the California Consumer Privacy Act (CCPA)

by Leave a Comment

Back in May of 2018 the General data Protection Regulation (GDPR) went into effect in the EU (see https://www.vault-solutions.com/gdpr-time-know-dont-know/).  This regulation applies consistent protection of data across the EU nations. 

On January 1 of this year, the California Consumer Privacy Act (CCPA) when into effect. 

The CCPA grants new rights to California consumers, including: 

  • The right to know what personal information is collected about them
  • The right to delete personal information held by businesses 
  • The right to opt-out of the sale of personal information 

It applies to legal entities that do business in the state of California and that have either: 

  • Gross annual revenue of more than $25 Million, 
  • Information on more than 50,000 consumers or households, or
  • Derive more then 50 percent of their annual revenues from selling consumer’s personal information/devices 

If you are a business that falls into this category, you must provide a way for consumers to request that you disclose what information of theirs you have, the category of information (e.g. Identifiers, Biometric), the source of the information, and the specific pieces of personal information you have collected. 

To respond, you will likely be scouring numerous internal data sources such as CRM and ERP systems, email, file shares and databases of customer information.   

If you have Veritas Enterprise Vault, you may need a way to search your archives. When searching Enterprise Vault as a larger company, you typically want to use Discovery Accelerator or the eDiscovery Platform.  In either case, you will be performing a manual search. 

Once more consumers realize they can request this information, the number of requests will increase to the point that it may be desirable to automate the process.   We’re working with one company now to provide them with an API into Enterprise Vault that would allow them to do just that and expect more companies will want to automate these searches in the future.  

Let us know if this is something that interests you!

An Interview with the Codefather, Alan Emtage

by Leave a Comment

One of the most useful aspects of the internet is the ability to search the whole thing in a split second.  Most people do that with Google or Bing these days.  Back when I worked at Digital Equipment Corporation, I used AltaVista which was a favorite search engine before Google came along.  AltaVista was developed at Digital back in 1995.  It was the first search engine used in Enterprise Vault, which was also developed at Digital.  (Enterprise Vault is used by many companies to archive their email and file systems, and provides a way to search the archived data, though it has since moved on to a different search engine.) 

But let’s go back even further… to 1990. At that time, there was an internet, but not yet a way to search what was out there.  Everything was done by using command line tools like Archie, Usenet and Gopher.  That is when I first used Archie… who’s name, by the way, comes from the word Archive without the ‘v’.    The ability to search this awesome new world of the internet so quickly was a game changer.  Archie was built by a student at McGill University in Montreal named Alan Emtage, who was working on his Masters at the time.  In 2017 Alan was inducted into the Internet Hall of Fame.  

Our own Tyler Reed recently caught up with Alan in Alan’s native country of Barbados and asked how it all started.  Many thanks to Tyler and Alan for doing the interview. 

At just under 6 minutes, it’s a short and sweet snippet of Internet history… Enjoy! 

For more about Alan’s history, check out this profile from 2013: 

https://www.caribbean-beat.com/issue-122/alan-emtage-codefather#axzz6CphzkEUx

There is still a web version of Archie maintained for posterities sake located here: 

http://archie.icm.edu.pl/archie_eng.html

Follow Up: Migrating Enterprise Vault Journal Data to Office 365

by Leave a Comment

This is a follow-up to my previous somewhat controversial post on Migrating Enterprise Vault data to Office 365.  I say controversial because there are people who agree with the strategy in that post, and those who strongly disagree.  The gist of that post was that when migrating journaled email from Enterprise Vault to Office 365, we suggest that the data be split and migrated to a mailbox per sender/receiver.  That means, if you have email from 10,000 employees in your Enterprise Vault Journal, you should migrate the data to 10,000 mailboxes, even if that means created shared mailboxes for ex employees. 

The typical Objections are: 

  • You don’t want to create thousands of shared mailboxes
  • It takes too long to search on shared mailboxes
  • You can’t put a legal hold on a shared mailbox
  • Because of duplicate messages in Enterprise Vault, you will end up migrating multiple copies of some messages to the same shared mailbox

So, with this follow up, I’ll address these objections. 

You don’t want to create thousands of shared mailboxes. 

This is something that should be automated (we provide a script to automate this.)  You don’t pay for the infrastructure to do it, since Microsoft does not charge for shared mailboxes.  Only mailboxes that people need to log into need a license.  You can do eDiscovery on shared mailboxes without the need to log into them. 

It takes too long to search on shared mailboxes. 

This is complaint I heard recently from a veteran in the business.  He told me of a company that had 6,000 mailboxes to search, and it took Office 365 over a week to do it.  I was surprised at that, until he told me this was a few years ago.  Microsoft continues to pour resources into its eDiscovery capabilities, and it is documented that, based on performance information they have collected from searches run by all Office 365 organizations, the average time to search 10,000 mailboxes is 4 minutes.  So, while it may have been slow several years ago, Microsoft has apparently taken full advantage of their own cloud services to dramatically step up their game. 

You can’t put a legal hold on a shared mailbox. 

Using the Exchange Admin Center, you’ll see this when you try to turn on litigation hold for shared mailboxes: 

Or, if you are using the ‘classic’ version of the Exchange Admin Console, it will appear to let you set it, but then give you this error when saving the changes: 

The workaround for this is to convert the shared mailbox to a regular mailbox temporarily, then put it on hold, then convert the mailbox back to being a shared mailbox.  The hold will remain.  You do have a wait some number of hours after giving a license to the mailbox, so this also needs to be automated for large environments. 

Because of duplicate messages in Enterprise Vault, you will end up migrating multiple copies of some messages to the same shared mailbox 

Duplicate messages can occur in your in the journal archives for a number of reasons.  Depending on how Exchange, journaling, and Enterprise Vault were configured, some environments may end up with multiple copies of many messages that were sent to a list of users.   

You need to make sure the migration software you are using takes this into consideration.  If the software was designed for this scenario, it should have a way to deal with this.  In the case of our Archive Accelerator Restore application, we take an inventory of all senders/recipients of the journal messages in a database and then only send one copy of each unique item to each relevant mailbox. 

Summary

I hope this clears up any confusion or controversy on how, with the right tools and the right process, migrating your Enterprise Vault data to Office 365 can be done efficiently so that you can take full advantage of the eDiscovery tools Office 365 has to offer.

The Magic Button: Outlook Add-In

by 1 Comment

“Holy crap!”, stated a customer as I was demonstrating one of our products. I wasn’t sure if he thought a feature I was showing him was a horrible idea or a great one!   

Mass Restore Solution

Customers from different industries, can have completely different requirements of a product.  In this case, I was showing him an Outlook Add-in that can be used in a unique way to do a mass restore of Enterprise Vault archived items back to folders in an Exchange mailbox. 

Quite a few law firms around the world use Enterprise Vault. Many of them also use a document management system to manage retention, such as NetDocuments ndMail or iManage Work. They typically have a way within Outlook, to push information from their mailbox into the document repository.  One of the problems if they have mailbox management with Enterprise Vault, is that a lot of the messages they want to put into the document management system could be in the form of Enterprise Vault ‘stubs’, aka ‘short-cuts’.  These stubs are not the full message and they never include the attachments. To resolve this issue, they will typically instruct their users to ‘restore’ the items from Enterprise Vault before putting them in the document management system. In Enterprise Vault, they can do that with a (non-magical) button in Outlook that looks like this:   

There are two problems with this: 

 1) If there are a lot of messages in the folder/subfolders of a case, it can be very tedious to restore all the items.  

2) Restoring the items leaves a copy in Enterprise Vault.  Some law firms don’t want multiple copies of important messages. They want the document management system to be the system of record.  If the document becomes expired there, they don’t want another copy to exist elsewhere (in Enterprise Vault, for example).   

Problem Solved

So, we came up with an Outlook Add-in which solves these two problems. It looks like this: 

Press it, and you get the option to restore all short-cuts in the current folder and its subfolders, as shown in this animated GIF: 

Magic Button

We have a customer that has been using this button for over 9 years. They coined it the ‘Magic Button’ because it saves their attorneys so much time. It is crucial for them. If anything goes wrong (issues with updates to Enterprise Vault, etc.) we hear about it fast. 

So, when I told this potential new customer that pressing the button would restore all items in the folder and subfolder, potentially thousands of items, he said “holy crap!”.  My first thought was that he was concerned it would fill up the user’s mailbox. That was not the case.  He was excited because he had been looking all over for a tool that would do this. This made my day and I realized we need to do a better job of getting the word out about this seemingly simple product that makes people’s daily tasks much easier.  Hence this blog post! 

Migrating Enterprise Vault Journal Data to Office 365

by Leave a Comment

If you’ve been journaling email from Microsoft Exchange to Veritas Enterprise Vault and are moving to Office 365, you have a few options about what to do going forward in terms of eDiscovery: 

  1. Switch to SMTP archiving and continue to use Enterprise Vault, either on-premise or hosted in the cloud. 
  2. Export the data from Enterprise Vault and transfer it to a cloud service such as Archive360 or VeritasEV.Cloud 
  3. Export the data  
  4. Migrate the existing data to Exchange Online and use the eDiscovery and Compliance capabilities of Office 365 

Though all options are valid, in this post, I’ll be talking in more detail about the last option. 

For example, you have 7 Enterprise Vault journal archives that are associated with 7 journal mailboxes in Exchange. These archives contain the email for all users that sent or received messages in your Exchange environment. The 7 mailboxes were only used as a conduit for the journaling of messages.  

One idea would be to create 7 shared mailboxes in Exchange On-line and migrate the 7 archives to these shared mailboxes.  Unfortunately, that isn’t going to work. It’s not possible to migrate the contents of these 7 journal archives to 7 individual mailboxes of any kind, even if unlimited archiving is enabled for the type of mailbox known as an archive mailbox.  

Let’s look at the Storage Limits across the Office 365 options… 

Feature Office 365
Enterprise
E1 		Office 365
Enterprise
E3 		Office 365
Enterprise
E5 		Office 365
Enterprise
F1 
User
mailboxes 		50 GB 100 GB 100 GB 2 GB 
Archive
mailboxes 		50 GB Unlimited Unlimited Not
available 
Shared
mailboxes 		50 GB 50 GB 50 GB 50 GB 
Resource
mailboxes 		50 GB 50 GB 50 GB 50 GB 
Site
mailboxes 		50 GB 50 GB 50 GB Not
available 
Public folder
mailboxes 		50 GB 100 GB 100 GB Not
available 
Group
mailboxes 		50 GB 50 GB 50 GB 50 GB 

Chart information up to date as of October 13th, 2019 and is subject to change 

You may look at this and see that the Archive mailboxes can have ‘Unlimited’ storage, thinking it would make a good place to put what are likely terabyte size archives. There are a few problems with that: 

  1. Microsoft says that “Using archive mailboxes as a means of storing mail from multiple users or entities is prohibited.”  That is a deal breaker for this option 
  2. To get past 100 GB in an Archive Mailbox, you need to enable auto-expanding archiving, which can take up to 30 days to take effect. 
  3. An auto-expanding archive has a max growth rate of 1 GB per day, so to add 1 TB would require nearly 3 years! 
  4. There is a 1 TB limit, on top of the 100 GB available in an auto-expanding archive 
  5. See #1

“Microsoft reserves the right to deny unlimited archiving in instances where a user’s archive mailbox is used to store archive data for other users.” 

Given the above points, it’s clearly not possible to use ‘Unlimited’ archive mailboxes for the purpose of storing data from a large journal archive. 

“Using journaling, transport rules, or auto-forwarding rules to copy messages to an archive mailbox for the purposes of archiving is not permitted. A user’s archive mailbox is intended for just that user. “ 

From a storage standpoint, it is important to split the journal data per mailbox to put it into Office 365.  Why? As we will see later, this will also give you much better search performance when doing eDiscovery. 

“Using archive mailboxes as a means of storing mail from multiple users or entities is prohibited.” 

The approach we take to transition from Journal archives to individual mailboxes is to determine which messages belong in which mailboxes and then copy the messages to one of the following: 

Active Mailbox – If there is room 

Archive Mailbox – If they are set up and active 

Shared Mailbox – If the users no longer work for the company, meaning the users are no longer active.  Note: In addition to being available for eDiscovery, there’s no charge in Office 365 for shared mailboxes (since the user can’t log in).

The architecture of Exchange Online.   

Exchange Online is essentially Exchange 2016 with each companies’ mailboxes spread across many different mailbox databases.   Each of those mailbox databases is shared with other companies.    This explains why there are limits to both the size of each mailbox, and the rate at which it is possible to put messages into each mailbox.  The goal is to prevent disruption to the other mailboxes being stored in the same mailbox database. 

 “Auto-expanding archiving is only supported for mailboxes used for individual users or shared mailboxes with a growth rate that doesn’t exceed 1 GB per day.”  

If you reach the 50 GB limit for a shared mailbox, you can assign an E3 license to then have a 100 GB limit.  If that’s not enough, you can enable the Online Archive option for that mailbox to provide an additional 100 GB.  If this still isn’t enough, you can enable unlimited archiving to provide a further 1 TB of space.  

Doing eDiscovery on your migrated data 

The way eDiscovery is done in Office 365 is very different than Enterprise Vault.  With Enterprise Vault, if you are expecting to do eDiscovery, the chances are you are using journaling, in which case you would typically use Discovery Accelerator to search the journal archives for users email, specifying search criteria such as custodian, date range, and keywords. 

In Office 365 you specify the search criteria, but rather than point to the journal archives, you specify the mailboxes associated with the custodians.  It is possible to have Office 365 search all locations.  You may think searching thousands of mailboxes would be slow, but Microsoft claims the average time to search 1000 mailboxes is only 45 seconds. (Based on performance information they have collected for searches run by all Office 365 organizations.) 

Here is a table of the average Search times in Office 365, according to Microsoft: 

Number of mailboxes Average search time 
                                       100  30 seconds 
                                     1,000  45 seconds 
                                   10,000  4 minutes 
                                   25,000  10 minutes 
                                   50,000  20 minutes 
                                 100,000  25 minutes 

Summary 

For those customers using Enterprise Vault journaling and planning to move to Office 365, it is important to rethink how data is stored to play nicely with the Exchange Online architecture.  If done properly, you too can take full advantage of the powerful and efficient eDiscovery capabilities in Office 365.