What is Dataverse Long Term Data Retention?
Currently in Public Preview (Q1 2024), Dataverse Long Term Data Retention allows your organisation to archive data to long term storage. But what exactly does this mean?
This means that you can achieve a number of goals:
- Reduce your overall Dataverse Database Capacity (more on this later)
- Have more control over data that will only primarily be used for auditing or occasional read-only access. It is designed as an interim stage between having live data, and deleted data. Microsoft calls this ‘retained data’.
- The ability to query retained data on demand.
- Build long term retention policies that will work within your ALM, meaning that you can define as a solution layer and carry your long term retention policies between your higher environments.
Note, that this feature should only be enabled in non-production environments for testing purposes at this stage (see Known Issues).
How do I set up Long Term Data Retention?
Firstly, you need to enable Long Term Data Retention by editing the properties of the table in which you wish to enable Long Term Data Retention.
Once this has been enabled, you can then define Retention Policies from the maker portal. Create a new Policy, select the intended Table, provide a meaningful name for this policy, select a view as your criteria for long term retention, and select a start date along with a cadence.
It is also worth noting that Data Retention policies are also solution aware, so they may be transported between environments and follow your existing ALM processes.
How do I access Long Term Data Retention?
In order to access data that resides in Long Term Data Retention, there are a few ways you may do so. The Advanced Find functionality provides the ability to query long-term data retention (a new option appears allow you to “Change to retained data” in the Advanced Find window).
Note however, you cannot see Retained Data and Live Data together in the same results screen.
In order to do this, advanced users can use the SQL TDS EndPoint (easily accessible from tools such as SQL 4 CDS available on XrmToolBox) and perform a join operation.
What is the impact of Long Term Data Retention on Dataverse Database Capacity?
When Long Term Retention is active, it will still count towards your database capacity as all data (live and retained) still remains in the Dataverse.
So you may ask, what is the point of it, if it still contributes to your database storage capacity? Well because all data in long-term retention is compressed, Microsoft says you can expect about a 50% reduction in storage.
What licensing or other requirements will I need to use Long Term Data Retention?
In order to use Dataverse Long Term Data Retention, you must have:
- A Minimum of 1000 Dynamics 365 or Power Apps Per User license. It may be any combination of the above, so long as the total count is over 1000.
- Additionally, the above licensing must come from the same tenant.
- The Power Platform environment must also be a managed environment.
The intended audience for this feature is mid-size to enterprise-sized organisations that would require such functionality.
Known Issues
Note that this feature is available only for preview in non-production environments.
With all preview functionality, there are some caveats and should not be enabled within Production environments. But in this case, it is worth calling out specifically some of the issues that users may expect to see during the preview phase. The following have been reported.
- The process of moving data to Long Term Data Retention is one-way and cannot be reversed.
- Moving parent records to Long Term Data Retention can adversely impact the behaviour of child records.
- There are reports of dependencies being inadvertently created within Solution packages with this feature enabled.
Hopefully you enjoyed this overview of the upcoming Long Term Data Retention, and provides you with a clear option for managing your long-term storage requirements. Any feedback please do not hesitate to reach out to me.