Cleaning Up Active Directory Metadata

I’m not 100% satisfied with the video associated with this post, but I’m including it since I think it’s important to AD Administrators to be familiar with this process.  Yesterday, while trying to do something in my lab, I realized I hadn’t brought one of my DCs online in a very long time, and that the tombstone lifetime for the forest had expired.

Deleted Objects, Tombstones and Garbage Collection

As I explain in the video, when an object is deleted in AD it isn’t actually removed from the directory.  Instead, it’s marked as an object to be deleted, it’s hidden and it’s put in a special deleted items container.  The purpose of this is to allow the change to the object (the fact that it’s been removed from AD) to replicate to all other DCs so they can remove their copy of it from their instance of NTDS.DIT.  This “hidden but not-yet-deleted” object is called a tombstoned object.

Additionally, because Active Directory doesn’t allow objects to remain in a “hidden but not-yet-deleted” state, it assigns a lifetime to that tombstoned object measured in days.  The default number of days that the deleted object will be retained is:

• Windows 2000 (all versions): 60 days
• 2003 RTM: 60 days
• Windows 2003 SP1: 180 days
• Windows 2008/Windows 2008 R2: 180 days

You can see (and, if necessary, change) this value by taking the following steps:

1. Connect to a domain controller using ADSI Edit
2. Connect to the Configuration Partition
3. Navigate to the following path:

CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,CN=<domain name>

Once at this object, right-click CN=Directory Service and select “properties”.  In the list of attributes, scroll down to “tombstoneLifetime” and you’ll see how many days the deleted objects will be retained in your directory before they’re deleted.

Once a deleted object has been in a deleted state for longer than its tombstone lifetime, it will be permanently removed from the directory.  This happens when a process known as Garbage Collection runs.  This process looks for deleted objects that have exceeded the tombstone lifetime.  If one is found, this object is permanently deleted.

By default, the Garbage Collection process runs once ever 12 hours (known as the Garbage Collection Interval), though the frequency of this process can be changed by going to the same object listed above (CN=Directory Service) and modifying the “garbageCollPeriod”.

By default, a longer tombstone lifetime and a longer Garbage Collection Interval will result in a larger NTDS.DIT file, but it might also allow a longer time to recover accidentally deleted objects. 

WARNING!!!

You’ll have to do a bit of experimenting to see the effect of changing these values, though the strong recommendation is to leave them at their defaults unless you have a valid business need AND you’ve fully tested and understand what these changes will mean to your environment.

Exceeding the Tombstone Lifetime

You can probably already tell why this is relevant to Active Directory, but just in case I’ll explain.  Let’s assume that you have two DCs (DC1 and DC2).  In DC1 you create a user named Steve.  Steve’s user object is replicated to DC2.  Then DC2 goes offline due to a problem and during that outage, Steve leaves the company and you delete his user object.  Time goes by and the replacement for DC2 is delayed and it’s quite awhile before you are able to bring it back online (let’s say that it’s 182 days later).  If DC2 were allowed to begin replicating with DC1 again, we’d have a problem.  Why?

Remember that DC2 received a copy of Steve’s user object, but it never received word that Steve had left the company.  Further, since the tombstone lifetime has expired for Steve’s user object, it’s been completely removed from DC1 and thus there is nothing to replicate to DC2.  So we end up with an inconsistent directory.  DC1 has no record of Steve while DC2 still considers Steve to be a valid user object.

To correct for this problem, Microsoft has designed Active Directory so that a DC can no longer participate in replication if it’s been offline for longer than the tombstone lifetime.  This prevents the sort of scenario I’ve just described from happening.

But it also means that if you need to remove that DC from the Active Directory forest, you must use a manual process.

Removing the Failed DC

All of that explanation is background to get us to the video I’ve created for this post.  What I’ve done in the video is start at the point that DC2 has to be removed from Active Directory since it’s exceeded the tombstone lifetime and can no longer replicate with DC1.  Below, I walk you through the steps of removing DC2 from the Active Directory forest using a process known as Metadata Cleanup.

Though I took one step while the video was not running (and thus wasn’t able to demonstrate it), I fully explain it so that you can understand what will happen when you need to do Metadata Cleanup on your own.

Sometimes when things happen, it opens the door to explain a principle that AD Admins should know about.  Hopefully this post was one of those instances.  Until next time…

For further information on this process, the following articles can provide assistance:

• The Active Directory database garbage collection process
• Clean Up Server Metadata