Understanding the concept
As like previous versions of windows servers, there are two types of defragmentation online and offline. By default, online defragmentation automatically happens in every 12 hours as part of AD's garbage-collection process and this process runs independently on each DC(Read to know more: http://support.microsoft.com/?kbid=198793). However the online defragmentation will not reduce the size of database file(Ntds.dit) and in such cases you need to perform an offline defragmentation to achieve this. Performing an offline defragmentation creates a new, compacted version of the database file and depending on how the database is fragmented the original database file and the new file may be considerably smaller. Remember that because DCs only replicate changes, performing an offline defragmentation on ntds.dit on one DC won't affect ntds.dit on other DCs. Hence, you must manually perform an offline defragmentation on each DC. You can perform the below steps to determine whether offline defragmentation will help you to reduce the database size.
When to perform an offline defragmentation
If you are sure that performing offline defragmentation will reduce the database size or if there are errors which report to perform offline defragmentation skip this step and start from How to perform offline defragmentation.
1.The actual size of the ntds.dit could be easily studied through windows explorer. In windows server 2003 the location of AD database is 'C:\WINNT\NTDS' and windows server 2008 it will be 'C:\Windows\NTDS'. This will help us to determine if the database has any corruption before offline defragmentation.
Note: Active Directory does not use a file level replication, so the file could be of various size on each Domain Controller in your domain.
2.We need to edit one of the registry key which will increase the logging from the Garbage Collection. Open registry editor from command prompt and locate the directory 'HKEY_LOCAL_MACHINE \ System \ CurrentControlSet \ Services \ NTDS \ Diagnostics'. Now double click on 6 Garbage Collection and edit the value from 0 to 1.
3.As like you know Garbage Collection process will occur twice a day. You need to wait for the next online defragmentation(happens as the part of Garbage collection) and check the Directory Service log in Event Viewer. After the online defragmentation time check for logs, Open Event viewer->Applications and services logs->Directory service.
When the online defragmentation is started you can see the event id 700, source Directory Service and when its finished you will see the event id 701, source Directory Service. In between these logs locate the log Event id 1646, source ActiveDirectory_DomainService. Read the description carefully and the size mentioned under Free hard disk space determines the MB that the offline defrag would recover. If the size is negligible then it is not necessary to perform offline defragmentation.
Note: By default a newly created DC will have a database size of 12MB.
How to perform offline defragmentation
Important: It is highly recommended to have latest backup of your server before performing this operation. If anything goes wrong during this operation we can restore it back from our system state backup.
In my case I am defragmenting the database to 'c:\windows\ntds\temp'. You can have the same scenario or can defragment to another location.
Note: Make a copy of the folder NTDS and its content to another directory before starting offline defragmentation.
If the compaction of the database does not work properly, you can then easily restore the database by copying it back to the original location. Do not delete the copy of the Ntds.dit file until you have at least verified that the domain controller starts properly.
1.As the initial stage we need to stop the AD DS services.
From Windows server 2008 onwards these services are restartable during the normal operation and it can be achieved by a single command or stop the services from services console.
Open command prompt and type : net stop ntds and press ENTER. Type Y to agree to stop additional services, and then press ENTER(Refer the link to know more : http://technet.microsoft.com/en-us/library/cc732714(v=ws.10).aspx).
For server 2003 you cannot manually stop those services, we need to boot the server in Directory service restore mode. Reboot the domain controller, select the appropriate installation from the boot menu, and press F8 to display the Advanced boot menu options. Choose Directory Services Restore Mode and press ENTER. Press ENTER again to start the boot process. Log on using the directory service restore account that is defined during the ADDS installation(This will automatically stop the ADDS services and its related services).
2.At the command prompt, type ntdsutil, and then press ENTER.
3.At the ntdsutil prompt, type activate instance ntds, and then press ENTER. This will set "NTDS" or a specific AD LDS instance as the active instance, here we have set to NTDS.
4.At the ntdsutil prompt, type files, and then press ENTER . This will cause NTDSUTIL to switch to the File Maintenance prompt which will help us to manage AD DS/LDS database files.
5.At the file maintenance prompt you need to type info and press ENTER. This will display information about the size and location of the Active Directory database.
You should make sure that the information that is displayed coincides with the size that you recorded earlier by window explorer. Otherwise, some corruption may exist.
6.In order to start the defragmentation at the file maintenance prompt type Compact to c:\Windows\NTDS\temp (which will create the defragment database to 'c:\Windows\NTDS\temp' ).
Note : When you compact the database to a local drive, you must provide a path. If the path contains any spaces, enclose the entire path in quotation marks (for example, compact to "c:\server labs"). If the directory does not exist, Ntdsutil.exe creates the directory and then creates the file named Ntds.dit in that location.
7.A new database named Ntds.dit is created in the path you specified. Here it will be on 'C:\Windows\NTDS\Temp'
8.Type q twice on command prompt to exit . Next, verify that Windows has created a copy of the Active Directory database in the 'C:\Windows\NTDS\Temp' folder. This copy is the defragmented version of the database. To use it, you must either delete or rename the original database, and then copy the defragmented database from 'C:\Windows\NTDS\Temp' to 'C:\Windows\NTDS'.
9.You must also either rename(If space allows) or delete the log files located in the C:\Windows\NTDS folder.
On the command prompt type del <drive>:\<pathToLogFiles>\*.log. Here considering fact that NTDS database and its log files are under the location 'C:\Windows\NTDS', the command will be del C:\Windows\NTDS\*.log and press ENTER.
10.The above steps will complete the NTDS offline defragmentation. Now restart the NTDS service that we have stopped at step 1. Type net statrt ntds and press ENTER.
11.If you have done the above operations in windows server 2003 you need to reboot the server normally and if it is in server 2008 you can continue with normal operations after starting the AD services.
If the offline defragmentation does not help you to reduce the NTDS database size, you can move this database to another disk drive. Read the below article to know more.