Use rsync to delete mass files quickly in Linux
Yesterday I encountered an issue to delete many log files in my Linux workstation, there were around 100,000 files to be deleted. These are log files and they grow very fast, and we need to delete them frequently. Usually we would use rm -rf * to delete these files, but we may need to wait for a relative long time if there are too many files. So we must adopt some unusual way. Fortunately, we can use rsync to delete mass files in one shot.
1. Install rsync
yum install rsync
2. Create a new empty folder
3. Use rsync to delete target folder
rsync --delete-before -a -H -v --progress --stats /tmp/test/ log/
Now the log folder we be cleared if we run this command, the speed is very fast. Actually what rsync does is to replace all files in the target folder.
Option explanation for command in step 3:
–delete-before : do deletion before the transmission starts in target folder
–progress : Show the transmission progress
-a :Archive mode, use archive method to transmit files
-H keep the hard linked files
-v Verbose output
–stats : Show transmission status of some files