Use rsync to delete mass files quickly in Linux

15,360 0
Summary

When faced with deleting a large number of files, such as 100,000 log files, the traditional rm -rf * command can be inefficient. A significantly faster method involves leveraging rsync to quickly clear a directory. This process requires installing rsync, creating an empty temporary folder, and then using rsync --delete-before -a -H -v --progress --stats /tmp/empty_folder/ target_folder/. This command effectively replaces all files in the target directory with the empty source, resulting in rapid deletion.

Yesterday I encountered an issue to delete many log files in my Linux workstation, there were around 100,000 files to be deleted. These are log files and they grow very fast, and we need to delete them frequently. Usually we would use rm -rf * to delete these files, but we may need to wait for a relative long time if there are too many files. So we must adopt some unusual way. Fortunately, we can use rsync to delete mass files in one shot.

1. Install rsync

yum install rsync

2. Create a new empty folder

mkdir /tmp/test

3. Use rsync to delete target folder

rsync --delete-before -a -H -v --progress --stats /tmp/test/ log/

Now the log folder we be cleared if we run this command, the speed is very fast. Actually what rsync does  is to replace all files in the target folder.

Option explanation for command in step 3:

–delete-before : do deletion before the transmission starts in target folder

–progress : Show the transmission progress

-a :Archive mode, use archive method to transmit files

-H keep the hard linked files

-v Verbose output

–stats : Show transmission status of some files

Original author : è°‹ä¸‡ä¸–全局者 Source ： http://www.ha97.com/4107.html

DEMO RSYNC DELETE FILE

  RELATED

  COMMENTS

0

No comment for this article.