Monday, April 25, 2011

How to download videos from your favorite website automatically everyday using wget

Suppose that you have a favorite website that allows you to download videos in the mp4 format. Suppose that this website is updated everyday. Now, if you, like most people, have an always on internet connection, you might want to automate the task of downloading these videos and watching them at your convenience, or to transfer them to a portable device like a mobile phone or iPad or whatever and watch it later, instead of having to manually "right click, save, right click, save" many, many times. Here's a simpler way.

For our example, suppose that http://www.example.org is a video hosting site and there is a little button "Download" below the videos.

We tell wget to download the page recursively (r), upto 2 levels, without making directories for domains (nd):
wget -r -l 2 -nd http://www.example.org

Now, this will also download all html pages linked with this page. We can exclude them if we want:
wget -r -l 2 -nd http://www.example.org -R *.html

If you have a slow system, make wget use less CPU power:
nice wget -r -l 2 -nd http://www.example.org

Make this process automatically occur everyday at 6 in the morning - open your crontab file using the command
crontab -e

and add the following line
0 6 0 0 0 nice wget -r -l 2 -nd http://www.example.org

No comments:

Post a Comment