Wget follows the links in the HTML and CSS pages and works recursively. Which makes it smart for cloning remote websites and running them on local machines. This is sometimes referred to as “recursive downloading.”

How to Install Wget on Linux

Generally, the Wget package is pre-installed on Linux distributions. But in some cases of minimal installations, It may not be installed.

Open a terminal and execute the following commands to install or upgrade the Wget package from the default package manager.

• On Ubuntu, Debian & Linux Mint Systems

  sudo apt update && apt install wget -y 
  

• On CentOS, RHEL & Fedora Systems

  sudo dnf install wget -y
  

Wget Command Syntax

A simple Wget command follows the below syntax.

wget [option]... [URL]...

A large number of command-line options make it more usable. Wget uses GNU getopt to process command-line arguments, which means all options have a long-form along with a short one.

Wget required a URL to an archive, IOS, or webpage to download.

Wget Command Examples

Here are some frequently used wget commands with examples.

1. Downloading a file using Wget

Open a terminal and type wget followed by the remote file URL to download on the local machine. No additional parameters are required to download a file.

wget https://wordpress.org/latest.zip 

The above command will download the file in the current working directory. The filename will remain the same in the local system as on the remote machine.

2. Downlaod file with a New Name

Default wget downloads the file with the same name on the local system. Use -O (Capital O) command-line option followed by a new name to write file on local system.

wget -O local.zip https://wordpress.org/latest.zip 

See the below screenshot, showing the local file is created with a new name.

3. Downlaod Large files with Resume Option

The Wget allows us to resume downloading for a partially downloaded file. It is helpful for downloading large files from a remote. In any case, the download interrupts, can resume downloading remaining content only instead of full download.

Use -c or --continue switch with the file.

wget -c //mirrors.edge.kernel.org/linuxmint/stable/20.1/linuxmint-20.1-cinnamon-64bit.isop 

See the below example, Downloading a large file with -c option. In the first attempt, once the downloading started killed the download process with CTRL+C. Now again execute the same command and you can see the downloading resumes.

4. Wget to Execute Remote Scripts without Creating Local File

Wget is also widely used for executing remote scripts through scheduled jobs like crontab. But we found that it creates a new file with each run under the home directory. We can instruct wget to redirect all content to /dev/null and ignore creating files.

wget -q -O /dev/null https://google.com 

Here -q will suppress all the output on-screen and -O will redirect all the content to /dev/null file.

5. How to Mirror a Website in Wget

Wget allows us to download website content recursively. It follows the internal links available in HTML content. Use --recursive option with wget command to download entire site in your local system.

wget --recursive https://google.com 

You can also set the maximum depth for recursion with -l opiton.

wget --recursive -l 2 https://google.com 

The above commands will create a directory with the same name to the domain in the current directory and place all files under it.

6. How to Authorize Requests with User & Password

Most of the files over remote FTP servers are secured with authentication. In some cases, content over the HTTP can be secured with authentication. Wget allows us to pass authentication details with this request.

wget --user=USER --password=PASS https://example.com/backup.zip 

You can use --user and --password for both FTP and HTTP authentications.

Conclusion

In this tutorial, you have learned the basics of the Linux wget command with examples.

The post Wget Command in Linux with Examples appeared first on TecAdmin.

The default wget download files under the current working directory. In this tutorial, we will describe you to how to download files to a specific directory using wget.

Using wget -O Option

Use -O or --output-document=FILE option will truncate FILE immediately, and all downloaded content will be written to FILE. Here the wget -O FILE http://path is intended to work like wget -O - http://path > FILE;

For example:

wget -O /tmp/Ubuntu.iso https://releases.ubuntu.com/20.04/ubuntu-20.04-desktop-amd64.iso  

The above command will download the remote ISO file with the Wget command and save it as /tmp/Ubuntu.iso in the local system. No matter what is the current working directory.

Using wget -P Option

Alternatively, Use -P or --directory-prefix=PREFIX. Here the PREFIX is a directory where all other files and subdirectories will be saved to. The default download directory is “.” the current directory.

For example:

wget -P /tmp https://releases.ubuntu.com/20.04/ubuntu-20.04-desktop-amd64.iso

The above command will download the iso file under the /tmp directory.

Conclusion

In this guide, you have learned two methods to download files to a specified directory using the Wget command.

The post Wget – Download files to Specific Directory appeared first on TecAdmin.

Problem

While using the wget with cron job saved the downloaded files under home directory. Due to this a large number of junk files get created in your system.

Solution

Use -O option with wget command to write the result file (data) to specific file and location. Next select /dev/null device file as target file. This will discard anything written to it. In result, no junk files will be created in your home directory.

For example, the original cron job command is:

wget https://www.example.com/cron.php

Update above command to:

 wget -q -O /dev/null https://www.example.com/cron.php

Here:

• -q Turn off wget command output
• -O /dev/null Write downloaded content (file) to /dev/null device.

That’s it. Hope this tutorial helps you to avoid unwanted files on root generated by wget cron jobs.

The post [SOLVED] Cron job wget writing files to root directory appeared first on TecAdmin.

For example – below command will download /remote/dir directory and its subdirectory from example.com ftp server.

$ wget -r ftp://ftpuser:password@example.com/remote/dir/

You can define username and password for authenticated ftp server like below example command.

$ wget -r --user="ftpuser" --password="password" ftp://example.com/subdir/

Additional Parameters
You may used some additional parameters to wget command to use more features in files downloading.

–reject jpg,png – To exclude jpg and png files from downloading.
-nH – avoids the creation of a directory named after the server name
-nc – avoids creating a new file if it already exists on the destination (it is just skipped)

The post How to Recursively Download Files from FTP appeared first on TecAdmin.

Tip 1: Download File using Wget

Below example will download file from server to current local directory.

$ wget https://tecadmin.net/file.zip

Tip 2: Download File & Save to Specific Location

Below command will download zip file in /opt folder with name file.zip. -O is used for specify destination folder

# wget https://tecadmin.net/file.zip -O /opt/file.zip

Tip 3: Download File from FTP

Some time you required to download file from ftp server, so wget can easily download file from ftp url like below.

# wget ftp://ftp.tecadmin.net/file.zip

Tip 4: Download File from Password Protected URLs

Sometimes we required to specify username and password to download a file. While using browser its easy but using command line it doesn’t prompt for login credentials. Below examples will show to how to use username,password while downloading files from password protected sources.

Tip 4.1: Download file from Password protected ftp server.

$ wget --ftp-user=username --ftp-password=secretpassword ftp://ftp.tecadmin.net/file.zip

or

$ wget ftp://username:secretpassword@ftp.tecadmin.net/file.zip

Tip 4.2: Download file from password protected http server.

# wget --http-user=username --http-password=secretpassword https://tecadmin.net/file.zip
or
# wget --user=username --password=secretpassword https://tecadmin.net/file.zip

Tip 4.3: Download file behind password protected proxy server.

$ wget --proxy-user=username --proxy-password=secretpassword https://tecadmin.net/file.zip

Tip 5: Download File from Untrusted Secure URL.

If any download url is using untrusted ssl certificate, wget will not download that file. But we can download it by using –no-check-certificate parameter in url.

$ wget  https://tecadmin.net/file.zip  --no-check-certificate

The post 5 Wget Examples to Download Files on Linux Command Line appeared first on TecAdmin.