How to use wget behind a proxy

If wget is your go-to download command on your Linux servers, and your machines are behind a proxy, Jack Wallen has the solution to get this setup working properly.

securityhacker-istock.jpg

Image: iStock/iBrave

More about cybersecurity

The wget command is one of the best ways to transfer files from a remote server to a local machine without a GUI. I use this tool constantly on Linux servers and it never fails to pull down the files I need. Unless I’m behind a proxy. When that’s the case, if you simply issue the standard wget command, you’ll receive errors and the file in question will fail to download.

What do you do?

Fortunately, the developers of wget considered this and built in the necessary options for using the tool when behind a proxy.

You might think it’s as simple as issuing a command like: 

wget --proxy=PROXY FILE

Where PROXY is the proxy server you’re machine is behind, and FILE is the name of the remote file to download. That’s not exactly how this works. Fear not, I’m going to show you how to make it possible.

SEE: Security incident response policy (TechRepublic Premium)

What you’ll need

How to configure wget for a proxy

To allow wget behind a proxy, you must edit a configuration file and add the necessary addresses. The best way to do this is via the global wget configuration file. To open the configuration file for editing, issue the command: 

sudo nano /etc/wgetrc

In that file, you’ll find three lines: 

#https_proxy = http://proxy.yoyodyne.com:18023/
#http_proxy = http://proxy.yoyodyne.com:18023/
#ftp_proxy = http://proxy.yoyodyne.com:18023/

You’ll need to uncomment (remove the # character) each necessary line for your proxy. If you are behind both HTTP and HTTPS proxies you’d configure those lines like: 

https_proxy = HTTPSPROXY:PORT
http_proxy = HTTPPROXY:PORT

Where HTTPSPROXY is your HTTPS proxy server address, HTTPPROXY is your HTTP proxy address, and PORT is the required port for your proxy. Save and close the file.

There might be a situation where you don’t want all users to be able to get beyond the proxy. For that, you could set the proxy variables in a specific user’s .bashrc file. For that, you’d open the file for editing with the command: 

sudo nano /home/USER/.bashrc

Where USER is the username.

At the bottom of the file, add the following: 

export http_proxy=HTTPPROXY:PORT
export https_proxy=HTTPSPROXY:PORT
export ftp_proxy=FTPPROXY:PORT

Where:

  • HTTPSPROXY is your HTTPS proxy server address
  • HTTPPROXY is your HTTP proxy address
  • FTPPROXY is your FTP proxy
  • PORT is the required port for your proxy

Save and close the file.

Finally, if your proxy requires user authentication, the configuration lines would look like: 

export http_proxy="http://USERNAME:PASSWORD@HTTPPROXY:PORT"
export https_proxy="http://USERNAME:PASSWORD@HTTPSPROXY:PORT"
export ftp_proxy="http://USERNAME:PASSWORD@FTPPROXY:PORT"

Where:

  • USERNAME and PASSWORD are the credentials used for proxy authentication
  • HTTPSPROXY is your HTTPS proxy server address
  • HTTPPROXY is your HTTP proxy address
  • FTPPROXY is your FTP proxy
  • PORT is the required port for your proxy

At this point, you should be able to use wget, in the usual method, without the proxy preventing you from downloading the file.

Subscribe to TechRepublic’s How To Make Tech Work on YouTube for all the latest tech advice for business pros from Jack Wallen.

Also see