Summary

Downloading a whole website is useful if you want your own copy of the text, images, and other content on it, just in case the author deletes it or the website shuts down. However, web browsers don’t include this functionality out-of-the box. Here’s how to download a whole site using the wget tool.

Before You Start

Before you start, ensure you have enough disk space to save the site you want to download. Some sites have alotof pages and alotof content that can add up fast, so keep in mind what you’ll be downloading, so you don’t run out of storage. If you’re on a laptop, make sure you’re plugged in to a power outlet, as downloading potentially thousands of files might take a while.

If you’re on Windows, you’ll need toinstall the Windows Subsystem for Linux(WSL) to get access to wget.

Also, understand that a lot of modern websites are dynamically generated, meaning that they process data on their servers to generate pages that change when new content is added, or when you refresh the page. Downloading a website or page creates a snapshot of it at that moment in time, so you’ll need to re-download the site if you want an up-to-date copy.

How to Copy a Whole Website Locally Using Wget

Wget is a command line tool for downloading data from the web. It’s installed by default on Ubuntu (this means WSL as well) and Fedora. If you’re on an Arch-based distribution, you’ll need toinstall it. If you’re on macOS, wget is available throughMacPorts.

If you’re on Linux or MacOS, you’ll need to open up a Terminal window. If you’re on Windows using WSL,launch whichever Linux distro you’d like to use.

You can confirm that wget is installed by running:

If wget is available on your system, the command will return information about the wget command.

Now, you’re ready to start downloading. This is the command to download a whole website:

This command will attempt to grab everything it can from the website athttps://example.com, scan all the links on the first page, download those linked pages and everything on them, and then continue on through the pages linked from those pages.

Here’s what each of the command flags (the letters following the dash in the wget command above) mean:

You’ll know the website is being successfully downloaded when you see output resembling the following:

Note that while this is a recursive download, only links from the specified website will be downloaded. Pages on other websites will remain online and require an internet connection to view.

You can see a complete list of all the wget command flags and what they do in thewget user manual.

Remember, this might take a while and there’s no real way to estimate how long large websites will take to download, so it’s worth checking progress occasionally to make sure that things are still progressing.

What About Downloading Just One Page?

If you want to download a single web page, your web browser can do that in a much easier process: just go to the File menu and then click the “Save” button (the wording might be a bit different for your browser, onFirefoxit’s “Save page as…").

If you want to use wget to download a single page, run the following command:

This omits the options that tell wget to recursively download pages, and adjust extensions and links.

Downloading an entire site is useful in many situations, including:

How Do You Access the Site You Downloaded?

A site you download using wget will be saved as an HTML file on your PC. If you used WSL, you’ll need toaccess your WSL files through File Explorer.

In our example, the file was located at “\wsl.localhost\Ubuntu-22.04\home\example” since we used Ubuntu 22.04 and our Linux username is “example.” Wget will save the folders to whichever directory you have active in the Terminal when you run the command, unless you specify another path manually.

HTML files can be opened by any browser, though there are other programs that can do it too, if you prefer one of those.

Other Tools for Downloading Web Content for Offline use

While wget one of the most common ways to download web content, there are others that you’re able to use if you have different requirements, or don’t want to deal with the command line.

Whatever your reason for wanting to copy a whole website to your own computer, now you know how! If you’re planning on doing some offline reading, adecent eReader is an absolute must.