Questions tagged [wget]

GNU Wget (or just wget) is a command line-based program for downloading content from the Internet using HTTP, HTTPS and FTP.

Filter by
Sorted by
Tagged with
0
votes
1answer
51 views

What does the -r option do when using wget?

I am having trouble understanding what exactly the -r option does when using wget. The man page says that it'll Turn on recursive retrieving. The default maximum depth is 5. What exactly is meant ...
0
votes
2answers
43 views

Can I force wget to use the secondary dns

I am trying to download something from NVIDIA website on a remote server using ssh. Iran is banned by the NVIDIA website. My server's DNS servers are: IP4.DNS[1]: 192.168....
0
votes
2answers
36 views

How to specify location of file with wget download?

I try to download file so wget -P /var/www/html/wp-content/uploads/movies/2019/ -O love.mp4 https://example.com/video.mp4 But it locates the file in root and not in /var/www/html/wp-content/uploads/...
-1
votes
1answer
56 views

I do not know how to create a new directory CSIS536 under my home directory and change to the in directory [closed]

In Ubuntu, Create a new directory CSIS536 under your home directory, and change to the directory. Use the wget command to download VUST catalog and save it to the CSIS536 directory as VUST_Catalog....
0
votes
0answers
51 views

wget permission denied error

Task: Hi I am trying to download apache spark using wget. However I get error: Problem : Permission denied. Reason: No write permission for folder spark1 What do I want: How Do I enable write ...
0
votes
1answer
37 views

Downloads in ubuntu terminal

I am using the Ubuntu terminal on my windows 10. I downloaded a file from the ubuntu terminal into a directory I created on the terminal. Right now I can't search for the file on my computer. Any ...
0
votes
1answer
28 views

wget converting or altering .jar file on the fly?

I am very new to Linux and trying to use this application server named Lucee. I am using Ubuntu 18.04. The instructions I am following say to download the application using wget. When I run this ...
0
votes
0answers
27 views

Using wget to download SRA file

We used wget to download SRA sequcing file. since the network was not stable, we used wget -c to continue downloading the not finished ones. When it was finished, we found that the size of downloaded ...
0
votes
0answers
22 views

How to throttle jigdo-lite's download rate

By using this command: jigdo-lite --noask http://cdimage.debian.org/image jigdo issues a call to wget like this /bin/sh /usr/bin/wget --user-agent=jigdo-lite/0.7.3 (GNU Wget 1.17.1 built on linux-...
0
votes
2answers
91 views

wget on ftpes://

When i run the following command line: wget --user=anonymous ftpes://x.x.x.x:x/FTP/image.bin.gz It returns that ftpes is not supported. Is there a package to install ?
1
vote
1answer
134 views

how to make wget not to download the same file more than once (with and without extension)

I'm using wget to download some pages and I don't want it to download the same page if it has already been downloaded. I use the following command: wget --no-clobber --input text04.txt --directory-...
0
votes
0answers
19 views

wget --convert-links does not convert index.html

Im using wget --mirror -p --convert-links https://my.host.com It downloads everything and converts all links to local relative path except in index.html. I can open some-sub-page.html and all the ...
0
votes
0answers
144 views

What could be reason behind “wget: unable to resolve host address ‘archive.ubuntu.com’” error

I'm trying to download the netboot ubuntu 18.04 image and using this link It has directories and file directly. So trying to download all the files and sub-directory recursively... For that, I'm ...
1
vote
1answer
106 views

How to crawl a website using wget until 300 html pages are saved

I want to crawl a website recursively using wget in Ubuntu and stop it after 300 pages are downloaded. I only save the html file of a page. Currently, this is the command I am using: wget -r --mirror ...
0
votes
2answers
139 views

bash - Unexcepted Token

I'm writing a bash script to automatize my downloading process. In a section, it gaves a error: syntax error near unexpected token `newURL=$(echo $URL | sed 's/&:/$index/')' That section of ...
0
votes
1answer
46 views

Bash script to download any new version of a file from a webpage of a particular software

Let's say I'm creating a script to automate downloading & compiling Nginx from source but one improvement I would like to make is make sure the script would work with new Nginx versions without me ...
0
votes
0answers
48 views

Downloading multiple files that have their URLs stored in a file but with wildcards inside the file using wget

I have to download multiple log files from an internet page but only the files that have a specific name that I have on my file: In my file called "net" I have the following names but not the address ...
-4
votes
1answer
170 views

Proxy authentication using wget on cygwin [closed]

This question was also posted on SO. My institute recently installed a new proxy server for our network. I am trying to configure my Cygwin environment to be able to run wget and download data from a ...
1
vote
1answer
189 views

wget of redis is not working in docker container with Debian 9 on Ubuntu host

I have a very simple Docker file FROM openjdk:11 ENTRYPOINT ["/bin/bash"] And I have 2 machines with Ubuntu. First machine: Ubuntu 18.04.1, Docker version 18.06.0-ce, build 0ffa825 In the docker ...
0
votes
0answers
53 views

How to find the correct page for wget to download from?

I want to download all *.pdf files on a website recursively. What I could achieve so far is: Store the cookies needed for the login page with this question Download a specific file by copy and ...
2
votes
2answers
3k views

wget or curl gives 403:Forbidden while downloading file from OneDrive for business

I'm trying to download software for a client which resides in my bussiness OneDrive account. Through the webinterface of OneDrive I do: Right click the file and select Share List item Change the ...
0
votes
1answer
47 views

How to download big sites with several interrupted wget:s?

I'm trying to download a site with wget, but the site is so big that I can't do it all at once. The problem is that the server is always reporting a new file and when I resume the download wget ...
1
vote
1answer
102 views

Wget batch of files fails, curl works, what am I doing wrong?

I am trying to download the entire directory from this website: https://data.geobasis-bb.de/geobasis/daten/dgm/xyz/ What I tried is: wget --show-progress -A 'dgm_*.zip' https://data.geobasis-bb.de/...
1
vote
0answers
26 views

standalone repo for upgrading

I am new to ubuntu I have an ubuntu 14.04 system on an isolated network (no access to internet) that I need to upgrade. I would like to pull a copy of a repo to use on this isolated network. I ...
0
votes
0answers
146 views

wget downloading speed gradually slows down

Today I attempted downloading a file using $ wget -P some_directory URL. The first few minutes of downloading went smoothly, downloading 500KBPS but later, its speed gradually decreased, reaching ...
0
votes
2answers
347 views

How to download website offline with authenticated username and password?

I have an account of this tutorial website testdriven.io, and I would like to download the tutorial offline for my team member to lean without having to login the credential. So, I tried several ways ...
1
vote
0answers
34 views

Wget website “drilling” for radio show elements

I run a Low Power FM community radio station and in addition to the live shows, we carry syndicated programs from producers around the world. Every week I spend way too much time downloading show ...
-1
votes
2answers
257 views

wget from file and save as acording to file name

I have multiple files, each file contains a URL. How can I use wget to read each file and download the URL inside it, save it as the container file name? EX : file name.frm contains : http://...
0
votes
0answers
107 views

wget & curl are unable to download but aria2c is able? why?

I'm trying to retrieve the content-length of a file by using below commands, they are all OK, but for this file which belongs to an online store in Iran: https://www.digikala.com/mag/wp-content/...
0
votes
3answers
376 views

difference between installation options for NodeJS on Ubuntu 18.04 LTS

I'm a brand new to Linux so this may be a rookie question... On a new installation of Ubuntu 18.04 LTS I want to install NodeJS. I ran node --version to verify that it is not installed: Command '...
0
votes
1answer
428 views

Error when using wget to download a list of URLs in a .txt file

My question is about downloading a list of URLs using wget. The URLs are listed in a text file named url-list.txt, which I have saved on my desktop. Whereas I'm using the bionic distro, on a 64-bit ...
1
vote
0answers
43 views

How to download specific files from ftp?

I wonder about how to download files, not all into index of directory. For example, my question is how to download specific files, which its name start "mat1", from 2015 to 2018 or index of 2018, ...
0
votes
0answers
10 views

Reduce size of WGET downloade HTML pages

Im using wget loop to download Html pages from local website wget example.com/imageId/{1..4000000} So i got a large number of files, in total 20 GB size. So how to reduce this size, as i need only ...
4
votes
2answers
1k views

How to get link of file to download with wget

I am using Ubuntu 19.10. For automation purposes, I need to download this file with a script. But I am not able to get the link. How can I achieve this?
0
votes
3answers
42 views

How to download certain folders from url

when I use this command: wget -r -np -nH --cut-dirs=6 -R "index.html*" -e robots=off "https://physionet.org/physiobank/database/challenge/2018/training/" It will download all the folders (994 ...
1
vote
1answer
301 views

Why does wget fail with “wget: symbol lookup error: wget: undefined symbol: psl_str_to_utf8lower”?

On a brand new installation of Ubuntu Linux 18.04, wget <URL> fails with, wget: symbol lookup error: wget: undefined symbol: psl_str_to_utf8lower I have tested that the resource exists at the ...
2
votes
2answers
2k views

install node latest version in linux

i am new in linux and want install node with bellow command: sudo apt-get install curl curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash - sudo apt-get install nodejs and angular 8 ...
0
votes
0answers
105 views

SSL not working for certain sites on ubuntu 19.04 server

So, on my new Ubuntu server I got recently, I cannot, for the life of me, get ssl to work with certain sites. I have tried adding their Root, Intermediate, and Client certificates to /usr/local/share/...
0
votes
1answer
1k views

wget not downloading entire file

I got historic house prices in UK from: https://www.gov.uk/government/statistical-data-sets/price-paid-data-downloads I ran the link for all historic house prices: wget http://prod.publicdata....
0
votes
0answers
128 views

not sure about proxy is working or not

I have recently configured proxy setting in Ubuntu. But I have a doubt that it is not working as per expectation. When I execute the command sudo apt-get update, it shows When I execute curl https:...
0
votes
2answers
756 views

Download a list of files with wget using wildcards

I have a list of files like: test.com/image01.png test.com/image02.png test.com/image03.png ... test.com/image50.png and want to download them all. One way is to write all names in a file and then: ...
0
votes
0answers
331 views

How to exclude folders from downloading using wget?

How can i exclude folders from being downloaded using wget ? For example if i wanted to download all of the files and folders of a subfolder: https://index.org has the following folders Folder1 -...
0
votes
0answers
99 views

Download Speed Rate Drops to 0 kb/s after 2 seconds!

this is ridiculous but my download speed drops to 0 kb/s after downloading about 200KB of a file... I've tested my network with other devices and it's fine but I still can't download with my laptop. ...
0
votes
1answer
3k views

Certificate problems

Many shell applications are throwing SSL certificate errors. I'm not aware of anything that could have caused this. apt, Google Chrome, Discord all work fine. Already tried: Problem with certificates ...
1
vote
0answers
807 views

wget unable to locally verify the issuer's authority but curl works

After updating from Ubuntu 16 to 18, wget stopped working (among other stuff), while curl works properly. $ curl -v https://google.com ... * Server certificate: * subject: C=US; ST=California; L=...
0
votes
0answers
202 views

How to use the command “wget” to download all files which satisfy to a URL that contains 2 wildcards “*”?

For example: This is all URLs of files in website.com : $ wget website.com/image/2007/dogs.jpg $ wget website.com/video/2007/cat.mp4 $ wget website.com/music/2007/sadsong.mp3 How can I make this ...
0
votes
1answer
1k views

Unable to download pdf file using wget

The website contains a button which I have to click to get the pdf to open. What I want to do is to use "wget" and the link of that website to download the pdf without me clicking on the button, and ...
1
vote
2answers
151 views

How to replace cp command with wget?

I am very newbie to linux and its commands. What I want is use cp command instead of wget to request from browser. Because I'm using docker and the mongo site filtered our country's IP. I have ...
1
vote
1answer
199 views

Downloading a repository from Github

I want to download and install fuse, I am following this tutorial Download and compile latest version of fuse source code. For this article, we are using fuse version 3.5. Following the set of ...
1
vote
1answer
77 views

wget mirror into my git direcotry

I want to wget --mirror into a specified directory. I can use -P <PATH> but it still makes a subdirectory with the website name. My Git directory name is buddhadust The website is http://...

1
2 3 4 5
9