Questions tagged [wget]
GNU Wget (or just wget) is a command line-based program for downloading content from the Internet using HTTP, HTTPS and FTP.
417
questions
0
votes
1answer
51 views
What does the -r option do when using wget?
I am having trouble understanding what exactly the -r option does when using wget. The man page says that it'll Turn on recursive retrieving. The default maximum depth is 5.
What exactly is meant ...
0
votes
2answers
43 views
Can I force wget to use the secondary dns
I am trying to download something from NVIDIA website on a remote server using ssh. Iran is banned by the NVIDIA website.
My server's DNS servers are:
IP4.DNS[1]: 192.168....
0
votes
2answers
36 views
How to specify location of file with wget download?
I try to download file so wget -P /var/www/html/wp-content/uploads/movies/2019/ -O love.mp4 https://example.com/video.mp4
But it locates the file in root and not in /var/www/html/wp-content/uploads/...
-1
votes
1answer
56 views
I do not know how to create a new directory CSIS536 under my home directory and change to the in directory [closed]
In Ubuntu,
Create a new directory CSIS536 under your home directory, and change to the directory.
Use the wget command to download VUST catalog and save it to the CSIS536 directory as
VUST_Catalog....
0
votes
0answers
51 views
wget permission denied error
Task: Hi I am trying to download apache spark using wget. However I get error:
Problem : Permission denied.
Reason: No write permission for folder spark1
What do I want: How Do I enable write ...
0
votes
1answer
37 views
Downloads in ubuntu terminal
I am using the Ubuntu terminal on my windows 10. I downloaded a file from the ubuntu terminal into a directory I created on the terminal. Right now I can't search for the file on my computer. Any ...
0
votes
1answer
28 views
wget converting or altering .jar file on the fly?
I am very new to Linux and trying to use this application server named Lucee. I am using Ubuntu 18.04. The instructions I am following say to download the application using wget.
When I run this ...
0
votes
0answers
27 views
Using wget to download SRA file
We used wget to download SRA sequcing file. since the network was not stable, we used wget -c to continue downloading the not finished ones. When it was finished, we found that the size of downloaded ...
0
votes
0answers
22 views
How to throttle jigdo-lite's download rate
By using this command:
jigdo-lite --noask http://cdimage.debian.org/image
jigdo issues a call to wget like this
/bin/sh /usr/bin/wget --user-agent=jigdo-lite/0.7.3 (GNU Wget 1.17.1 built on linux-...
0
votes
2answers
91 views
wget on ftpes://
When i run the following command line:
wget --user=anonymous ftpes://x.x.x.x:x/FTP/image.bin.gz
It returns that ftpes is not supported.
Is there a package to install ?
1
vote
1answer
134 views
how to make wget not to download the same file more than once (with and without extension)
I'm using wget to download some pages and I don't want it to download the same page if it has already been downloaded. I use the following command:
wget --no-clobber --input text04.txt --directory-...
0
votes
0answers
19 views
wget --convert-links does not convert index.html
Im using
wget --mirror -p --convert-links https://my.host.com
It downloads everything and converts all links to local relative path except in index.html.
I can open some-sub-page.html and all the ...
0
votes
0answers
144 views
What could be reason behind “wget: unable to resolve host address ‘archive.ubuntu.com’” error
I'm trying to download the netboot ubuntu 18.04 image and using this link
It has directories and file directly. So trying to download all the files and sub-directory recursively...
For that, I'm ...
1
vote
1answer
106 views
How to crawl a website using wget until 300 html pages are saved
I want to crawl a website recursively using wget in Ubuntu and stop it after 300 pages are downloaded. I only save the html file of a page. Currently, this is the command I am using:
wget -r --mirror ...
0
votes
2answers
139 views
bash - Unexcepted Token
I'm writing a bash script to automatize my downloading process. In a section, it gaves a error:
syntax error near unexpected token `newURL=$(echo $URL | sed 's/&:/$index/')'
That section of ...
0
votes
1answer
46 views
Bash script to download any new version of a file from a webpage of a particular software
Let's say I'm creating a script to automate downloading & compiling Nginx from source but one improvement I would like to make is make sure the script would work with new Nginx versions without me ...
0
votes
0answers
48 views
Downloading multiple files that have their URLs stored in a file but with wildcards inside the file using wget
I have to download multiple log files from an internet page but only the files that have a specific name that I have on my file:
In my file called "net" I have the following names but not the address ...
-4
votes
1answer
170 views
Proxy authentication using wget on cygwin [closed]
This question was also posted on SO.
My institute recently installed a new proxy server for our network. I am trying to configure my Cygwin environment to be able to run wget and download data from a ...
1
vote
1answer
189 views
wget of redis is not working in docker container with Debian 9 on Ubuntu host
I have a very simple Docker file
FROM openjdk:11
ENTRYPOINT ["/bin/bash"]
And I have 2 machines with Ubuntu.
First machine: Ubuntu 18.04.1, Docker version 18.06.0-ce, build 0ffa825
In the docker ...
0
votes
0answers
53 views
How to find the correct page for wget to download from?
I want to download all *.pdf files on a website recursively.
What I could achieve so far is:
Store the cookies needed for the login page with this question
Download a specific file by copy and ...
2
votes
2answers
3k views
wget or curl gives 403:Forbidden while downloading file from OneDrive for business
I'm trying to download software for a client which resides in my bussiness OneDrive account. Through the webinterface of OneDrive I do:
Right click the file and select Share
List item
Change the ...
0
votes
1answer
47 views
How to download big sites with several interrupted wget:s?
I'm trying to download a site with wget, but the site is so big that I can't do it all at once. The problem is that the server is always reporting a new file and when I resume the download wget ...
1
vote
1answer
102 views
Wget batch of files fails, curl works, what am I doing wrong?
I am trying to download the entire directory from this website: https://data.geobasis-bb.de/geobasis/daten/dgm/xyz/
What I tried is:
wget --show-progress -A 'dgm_*.zip' https://data.geobasis-bb.de/...
1
vote
0answers
26 views
standalone repo for upgrading
I am new to ubuntu
I have an ubuntu 14.04 system on an isolated network (no access to internet) that I need to upgrade. I would like to pull a copy of a repo to use on this isolated network. I ...
0
votes
0answers
146 views
wget downloading speed gradually slows down
Today I attempted downloading a file using $ wget -P some_directory URL.
The first few minutes of downloading went smoothly, downloading 500KBPS but later, its speed gradually decreased, reaching ...
0
votes
2answers
347 views
How to download website offline with authenticated username and password?
I have an account of this tutorial website testdriven.io, and I would like to download the tutorial offline for my team member to lean without having to login the credential.
So, I tried several ways ...
1
vote
0answers
34 views
Wget website “drilling” for radio show elements
I run a Low Power FM community radio station and in addition to the live shows, we carry syndicated programs from producers around the world. Every week I spend way too much time downloading show ...
-1
votes
2answers
257 views
wget from file and save as acording to file name
I have multiple files, each file contains a URL.
How can I use wget to read each file and download the URL inside it, save it as the container file name?
EX : file name.frm contains :
http://...
0
votes
0answers
107 views
wget & curl are unable to download but aria2c is able? why?
I'm trying to retrieve the content-length of a file by using below commands, they are all OK, but for this file which belongs to an online store in Iran:
https://www.digikala.com/mag/wp-content/...
0
votes
3answers
376 views
difference between installation options for NodeJS on Ubuntu 18.04 LTS
I'm a brand new to Linux so this may be a rookie question...
On a new installation of Ubuntu 18.04 LTS I want to install NodeJS. I ran node --version to verify that it is not installed:
Command '...
0
votes
1answer
428 views
Error when using wget to download a list of URLs in a .txt file
My question is about downloading a list of URLs using wget. The URLs are listed in a text file named url-list.txt, which I have saved on my desktop. Whereas I'm using the bionic distro, on a 64-bit ...
1
vote
0answers
43 views
How to download specific files from ftp?
I wonder about how to download files, not all into index of directory.
For example, my question is how to download specific files, which its name start "mat1", from 2015 to 2018 or index of 2018, ...
0
votes
0answers
10 views
Reduce size of WGET downloade HTML pages
Im using wget loop to download Html pages from local website
wget example.com/imageId/{1..4000000}
So i got a large number of files, in total 20 GB size.
So how to reduce this size, as i need only ...
4
votes
2answers
1k views
How to get link of file to download with wget
I am using Ubuntu 19.10. For automation purposes, I need to download this file with a script.
But I am not able to get the link.
How can I achieve this?
0
votes
3answers
42 views
How to download certain folders from url
when I use this command:
wget -r -np -nH --cut-dirs=6 -R "index.html*" -e robots=off "https://physionet.org/physiobank/database/challenge/2018/training/"
It will download all the folders (994 ...
1
vote
1answer
301 views
Why does wget fail with “wget: symbol lookup error: wget: undefined symbol: psl_str_to_utf8lower”?
On a brand new installation of Ubuntu Linux 18.04, wget <URL> fails with,
wget: symbol lookup error: wget: undefined symbol: psl_str_to_utf8lower
I have tested that the resource exists at the ...
2
votes
2answers
2k views
install node latest version in linux
i am new in linux and want install node with bellow command:
sudo apt-get install curl
curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
sudo apt-get install nodejs
and angular 8 ...
0
votes
0answers
105 views
SSL not working for certain sites on ubuntu 19.04 server
So, on my new Ubuntu server I got recently, I cannot, for the life of me, get ssl to work with certain sites. I have tried adding their Root, Intermediate, and Client certificates to /usr/local/share/...
0
votes
1answer
1k views
wget not downloading entire file
I got historic house prices in UK from: https://www.gov.uk/government/statistical-data-sets/price-paid-data-downloads
I ran the link for all historic house prices:
wget http://prod.publicdata....
0
votes
0answers
128 views
not sure about proxy is working or not
I have recently configured proxy setting in Ubuntu. But I have a doubt that it is not working as per expectation.
When I execute the command sudo apt-get update, it shows
When I execute curl https:...
0
votes
2answers
756 views
Download a list of files with wget using wildcards
I have a list of files like:
test.com/image01.png
test.com/image02.png
test.com/image03.png
...
test.com/image50.png
and want to download them all. One way is to write all names in a file and then:
...
0
votes
0answers
331 views
How to exclude folders from downloading using wget?
How can i exclude folders from being downloaded using wget ? For example if i wanted to download all of the files and folders of a subfolder:
https://index.org has the following folders
Folder1
-...
0
votes
0answers
99 views
Download Speed Rate Drops to 0 kb/s after 2 seconds!
this is ridiculous but my download speed drops to 0 kb/s after downloading about 200KB of a file...
I've tested my network with other devices and it's fine but I still can't download with my laptop. ...
0
votes
1answer
3k views
Certificate problems
Many shell applications are throwing SSL certificate errors. I'm not aware of anything that could have caused this. apt, Google Chrome, Discord all work fine.
Already tried: Problem with certificates
...
1
vote
0answers
807 views
wget unable to locally verify the issuer's authority but curl works
After updating from Ubuntu 16 to 18, wget stopped working (among other stuff), while curl works properly.
$ curl -v https://google.com
...
* Server certificate:
* subject: C=US; ST=California; L=...
0
votes
0answers
202 views
How to use the command “wget” to download all files which satisfy to a URL that contains 2 wildcards “*”?
For example:
This is all URLs of files in website.com :
$ wget website.com/image/2007/dogs.jpg
$ wget website.com/video/2007/cat.mp4
$ wget website.com/music/2007/sadsong.mp3
How can I make this ...
0
votes
1answer
1k views
Unable to download pdf file using wget
The website contains a button which I have to click to get the pdf to open.
What I want to do is to use "wget" and the link of that website to download the pdf without me clicking on the button, and ...
1
vote
2answers
151 views
How to replace cp command with wget?
I am very newbie to linux and its commands. What I want is use cp command instead of wget to request from browser. Because I'm using docker and the mongo site filtered our country's IP. I have ...
1
vote
1answer
199 views
Downloading a repository from Github
I want to download and install fuse, I am following this tutorial
Download and compile latest version of fuse source code. For this
article, we are using fuse version 3.5. Following the set of ...
1
vote
1answer
77 views
wget mirror into my git direcotry
I want to wget --mirror into a specified directory.
I can use -P <PATH> but it still makes a subdirectory with the website name.
My Git directory name is buddhadust
The website is http://...
