curl

What is curl?

curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.

curl offers a busload of useful tricks like proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume, Metalink, and more. The number of features will make your head spin!

How to use curl - general usage

# syntax
curl <OPTIONS> <URL>

##### examples with parameters
curl http://127.0.0.1/test/test.php # gets the content of a file, but does not download anything
curl -o /root/test/test-php.txt http://192.168.0.110/test/test.php # writes the output to the file test-php.txt in the directory /root/test/

##### Download multiple files
curl -O http://mirrors.edge.kernel.org/archlinux/iso/2018.06.01/archlinux-2018.06.01-x86_64.iso  \
     -O https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso

##### Get HTTP headers of a URL
curl -I --http2 https://www.ubuntu.com/

##### test if a website supports http/2
curl -I --http2 -s https://linuxize.com/ | grep HTTP
# support = HTTP/2 200
# no support = HTTP/1.1 200 OK

##### follow redirects
# try curl google.com and curl -L google.com to see the differences.
curl -L google.com

##### change user agent (act as another browser)
curl -A "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0" https://getfedora.org/

##### specify a maximum transfer rate (so you will decide how much data is going to be used at once to retrieve a file/page)
curl --limit-rate 1m -O https://dl.google.com/go/go1.10.3.linux-amd64.tar.gz

# normally it will be the max your internet provider can handle, in above example it will be 1MB

How to use curl - cookies & proxies

# syntax = default

##### send cookies with curl
curl -L -b "oraclelicense=a" -O http://download.oracle.com/otn-pub/java/jdk/10.0.2+13/19aef61b38124481863b1413dce1855f/jdk-10.0.2_linux-x64_bin.rpm

##### use proxy with curl
curl -x 192.168.44.1:8888 http://linux.com/ # unauthenticated proxy
curl -U username:password -x 192.168.44.1:8888 http://linux.com/ # authenticated proxy

How to use curl - post requests

# syntax 
curl -X POST <OPTIONS> <URL>

##### send data using the multipart/form-data content-type
curl -X POST -F 'name=linuxize' -F 'email=[e-mail]' https://example.com/contact.php

##### send data using the application/x-www-form-urlencoded content-type
curl -X POST -d 'name=linuxize' -d 'email=[e-mail]' https://example.com/contact.php
curl -X POST -d 'name=linuxize&email=[e-mail]' https://example.com/contact.php # If the -d option is used more than once you can merge the data using the & symbol 

##### uploading an image/file (use multipart/form-data content-type)
curl -X POST -F 'image=@/home/user/Pictures/wallpaper.jpg' http://example.com/upload

How to use curl - ftp usage

# syntax = default

##### access a protected ftp server (this can also be done with the linux protocol tool : ftp)
curl -u FTP_USERNAME:FTP_PASSWORD ftp://ftp.example.com/

##### download a file from the remote, protected ftp server
curl -u FTP_USERNAME:FTP_PASSWORD ftp://ftp.example.com/file.tar.gz

##### upload a file to the remote, protected ftp server
curl -T newfile.tar.gz -u FTP_USERNAME:FTP_PASSWORD ftp://ftp.example.com/