cURL Command Line tool in Unix Restful Web Services

cURL is a command line tool for transferring data with URL syntax and supporting different network protocols. It can be applied as a client using web services.

Command Syntax
In order to use cURL, you should type curl at the command line followed by additional options and URL of the output you wish to retrieve or post:

curl options URL

Options
The following options may be applied in cURL in order to use Restful-Web Services:

Command                Description
-u user:password   Specifies the user name and password to use for server authentication.
-X command            Specifies a custom request method to use when communicating with the HTTP                                    server. For details see HTTP Methods section below.
-d @ filename            Sends the data contained a specified file to the HTTP server. @ followed be a                                      file name specifies the file to read the data from.
-H header                   Extra header to use when getting a web page.
HTTP Methods
There are five different HTTP methods available that may be used by cURL.

GET
The GET method requests to retrieve a content of a resource specified in the URL request.

curl -u user:password uri/resources 

PUT
The PUT method requests to store a content of a specified file in the location indicated in URL. There are two options possible:
    •Creates a new resource if you send the full content of the specified resource,
    •Updates a resource if you update the full content of the specified resource.

curl -u user:password -X PUT -d @file.xml -H "Content-type: application/xml" uri/resource 

POST
The POST method requests to store a content of a specified file in the location indicated in URL. There are two options possible:
    •Create a subordinate of the specified resource,
    •Update a resource if you request the server to update one or more subordinates of the specified resource.
On the contrary to PUT method, you do not need to send the full content of the specified resource.
curl -u user:password -X POST -d @file.xml -H "Content-type: application/xml" uri/resources 

DELETE
The DELETE method requests that the HTTP server removes a resource specified by URL.

curl -u user:password -X DELETE uri/resource 

OPTIONS
The OPTIONS method requests for information about the communication options available on the request-response chain specified by the URL.

curl -u user:password -X OPTIONS uri/resources 

For more details about this tool just type the command


[tutorialbyexample@tutorialbyexample-host]$ man curl | more
curl(1)                           Curl Manual                          curl(1)

NAME
       curl - transfer a URL

SYNOPSIS
       curl [options] [URL...]

DESCRIPTION
       curl  is  a tool to transfer data from or to a server, using one of the
       supported protocols (HTTP, HTTPS, FTP, FTPS,  SCP,  SFTP,  TFTP,  DICT,
       TELNET,  LDAP  or  FILE).  The command is designed to work without user
       interaction.

       curl offers a busload of useful tricks like proxy support, user authen-
       tication,  FTP upload, HTTP post, SSL connections, cookies, file trans-
       fer resume and more. As you will see below, the number of features will
       make your head spin!

       curl  is  powered  by  libcurl  for  all transfer-related features. See
       libcurl(3) for details.

URL
       The URL syntax is protocol-dependent. You’ll find a  detailed  descrip-
       tion in RFC 3986.

       You  can  specify  multiple  URLs or parts of URLs by writing part sets
       within braces as in:

        http://site.{one,two,three}.com

       or you can get sequences of alphanumeric series by using [] as in:

        ftp://ftp.numericals.com/file[1-100].txt
        ftp://ftp.numericals.com/file[001-100].txt    (with leading zeros)
        ftp://ftp.letters.com/file[a-z].txt

       No nesting of the sequences is supported at the moment, but you can use
       several ones next to each other:

        http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html

       You  can  specify  any amount of URLs on the command line. They will be
       fetched in a sequential manner in the specified order.

       Since curl 7.15.1 you can also specify a step counter for  the  ranges,
       so that you can get every Nth number or letter:

        http://www.numericals.com/file[1-100:10].txt
        http://www.letters.com/file[a-z:2].txt

       If  you  specify  URL  without protocol:// prefix, curl will attempt to
       guess what protocol you might want. It will then default  to  HTTP  but
       try  other  protocols based on often-used host name prefixes. For exam-
       ple, for host names starting with "ftp." curl will assume you  want  to
       speak FTP.
       curl  will  do  its best to use what you pass to it as a URL. It is not
       trying to validate it as a syntactically correct URL by any  means  but
       is instead very liberal with what it accepts.

       Curl will attempt to re-use connections for multiple file transfers, so
       that getting many files from the same server will not do multiple  con-
       nects / handshakes. This improves speed. Of course this is only done on
       files specified on a single command line and  cannot  be  used  between
       separate curl invokes.

PROGRESS METER
       curl  normally  displays a progress meter during operations, indicating
       the amount of transferred data,  transfer  speeds  and  estimated  time
       left, etc.

       However,  since  curl displays this data to the terminal by default, if
       you invoke curl to do an operation and it is about to write data to the
       terminal,  it disables the progress meter as otherwise it would mess up
       the output mixing progress meter and response data.

       If you want a progress meter for HTTP POST or PUT requests, you need to
       redirect  the  response  output to a file, using shell redirect (>), -o
       [file] or similar.

       It is not the same case for FTP upload as that operation does not  spit
       out any response data to the terminal.

       If you prefer a progress "bar" instead of the regular meter, -# is your
       friend.

OPTIONS
       In general, all boolean options are enabled with --option and yet again
       disabled  with --no-option. That is, you use the exact same option name
       but prefix it with "no-". However, in this list we mostly only list and
       show  the --option version of them. (This concept with --no options was
       added in  7.19.0.  Previously  most  options  were  toggled  on/off  on
       repeated use of the same command line option.)

       -a/--append
              (FTP/SFTP) When used in an upload, this will tell curl to append
              to the target file  instead  of  overwriting  it.  If  the  file
              doesn’t  exist,  it  will  be  created.   Note that this flag is
              ignored by some SSH servers (including OpenSSH).

       -A/--user-agent <agent string>
              (HTTP) Specify the User-Agent string to send to the HTTP server.
              Some   badly   done  CGIs  fail  if  this  field  isn’t  set  to
              "Mozilla/4.0". To encode blanks  in  the  string,  surround  the
              string  with  single  quote marks. This can also be set with the
              -H/--header option of course.

              If this option is set more than once, the last one will  be  the
              one that’s used.

       --anyauth
              (HTTP) Tells curl to figure out authentication method by itself,
              and use the most secure one the remote site claims  to  support.
              This is done by first doing a request and checking the response-
              headers, thus possibly inducing  an  extra  network  round-trip.
              This  is  used  instead  of  setting  a  specific authentication
              method, which you can do with  --basic,  --digest,  --ntlm,  and
              --negotiate.

              Note  that  using --anyauth is not recommended if you do uploads
              from stdin, since it may require data to be sent twice and  then
              the client must be able to rewind. If the need should arise when
              uploading from stdin, the upload operation will fail.

       -b/--cookie <name=data>
              (HTTP) Pass the data to the HTTP server as a cookie. It is  sup-
              posedly  the data previously received from the server in a "Set-
              Cookie:" line.  The data should be in the format  "NAME1=VALUE1;
              NAME2=VALUE2".

              If  no  ’=’ symbol is used in the line, it is treated as a file-
              name to use to read previously stored cookie lines  from,  which
              should  be used in this session if they match. Using this method
              also activates the "cookie parser" which will make  curl  record
              incoming cookies too, which may be handy if you’re using this in
              combination with the -L/--location option. The  file  format  of
              the  file  to  read cookies from should be plain HTTP headers or
              the Netscape/Mozilla cookie file format.

              NOTE that the file specified with -b/--cookie is  only  used  as
              input.  No cookies will be stored in the file. To store cookies,
              use the -c/--cookie-jar option or you could even save  the  HTTP
              headers to a file using -D/--dump-header!

              If  this  option is set more than once, the last one will be the
              one that’s used.

       -B/--use-ascii
              Enable ASCII transfer when using FTP or LDAP. For FTP, this  can
              also  be enforced by using an URL that ends with ";type=A". This
              option causes data sent to stdout to be in text mode  for  win32
              systems.

       --basic
              (HTTP)  Tells curl to use HTTP Basic authentication. This is the
              default and this option is usually pointless, unless you use  it
              to  override  a  previously  set  option  that  sets a different
              authentication method (such as --ntlm,  --digest,  or  --negoti-
              ate).

       --ciphers <list of ciphers>
              (SSL) Specifies which ciphers to use in the connection. The list
              of ciphers must specify valid ciphers. Read  up  on  SSL  cipher
              list           details           on           this          URL:
              http://www.openssl.org/docs/apps/ciphers.html

              NSS ciphers are done differently than OpenSSL  and  GnuTLS.  The
              full  list of NSS ciphers is in the NSSCipherSuite entry at this
              URL: http://directory.fedora.redhat.com/docs/mod_nss.html#Direc-
              tives

              If this option is used several times, the last one will override
              the others.

       --compressed
              (HTTP) Request a compressed response using one of the algorithms
              libcurl supports, and return the uncompressed document.  If this
              option is used and the server  sends  an  unsupported  encoding,
              curl will report an error.

       --connect-timeout <seconds>
              Maximum  time  in  seconds  that you allow the connection to the
              server to take.  This only limits  the  connection  phase,  once
              curl  has  connected this option is of no more use. See also the
              -m/--max-time option.

              If this option is used several times, the last one will be used.

       -c/--cookie-jar <file name>
              Specify to which file you want curl to write all cookies after a
              completed operation. Curl writes  all  cookies  previously  read
              from  a  specified  file  as  well  as all cookies received from
              remote server(s). If no cookies are known, no file will be writ-
              ten.  The  file  will  be written using the Netscape cookie file
              format. If you set the file name to  a  single  dash,  "-",  the
              cookies will be written to stdout.

              NOTE If the cookie jar can’t be created or written to, the whole
              curl operation won’t fail or even report an error clearly. Using
              -v  will  get  a warning displayed, but that is the only visible
              feedback you get about this possibly lethal situation.

              If this option is used several times, the  last  specified  file
              name will be used.

       -C/--continue-at <offset>
              Continue/Resume  a  previous  file transfer at the given offset.
              The given offset is the exact  number  of  bytes  that  will  be
              skipped,  counting  from the beginning of the source file before
              it is transferred to the destination.  If used with uploads, the
              FTP server command SIZE will not be used by curl.

              Use  "-C  -" to tell curl to automatically find out where/how to
              resume the transfer. It then uses the given  output/input  files
              to figure that out.

              If this option is used several times, the last one will be used.

       --create-dirs
              When used in conjunction with the -o option,  curl  will  create
              the  necessary  local directory hierarchy as needed. This option
              creates the dirs mentioned with the -o option, nothing else.  If
              the  -o file name uses no dir or if the dirs it mentions already
              exist, no dir will be created.

              To create remote directories when using FTP or SFTP, try  --ftp-
              create-dirs.

       --crlf (FTP) Convert LF to CRLF in upload. Useful for MVS (OS/390).

       --crlfile <file>
              (HTTPS/FTPS)  Provide a file using PEM format with a Certificate
              Revocation List that may specify peer certificates that  are  to
              be considered revoked.

              If this option is used several times, the last one will be used.

              (Added in 7.19.7)

       -d/--data <data>
              (HTTP) Sends the specified data in a POST request  to  the  HTTP
              server,  in  the  same  way  that a browser does when a user has
              filled in an HTML form and presses the submit button. This  will
              cause curl to pass the data to the server using the content-type
              application/x-www-form-urlencoded.  Compare to -F/--form.

              -d/--data is the same  as  --data-ascii.  To  post  data  purely
              binary, you should instead use the --data-binary option. To URL-
              encode the value of a form field you may use --data-urlencode.

              If any of these options is used more than once on the same  com-
              mand  line,  the  data  pieces specified will be merged together
              with a separating  &-symbol.  Thus,  using  ’-d  name=daniel  -d
              skill=lousy’  would  generate  a  post  chunk  that  looks  like
              ’name=daniel&skill=lousy’.

              If you start the data with the letter @, the rest  should  be  a
              file  name  to read the data from, or - if you want curl to read
              the data from stdin.  The contents of the file must  already  be
              URL-encoded.  Multiple files can also be specified. Posting data
              from a file named  ’foobar’  would  thus  be  done  with  --data
              @foobar.

       --data-binary <data>
              (HTTP)  This  posts data exactly as specified with no extra pro-
              cessing whatsoever.

              If you start the data with the letter @, the rest  should  be  a
              filename.   Data  is  posted in a similar manner as --data-ascii
              does, except that newlines are  preserved  and  conversions  are
              never done.

              If  this  option  is  used several times, the ones following the
              first will append data as described in -d/--data.

       --data-urlencode <data>
              (HTTP) This posts data, similar to the other --data options with
              the exception that this performs URL-encoding. (Added in 7.18.0)

              To be CGI-compliant, the <data> part should begin  with  a  name
              followed  by a separator and a content specification. The <data>
              part can be passed to curl using one of the following syntaxes:

              content
                     This will make curl URL-encode the content and pass  that
                     on.  Just  be careful so that the content doesn’t contain
                     any = or @ symbols, as that will  then  make  the  syntax
                     match one of the other cases below!

              =content
                     This  will make curl URL-encode the content and pass that
                     on. The preceding = symbol is not included in the data.

              name=content
                     This will make curl URL-encode the content part and  pass
                     that  on.  Note that the name part is expected to be URL-
                     encoded already.

              @filename
                     This will  make  curl  load  data  from  the  given  file
                     (including  any  newlines), URL-encode that data and pass
                     it on in the POST.

              name@filename
                     This will  make  curl  load  data  from  the  given  file
                     (including  any  newlines), URL-encode that data and pass
                     it on in the POST. The  name  part  gets  an  equal  sign
                     appended, resulting in name=urlencoded-file-content. Note
                     that the name is expected to be URL-encoded already.

       --delegation LEVEL
              Set LEVEL to tell the server what it is allowed to delegate when
              it comes to user credentials. Used with GSS/kerberos.

              none   Don’t allow any delegation.

              policy Delegates  if  and only if the OK-AS-DELEGATE flag is set
                     in the Kerberos service ticket,  which  is  a  matter  of
                     realm policy.

              always Unconditionally allow the server to delegate.

       --digest
              (HTTP) Enables HTTP Digest authentication. This is a authentica-
              tion that prevents the password from being sent over the wire in
              clear  text.  Use  this in combination with the normal -u/--user
              option  to  set  user  name  and  password.  See  also   --ntlm,
              --negotiate and --anyauth for related options.

              If  this option is used several times, the following occurrences
              make no difference.

       --disable-eprt
              (FTP) Tell curl to disable the use of the EPRT and LPRT commands
              when doing active FTP transfers. Curl will normally always first
              attempt to use EPRT, then LPRT before using PORT, but with  this
              option,  it  will  use PORT right away. EPRT and LPRT are exten-
              sions to the original FTP protocol, and  may  not  work  on  all
              servers, but they enable more functionality in a better way than
              the traditional PORT command.

              Since curl 7.19.0, --eprt can be used to explicitly enable  EPRT
              again and --no-eprt is an alias for --disable-eprt.

              Disabling  EPRT only changes the active behavior. If you want to
              switch to passive mode you need  to  not  use  -P/--ftp-port  or
              force it with --ftp-pasv.

       --disable-epsv
              (FTP)  Tell  curl  to  disable  the use of the EPSV command when
              doing passive FTP transfers. Curl  will  normally  always  first
              attempt  to  use EPSV before PASV, but with this option, it will
              not try using EPSV.

              Since curl 7.19.0, --epsv can be used to explicitly enable  EPRT
              again and --no-epsv is an alias for --disable-epsv.

              Disabling EPSV only changes the passive behavior. If you want to
              switch to active mode you need to use -P/--ftp-port.

       -D/--dump-header <file>
              Write the protocol headers to the specified file.

              This option is handy to use when you want to store  the  headers
              that  a  HTTP  site sends to you. Cookies from the headers could
              then  be  read  in  a  second  curl  invocation  by  using   the
              -b/--cookie option! The -c/--cookie-jar option is however a bet-
              ter way to store cookies.

              When used in FTP, the FTP server response lines  are  considered
              being "headers" and thus are saved there.

              If this option is used several times, the last one will be used.

       -e/--referer <URL>
              (HTTP) Sends the "Referer Page" information to the HTTP  server.
              This  can also be set with the -H/--header flag of course.  When
              used with -L/--location you can append ";auto" to the  --referer
              URL to make curl automatically set the previous URL when it fol-
              lows a Location: header. The ";auto" string can be  used  alone,
              even if you don’t set an initial --referer.

              If this option is used several times, the last one will be used.

<<more >>


Reference: 
Unix

1 comment:

  1. good...curl for rest webservices example

    ReplyDelete

Please provide your input, it will honer for me...