HTTP_REQUEST

Sends an HTTP request to the specified URL.

Syntax

CC "AdoScript" HTTP_REQUEST strValue [ method:strValue ]
    [ username:strValue ] [ password:strValue ]
    [ reqheader:mapValue ] [ reqbody:strValue ]
    [ base64-request:boolValue ] [ binary-request:boolValue ]
    [ base64:boolValue ] [ binary:boolValue ]
    [ certver-off:boolValue ] [ timeouts:tokenStr ] [ tries:intValue ]
    [ proxy:strValue ] [ proxyuser:strValue ] [ proxypw:strValue ] .
# --> RESULT ecode:intValue errmsg:strValue statuscode:intValue
#     header:mapValue response:strValue charset:strValue host:strValue

Parameters

• <main-parameter> (strValue) - The URL to send the request to.
• method (strValue, optional) - The HTTP method / verb for the request, e.g. "GET", "POST", "PUT" etc. Should be specified in upper-case. The default is "GET".
• username (strValue, optional) - The user name for basic authentication. The default is "".
• password (strValue, optional) - The password for basic authentication. The default is "".
• reqheader (mapValue, optional) - Additional headers to send with the request. The default is an empty map.
• reqbody (strValue, optional) - The body to be sent with the request. The default is "".
• base64-request (boolValue, optional) - When true, then the request body is first decoded using base64 before it is sent. The default is false.
• binary-request (boolValue, optional) - When true, then the request body is treated as binary data instead of text. The default is false.
• base64 (boolValue, optional) - When true, then the response body is encoded using base64 before it is returned. The default is false.
• binary (boolValue, optional) - When true, then the response body is treated as binary data instead of text. The default is false.
• certver-off (boolValue, optional) - When true, then verification of certificates (HTTPS) is turned off. The default is false.
• timeouts (tokenStr, optional) - Specifies the total timeout and the connection timeout for the request in milliseconds. The two values should be separated with a single white space. The default value is "90000 60000".
• tries (intValue, optional) - Set the maximum number of tries to perform the request. The default is 1.
• proxy (strValue, optional) - The proxy server to use. An empty value "" means to not use any proxy. The default is "".
• proxyuser (strValue, optional) - The user name to use with the proxy. The default is "".
• proxypw (strValue, optional) - The password to use with the proxy. The default is "".

Returns

• ecode (intValue) - Contains the error code or 0 in case of success. This only covers errors whether the request could be performed and a response received. Even if it is 0, the response can have an HTTP status code of 404, 500 etc.
• errmsg (strValue) - When ecode is non-0, then this contains a short description of the encountered error.
• statuscode (intValue) - The HTTP response status code, e.g. 200, 404 etc.
• header (mapValue) - The headers from the response.
• response (strValue) - The body from the response.
• host (strValue) - The "host" part of the URL provided as input.

Details

The "proxy" settings have not been tested yet.

When sending a "GET" HTTP request, then the parameters have to be put as part of the "query string" of the URL.

There is a limit for the size of data / files that can be downloaded through this command. It is slower than downloading for example through a web-browser or dedicated program like curl. Things we have managed to Download (and store using "AdoScript" FWRITE) and superficially test (times can vary based on used hardware):

• Python 3.9.12 as Gzipped source tarball: .tgz file ~25MB in 8 seconds
• Python 3.9.12 Windows installer (64-bit): .exe file ~28MB in 10 seconds
• SAP Scenes Basic Set: .zip file ~50MB in 30 seconds

Basic authentication is only used when both the username and password parameters are explicitly specified, although they can be empty. The same applies to the proxy authentication with proxyuser and proxypw.

The method parameter specifies what type of HTTP request methods should be sent. Any upper-case string can be specified, but typical HTTP methods are: "GET", "HEAD", "POST", "PUT", "DELETE", "CONNECT", "OPTIONS", "TRACE", and "PATCH".

The map provided through reqheaders is used to add headers to the request. The keys and values of the map are transformed into strings before building the sent header portion. By default the header "User-Agent: ADOxx" is added if no other value is specified for it through this map. The "Content-Length" header is also set by default based on the provided reqbody. The "Content-Type" header must however be specified manually.

The reqbody should be provided according to the used HTTP method and what the server expects to receive. It can contain text content or other type of content, for example the binary data from an image loaded with "AdoScript" FREAD. Also the "Content-Type" header should be specified as part of reqheaders according to the type of data that is sent, e.g. "text/plain", "application/json", "application/xml", "image/png" etc.

The four parameters base64-request, binary-request, base64 and binary are relevant when dealing with data that is not text, like image files. When data is considered to be text, then all the line breaks are transformed (to \n inside ADOxx, to \r\n outside ADOxx) and the string is considered to end at the NULL byte (00). These parameters are needed when the sent / received data should not be changed.

The timeouts are specified as a token string with two integer numbers: The first number should specify the total timeout for the request and the second number the timeout for establishing the connection (which is part of the total timeout). The timeouts are specified in milliseconds. A value of 0 indicates "indefinite". The previously used structure for timeouts (4 semicolon-separated integer) is translated as best as possible to the new timeouts.

The maximum number of tires to perform is specified with the tries parameter. A value of 0 means that the request is not sent at all. Each next try is also separated from the previous one by a pause. This pause increases with each retry, starting at ~2 seconds and doubling each retry up to a maximum of slightly over 2 minutes. This parameter has been renamed from the previous retry to better fit the semantics. retry can still be used, but should be avoided.

The proxy parameter can be used to specify a proxy server. When authentication with the proxy server is necessary, then the user name and password can be provided with proxyuser and proxypw respectively.

Avoid using debug as part of the command, as this can cause issues or crashes. Furthermore, when downloading / receiving binary data that can contain NULL bytes (00) as part of its normal data (e.g. PNG images) the use of debug will alter the received data and lead to errors during execution. Also do not use debug as part of the "AdoScript" FWRITE command to write the data to a file.

For backwards compatibility the charset return value is still provided. Just like before the value of is always an empty string.

Examples

Perform a simple HTTP GET request.

# Perform a simple HTTP request.
CC "AdoScript" HTTP_REQUEST ("https://httpbin.org/anything")
# Show the response received from the server.
CC "AdoScript" VIEWBOX text:("Response code (" + type(statuscode) + "): " +
    STR statuscode + "\n" + "Host (" + type(host) + "): " + host + "\n" +
    "Headers (" + type(header) + "):\n" + STR header + "\n\n" +
    "Response body (" + type(response) + "):\n" + response)

Perform an HTTP POST request with a textual body and custom headers.

# Perform the HTTP POST request.
CC "AdoScript" HTTP_REQUEST ("https://httpbin.org/anything")
    method:("POST")
    reqheader:({"additional-header":42, "another-header":"Hello World!"})
    reqbody:("A simple request body.\nWell, it does have a new line.")
# Show the response received from the server.
CC "AdoScript" VIEWBOX text:("Response code (" + type(statuscode) + "): " +
    STR statuscode + "\n" + "Host (" + type(host) + "): " + host + "\n" +
    "Headers (" + type(header) + "):\n" + STR header + "\n\n" +
    "Response body (" + type(response) + "):\n" + response)

Perform an HTTP POST request with a textual body and custom headers. The response body is encoded using base64.

# Perform the HTTP POST request.
CC "AdoScript" HTTP_REQUEST ("https://httpbin.org/anything")
    method:("POST")
    reqheader:({"additional-header":42, "another-header":"Hello World!"})
    reqbody:("A simple request body.\nWell, it does have a new line.")
    base64:(1)
# Show the response received from the server with the body decoded (base64).
CC "AdoScript" VIEWBOX text:("Response code (" + type(statuscode) + "): " +
    STR statuscode + "\n" + "Host (" + type(host) + "): " + host + "\n" +
    "Headers (" + type(header) + "):\n" + STR header + "\n\n" +
    "Response body (" + type(response) + "):\n" + response + "\n\n" +
    "Response body (decoded):\n" + base64decode(response))

Retrieve an image through an HTTP request, store it in a temporary file and open the image using the default system's image viewer. Also shows the path to the image file.

# Get the image data from an HTTP request.
CC "AdoScript" HTTP_REQUEST ("https://www.omilab.org/assets/images/omilab_logo_.png")
    binary:(1)
SETL binImageData:(response)
# Get a new temporary file name.
CC "AdoScript" GET_TEMP_FILENAME
SETL sTempImageFilepath:(filename + ".png")
# Store the image data in the temporary file.
CC "AdoScript" FWRITE file:(sTempImageFilepath) text:(binImageData) binary:(1)
# Open the file using the system's default application.
START (sTempImageFilepath)
# Show the path to the file.
CC "AdoScript" VIEWBOX text:("Image file location: " + sTempImageFilepath)

See Also

Work in progress.

Versions and Changes

Available since ADOxx 1.3

ADOxx 1.8.0

• The returned header value is now a map containing only the headers. The first line from the HTTP response is not provided. Instead, the status code is instead returned as statuscode.
• The timeout works a bit different now due to the differences between used implementations. The new ones only allow for a "total timeout" and a "connection timeout". Old timeout settings are translated as best as possible to the new ones.
• Previously it seemed to be necessary to call the fromutf8() function twice to get readable values from the response. Now once is enough.