Skip to content

Piped URL to httpie with xargs creates a POST request unexpectedly #1017

@webdog

Description

@webdog

Hello!

Thank you for this utility! Has made requesting/testing URLs in development very handy for me.

I came across a bug that was resolved by using the instructions in #150 and #660 , but I am seeing unexpected behavior in the request itself that I couldn't find in reported in another issue.

When parsing a URL from a pipe, I expected httpie to create a GET request to the specified URL:

http https://api.weather.gov/points/41.9752,-87.9035 | jq '.properties.forecast | tostring' | sed 's/"//g' | xargs http

Translates to: http https://api.weather.gov/gridpoints/LOT/65,76/forecast

When I run the piped request, the URL returns 403 Forbidden, indicating an unauthorized request. Running the request with debug indicates I've sent a POST method to the URL (Refer to requests.request section)

❯ http  https://api.weather.gov/points/41.9752,-87.9035 | jq '.properties.forecast | tostring' | sed 's/"//g' | xargs http --debug                                                                                                      
HTTPie 2.2.0
Requests 2.24.0
Pygments 2.6.1
Python 3.9.0 (default, Dec  6 2020, 18:02:34) 
[Clang 12.0.0 (clang-1200.0.32.27)]
/usr/local/Cellar/httpie/2.2.0_1/libexec/bin/python
Darwin 19.6.0

<Environment {'colors': 256,
 'config': {'default_options': []},
 'config_dir': PosixPath('/Users/cwebdog/.config/httpie'),
 'is_windows': False,
 'log_error': <function Environment.log_error at 0x103ad0700>,
 'program_name': 'http',
 'stderr': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>,
 'stderr_isatty': True,
 'stdin': <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>,
 'stdin_encoding': 'utf-8',
 'stdin_isatty': False,
 'stdout': <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>,
 'stdout_encoding': 'utf-8',
 'stdout_isatty': True}>

>>> requests.request(**{'auth': None,
 'data': b'',
 'files': RequestFilesDict(),
 'headers': {'User-Agent': b'HTTPie/2.2.0'},
 'method': 'post',
 'params': RequestQueryParamsDict(),
 'url': 'https://api.weather.gov/gridpoints/LOT/65,76/forecast'})

HTTP/1.1 403 Forbidden
Cache-Control: max-age=0
Connection: close
Content-Length: 309
Content-Type: text/html
Date: Tue, 12 Jan 2021 16:42:18 GMT
Expires: Tue, 12 Jan 2021 16:42:18 GMT
Mime-Version: 1.0
Server: AkamaiGHost
Strict-Transport-Security: max-age=31536000 ; includeSubDomains ; preload
Vary: Accept,Feature-Flags,Accept-Language
X-Edge-Request-ID: 3d0c2d7

<HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
 
You don't have permission to access "http&#58;&#47;&#47;api&#46;weather&#46;gov&#47;gridpoints&#47;LOT&#47;65&#44;76&#47;forecast" on this server.<P>
Reference&#32;&#35;18&#46;6fe13217&#46;1610469738&#46;3d0c2d7
</BODY>
</HTML>

By appending --ignore-stdin to the request URL, or by declaring GET as the method in the first argument following http maintains the request as a GET:

❯ http  https://api.weather.gov/points/41.9752,-87.9035 | jq '.properties.forecast | tostring' | sed 's/"//g' | xargs http --ignore-stdin --debug                                                                                         
HTTPie 2.2.0
Requests 2.24.0
Pygments 2.6.1
Python 3.9.0 (default, Dec  6 2020, 18:02:34) 
[Clang 12.0.0 (clang-1200.0.32.27)]
/usr/local/Cellar/httpie/2.2.0_1/libexec/bin/python
Darwin 19.6.0

<Environment {'colors': 256,
 'config': {'default_options': []},
 'config_dir': PosixPath('/Users/cwebdog/.config/httpie'),
 'is_windows': False,
 'log_error': <function Environment.log_error at 0x1025a2700>,
 'program_name': 'http',
 'stderr': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>,
 'stderr_isatty': True,
 'stdin': <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>,
 'stdin_encoding': 'utf-8',
 'stdin_isatty': False,
 'stdout': <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>,
 'stdout_encoding': 'utf-8',
 'stdout_isatty': True}>

>>> requests.request(**{'auth': None,
 'data': RequestJSONDataDict(),
 'files': RequestFilesDict(),
 'headers': {'User-Agent': b'HTTPie/2.2.0'},
 'method': 'get',
 'params': RequestQueryParamsDict(),
 'url': 'https://api.weather.gov/gridpoints/LOT/65,76/forecast'})

HTTP/1.1 200 OK
Access-Control-Allow-Headers: Feature-Flags
Access-Control-Allow-Origin: *
Cache-Control: public, max-age=842, s-maxage=3600
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 1514
Content-Type: application/geo+json
Date: Tue, 12 Jan 2021 16:56:32 GMT
Expires: Tue, 12 Jan 2021 17:10:34 GMT
Last-Modified: Sun, 26 Dec 2021 06:21:50 GMT
Server: nginx/1.16.1
Strict-Transport-Security: max-age=31536000 ; includeSubDomains ; preload
Vary: Accept-Encoding
Vary: Accept,Feature-Flags,Accept-Language
X-Correlation-ID: 8942acf
X-Edge-Request-ID: 84a76f69
X-Request-ID: c9da5813-06a6-4b4d-ab6a-1789ae353037
X-Server-ID: vm-lnx-nids-apiapp3.ncep.noaa.gov

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions