help-bash
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How is http request via /dev/tcp different from that of curl?

From: Peng Yu
Subject: Re: How is http request via /dev/tcp different from that of curl?
Date: Thu, 16 May 2024 18:33:44 -0500 
When I try the same bash code on httpbun.com, it works fine.

It seems that bash leaves some signature so httpbin.org may identify
it as not a normal curl command.

On Thu, May 16, 2024 at 6:24 PM Peng Yu <pengyu.ut@gmail.com> wrote:
>
> Hi,
>
> $ curl 'http://httpbin.org/get' -H 'User-Agent: Mozilla/5.0
> (Macintosh; Intel Mac OS X 10.15; rv:125.0) Gecko/20100101
> Firefox/125.0' -H 'Accept: application/json' -H 'Accept-Language:
> en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' -H 'Referer:
> http://httpbin.org/' -H 'DNT: 1' -H 'Sec-GPC: 1' -H 'Connection:
> keep-alive'
> {
>   "args": {},
>   "headers": {
>     "Accept": "application/json",
>     "Accept-Encoding": "gzip, deflate",
>     "Accept-Language": "en-US,en;q=0.5",
>     "Dnt": "1",
>     "Host": "httpbin.org",
>     "Referer": "http://httpbin.org/";,
>     "Sec-Gpc": "1",
>     "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15;
> rv:125.0) Gecko/20100101 Firefox/125.0",
>     "X-Amzn-Trace-Id": "Root=1-66469337-0aab56e96f4fee271f6a09ce"
>   },
>   "origin": "xxx.xxx.xxx.xxx",
>   "url": "http://httpbin.org/get";
> }
>
> I want to replicate the same thing in bash below. But the result is
> not the same. How is the bash http request via /dev/tcp different from
> that of curl?
>
> exec {fd}<>/dev/tcp/httbin.org/80
>
> echo -e "GET /get HTTP/1.1\r
> Host: httpbin.org\r
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:125.0)
> Gecko/20100101 Firefox/125.0\r
> Accept: application/json\r
> Accept-Language: en-US,en;q=0.5\r
> Accept-Encoding: gzip, deflate\r
> Referer: http://httpbin.org/\r
> DNT: 1\r
> Sec-GPC: 1\r
> Connection: keep-alive\r
> \r" >&"$fd"
>
> function httpprotocol/recv {
>   while IFS= read -u "$fd" -r; do
>     if [[ $REPLY =~ ^([^:]+):\ *(.*)$'\r'$ ]]; then
>       headers[${BASH_REMATCH[1],,}]=${BASH_REMATCH[2]}
>     elif [[ $REPLY = $'\r' ]]; then
>       LC_ALL=C read -u "$fd" -r -d '' -N "${headers[content-length]}" body
>       break
>     fi
>  printf -v REPLY '%q' "$REPLY"
>  echo "$REPLY"
>   done
> }
>
> declare -A headers
> $ httpprotocol/recv
> $'HTTP/1.1 200 OK\r'
> $'accept-ch: Sec-CH-UA, Sec-CH-UA-Platform,
> Sec-CH-UA-Platform-Version, Sec-CH-UA-Mobile\r'
> $'cache-control: max-age=0, private, must-revalidate\r'
> $'connection: close\r'
> $'content-length: 475\r'
> $'content-type: text/html; charset=utf-8\r'
> $'date: Thu, 16 May 2024 23:18:56 GMT\r'
> $'server: nginx\r'
> $'set-cookie: sid=ab0e69e8-13da-11ef-911e-ea384f7d4599; path=/;
> domain=.httpbin.org; expires=Wed, 04 Jun 2092 02:33:03 GMT;
> max-age=2147483647; HttpOnly\r'
>
> $ declare -p headers
> declare -A headers=([server]="nginx" [connection]="close"
> [content-type]="text/html; charset=utf-8" [cache-control]="max-age=0,
> private, must-revalidate"
> [set-cookie]="sid=ab0e69e8-13da-11ef-911e-ea384f7d4599; path=/;
> domain=.httpbin.org; expires=Wed, 04 Jun 2092 02:33:03 GMT;
> max-age=2147483647; HttpOnly" [content-length]="475"
> [accept-ch]="Sec-CH-UA, Sec-CH-UA-Platform,
> Sec-CH-UA-Platform-Version, Sec-CH-UA-Mobile" [date]="Thu, 16 May 2024
> 23:18:56 GMT" )
> $ echo "$body"
> <html><head><title>Loading...</title></head><body><script
> type='text/javascript'>window.location.replace('http://httpbin.org/get?ch=1&js=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJKb2tlbiIsImV4cCI6MTcxNTkwODczNiwiaWF0IjoxNzE1OTAxNTM2LCJpc3MiOiJKb2tlbiIsImpzIjoxLCJqdGkiOiIydjgxbW1wajdvc2w5b2M4M2swMHBwNmMiLCJuYmYiOjE3MTU5MDE1MzYsInRzIjoxNzE1OTAxNTM2NTQzNzg3fQ.Bc07CLhZ_HLw5QwfQoWotZPfoegQMRYc84bD-Wqg96k&sid=ab0e69e8-13da-11ef-911e-ea384f7d4599');</script></body></html>
>
> --
> Regards,
> Peng



-- 
Regards,
Peng
reply via email to
[Prev in Thread] Current Thread [Next in Thread]