Re: How is http request via /dev/tcp different from that of curl?

[Peng Yu]

But the bash script is fixed. I need to be able to stop without terminating
the tcp connection, then manually do something then send the response back.
It appears to be /dev/tcp/host/port is the only way to achieve this goal.

On Mon, May 20, 2024 at 1:33 AM alex xmb sw ratchev <fxmbsw7@gmail.com>
wrote:

>
>
> On Mon, May 20, 2024, 08:06 Peng Yu <pengyu.ut@gmail.com> wrote:
>
>> http://www.dest-unreach.org/socat/doc/socat.html#EXAMPLES
>>
>>
>> Let’s take socat as an example. I see none of the examples shown above
>> allows you make a TCP request, get the response, process it, then send
>> another request. The process part must allow arbitrary operations. It has
>> fixed ways of processing the responses but not arbitrary ones.
>>
>
> i said its ur job
> u'd do , socat tcp4:host:port exec:'bash myscript'
> and inside myscript u do the actions
> thats ur part
> the part of socat and so is to make conn and pass data thru .. the script
> is the data reader / sender
>
> Please clarify which example it allows you process it arbitrarily.
>>
>
> tell me a http2 url u work with i can try code something for
>
> On Sun, May 19, 2024 at 7:35 PM alex xmb sw ratchev <fxmbsw7@gmail.com>
>> wrote:
>>
>>> if u gimme an official example
>>> say , some simple http2 keep alive server url , u name it ..
>>> i can show u
>>>
>>> On Mon, May 20, 2024, 02:33 alex xmb sw ratchev <fxmbsw7@gmail.com>
>>> wrote:
>>>
>>>> two comments about ur first post
>>>> first , the last -H with newline and json data looks suspicious , ..
>>>> that it may not be the last sent ( in the list ) , whenever it makes a
>>>> difference
>>>>
>>>> and second , ur bash doesnt contain that json
>>>>
>>>> On Mon, May 20, 2024, 02:28 alex xmb sw ratchev <fxmbsw7@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Sun, May 19, 2024, 22:23 Peng Yu <pengyu.ut@gmail.com> wrote:
>>>>>
>>>>>> That is a simple example in my original post. For example, in a real
>>>>>> application, I may get some info from the server and then send a new
>>>>>> request based on the info I received. This will require full fledged
>>>>>> programming ability.
>>>>>>
>>>>>> The tools that you don’t seem to have such functionality. Nor it is
>>>>>> easy to access external programs in those tools like bash.
>>>>>>
>>>>>
>>>>> no ...
>>>>> its not as u describe it
>>>>>
>>>>> u use http2 keep alive
>>>>> u , with every coding language and tool , have to send tx1 then read
>>>>> rx1 then again for tx2 rx2
>>>>> if http'ed correctly , u can do all these
>>>>> u dont , it seems to me , cause u got some points complety wrong
>>>>>
>>>>> the tools , i said here , as also bash , do make tcp or udp or
>>>>> whatever conns
>>>>> example : socat or tcpclient or bash opens tcp conn to httpd
>>>>> this , only this , the tools do
>>>>> they dont parse the conns data much
>>>>> this is UR JOB , not conn making apps
>>>>> its : if u code right it works
>>>>> again : the tools make tcp conns
>>>>> if u dont exit the conn , or the httpd , it may still be connected
>>>>> if u get the sent headers and reads right , it all works
>>>>>
>>>>> On Sun, May 19, 2024 at 11:14 AM alex xmb sw ratchev <
>>>>>> fxmbsw7@gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sun, May 19, 2024, 13:46 Peng Yu <pengyu.ut@gmail.com> wrote:
>>>>>>>
>>>>>>>> I said I need to process the responses after the sentence you
>>>>>>>> quoted. This means that I can do whatever operations with those 
>>>>>>>> responses.
>>>>>>>> Can those tools have the ability allowing me to do so?
>>>>>>>>
>>>>>>>
>>>>>>> they make tcp conn , what u printf in is ur thing
>>>>>>> and when to read or printf more first
>>>>>>>
>>>>>>> i dont get ur http example much
>>>>>>>
>>>>>>> On Sat, May 18, 2024 at 3:29 AM alex xmb sw ratchev <
>>>>>>>> fxmbsw7@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> both tools i mentioned can put multiple user http requests ,
>>>>>>>>> depending on httpd 2's
>>>>>>>>>
>>>>>>>>> they connect , and send user data , and recieve
>>>>>>>>> say u can send 3 requests , then read ..
>>>>>>>>>
>>>>>>>>> ita not ' those tools dont support multi request ' - thats wrong
>>>>>>>>> knowledge
>>>>>>>>>
>>>>>>>>> On Fri, May 17, 2024, 20:03 Peng Yu <pengyu.ut@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Please use proper grammar. I am not following you.
>>>>>>>>>>
>>>>>>>>>> On Fri, May 17, 2024 at 10:06 AM alex xmb sw ratchev <
>>>>>>>>>> fxmbsw7@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, May 17, 2024, 13:44 Peng Yu <pengyu.ut@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I know. But I need to establish one tcp connection and submit
>>>>>>>>>>>> multiple requests using the same connection. I don’t want the 
>>>>>>>>>>>> connection be
>>>>>>>>>>>> closed while I process the response. None external tools can 
>>>>>>>>>>>> achieve this.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> what , ur dreaming ..
>>>>>>>>>>> if u http request , and the httpd is valid , u can send all
>>>>>>>>>>> kinds of http supported multi-things*
>>>>>>>>>>> no matter what tool all work
>>>>>>>>>>> its up to u having to make the http request n headers
>>>>>>>>>>> if u do well , and the httpd is not too junk , it ll work , no
>>>>>>>>>>> matter which tool
>>>>>>>>>>>
>>>>>>>>>>> On Fri, May 17, 2024 at 5:40 AM alex xmb sw ratchev <
>>>>>>>>>>>> fxmbsw7@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> other tools to make tcp scripts are socat
>>>>>>>>>>>>> http://www.dest-unreach.org/socat/ and tcpclient and
>>>>>>>>>>>>> tcpserver from ucspi-tcp https://cr.yp.to/ucspi-tcp.html
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, May 17, 2024, 01:25 Peng Yu <pengyu.ut@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> $ curl 'http://httpbin.org/get' -H 'User-Agent: Mozilla/5.0
>>>>>>>>>>>>>> (Macintosh; Intel Mac OS X 10.15; rv:125.0) Gecko/20100101
>>>>>>>>>>>>>> Firefox/125.0' -H 'Accept: application/json' -H
>>>>>>>>>>>>>> 'Accept-Language:
>>>>>>>>>>>>>> en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' -H
>>>>>>>>>>>>>> 'Referer:
>>>>>>>>>>>>>> http://httpbin.org/' -H 'DNT: 1' -H 'Sec-GPC: 1' -H
>>>>>>>>>>>>>> 'Connection:
>>>>>>>>>>>>>> keep-alive'
>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>   "args": {},
>>>>>>>>>>>>>>   "headers": {
>>>>>>>>>>>>>>     "Accept": "application/json",
>>>>>>>>>>>>>>     "Accept-Encoding": "gzip, deflate",
>>>>>>>>>>>>>>     "Accept-Language": "en-US,en;q=0.5",
>>>>>>>>>>>>>>     "Dnt": "1",
>>>>>>>>>>>>>>     "Host": "httpbin.org",
>>>>>>>>>>>>>>     "Referer": "http://httpbin.org/";,
>>>>>>>>>>>>>>     "Sec-Gpc": "1",
>>>>>>>>>>>>>>     "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X
>>>>>>>>>>>>>> 10.15;
>>>>>>>>>>>>>> rv:125.0) Gecko/20100101 Firefox/125.0",
>>>>>>>>>>>>>>     "X-Amzn-Trace-Id":
>>>>>>>>>>>>>> "Root=1-66469337-0aab56e96f4fee271f6a09ce"
>>>>>>>>>>>>>>   },
>>>>>>>>>>>>>>   "origin": "xxx.xxx.xxx.xxx",
>>>>>>>>>>>>>>   "url": "http://httpbin.org/get";
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I want to replicate the same thing in bash below. But the
>>>>>>>>>>>>>> result is
>>>>>>>>>>>>>> not the same. How is the bash http request via /dev/tcp
>>>>>>>>>>>>>> different from
>>>>>>>>>>>>>> that of curl?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> exec {fd}<>/dev/tcp/httbin.org/80
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> echo -e "GET /get HTTP/1.1\r
>>>>>>>>>>>>>> Host: httpbin.org\r
>>>>>>>>>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15;
>>>>>>>>>>>>>> rv:125.0)
>>>>>>>>>>>>>> Gecko/20100101 Firefox/125.0\r
>>>>>>>>>>>>>> Accept: application/json\r
>>>>>>>>>>>>>> Accept-Language: en-US,en;q=0.5\r
>>>>>>>>>>>>>> Accept-Encoding: gzip, deflate\r
>>>>>>>>>>>>>> Referer: http://httpbin.org/\r
>>>>>>>>>>>>>> DNT: 1\r
>>>>>>>>>>>>>> Sec-GPC: 1\r
>>>>>>>>>>>>>> Connection: keep-alive\r
>>>>>>>>>>>>>> \r" >&"$fd"
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> function httpprotocol/recv {
>>>>>>>>>>>>>>   while IFS= read -u "$fd" -r; do
>>>>>>>>>>>>>>     if [[ $REPLY =~ ^([^:]+):\ *(.*)$'\r'$ ]]; then
>>>>>>>>>>>>>>       headers[${BASH_REMATCH[1],,}]=${BASH_REMATCH[2]}
>>>>>>>>>>>>>>     elif [[ $REPLY = $'\r' ]]; then
>>>>>>>>>>>>>>       LC_ALL=C read -u "$fd" -r -d '' -N
>>>>>>>>>>>>>> "${headers[content-length]}" body
>>>>>>>>>>>>>>       break
>>>>>>>>>>>>>>     fi
>>>>>>>>>>>>>>  printf -v REPLY '%q' "$REPLY"
>>>>>>>>>>>>>>  echo "$REPLY"
>>>>>>>>>>>>>>   done
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> declare -A headers
>>>>>>>>>>>>>> $ httpprotocol/recv
>>>>>>>>>>>>>> $'HTTP/1.1 200 OK\r'
>>>>>>>>>>>>>> $'accept-ch: Sec-CH-UA, Sec-CH-UA-Platform,
>>>>>>>>>>>>>> Sec-CH-UA-Platform-Version, Sec-CH-UA-Mobile\r'
>>>>>>>>>>>>>> $'cache-control: max-age=0, private, must-revalidate\r'
>>>>>>>>>>>>>> $'connection: close\r'
>>>>>>>>>>>>>> $'content-length: 475\r'
>>>>>>>>>>>>>> $'content-type: text/html; charset=utf-8\r'
>>>>>>>>>>>>>> $'date: Thu, 16 May 2024 23:18:56 GMT\r'
>>>>>>>>>>>>>> $'server: nginx\r'
>>>>>>>>>>>>>> $'set-cookie: sid=ab0e69e8-13da-11ef-911e-ea384f7d4599;
>>>>>>>>>>>>>> path=/;
>>>>>>>>>>>>>> domain=.httpbin.org; expires=Wed, 04 Jun 2092 02:33:03 GMT;
>>>>>>>>>>>>>> max-age=2147483647; HttpOnly\r'
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> $ declare -p headers
>>>>>>>>>>>>>> declare -A headers=([server]="nginx" [connection]="close"
>>>>>>>>>>>>>> [content-type]="text/html; charset=utf-8"
>>>>>>>>>>>>>> [cache-control]="max-age=0,
>>>>>>>>>>>>>> private, must-revalidate"
>>>>>>>>>>>>>> [set-cookie]="sid=ab0e69e8-13da-11ef-911e-ea384f7d4599;
>>>>>>>>>>>>>> path=/;
>>>>>>>>>>>>>> domain=.httpbin.org; expires=Wed, 04 Jun 2092 02:33:03 GMT;
>>>>>>>>>>>>>> max-age=2147483647; HttpOnly" [content-length]="475"
>>>>>>>>>>>>>> [accept-ch]="Sec-CH-UA, Sec-CH-UA-Platform,
>>>>>>>>>>>>>> Sec-CH-UA-Platform-Version, Sec-CH-UA-Mobile" [date]="Thu, 16
>>>>>>>>>>>>>> May 2024
>>>>>>>>>>>>>> 23:18:56 GMT" )
>>>>>>>>>>>>>> $ echo "$body"
>>>>>>>>>>>>>> <html><head><title>Loading...</title></head><body><script
>>>>>>>>>>>>>> type='text/javascript'>window.location.replace('
>>>>>>>>>>>>>> http://httpbin.org/get?ch=1&js=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJKb2tlbiIsImV4cCI6MTcxNTkwODczNiwiaWF0IjoxNzE1OTAxNTM2LCJpc3MiOiJKb2tlbiIsImpzIjoxLCJqdGkiOiIydjgxbW1wajdvc2w5b2M4M2swMHBwNmMiLCJuYmYiOjE3MTU5MDE1MzYsInRzIjoxNzE1OTAxNTM2NTQzNzg3fQ.Bc07CLhZ_HLw5QwfQoWotZPfoegQMRYc84bD-Wqg96k&sid=ab0e69e8-13da-11ef-911e-ea384f7d4599')
>>>>>>>>>>>>>> ;</script></body></html>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Peng
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Peng
>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>> Regards,
>>>>>>>>>> Peng
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Peng
>>>>>>>>
>>>>>>> --
>>>>>> Regards,
>>>>>> Peng
>>>>>>
>>>>> --
>> Regards,
>> Peng
>>
> --
Regards,
Peng