10

when i execute

curl "http://weather.yahooapis.com/forecastrss?w=1225955&u=c"

it returns me response with incorrect encoding:

khan@khan-P55A-UD3P:~$ curl "http://weather.yahooapis.com/forecastrss?w=1225955&u=c" 
���dž��ud@3��v(
����$j$��~����4(���Xy����wH�o�9<q��,�s\��e"�tA�\݌h�ʄ���
                                                             �����h��M���{���J=�m93W
                                                                                      �S�)�e�[sv,�҉eAKM�z{ǔ��g��:���*�����(n�m��&�Jꟈ��Mg�,yn?F�&��_��
ik6                                                                                                                                                      >��0�e&X��簺
sQ~�:�Z;*9�.a"ߕ|��EO[�5"�׫[�k�����1ӆ�n?}r1�u�d��Cڐ��X��`�NF�g!�c��W��G��1�o����Z��53<z`���.��w� s׃����+�vh��3yt�b}�9
�6�s3K
�W�  �0�هF@���>�X֥Qh�ʰv�BQ�R
ʮ�<�4;�ڊ2�8y� �g���6M(��]�|'�U@�ș�B
�8du!�&'�NOB��ț��3�K��fW��
                           \Rheg�=��F�R;�u�F�s9���&����,��|r��o�E۲�T��V$&�����uf\������v��Z~&�Au��{��ى"m�ʨ���U����2�8�#0F@'������
                                                                                                                                           l���R�XL��~A��̱���p��9��8�iH��nC�i4��^t;����۪���d�V�����7��=S&��2�u�#v~�L`�k���v�0
                            �[���"<���~�z��j,���X=�zmKD/|���(�p��M���⥁}_�!��GџC��2|�G��<ফe��nb"x ?�e�s��;���r;ﲃ�]�9"l��;�}�w�ٮjwR[�C����#O�
                                                                                                                                                      �������#a����s�km���$a�����\)�$�o��Ә�K��FR�*�ý�l�Z
            �
             &�`_�D�WӠ�>`T��0��| c��⿎K%��n:���~(�����.{��}< /~�^!A��$\���c�<�Á
"��k�_��t����t�n�5�^t�وF��l<V�����oo?
                                        `O���3p��ĝ�S�X�G�x��Ź+�
khan@khan-P55A-UD3P:~$

However, the same command works just fine in another computer. is there anything i need to be setting in shell in order to get this in correct format ?

i m using ubuntu 14.04 64bits.(Linux khan-P55A-UD3P 3.13.0-40-generic #69-Ubuntu SMP Thu Nov 13 17:53:56 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux)

any ideas? a screenshot of the command can be seen here as well: https://i.sstatic.net/MlV2Q.png

Improve this question
1
  • 1
    Marcin asked you to save the output to a file; Does "cat output.xml |gunzip" produce the correct output? Try "hexdump -C output.xml" and post the first line or two. Try "curl -v url; echo $?" and post the headers and return value too. (My guess would be that either the website or a caching proxy is sending it compressed but not sending the correct headers indicating that for some reason or curl is not decompressing it despite getting the headers for some reason (Not compiled with libz support?))
    – Aleksi Torhamo
    Commented Nov 29, 2014 at 10:52

5 Answers 5

Reset to default
7

curl will automatically decompress the response if you set the --compressed flag:

curl --compressed "http://example.com"

--compressed (HTTP) Request a compressed response using one of the algorithms libcurl supports, and save the uncompressed document. If this option is used and the server sends an unsupported encoding, curl will report an error.

refrence of answer

if you need use this option only for gzip content encoding, use this command

curl -sH 'Accept-encoding: gzip' http://example.com/ | gunzip -
Improve this answer
3

I think is connected with default encoding of your terminal (which default is UTF-8). You can try to pass the stream to the file, for instance:

curl "http://weather.yahooapis.com/forecastrss?w=1225955&u=c" > response

I had the same problem with rest webservice, when I was passing bytes (Pdf content format inside Data Handler). Without passing stream I was receiving data encoded in UTF-8 in terminal and also when I was using soapUi.

Improve this answer
2
  • unfortunately it returns the same result ! khan@khan-P55A-UD3P:~$ curl "weather.yahooapis.com/forecastrss?w=1225955&u=c" > output.xml khan@khan-P55A-UD3P:~$ cat output.xml �Vێ�6}b¾��Z�,��m /�-�-�������THj�}�������C�b�23g.g�}+ x�%�$�|\&*2[�?�o�3�2��BI�${��G/^Dژ���i�-i�g6�zIrk��x�ugޞ�Jy�*�4-�࠘q5��v;ozJgt��!���t���tS)�q>����I�
    – Shrouk Khan
    Commented Nov 29, 2014 at 8:56
  • What if you open this file using for instance VIM?
    – Marcin Erbel
    Commented Nov 29, 2014 at 12:06
1

Try setting the charset of the terminal to utf-8. A google got me this: https://unix.stackexchange.com/questions/28136/set-gnome-terminals-default-charset-to-utf8

Before you set the encoding check to make sure that indeed is the issue by determining the current charset as in:

How to get terminal's Character Encoding

Improve this answer
1

There is one more solution to add Accept-Charset header:

 curl -vvv -H "Accept-Charset: utf-8" http://example.com/ > tmpFile
Improve this answer
-1

Sometimes is as easy as removing Accept-encoding: gzip from your request.

Improve this answer
1
  • But there is no such thing in the OP's request. And that "Accept-Encoding" field has already been covered in previous answers. Be sure when answering old question, with already upvoted answers, to bring new information.
    – chrslg
    Commented Nov 5, 2022 at 10:25

Not the answer you're looking for? Browse other questions tagged or ask your own question.