Kibana returns "Connection Failed"

Question

I am trying to run ElasticSearch with Kibana in Windows 2008 R2.

I followed this article: Install-logstash-on-a-windows-server-with-kibana

Step by step, but all I get is:

Connection Failed
Possibility #1: Your elasticsearch server is down or unreachable

This can be caused by a network outage, or a failure of the Elasticsearch process. If you have recently run a query that required a terms facet to be executed it is possible the process has run out of memory and stopped. Be sure to check your Elasticsearch logs for any sign of memory pressure.
Possibility #2: You are running Elasticsearch 1.4 or higher

Elasticsearch 1.4 ships with a security setting that prevents Kibana from connecting. You will need to set http.cors.allow-origin in your elasticsearch.yml to the correct protocol, hostname, and port (if not 80) that your access Kibana from. Note that if you are running Kibana in a sub-url, you should exclude the sub-url path and only include the protocol, hostname and port. For example, http://mycompany.com:8080, not http://mycompany.com:8080/kibana.
Click back, or the home button, when you have resolved the connection issue

When I go to

http://XXX.XXX.XXX.XXX:9200/

I get:

{
  "status" : 200,
  "name" : "Benazir Kaur",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "1.4.0",
    "build_hash" : "bc94bd81298f81c656893ab1ddddd30a99356066",
    "build_timestamp" : "2014-11-05T14:26:12Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.2"
  },
  "tagline" : "You Know, for Search"
}

So it seems that the ElasticSearch is running, but for some reason the Kibana cannot connect to it.

The ElasticSearch logs contains an error:

[2014-11-08 13:02:41,474][INFO ][node                     ] [Virako] version[1.4.0], pid[5556], build[bc94bd8/2014-11-05T14:26:12Z]
[2014-11-08 13:02:41,490][INFO ][node                     ] [Virako] initializing ...
[2014-11-08 13:02:41,490][INFO ][plugins                  ] [Virako] loaded [], sites []
[2014-11-08 13:02:46,872][INFO ][node                     ] [Virako] initialized
[2014-11-08 13:02:46,872][INFO ][node                     ] [Virako] starting ...
[2014-11-08 13:02:47,402][INFO ][transport                ] [Virako] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.0.14:9300]}
[2014-11-08 13:02:47,558][INFO ][discovery                ] [Virako] elasticsearch/XyAjXnofTnG1CXgDoHrNsA
[2014-11-08 13:02:51,412][INFO ][cluster.service          ] [Virako] new_master [Virako][XyAjXnofTnG1CXgDoHrNsA][test04][inet[/192.168.0.14:9300]], reason: zen-disco-join (elected_as_master)
[2014-11-08 13:02:51,521][INFO ][gateway                  ] [Virako] recovered [0] indices into cluster_state
[2014-11-08 13:02:51,552][INFO ][http                     ] [Virako] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.0.14:9200]}
[2014-11-08 13:02:51,552][INFO ][node                     ] [Virako] started
[2014-11-08 13:11:04,781][WARN ][transport.netty          ] [Virako] exception caught on transport layer [[id: 0x3984a6b4, /192.168.0.14:58237 => /192.168.0.14:9300]], closing connection
java.io.StreamCorruptedException: invalid internal transport message format, got (47,45,54,20)
    at org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:47)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
    at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Any idea what am I doing wrong?

Answer 1

I have faced similar kind of issue.

If you are using elasticsearch-1.4 with Kibana-3 then add following parameters in elasticsearch.yml file

http.cors.allow-origin: "/.*/"
http.cors.enabled: true

Reference, https://gist.github.com/rmoff/379e6ce46eb128110f38

Answer 2

In my case the problem was caused by the HTTP_PROXY environment variable being set and the proxy being down.

It's not the most obvious cause, and there is no obvious way from the error message that you would think to look at that.

Unsetting HTTP_PROXY and restarting Kibana did the trick.