I am not able to connect to elasticsearch in kubernetes inside docker. My elasticsearch is accessed via kubernetes and I have an index called 'radius_ml_posts'. I am using elasticsearch's python library to connect to elasticsearch. When I run the whole process on my python IDE (Spyder), it works just fine. However, when I try to run it inside a docker container, I get connection issues. What am I missing? Below are my configs and code:
The localhost:9200:
{
"name" : "elasticsearch-dev-client-6858c5f9dc-zbz8p",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "lJJbPJpJRaC1j7k5IGhj7g",
"version" : {
"number" : "6.7.0",
"build_flavor" : "oss",
"build_type" : "docker",
"build_hash" : "8453f77",
"build_date" : "2019-03-21T15:32:29.844721Z",
"build_snapshot" : false,
"lucene_version" : "7.7.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
My python code to connect to elasticsearch host:
def get_data_es(question):
es = Elasticsearch(hosts=[{"host": "elastic", "port": 9200}], connection_class=RequestsHttpConnection, max_retries=30,
retry_on_timeout=True, request_timeout=30)
#es = Elasticsearch(hosts='http://host.docker.internal:5000', connection_class=RequestsHttpConnection, max_retries=30, timeout=30)
doc = {'author': 'gunner','text': 'event', "timestamp": datetime.now()}
es.indices.refresh(index="radius_ml_posts")
res = es.index(index="radius_ml_posts", id = 1, body = doc)
res = es.search(index="radius_ml_posts", size = 30, body={ "query": {
"query_string": {
"default_field": "search_text",
"query": question
}
}
}
)
return res
My docker-compose.yml file:
version: '2.2'
services:
elastic:
image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.7.0
container_name: elastic
environment:
- discovery.type=single-node
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
ports:
- 9300:9300
- 9200:9200
networks:
- elastic
myimage:
image: myimage:myversion
ports:
- 5000:5000
expose:
- 5000
networks:
- elastic
volumes:
data01:
driver: local
networks:
elastic:
driver: bridge
My Dockerfile:
FROM python:3.7.4
COPY . /app
WORKDIR /app
RUN pip install --upgrade pip
RUN pip3 install -U nltk
RUN python3 -m nltk.downloader all
RUN pip --default-timeout=100 install -r requirements.txt
EXPOSE 5000
ENTRYPOINT ["python"]
CMD ["main.py"]
The docker commands I am running stepwise:
docker build -t myimage:myversion .docker-compose up
The error I am getting:
myimage_1 | Traceback (most recent call last):
myimage_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
myimage_1 | response = self.full_dispatch_request()
myimage_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
myimage_1 | rv = self.handle_user_exception(e)
myimage_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
myimage_1 | reraise(exc_type, exc_value, tb)
myimage_1 | File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
myimage_1 | raise value
myimage_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
myimage_1 | rv = self.dispatch_request()
myimage_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
myimage_1 | return self.view_functions[rule.endpoint](**req.view_args)
myimage_1 | File "main.py", line 41, in launch_app
myimage_1 | ques = get_data_es(ques1)
myimage_1 | File "/app/Text_Cleaning.py", line 32, in get_data_es
myimage_1 | es.indices.refresh(index="radius_ml_posts")
myimage_1 | File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 92, in _wrapped
myimage_1 | return func(*args, params=params, headers=headers, **kwargs)
myimage_1 | File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/indices.py", line 42, in refresh
myimage_1 | "POST", _make_path(index, "_refresh"), params=params, headers=headers
myimage_1 | File "/usr/local/lib/python3.7/site-packages/elasticsearch/transport.py", line 362, in perform_request
myimage_1 | timeout=timeout,
myimage_1 | File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_requests.py", line 157, in perform_request
myimage_1 | raise ConnectionError("N/A", str(e), e)
myimage_1 | elasticsearch.exceptions.ConnectionError: ConnectionError(HTTPConnectionPool(host='elastic', port=9200): Max retries exceeded with url: /radius_ml_posts/_refresh (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f967a9b1710>: Failed to establish a new connection: [Errno -2] Name or service not known'))) caused by: ConnectionError(HTTPConnectionPool(host='elastic', port=9200): Max retries exceeded with url: /radius_ml_posts/_refresh (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f967a9b1710>: Failed to establish a new connection: [Errno -2] Name or service not known')))
Please help in fixing the issue.
Thanks in advance.
