Questions tagged [elasticsearch]

Elasticsearch is an Open Source (Apache 2), Distributed, RESTful, Search Engine based on Lucene.

0
votes
0answers
6 views

elasticsearch: denormalization and aggregation

I'm new to elasticsearch and having trouble to come up with a data model. In relational terms I have 3 layers of hierarchy, each one-to-many. If I denormalize I would simply repeat the top 2 levels. ...
0
votes
0answers
11 views

How to use fuzziness just when the term is misspelled?

I am using elasticsearch with a fuzziness 'AUTO', what I observed is that the queries matches so many words that has no sens with the wanted ones. My question is that is there any trick to limit the ...
0
votes
0answers
10 views

Querying from ElasticSearch with props (React)

I am attempting to pass in variables as the arguments for an Elasticsearch query, but the response is coming up empty. Just trying to find out if there's any reason why a var passed into the query ...
0
votes
0answers
12 views

how to index two mongodb collections in one Elasticsearch index

I have two mongodb collections, Transactions and Users,this is transaction example { "_id" : ObjectId("5cdd391e1e4b8f0cb8e17d0f"), "txId" : "...
0
votes
0answers
4 views

td-agent appends “message” to the data

I'm on windows environment, I have td-agent (made by treasuredata.com), elastic, kibana. I wrote a very simple client to send a simple json: "name:aaaaa" using tcp protocol to td-agent that is ...
0
votes
0answers
7 views

shingle in nested object

In Elasticsearch 6.3, I'm trying to apply bigram query in the nested object field which generates bigram data by combining all values in the nested field. Below is the mappings. PUT /testindex { "...
1
vote
0answers
14 views

Avoid joins by multiple select statements - Logstash

I'm using Logstash to migrate data from mysql to elasticsearch. My mysql database has a primary table called product that has many relations the query to select it contains around 46 left outer join ...
0
votes
0answers
19 views

How to get BoolQueryBuilder object from String in Elasticsearch 2.4.4 Java API?

I have a bool query in String form as: { "bool" : { "must" : [ { "term" : { "flag" : "N" } }, { "bool" : { "must" : { "exists" : { "...
1
vote
1answer
11 views

Elasticsearch term query to number token

I need to explain some weird behavior of term query to Elasticsearch database which contains number part in the string. Query is pretty simple: { "query": { "bool": { "should":...
1
vote
0answers
14 views

ElasticSearch on AWS: Token filters that require files

We use the ElasticSearch Service on AWS and we want to use the hyphenation decompounder token filter (https://www.elastic.co/guide/en/elasticsearch/reference/6.0/analysis-compound-word-tokenfilter....
0
votes
0answers
13 views

What's the most efficient way, to extract tables from specific pages from PDF to CSV

I want to extract specific pages from PDF's to tables into CSV. For this task I use tabula-py, textract(I get better results from PyPDF2 method), PyPDF2. For each PDF, I split each page, then I use ...
0
votes
0answers
14 views

Get a list of all duplicate documents in Elastic Search index

Imagine I have documents like this: { "customerId" : "some-id", ... } How can I retrieve a list of all documents that have a non-unique value in the customerId field? So, for this collection: {...
0
votes
0answers
7 views

upload the data into elastic search-kibana through file

I have data files. one of them is this https://drive.google.com/open?id=1JRZj8myVu1UHJ3jxZzzb8LSKKMicY0Qi. I want to upload them into elastic search through kibana what is the best way to do it?
0
votes
1answer
18 views

Elastic Search Make Case Insensitive

I am new to elastic search. I have below query GET deals2/_search { "size": 200, "_source": ["acquireInfo"], "query": { "bool": { "must": [ { "query_string": { ...
0
votes
0answers
23 views

SearchResponse of Elasticsearch java high level rest client is bringing an empty set of values as result

I am using Elasticsearch Java rest high level client in my spring boot application. I have successfully indexed the data using BulkRequest. But when I want to implement search I am unable to do so. I ...
0
votes
1answer
17 views

Sync MongoDB with ElasticSearch [Losgtach]

I want to sync my MongoDB data to ElasticSearch, I read a lot of posts talking about elasticsearch river plugin and mongo connector, but all of them are deprecated for mongo 4 and elasticsearch 7! As ...
0
votes
0answers
15 views

POC implementation of document ingestion into Elastic Search very slow

I am using the code (skeleton) below to ingest various documents such as emails (with potential attachments), pdfs, word documents etc. The List of documents below is populated by another piece of ...
0
votes
0answers
14 views

Elasticsearch - load ILM policy from json file

I have implemented through JAVA api an ILM policy. my code looks something like this: Map<String, Phase> phases = new HashMap<>(); Map<String, LifecycleAction> ...
0
votes
0answers
7 views

How can I update bonsai elastic search records, when I get a new entry in database?

How can I update bonsai elastic search records, when I get a new entry in database? https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html
1
vote
2answers
22 views

docker logs filebeat > file.log don't work

With Docker version 18.09.5, build e8ff056 and filebeat:7.1.1 (by elasticsearch) when I type: $ docker logs filebeat > filebeat.log I see the logs but the filebeat.log is empty. If I try: $ ...
0
votes
0answers
24 views

How to write unit tests in spring boot for elasticsearch

I have to write unit tests of spring boot application with elasticsearch. I've read a lot of documentation, and I've discovered that an embedded elasticsearch can be used, a bit like using H2 for ...
0
votes
0answers
17 views

How can I load data inside elasticsearch?

I create a dictionary, when I tried to load data in ES I get this ERROR, Please can someone explain what is mean the ERROR? final_adress = {} for i in DF['adress'].unique(): ...
1
vote
0answers
22 views

Suggest some database that supports analytics and large data fetching on complex conditions

I have a bulk data of users and tags. Each user can have multiple tags. each tag contains tagName and tagValue. tagValue can be number,string or array. Few operations I am performing on this data ...
3
votes
0answers
39 views

Docker for windows: docker pull elasticsearch read: connection reset by peer

I am using docker for windows (windows 10) with Linux containers. I am unable to pull out elastic search image. I am trying below: docker.elastic.co/elasticsearch/elasticsearch-oss:7.1.0 It is ...
0
votes
1answer
18 views

elasticdump How do I use offset?

When elasticdump is stopped and restarted, it tries to execute after offset. but an error occurs. [Execute command] nohup ./elasticdump --input=http://host/common --output=http://host/common --type=...
0
votes
0answers
17 views

Upload multiple files in kibana in same index

I have json files.I want to upload them into kibana. They have option for uploading a file. Which creates a new index every time.I want to upload file data into existing index how can I do that? ...
0
votes
1answer
15 views

ElasticSearch/Kibana: The correct syntax to exclude a value?

I'm racking my brain trying to work out why this search doesn't work properly. I want to find all documents where instance=ko14274 but exclude any that are maxlevel=UNKNOWN. I have tried the following:...
0
votes
1answer
22 views

Convert date to year-month-day format with Painless

I have dates in this format: 2018-07-24T08:27:59.259Z. What is the best way to convert this to 2018-07-24 in Painless? Looking through the Painless API reference, I realise there are methods such as ...
1
vote
1answer
24 views

Correct way to store MD5 hashes in an Elasticsearch index

I'm using Elasticsearch for creating a custom search API. I need to store MD5 hashes in one of the indices. I also need the field to be searchable. How should I index/analyze the field so that the ...
0
votes
0answers
32 views

Elasticsearch kubernetes pods failing with crashloopbackoff - Back-off restarting failed container

I am using these steps https://vocon-it.com/2019/03/04/kubernetes-9-installing-elasticsearch-using-helm-charts/ to install elasticsearch using helm charts. I am using these elasticsearch helm charts - ...
1
vote
2answers
27 views

Do all shards (within index) have the same content?

Do all shards (within index) have the same content? If yes, more shards = longer propagation (save) time? If no, when one of shards failed = data is incomplete when merging?
0
votes
1answer
19 views

Elasticsearch retrieve matched field in multi_match

Given a data structure where multiple fields are searched over, how can I retrieve the one that matched? Example data: person { "id": 123, "name": Bill, "name": William, "surname"...
0
votes
0answers
14 views

Set existing index as alias to another index in elasticsearch

I'm using an application that creates an index per month in elasticsearch. In a normal scenario the index and aliases should be as follow: { "api-umbrella-logs-v1-2019-06" : { "aliases" ...
2
votes
0answers
24 views

How you configure correctly elasticsearch 7.1.1 with spring boot 2.15?

I am using Elasticsearch 7.1.1 with spring-boot 2.1.5. I downloaded Elasticsearch and run it. When I start the spring project I am getting an error. The is "failed to load Elasticsearch nodes : org....
1
vote
1answer
18 views

How to set kibana index pattern from filebeat?

I am using elk stack with a node application. I am sending logs from host to logstash with filebeat, logsstash formats and send data to elastic and kibana reads from elastic. In kibana i see default ...
0
votes
1answer
32 views

Query on multiple range of document

What I want to search is to extract documents among certain range of documents, not the whole documents. I know ids of documents. For example, I want to query matching some sentences with query field -...
0
votes
1answer
21 views

Associating each document with a function to be satisfied by search parameters in Elasticsearch

In Elasticsearch, can I associate each document with a (different) function that must be satisfied by parameters I supply on a search, in order to be returned on that search? The particular functions ...
0
votes
1answer
29 views

what does ElasticSearch unlike Solr designed from the ground up to be a distributed index mean?

In a talk, I heard that ElasticSearch Unlike Solr, was designed from the ground up to be a distributed index I was wondering what that means by ElasticSearch designed from the ground up to be a ...
0
votes
0answers
34 views

What is the best way to read from existing Elasticsearch instances in Pyspark preventing mapping/parsing errors?

I have a lot of Elasticsearch indices with many fields that I want to use in Spark. As those indices don't have any mapping specified, using the data gets multiple parsing errors. What is the best way ...
0
votes
1answer
32 views

Is searching dynamic (webpage) content with Elasticsearch possible?

I want to implement a website search feature and I'm considering Elasticsearch for it. However, each webpage consists of various blocks of text which may or may not be viewable to the user, depending ...
1
vote
1answer
30 views

ES query from a golang server to ES returning error while the postman request directly to ES returns the expected results

This is the request body that I for this endpoint using Postman localhost:9201/response_v2_862875ee3a88a6d09c95bdbda029ce2b/_search { "_source": ["_id"], "from": 1, "size": 10, "query": { "bool": ...
0
votes
1answer
29 views

Elastic search advanced query for showing price changes on time series indexes

I have separated my time-series data into day wise indexes. Let's say I have 2 days indexes. 1) index1 data-timeseries-14-04-2019 : example data: {{"_id": 1, "product_name": "mobile1", "...
0
votes
1answer
32 views

How do I do an Anti Match Pattern on Keyword Field Elasticsearch Query 6.4.2

The problem: Our log data has 27-34 million entries for a /event-heartbeat. I need to filter those entries out to see just viable log messages in Kibana. Using Kibana filters with wildcards does ...
1
vote
1answer
28 views

How to declare Nested fieldtype for a list of custom object?

I have a class defined with nested type for SearchFields. I am able to save the documents but while retrieving using repository.findAll() I get the error: org.springframework.data.elasticsearch....
1
vote
2answers
25 views

ElasticSearch nesting Must and Should

I'm trying create some filters for a web application. Users need to be able to filter by users (checkboxes) and by types (also checkboxes). I need something like this (in T-SQL) to Elasticsearch : ...
-2
votes
1answer
25 views

Extract fields and its path from json object using python

I got json object from elasticsearch index mapping I want to group index fields from json object based on its type. https://gist.github.com/akthodu/47404880d2e5b6480a881214d41feb58 long field act....
0
votes
1answer
14 views

Wait for completion of UpdateByQuery with the elasticsearch DSL

I'm working with a really large dataset and I need to clean (remove) some properties of some documents, and immediately after to add such a property to other documents. Sometimes, the documents with ...
0
votes
0answers
19 views

ElastAlert New Term with Frequency

So I'm wanting to set up an alert that looks at javascript error messages. I want the alert to kick off if the error message is thrown 20 times, and within a 7 day time period. I've tried this ...
1
vote
2answers
16 views

How to get filebeat to ignore certain container logs

I am using elasticserach 6.8 and filebeat 6.8.0 in a Kubernetes cluster. I want filebeat to ignore certain container logs but it seems almost impossible :). This is my autodiscover config filebeat....
0
votes
0answers
17 views

How to put a json file into elastic search using logstash

I currently and attempting to place a json file into elastic-search using logstash. Im not sure if I am doing it correctly and I am not really sure how to check my elastic-search instance to make sure ...