i trying records elasticsearch using java api. receive below error
n[[wild thing][localhost:9300][indices:data/read/search[phase/dfs]]]; nested: queryphaseexecutionexception[result window large, + size must less or equal to: [10000] [10101].
my code below
client client; try { client = transportclient.builder().build(). addtransportaddress(new inetsockettransportaddress(inetaddress.getbyname("localhost"), 9300)); int = 1; int = 100; while (from <= 131881) { searchresponse response = client .preparesearch("demo_risk_data") .setsearchtype(searchtype.dfs_query_then_fetch).setfrom(from) .setquery(querybuilders.boolquery().mustnot(querybuilders.termquery("user_agent", ""))) .setsize(to).setexplain(true).execute().actionget(); if (response.gethits().gethits().length > 0) { (searchhit searchdata : response.gethits().gethits()) { jsonobject value = new jsonobject(searchdata.getsource()); system.out.println(value.tostring()); } } } } total number of records present 131881 ,so start from = 1 , to = 100 , 100 records until from <= 131881. there way can check records in set of 100 until there no further records in elasticsearch.
yes, can using scroll api, java client also supports.
you can this:
client client; try { client = transportclient.builder().build(). addtransportaddress(new inetsockettransportaddress(inetaddress.getbyname("localhost"), 9300)); querybuilder qb = querybuilders.boolquery().mustnot(querybuilders.termquery("user_agent", "")); searchresponse scrollresp = client.preparesearch("demo_risk_data") .addsort(sortparseelement.doc_field_name, sortorder.asc) .setscroll(new timevalue(60000)) .setquery(qb) .setsize(100).execute().actionget(); //scroll until no hits returned while (true) { //break condition: no hits returned if (scrollresp.gethits().gethits().length == 0) { break; } // otherwise read results (searchhit hit : scrollresp.gethits().gethits()) { jsonobject value = new jsonobject(searchdata.getsource()); system.out.println(value.tostring()); } // prepare next query scrollresp = client.preparesearchscroll(scrollresp.getscrollid()).setscroll(new timevalue(60000)).execute().actionget(); } } 