驽马十驾 驽马十驾

驽马十驾,功在不舍

目录
【ELK】ES JPA 下
/  

【ELK】ES JPA 下

上文提到了SpringBoot如何集成 ES JPA和其基本使用,接下来我们来看看更底层的类:ElasticsearchRestTemplate的使用。它其实是一个更具有灵活性的基于 ES HighLevelAPI 的 CRUD模板库,使用起来也非常方便。

增删改

通过新增 2 条数据进行操作

@Test
    public void save1() {
        IndexCoordinates index = IndexCoordinates.of("art");

        Article mock = new Article();
        mock.setId(1L);
        mock.setAuthor("王天黑");
        mock.setTitle("发现Kotlin一个神奇的bug");
        mock.setBody("本文将会通过具体的业务场景,由浅入深的引出Kotlin的一个bug,并告知大家这个bug的神奇之处,接着会带领大家去查找bug出现的原因,最后去规避这个bug。");
        mock.setTimes(568);
        mock.setPublishDate(LocalDate.of(2021, 1, 27));
        mock.setImages("https://juejin.cn/post/6921359126438084621");

        Article art = esRestTemplate.save(mock, index);
        System.out.println(art);

        //第二个
        Article article2 = new Article();
        article2.setId(2L);
        article2.setAuthor("刘禹锡");
        article2.setTitle("自从上了K8S,项目更新都不带停机的!");
        article2.setBody("如果你看了《Kubernetes太火了!花10分钟玩转它不香么?》一文的话,基本上已经可以玩转K8S了。其实K8S中还有一些高级特性也很值得学习,比如弹性扩缩应用、滚动更新、配置管理、存储卷、网关路由等。今天我们就来了解下这些高级特性,希望对大家有所帮助!");
        article2.setTimes(120);
        article2.setPublishDate(LocalDate.of(2020, 11, 27));
        article2.setImages("https://juejin.cn/post/6922236829278306311");
        esRestTemplate.save(article2, index);
        System.out.println("over...");
    }

通过日志可以看出对应的请求是

curl -iX GET 'http://my.search.com:9200/'
curl -iX HEAD 'http://my.search.com:9200/art?ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false'
curl -iX PUT 'http://my.search.com:9200/art?master_timeout=30s&timeout=30s' -d '{"settings":{"index":{"analysis":{"filter":{"my_pinyin":{"keep_joined_full_pinyin":"true","lowercase":"true","none_chinese_pinyin_tokenize":"false","keep_original":"true","keep_none_chinese_together":"true","keep_first_letter":"true","trim_whitespace":"true","type":"pinyin","keep_none_chinese":"true","keep_full_pinyin":"false"}},"char_filter":{"ue_char_filter":{"type":"mapping","mappings":["- => ,","— => ,"]}},"analyzer":{"ue_ik_pinyin_analyzer":{"filter":["my_pinyin"],"char_filter":["html_strip","ue_char_filter"],"type":"custom","tokenizer":"ik_max_word"},"ue-ngram":{"type":"custom","char_filter":["html_strip","ue_char_filter"],"tokenizer":"ngram_tokenizer"}},"tokenizer":{"ngram_tokenizer":{"token_chars":["letter","digit"],"min_gram":"2","type":"ngram","max_gram":"3"}}},"number_of_replicas":"1"}},"aliases":{}}'
curl -iX PUT 'http://my.search.com:9200/art/_mapping?master_timeout=30s&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false&ignore_throttled=false&timeout=30s' -d '{"properties":{"id":{"type":"keyword","index":true},"author":{"type":"text","analyzer":"ue-ngram","search_analyzer":"ue-ngram"},"title":{"type":"text","analyzer":"ik_max_word","search_analyzer":"ik_smart"},"body":{"type":"text","analyzer":"ik_max_word","search_analyzer":"ik_smart"},"times":{"type":"long"},"images":{"type":"keyword","index":false},"publishDate":{"type":"date","format":"basic_date"}}}'
curl -iX HEAD 'http://my.search.com:9200/book?ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false'
curl -iX PUT 'http://my.search.com:9200/art/_doc/1?timeout=1m' -d '{"_class":"club.hicode.dockerhi.entity.Article","id":1,"author":"王天黑","title":"发现Kotlin一个神奇的bug","body":"本文将会通过具体的业务场景,由浅入深的引出Kotlin的一个bug,并告知大家这个bug的神奇之处,接着会带领大家去查找bug出现的原因,最后去规避这个bug。","times":568,"images":"https://juejin.cn/post/6921359126438084621","publishDate":"20210127"}'
curl -iX PUT 'http://my.search.com:9200/art/_doc/2?timeout=1m' -d '{"_class":"club.hicode.dockerhi.entity.Article","id":2,"author":"刘禹锡","title":"自从上了K8S,项目更新都不带停机的!","body":"如果你看了《Kubernetes太火了!花10分钟玩转它不香么?》一文的话,基本上已经可以玩转K8S了。其实K8S中还有一些高级特性也很值得学习,比如弹性扩缩应用、滚动更新、配置管理、存储卷、网关路由等。今天我们就来了解下这些高级特性,希望对大家有所帮助!","times":120,"images":"https://juejin.cn/post/6922236829278306311","publishDate":"20201127"}'

分析后可以得出,保存的时候,进行了如下操作

  1. 检测ES服务是否正常
  2. 检测该index 是否存在
  3. 如果不存在,那么创建 index
  4. 最后插入数据

由此可以得出结论:<span>ElasticsearchRestTemplate</span>也是会自动创建索引的。

查询

下面给出一个最通用的 构建查询条件的示例

NativeSearchQuery build = new NativeSearchQueryBuilder()
    // 构建查询条件
    .withQuery(queryBuilder(obj))
    //构建过滤条件:不参与算法
    .withFilter(filterBuilder(obj))
    //排序
    .withSort(sortBuilder())
    //分页
    .withPageable(PageRequest.of((int) obj.getCurrent() - 1, (int) obj.getSize()))
    //高亮构建
    .withHighlightBuilder(builderHighlight())
    //高亮字段
    .withHighlightFields(builderHighlightFields()).build();

通过一个案例进行一次综合性讲解,实体类Article

@Setting(settingPath = "es/es-setting.json")
@Document(indexName = "art")
public class Article {

    @Id
    @Field(type = FieldType.Long)
    private Long id;

    @Field(type = FieldType.Text, searchAnalyzer = "ue-ngram", analyzer = "ue-ngram")
    private String author;

    @Field(type = FieldType.Text, searchAnalyzer = "ik_smart", analyzer = "ik_max_word")
    private String title;

    @Field(type = FieldType.Text, searchAnalyzer = "ik_smart", analyzer = "ik_max_word")
    private String body;

    @Field(type = FieldType.Keyword, index = false)
    private String images;

    @Field(type = FieldType.Date, format = DateFormat.date)
    private LocalDate publishDate;

    @Field(type = FieldType.Integer)
    private Integer asDelete;

    @Field(type = FieldType.Long)
    private Long readTimes;
    //...省略...//
}

上面是一个文章的数据结构,现在要实现一个需求:

查询出:阅读次数不小于 100 次的,且发布时间在 2020年10 月 1 日之后的,标题中带有 kotlin 或者作者是刘禹锡。

对应的 SQL 简化如下

where postDate > '2020-10-01' and (title like '%kotlin%' or author like '%禹锡%')  and readTimes>= 100

那么这个查询如何写了

/**
     * SQL:
     * WHERE postDate > '2020-10-01' AND (title LIKE '%kotlin%' OR author LIKE '%禹锡%')  AND readTimes>= 100
     */
    @Test
    public void testFind() {
        //0.构建 index
        IndexCoordinates index = IndexCoordinates.of("art");

        //1.1 通过 QueryBuilders 构建查询条件。将 title 的权重提高到 3
        MatchQueryBuilder titleQuery = QueryBuilders.matchQuery("title", "kotlin").boost(3.0f);
        MatchQueryBuilder authorQuery = QueryBuilders.matchQuery("author", "刘禹锡");
        BoolQueryBuilder complexQuery = QueryBuilders.boolQuery().should(titleQuery).should(authorQuery);

        //2.1 构建时间 Query。理论上应该作为 filter存在,此处为了综合 bool 查询,做此处理。
        RangeQueryBuilder dateQuery = QueryBuilders.rangeQuery("publishDate").gt("2020-10-01");

        //3.合并之后的 Query
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery().must(dateQuery).must(complexQuery);

        //4.构建 FilterBuilder,此处将阅读次数作为 filter 条件,提高效率,该字段不参与计分。理论上大于小于字段都应该作为 filter 存在
        RangeQueryBuilder readTimesQuery = QueryBuilders.rangeQuery("readTimes").gte(100);

        //5.设置高亮字段
        HighlightBuilder.Field[] highBuilder = new HighlightBuilder.Field[]{
                new HighlightBuilder.Field("title"), //title 高亮
                new HighlightBuilder.Field("author") //author 高亮
        };

        //6.构建分页
        PageRequest page = PageRequest.of(0, 5);

        //7.构建条件
        NativeSearchQuery query = new NativeSearchQueryBuilder()
                .withQuery(queryBuilder)
                .withFilter(readTimesQuery)
                .withPageable(page)
                .withHighlightFields(highBuilder)
                .build();

        //8.执行查询
        SearchHits<Article> result = esRestTemplate.search(query, Article.class, index);

        //9.打印结果
        result.getSearchHits().forEach(System.out::println);
    }

上述的代码注释已经比较完善了,可以自行理解。

关键难点:QueryBuilders的使用构建复杂的查询条件,后续可以自行根据上述例子进行摸索和查看。如果你的 ES基础不算差,那么很容易看懂的。

image.png

骐骥一跃,不能十步。驽马十驾,功在不舍。