网站建设费属于什么税目,简述网页设计的流程,wordpress试试手气,专业网站公司注#xff1a;此为笔者学习狂神说ElasticSearch的实战笔记#xff0c;其中包含个人的笔记和理解#xff0c;仅做学习笔记之用#xff0c;更多详细资讯请出门左拐B站#xff1a;狂神说!!!
七、ElasticSearch实战
仿京东商城搜索#xff08;高亮#xff09; 1、工程创建…注此为笔者学习狂神说ElasticSearch的实战笔记其中包含个人的笔记和理解仅做学习笔记之用更多详细资讯请出门左拐B站狂神说!!!
七、ElasticSearch实战
仿京东商城搜索高亮 1、工程创建springboot 目录结构
2、基本编码
①导入依赖 propertiesjava.version1.8/java.versionproject.build.sourceEncodingUTF-8/project.build.sourceEncodingproject.reporting.outputEncodingUTF-8/project.reporting.outputEncodingspring-boot.version2.3.7.RELEASE/spring-boot.version/propertiesdependencies!-- jsoup解析页面 --!-- 解析网页 爬视频可 研究tiko --dependencygroupIdorg.jsoup/groupIdartifactIdjsoup/artifactIdversion1.10.2/version/dependency!-- fastjson --dependencygroupIdcom.alibaba/groupIdartifactIdfastjson/artifactIdversion1.2.70/version/dependencydependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-data-elasticsearch/artifactId
!-- version7.6.1/version--/dependencydependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-thymeleaf/artifactId/dependencydependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-web/artifactId/dependencydependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-devtools/artifactIdscoperuntime/scopeoptionaltrue/optional/dependencydependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-configuration-processor/artifactIdoptionaltrue/optional/dependencydependencygroupIdorg.projectlombok/groupIdartifactIdlombok/artifactIdoptionaltrue/optional/dependency②导入前端素材
③编写 application.preperties配置文件
# 更改端口防止冲突
server.port9999
# 关闭thymeleaf缓存
spring.thymeleaf.cachefalse④测试controller和view Controller
public class IndexController {GetMapping({/,index})public String index(){return index;}
} 到这里可以先去编写爬虫编写之后回到这里 ⑤编写Config
Configuration
public class ElasticSearchConfig {Beanpublic RestHighLevelClient restHighLevelClient(){RestHighLevelClient client new RestHighLevelClient(RestClient.builder(new HttpHost(127.0.0.1,9200,http)));return client;}
}⑥编写service 因为是爬取的数据那么就不走Dao以下编写都不会编写接口开发中必须严格要求编写 ContentService Service
public class ContentService {Autowiredprivate RestHighLevelClient restHighLevelClient;// 1、解析数据放入 es 索引中public Boolean parseContent(String keyword) throws IOException {// 获取内容ListContent contents HtmlParseUtil.parseJD(keyword);// 内容放入 es 中BulkRequest bulkRequest new BulkRequest();bulkRequest.timeout(2m); // 可更具实际业务是指for (int i 0; i contents.size(); i) {bulkRequest.add(new IndexRequest(jd_goods).id((i1)).source(JSON.toJSONString(contents.get(i)), XContentType.JSON));}BulkResponse bulk restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);restHighLevelClient.close();return !bulk.hasFailures();}// 2、根据keyword分页查询结果public ListMapString, Object search(String keyword, Integer pageIndex, Integer pageSize) throws IOException {if (pageIndex 0){pageIndex 0;}SearchRequest jd_goods new SearchRequest(jd_goods);// 创建搜索源建造者对象SearchSourceBuilder searchSourceBuilder new SearchSourceBuilder();// 条件采用精确查询 通过keyword查字段nameTermQueryBuilder termQueryBuilder QueryBuilders.termQuery(name, keyword);searchSourceBuilder.query(termQueryBuilder);searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));// 60s// 分页searchSourceBuilder.from(pageIndex);searchSourceBuilder.size(pageSize);// 高亮// ....// 搜索源放入搜索请求中jd_goods.source(searchSourceBuilder);// 执行查询返回结果SearchResponse searchResponse restHighLevelClient.search(jd_goods, RequestOptions.DEFAULT);restHighLevelClient.close();// 解析结果SearchHits hits searchResponse.getHits();ListMapString,Object results new ArrayList();for (SearchHit documentFields : hits.getHits()) {MapString, Object sourceAsMap documentFields.getSourceAsMap();results.add(sourceAsMap);}// 返回查询的结果return results;}
}⑦编写controller ⑧测试结果
1、解析数据放入 es 索引中 2、根据keyword分页查询结果 报错
java.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED一般是因为OTSClient被调用了shutDown其内部的I/O reactor均已被关闭。如果此时再调用OTSClient进行读写则会抛出这个错误。 此处是restHighLevelClient
3、爬虫jsoup 数据获取数据库、消息队列、爬虫、… ①搜索京东搜索页面并分析页面
http://search.jd.com/search?keywordjava页面如下 审查页面元素
页面列表idJ_goodsList 目标元素img、price、name ②爬取数据获取请求返回的页面信息筛选出可用的
创建HtmlParseUtil并简单编写 public class HtmlParseUtil {public static void main(String[] args) throws IOException {/// 使用前需要联网// 请求urlString url http://search.jd.com/search?keywordjava;// 1.解析网页(jsoup 解析返回的对象是浏览器Document对象)Document document Jsoup.parse(new URL(url), 30000);// 使用document可以使用在js对document的所有操作// 2.获取元素通过idElement j_goodsList document.getElementById(J_goodsList);// 3.获取J_goodsList ul 每一个 liElements lis j_goodsList.getElementsByTag(li);// 4.获取li下的 img、price、namefor (Element li : lis) {String img li.getElementsByTag(img).eq(0).attr(src);// 获取li下 第一张图片String name li.getElementsByClass(p-name).eq(0).text();String price li.getElementsByClass(p-price).eq(0).text();System.out.println();System.out.println(img : img);System.out.println(name : name);System.out.println(price : price);}}
}运行结果 原因是啥 一般图片特别多的网站所有的图片都是通过延迟加载的 发现img标签中并没有属性src的设置只是data-lazy-img设置图片加载的地址 创建HtmlParseUtil、改写
更改图片获取属性为 data-lazy-img与实体类结合实体类如下
Data
AllArgsConstructor
NoArgsConstructor
public class Content implements Serializable {private static final long serialVersionUID -8049497962627482693L;private String name;private String img;private String price;
}封装为方法
public class HtmlParseUtil {public static void main(String[] args) throws IOException {System.out.println(parseJD(java));}public static ListContent parseJD(String keyword) throws IOException {/// 使用前需要联网// 请求urlString url http://search.jd.com/search?keyword keyword;// 1.解析网页(jsoup 解析返回的对象是浏览器Document对象)Document document Jsoup.parse(new URL(url), 30000);// 使用document可以使用在js对document的所有操作// 2.获取元素通过idElement j_goodsList document.getElementById(J_goodsList);// 3.获取J_goodsList ul 每一个 liElements lis j_goodsList.getElementsByTag(li);
// System.out.println(lis);// 4.获取li下的 img、price、name// list存储所有li下的内容ListContent contents new ArrayListContent();for (Element li : lis) {// 由于网站图片使用懒加载将src属性替换为data-lazy-imgString img li.getElementsByTag(img).eq(0).attr(data-lazy-img);// 获取li下 第一张图片String name li.getElementsByClass(p-name).eq(0).text();String price li.getElementsByClass(p-price).eq(0).text();// 封装为对象Content content new Content(name,img,price);// 添加到list中contents.add(content);}
// System.out.println(contents);// 5.返回 listreturn contents;}
}结果展示 4、搜索高亮 在3、的基础上添加内容 ①ContentService // 3、 在2的基础上进行高亮查询
public ListMapString, Object highlightSearch(String keyword, Integer pageIndex, Integer pageSize) throws IOException {SearchRequest searchRequest new SearchRequest(jd_goods);SearchSourceBuilder searchSourceBuilder new SearchSourceBuilder();// 精确查询添加查询条件TermQueryBuilder termQueryBuilder QueryBuilders.termQuery(name, keyword);searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));searchSourceBuilder.query(termQueryBuilder);// 分页searchSourceBuilder.from(pageIndex);searchSourceBuilder.size(pageSize);// 高亮 HighlightBuilder highlightBuilder new HighlightBuilder();highlightBuilder.field(name);highlightBuilder.preTags(span stylecolor:red);highlightBuilder.postTags(/span);searchSourceBuilder.highlighter(highlightBuilder);// 执行查询searchRequest.source(searchSourceBuilder);SearchResponse searchResponse restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);// 解析结果 SearchHits hits searchResponse.getHits();ListMapString, Object results new ArrayList();for (SearchHit documentFields : hits.getHits()) {// 使用新的字段值高亮覆盖旧的字段值MapString, Object sourceAsMap documentFields.getSourceAsMap();// 高亮字段MapString, HighlightField highlightFields documentFields.getHighlightFields();HighlightField name highlightFields.get(name);// 替换if (name ! null){Text[] fragments name.fragments();StringBuilder new_name new StringBuilder();for (Text text : fragments) {new_name.append(text);}sourceAsMap.put(name,new_name.toString());}results.add(sourceAsMap);}return results;
}②ContentController
替换高亮的方法 ResponseBody
GetMapping(/h_search/{keyword}/{pageIndex}/{pageSize})
public ListMapString, Object highlightParse(PathVariable(keyword) String keyword,PathVariable(pageIndex) Integer pageIndex,PathVariable(pageSize) Integer pageSize) throws IOException {return contentService.highlightSearch(keyword,pageIndex,pageSize);
}③结果展示 最终页面 5、前后端分离简单使用Vue
①下载并引入Vue.min.js和axios.js
下载失败报错试试初始化一下npm npm install vue
npm install axios导入vue.min.js和axios.min.js ②修改静态页面
引入js
script th:src{/js/vue.min.js}/script
script th:src{/js/axios.min.js}/script实现数据的双向绑定 绑定点击事件去掉事件冒泡 遍历数据 !DOCTYPE html
html xmlns:thhttp://www.thymeleaf.org
headmeta charsetutf-8/title狂神说Java-ES仿京东实战/titlelink relstylesheet th:href{/css/style.css}/script th:src{/js/jquery.min.js}/script
/head
body classpg
div classpagediv idapp class mallist tmall- page-not-market !-- 头部搜索 --div idheader class header-list-appdiv classheaderLayoutdiv classheaderCon !-- Logo--h1 idmallLogoimg th:src{/images/jdlogo.png} alt/h1div classheader-extra!--搜索--div idmallSearch classmall-searchform namesearchTop classmallSearch-form clearfixfieldsetlegend天猫搜索/legenddiv classmallSearch-input clearfixdiv classs-combobox ids-combobox-685div classs-combobox-input-wrapinput v-modelkeyword typetext autocompleteoff idmqclasss-combobox-input aria-haspopuptrue/div/divbutton typesubmit click.preventsearchKey idsearchbtn搜索/button/div/fieldset/formul classrelKeyToplia狂神说Java/a/lilia狂神说前端/a/lilia狂神说Linux/a/lilia狂神说大数据/a/lilia狂神聊理财/a/li/ul/div/div/div/div/div!-- 商品详情页面 --div idcontentdiv classmain!-- 品牌分类 --form classnavAttrsFormdiv classattrs j_NavAttrs styledisplay:blockdiv classbrandAttr j_nav_branddiv classj_Brand attrdiv classattrKey品牌/divdiv classattrValuesul classav-collapse row-2lia href# 狂神说 /a/lilia href# Java /a/li/ul/div/div/div/div/form!-- 排序规则 --div classfilter clearfixa classfSort fSort-cur综合i classf-ico-arrow-d/i/aa classfSort人气i classf-ico-arrow-d/i/aa classfSort新品i classf-ico-arrow-d/i/aa classfSort销量i classf-ico-arrow-d/i/aa classfSort价格i classf-ico-triangle-mt/ii classf-ico-triangle-mb/i/a/div!-- 商品详情 --div classview grid-nosku div classproduct v-forresult in resultsdiv classproduct-iWrap!--商品封面--div classproductImg-wrapa classproductImgimg :srcresult.img/a/div!--价格--p classproductPriceem v-textresult.price/em/p!--标题--p classproductTitlea v-htmlresult.name/a/p!-- 店铺名 --div classproductShopspan店铺 狂神说Java /span/div!-- 成交信息 --p classproductStatusspan月成交em999笔/em/spanspan评价 a3/a/span/p/div/div/div/div/div/div
/div
script th:src{/js/vue.min.js}/script
script th:src{/js/axios.min.js}/script
scriptnew Vue({el:#app,data:{keyword: , // 搜索的关键字results:[] // 后端返回的结果},methods:{searchKey(){var keyword this.keyword;console.log(keyword);axios.get(h_search/keyword/0/20).then(response{console.log(response.data);this.resultsresponse.data;})}}});
/script
/body
/html测试 ElasticSearch-ElasticSearch实战-仿京东商城搜索高亮 到此完结笔者归纳、创作不易大佬们给个3连再起飞吧
