課程大綱 1. 默認的分詞器 > standard > standard tokenizer：以單詞邊界進行切分 > standard token filter：什么都不做 > lowercase token filter：將所有字母轉換為小寫 > stop token filer（默認被禁用）：移除停用詞，比如a the it等等 2. 修改分詞器的設置 啟用english停用詞token filter ~~~ PUT /my_index { "settings": { "analysis": { "analyzer": { "es_std": { "type": "standard", "stopwords": "_english_" } } } } } ~~~ 標準分詞器，安排空格分割字符串，得到a dog is in the house六個詞 ~~~ GET /my_index/_analyze { "analyzer": "standard", "text": "a dog is in the house" } ~~~ 自定義分詞器，去掉英文中常用詞，is the and等頻繁詞 得到 dog 和 house兩個詞 ~~~ GET /my_index/_analyze { "analyzer": "es_std", # 使用索引下，自定義的分詞器 "text":"a dog is in the house" } ~~~ 3. 定制化自己的分詞器 ~~~ PUT /my_index { "settings": { "analysis": { "char_filter": { "&_to_and": { "type": "mapping", "mappings": ["&=> and"] } }, "filter": { "my_stopwords": { # 自定義一個stop tokenizer，去掉 the is "type": "stop", "stopwords": ["the", "a"] } }, "analyzer": { "my_analyzer": { "type": "custom", "char_filter": ["html_strip", "&_to_and"], # 調用HTML和自定義轉化 "tokenizer": "standard", # 標準分詞器 "filter": ["lowercase", "my_stopwords"] # 所有字母變成小寫，調用自定義stop tokenizer } } } } } ~~~ char_filter：字符映射 filter：字符過濾 ~~~ GET /my_index/_analyze { "text": "tom&jerry are a friend in the house, <a>, HAHA!!", # &變成and，<a>變成a,HAHA變成haha "analyzer": "my_analyzer" } PUT /my_index/_mapping/my_type { "properties": { "content": { "type": "text", "analyzer": "my_analyzer" } } } ~~~