# Term Vectors(詞條向量)
回有關特定文檔字段中的詞條的信息和統計信息。文檔可以存儲在索引中或由用戶人工提供。詞條向量默認為[實時](Get_API.md#realtime),不是近實時。這可以通過將`realtime`參數設置為`false`來更改。
```
GET /twitter/tweet/1/_termvectors
```
可選的,您可以使用`url`中的參數指定檢索信息的字段:
```
GET /twitter/tweet/1/_termvectors?fields=message
```
或通過在請求主體中添加請求的字段(參見下面的示例)。也可以使用通配符指定字段,類似于[多匹配查詢](../Query_DSL/Full_text_queries/Multi_Match_Query.md)
> 警告
>
> 請注意`/_termvector`的使用方式在2.0中已廢棄,請使用`_termvectors`替代。
## 返回值
請求可以得到三種類型的值:詞條信息,詞條統計和字段統計。默認情況下,所有詞條信息與字段統計信息都會被返回,但不包含詞條統計信息。
### 詞條信息
- 在字段中的詞頻(總是返回)
- 詞條位置(`positions`: `true`)
- 開始與結束的偏移量(`offsets`: `true`)
- 詞條有效載荷(`payloads`: `true`),base64編碼的字節
如果請求的信息沒有存儲在索引中,如果可能它將被即時計算。另外,對于甚至不存在于索引中但由用戶提供的文檔,也可以計算詞條向量。
> 警告
>
> 開始與結束的偏移量假設UTF-16編碼被使用。如果要使用這些偏移量來從原始文本中獲取詞條,則應確保使用UTF-16對正在使用的子字符串進行編碼。
### 詞條統計
設置`term_statistics`為`true`(默認為`false`)將返回:
- 總詞頻(所有文件中的詞條頻率)
- 文檔頻率(包含詞條的文檔數)
默認情況下這些值不返回,因為詞條統計數據會嚴重影響性能。
### 字段統計
將`field_statistics`設置為`false`(默認值為true)將省略:
- 文檔數(包含此字段的文檔數)
- 文檔頻率的總和(本字段中所有詞條的文檔頻率的總和)
- 詞頻的總和(該字段中每個詞條的詞頻的總和)
### 詞條過濾
使用參數`filter`,返回的詞條也可以根據其`tf-idf`分數進行過濾。這可能是有用的良好特征向量,以便找到文檔。此功能的工作方式與[More Like This Query](../Query_DSL/Specialized_queries/More_Like_This_Query.md)的[第二章節](../Query_DSL/Specialized_queries/More_Like_This_Query.md#mlt-query-term-selection)相似。參見示[例5](#docs-termvectors-terms-filtering)的使用。 支持以下子參數:
參數名描述`max_num_terms`每個字段必須返回的最大詞條數。默認為`25`。`min_term_freq`在源文檔中忽略少于此頻率的單詞。默認為`1`。`max_term_freq`在源文檔中忽略超過此頻率的單詞。默認為無界。`min_doc_freq`忽略文檔頻率少于此參數的詞條。默認為`1`。`max_doc_freq`忽略文檔頻率大于此參數的詞條。默認為無界。`min_word_length`字詞長度低于此參數的將被忽略。默認為`0`。`max_word_length`字詞長度大于此參數的將被忽略。默認為無界(`0`)。## 行為
詞條和字段統計數據不準確。刪除的文件不被考慮。這些信息只能用于所請求文檔所在的分片。因此,術語和字段統計信息僅用作相對度量,而絕對數字在此上下文中無意義。默認情況下,當請求人造文檔的詞條向量時,隨機選擇獲取統計信息的分片。使用`routing`將命中特定的分片。
### 示例:返回存儲詞條向量
首先,我們創建一個存儲詞條向量、有效載荷等的索引:
```
PUT /twitter/
{ "mappings": {
"tweet": {
"properties": {
"text": {
"type": "text",
"term_vector": "with_positions_offsets_payloads",
"store" : true,
"analyzer" : "fulltext_analyzer"
},
"fullname": {
"type": "text",
"term_vector": "with_positions_offsets_payloads",
"analyzer" : "fulltext_analyzer"
}
}
}
},
"settings" : {
"index" : {
"number_of_shards" : 1,
"number_of_replicas" : 0
},
"analysis": {
"analyzer": {
"fulltext_analyzer": {
"type": "custom",
"tokenizer": "whitespace",
"filter": [
"lowercase",
"type_as_payload"
]
}
}
}
}
}
```
然后,我們添加一些文檔:
```
PUT /twitter/tweet/1
{
"fullname" : "John Doe",
"text" : "twitter test test test "
}
PUT /twitter/tweet/2
{
"fullname" : "Jane Doe",
"text" : "Another twitter test ..."
}
```
以下請求返回文檔`1`(John Doe)中字段`text`的所有信息和統計信息:
```
GET /twitter/tweet/1/_termvectors
{
"fields" : ["text"],
"offsets" : true,
"payloads" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}
```
響應:
```
{
"_id": "1",
"_index": "twitter",
"_type": "tweet",
"_version": 1,
"found": true,
"took": 6,
"term_vectors": {
"text": {
"field_statistics": {
"doc_count": 2,
"sum_doc_freq": 6,
"sum_ttf": 8
},
"terms": {
"test": {
"doc_freq": 2,
"term_freq": 3,
"tokens": [
{
"end_offset": 12,
"payload": "d29yZA==",
"position": 1,
"start_offset": 8
},
{
"end_offset": 17,
"payload": "d29yZA==",
"position": 2,
"start_offset": 13
},
{
"end_offset": 22,
"payload": "d29yZA==",
"position": 3,
"start_offset": 18
}
],
"ttf": 4
},
"twitter": {
"doc_freq": 2,
"term_freq": 1,
"tokens": [
{
"end_offset": 7,
"payload": "d29yZA==",
"position": 0,
"start_offset": 0
}
],
"ttf": 2
}
}
}
}
}
```
### 示例:自動生成詞條向量
未明確存儲在索引中的詞條向量將自動計算。以下請求返回文檔`1`中字段的所有信息和統計信息,即使詞條尚未明確存儲在索引中。請注意,對于字段`text`,術語不會重新生成。
```
GET /twitter/tweet/1/_termvectors
{
"fields" : ["text", "some_field_without_term_vectors"],
"offsets" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}
```
### 示例:人造文檔
還可以為人造文檔生成詞條向量,也就是生成索引中不存在的文檔。例如,以下請求將返回與示例1中相同的結果。所使用的映射由索引和類型確定。
如果動態映射打開(默認),則不在原始映射中的文檔字段將被動態創建。
```
GET /twitter/tweet/_termvectors
{
"doc" : {
"fullname" : "John Doe",
"text" : "twitter test test test"
}
}
```
#### Per-field 分析器
另外,可以通過使用`per_field_analyzer`參數來提供不同于當前的分析器。這對于以任何方式生成詞條向量是有用的,特別是在使用人造文檔時。當為已經存儲的詞條向量提供分析器時,將重新生成項向量。
```
GET /twitter/tweet/_termvectors
{
"doc" : {
"fullname" : "John Doe",
"text" : "twitter test test test"
},
"fields": ["fullname"],
"per_field_analyzer" : {
"fullname": "keyword"
}
}
```
響應:
```
{
"_index": "twitter",
"_type": "tweet",
"_version": 0,
"found": true,
"took": 6,
"term_vectors": {
"fullname": {
"field_statistics": {
"sum_doc_freq": 2,
"doc_count": 4,
"sum_ttf": 4
},
"terms": {
"John Doe": {
"term_freq": 1,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 8
}
]
}
}
}
}
}
```
### 示例:詞條過濾
最后,返回的詞條可以根據他們的`tf-idf`分數進行過濾。在下面的例子中,我們從具有給定“plot”字段值的人造文檔中獲取三個“interesting”的關鍵字。請注意,關鍵字“Tony”或任何停止詞不是響應的一部分,因為它們的`tf-idf`必須太低。
```
GET /imdb/movies/_termvectors
{
"doc": {
"plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
},
"term_statistics" : true,
"field_statistics" : true,
"positions": false,
"offsets": false,
"filter" : {
"max_num_terms" : 3,
"min_term_freq" : 1,
"min_doc_freq" : 1
}
}
```
響應:
```
{
"_index": "imdb",
"_type": "movies",
"_version": 0,
"found": true,
"term_vectors": {
"plot": {
"field_statistics": {
"sum_doc_freq": 3384269,
"doc_count": 176214,
"sum_ttf": 3753460
},
"terms": {
"armored": {
"doc_freq": 27,
"ttf": 27,
"term_freq": 1,
"score": 9.74725
},
"industrialist": {
"doc_freq": 88,
"ttf": 88,
"term_freq": 1,
"score": 8.590818
},
"stark": {
"doc_freq": 44,
"ttf": 47,
"term_freq": 1,
"score": 9.272792
}
}
}
}
}
```
- 入門
- 基本概念
- 安裝
- 探索你的集群
- 集群健康
- 列出所有索引庫
- 創建一個索引庫
- 索引文檔創建與查詢
- 刪除一個索引庫
- 修改你的數據
- 更新文檔
- 刪除文檔
- 批量處理
- 探索你的數據
- 搜索API
- 查詢語言介紹
- 執行搜索
- 執行過濾
- 執行聚合
- 總結
- Elasticsearch設置
- 安裝Elasticsearch
- .zip或.tar.gz文件的安裝方式
- Install Elasticsearch with .zip on Windows
- Debian軟件包安裝方式
- RPM安裝方式
- Install Elasticsearch with Windows MSI Installer
- Docker安裝方式
- 配置Elasticsearch
- 安全配置
- 日志配置
- 重要的Elasticsearch配置
- 重要的系統配置
- 系統設置
- 在jvm.options中設置JVM堆大小
- 禁用swapping
- 文件描述符
- 虛擬內存
- 線程數
- DNS cache settings
- 啟動前檢查
- 堆大小檢查
- 文件描述符檢查
- 內存鎖定檢查
- 最大線程數檢查
- 最大虛擬內存檢查
- Max file size check
- 最大map數檢查
- JVM Client模式檢查
- 串行收集使用檢查
- 系統調用過濾檢查
- OnError與OnOutOfMemoryError檢查
- Early-access check
- G1GC檢查
- Elasticsearch停機
- Elasticsearch升級
- 滾動升級
- 全集群重啟升級
- 索引重建升級
- Set up X-Pack
- Installing X-Pack
- X-Pack Settings
- Watcher Settings
- Configuring Security
- Breaking changes in 6.0
- X-Pack Breaking Changes
- 重大變化
- 6.0的重大變化
- 聚合變化
- Cat API變化
- 客戶端變化
- 集群變化
- 文檔API變化
- 索引變化
- 預處理變化
- 映射變化
- Packaging變化
- Percolator變化
- 插件變化
- 索引重建變化
- 信息統計變化
- DSL查詢變化
- 設置變化
- 腳本變化
- API約定
- 多索引語法
- 索引庫名稱的日期運算
- 常用選項
- URL-based訪問控制
- 文檔APIs
- 讀寫文檔
- 索引接口
- Get接口
- Delete API
- Delete By Query API
- Update API
- Update By Query API
- Multi Get API
- Bulk API
- Reindex API
- Term Vectors
- Multi termvectors API
- ?refresh
- 搜索APIs
- Search
- URI Search
- Request Body Search
- Query
- From / Size
- Sort
- Source filtering
- Fields
- Script Fields
- Doc value Fields
- Post filter
- Highlighting
- Rescoring
- Search Type
- Scroll
- Preference
- Explain
- Version
- Index Boost
- min_score
- Named Queries
- Inner hits
- Field Collapsing
- Search After
- Search Template
- Multi Search Template
- Search Shards API
- Suggesters
- Term suggester
- Phrase Suggester
- Completion Suggester
- Context Suggester
- Returning the type of the suggester
- Multi Search API
- Count API
- Validate API
- Explain API
- Profile API
- Profiling Queries
- Profiling Aggregations
- Profiling Considerations
- Field Capabilities API
- Aggregations
- Metrics Aggregations
- 平均值聚合
- 值計數聚合(Value Count Aggregation)
- Cardinality Aggregation
- Extended Stats Aggregation
- 地理邊界聚合
- 地理重心聚合
- Max Aggregation
- Min Aggregation
- Percentiles Aggregation
- Percentile Ranks Aggregation
- Scripted Metric Aggregation
- Stats Aggregation
- Sum Aggregation
- Top hits Aggregation
- Value Count Aggregation
- Bucket Aggregations
- 鄰接矩陣聚合
- Children Aggregation
- Date Histogram Aggregation
- Date Range Aggregation
- Significant Terms Aggregation
- Filter Aggregation(過濾器聚合)
- Filters Aggregation
- Geo Distance Aggregation(地理距離聚合) 轉至元數據結尾
- GeoHash grid Aggregation(GeoHash網格聚合)
- Global Aggregation(全局聚合) 轉至元數據結尾
- Histogram Aggregation
- IP Range Aggregation(IP范圍聚合)
- Missing Aggregation
- Nested Aggregation(嵌套聚合)
- Range Aggregation(范圍聚合)
- Reverse nested Aggregation
- Sampler Aggregation
- Significant Terms Aggregation
- Significant Text Aggregation
- Terms Aggregation
- Pipeline Aggregations
- Avg Bucket Aggregation
- Derivative Aggregation(導數聚合)
- Max Bucket Aggregation
- Min Bucket Aggregation
- Sum Bucket Aggregation
- Stats Bucket Aggregation
- Extended Stats Bucket Aggregation(擴展信息桶聚合)
- Percentiles Bucket Aggregation(百分數桶聚合)
- Moving Average Aggregation
- Cumulative Sum Aggregation(累積匯總聚合)
- Bucket Script Aggregation(桶腳本聚合)
- Bucket Selector Aggregation(桶選擇器聚合)
- Serial Differencing Aggregation(串行差異聚合)
- Matrix Aggregations
- Matrix Stats
- Caching heavy aggregations
- Returning only aggregation results
- Aggregation Metadata
- Returning the type of the aggregation
- Indices APIs
- Create Index /創建索引
- Delete Index /刪除索引
- Get Index /獲取索引
- Indices Exists /索引存在
- Open / Close Index API /啟動關閉索引
- Shrink Index /縮小索引
- Rollover Index/滾動索引
- Put Mapping /提交映射
- Get Mapping /獲取映射
- Get Field Mapping /獲取字段映射
- Types Exists
- Index Aliases
- Update Indices Settings
- Get Settings
- Analyze
- Explain Analyze
- Index Templates
- 索引統計信息
- 索引段
- 索引恢復
- 索引分片存儲
- 清理緩存
- 刷新
- 同步刷新
- 重新加載
- 強制合并
- Cat APIs
- cat aliases
- cat allocation
- cat count
- cat fielddata
- cat health
- cat indices
- cat master
- cat nodeattrs
- cat nodes
- cat pending tasks
- cat plugins
- cat recovery
- cat repositories
- cat segments
- cat shards
- cat thread pool
- cat snapshots
- cat templates
- Cluster APIs
- 集群健康
- 集群狀態
- 集群統計
- 掛起的集群任務
- 集群重新路由
- Cluster Update Settings
- Nodes Stats
- Nodes Info
- Nodes Feature Usage
- Remote Cluster Info
- Task Management API
- Nodes hot_threads
- Cluster Allocation Explain API
- Query DSL
- 查詢context與過濾context
- Match All Query
- 全文搜索
- 匹配查詢
- 短語匹配查詢
- 短語前綴匹配查詢
- 多字段查詢
- 常用術語查詢
- 查詢語句查詢
- 簡單查詢語句
- Term level queries
- Term Query
- Terms Query
- Range Query
- Exists Query
- Prefix Query
- Wildcard Query
- Regexp Query
- Fuzzy Query
- Type Query
- Ids Query
- 復合查詢
- Constant Score 查詢
- Bool 查詢
- Dis Max 查詢
- Function Score 查詢
- Boosting 查詢
- Joining queries
- Has Child Query
- Has Parent Query
- Nested Query(嵌套查詢)
- Parent Id Query
- Geo queries
- GeoShape Query(地理形狀查詢)
- Geo Bounding Box Query(地理邊框查詢)
- Geo Distance Query(地理距離查詢)
- Geo Polygon Query(地理多邊形查詢)
- Specialized queries
- More Like This Query
- Script Query
- Percolate Query
- Span queries
- Span Term 查詢
- Span Multi Term 查詢
- Span First 查詢
- Span Near 查詢
- Span Or 查詢
- Span Not 查詢
- Span Containing 查詢
- Span Within 查詢
- Span Field Masking 查詢 轉至元數據結尾
- Minimum Should Match
- Multi Term Query Rewrite
- Mapping
- Removal of mapping types
- Field datatypes
- Array
- Binary
- Range
- Boolean
- Date
- Geo-point datatype
- Geo-Shape datatype
- IP datatype
- Keyword datatype
- Nested datatype
- Numeric datatypes
- Object datatype
- Text
- Token數
- 滲濾型
- join datatype
- Meta-Fields
- _all field
- _field_names field
- _id field
- _index field
- _meta field
- _routing field
- _source field
- _type field
- _uid field
- Mapping parameters
- analyzer(分析器)
- normalizer(歸一化)
- boost(提升)
- Coerce(強制類型轉換)
- copy_to(合并參數)
- doc_values(文檔值)
- dynamic(動態設置)
- enabled(開啟字段)
- eager_global_ordinals
- fielddata(字段數據)
- format (日期格式)
- ignore_above(忽略超越限制的字段)
- ignore_malformed(忽略格式不對的數據)
- index (索引)
- index_options(索引設置)
- fields(字段)
- Norms (標準信息)
- null_value(空值)
- position_increment_gap(短語位置間隙)
- properties (屬性)
- search_analyzer (搜索分析器)
- similarity (匹配方法)
- store(存儲)
- Term_vectors(詞根信息)
- Dynamic Mapping
- Dynamic field mapping(動態字段映射)
- Dynamic templates(動態模板)
- default mapping(mapping中的_default_)
- Analysis
- Anatomy of an analyzer(分析器的分析)
- Testing analyzers(測試分析器)
- Analyzers(分析器)
- Configuring built-in analyzers(配置內置分析器)
- Standard Analyzer(標準分析器)
- Simple Analyzer(簡單分析器)
- 空白分析器
- Stop Analyzer
- Keyword Analyzer
- 模式分析器
- 語言分析器
- 指紋分析器
- 自定義分析器
- Normalizers
- Tokenizers(分詞器)
- Standard Tokenizer(標準分詞器)
- Letter Tokenizer
- Lowercase Tokenizer (小寫分詞器)
- Whitespace Analyzer
- UAX URL Email Tokenizer
- Classic Tokenizer
- Thai Tokenizer(泰語分詞器)
- NGram Tokenizer
- Edge NGram Tokenizer
- Keyword Analyzer
- Pattern Tokenizer
- Simple Pattern Tokenizer
- Simple Pattern Split Tokenizer
- Path Hierarchy Tokenizer(路徑層次分詞器)
- Token Filters(詞元過濾器)
- Standard Token Filter
- ASCII Folding Token Filter
- Flatten Graph Token Filter
- Length Token Filter
- Lowercase Token Filter
- Uppercase Token Filter
- NGram Token Filter
- Edge NGram Token Filter
- Porter Stem Token Filter
- Shingle Token Filter
- Stop Token Filter
- Word Delimiter Token Filter
- Word Delimiter Graph Token Filter
- Stemmer Token Filter
- Stemmer Override Token Filter
- Keyword Marker Token Filter
- Keyword Repeat Token Filter
- KStem Token Filter
- Snowball Token Filter
- Phonetic Token Filter
- Synonym Token Filter
- Synonym Graph Token Filter
- Compound Word Token Filters
- Reverse Token Filter
- Elision Token Filter
- Truncate Token Filter
- Unique Token Filter
- Pattern Capture Token Filter
- Pattern Replace Token Filter
- Trim Token Filter
- Limit Token Count Token Filter
- Hunspell Token Filter
- Common Grams Token Filter
- Normalization Token Filter
- CJK Width Token Filter
- CJK Bigram Token Filter
- Delimited Payload Token Filter
- Keep Words Token Filter
- Keep Types Token Filter
- Classic Token Filter
- Apostrophe Token Filter
- Decimal Digit Token Filter
- Fingerprint Token Filter
- Minhash Token Filter
- Character Filters(字符過濾器)
- HTML Strip Character Filter
- Mapping Character Filter
- Pattern Replace Character Filter
- 模塊
- Cluster
- 集群級路由和碎片分配
- 基于磁盤的分片分配
- 分片分配awareness
- 分片分配過濾
- Miscellaneous cluster settings
- Scripting
- Painless Scripting Language
- Lucene Expressions Language
- Advanced scripts using script engines
- Snapshot And Restore
- Thread Pool
- Index Modules(索引模塊)
- 預處理節點
- Pipeline Definition
- Ingest APIs
- Put Pipeline API
- Get Pipeline API
- Delete Pipeline API
- Simulate Pipeline API
- Accessing Data in Pipelines
- Handling Failures in Pipelines
- Processors
- Monitoring Elasticsearch
- X-Pack APIs
- X-Pack Commands
- How To
- Testing(測試)
- Glossary of terms
- Release Notes
- X-Pack Release Notes