##掃描和滾屏
`scan(掃描)`搜索類型是和`scroll(滾屏)`API一起使用來從Elasticsearch里高效地取回巨大數量的結果而不需要付出深分頁的代價。
___scroll(滾屏)___
一個滾屏搜索允許我們做一個初始階段搜索并且持續批量從Elasticsearch里拉取結果直到沒有結果剩下。這有點像傳統數據庫里的_cursors(游標)_。
滾屏搜索會及時制作快照。這個快照不會包含任何在初始階段搜索請求后對index做的修改。它通過將舊的數據文件保存在手邊,所以可以保護index的樣子看起來像搜索開始時的樣子。
___scan(掃描)___
深度分頁代價最高的部分是對結果的全局排序,但如果禁用排序,就能以很低的代價獲得全部返回結果。為達成這個目的,可以采用`scan(掃描)`搜索模式。掃描模式讓Elasticsearch不排序,只要分片里還有結果可以返回,就返回一批結果。
為了使用_scan-and-scroll(掃描和滾屏)_,需要執行一個搜索請求,將`search_type` 設置成`scan`,并且傳遞一個`scroll`參數來告訴Elasticsearch滾屏應該持續多長時間。
``` js
GET /old_index/_search?search_type=scan&scroll=1m (1)
{
"query": { "match_all": {}},
"size": 1000
}
```
(1)保持滾屏開啟1分鐘。
這個請求的應答沒有包含任何命中的結果,但是包含了一個Base-64編碼的`_scroll_id(滾屏id)`字符串。現在我們可以將`_scroll_id` 傳遞給`_search/scroll`末端來獲取第一批結果:
``` js
GET /_search/scroll?scroll=1m (1)
c2Nhbjs1OzExODpRNV9aY1VyUVM4U0NMd2pjWlJ3YWlBOzExOTpRNV9aY1VyUVM4U0 <2>
NMd2pjWlJ3YWlBOzExNjpRNV9aY1VyUVM4U0NMd2pjWlJ3YWlBOzExNzpRNV9aY1Vy
UVM4U0NMd2pjWlJ3YWlBOzEyMDpRNV9aY1VyUVM4U0NMd2pjWlJ3YWlBOzE7dG90YW
xfaGl0czoxOw==
```
--------------------------------------------------
(1) 保持滾屏開啟另一分鐘。
(2) `_scroll_id` 可以在body或者URL里傳遞,也可以被當做查詢參數傳遞。
注意,要再次指定`?scroll=1m`。滾屏的終止時間會在我們每次執行滾屏請求時刷新,所以他只需要給我們足夠的時間來處理當前批次的結果而不是所有的匹配查詢的document。
這個滾屏請求的應答包含了第一批次的結果。雖然指定了一個1000的`size` ,但是獲得了更多的document。當掃描時,`size`被應用到每一個分片上,所以我們在每個批次里最多或獲得`size * number_of_primary_shards(size*主分片數)`個document。
> ####注意:
> 滾屏請求也會返回一個_新_的`_scroll_id`。每次做下一個滾屏請求時,必須傳遞前一次請求返回的`_scroll_id`。
如果沒有更多的命中結果返回,就處理完了所有的命中匹配的document。
> ####提示:
> 一些[Elasticsearch官方客戶端](http://www.elasticsearch.org/guide)提供_掃描和滾屏_的小助手。小助手提供了一個對這個功能的簡單封裝。
<!--
[[scan-scroll]]
=== scan and scroll
The `scan` search type and the `scroll` API((("scroll API", "scan and scroll"))) are used together to retrieve
large numbers of documents from Elasticsearch efficiently, without paying the
penalty of deep pagination.
`scroll`::
+
--
A _scrolled search_ allows us to((("scrolled search"))) do an initial search and to keep pulling
batches of results from Elasticsearch until there are no more results left.
It's a bit like a _cursor_ in ((("cursors")))a traditional database.
A scrolled search takes a snapshot in time. It doesn't see any changes that
are made to the index after the initial search request has been made. It does
this by keeping the old data files around, so that it can preserve its ``view''
on what the index looked like at the time it started.
--
`scan`::
The costly part of deep pagination is the global sorting of results, but if we
disable sorting, then we can return all documents quite cheaply. To do this, we
use the `scan` search type.((("scan search type"))) Scan instructs Elasticsearch to do no sorting, but
to just return the next batch of results from every shard that still has
results to return.
To use _scan-and-scroll_, we execute a search((("scan-and-scroll"))) request setting `search_type` to((("search_type", "scan and scroll")))
`scan`, and passing a `scroll` parameter telling Elasticsearch how long it
should keep the scroll open:
[source,js]
--------------------------------------------------
GET /old_index/_search?search_type=scan&scroll=1m <1>
{
"query": { "match_all": {}},
"size": 1000
}
--------------------------------------------------
<1> Keep the scroll open for 1 minute.
The response to this request doesn't include any hits, but does include a
`_scroll_id`, which is a long Base-64 encoded((("scroll_id"))) string. Now we can pass the
`_scroll_id` to the `_search/scroll` endpoint to retrieve the first batch of
results:
[source,js]
--------------------------------------------------
GET /_search/scroll?scroll=1m <1>
c2Nhbjs1OzExODpRNV9aY1VyUVM4U0NMd2pjWlJ3YWlBOzExOTpRNV9aY1VyUVM4U0 <2>
NMd2pjWlJ3YWlBOzExNjpRNV9aY1VyUVM4U0NMd2pjWlJ3YWlBOzExNzpRNV9aY1Vy
UVM4U0NMd2pjWlJ3YWlBOzEyMDpRNV9aY1VyUVM4U0NMd2pjWlJ3YWlBOzE7dG90YW
xfaGl0czoxOw==
--------------------------------------------------
<1> Keep the scroll open for another minute.
<2> The `_scroll_id` can be passed in the body, in the URL, or as a
query parameter.
Note that we again specify `?scroll=1m`. The scroll expiry time is refreshed
every time we run a scroll request, so it needs to give us only enough time
to process the current batch of results, not all of the documents that match
the query.
The response to this scroll request includes the first batch of results.
Although we specified a `size` of 1,000, we get back many more
documents.((("size parameter", "in scanning"))) When scanning, the `size` is applied to each shard, so you will
get back a maximum of `size * number_of_primary_shards` documents in each
batch.
NOTE: The scroll request also returns a _new_ `_scroll_id`. Every time
we make the next scroll request, we must pass the `_scroll_id` returned by the
_previous_ scroll request.
When no more hits are returned, we have processed all matching documents.
TIP: Some of the http://www.elasticsearch.org/guide[official Elasticsearch clients]
provide _scan-and-scroll_ helpers that provide an easy wrapper around this
functionality.((("clients", "providing scan-and-scroll helpers")))
-->
- Introduction
- 入門
- 是什么
- 安裝
- API
- 文檔
- 索引
- 搜索
- 聚合
- 小結
- 分布式
- 結語
- 分布式集群
- 空集群
- 集群健康
- 添加索引
- 故障轉移
- 橫向擴展
- 更多擴展
- 應對故障
- 數據
- 文檔
- 索引
- 獲取
- 存在
- 更新
- 創建
- 刪除
- 版本控制
- 局部更新
- Mget
- 批量
- 結語
- 分布式增刪改查
- 路由
- 分片交互
- 新建、索引和刪除
- 檢索
- 局部更新
- 批量請求
- 批量格式
- 搜索
- 空搜索
- 多索引和多類型
- 分頁
- 查詢字符串
- 映射和分析
- 數據類型差異
- 確切值對決全文
- 倒排索引
- 分析
- 映射
- 復合類型
- 結構化查詢
- 請求體查詢
- 結構化查詢
- 查詢與過濾
- 重要的查詢子句
- 過濾查詢
- 驗證查詢
- 結語
- 排序
- 排序
- 字符串排序
- 相關性
- 字段數據
- 分布式搜索
- 查詢階段
- 取回階段
- 搜索選項
- 掃描和滾屏
- 索引管理
- 創建刪除
- 設置
- 配置分析器
- 自定義分析器
- 映射
- 根對象
- 元數據中的source字段
- 元數據中的all字段
- 元數據中的ID字段
- 動態映射
- 自定義動態映射
- 默認映射
- 重建索引
- 別名
- 深入分片
- 使文本可以被搜索
- 動態索引
- 近實時搜索
- 持久化變更
- 合并段
- 結構化搜索
- 查詢準確值
- 組合過濾
- 查詢多個準確值
- 包含,而不是相等
- 范圍
- 處理 Null 值
- 緩存
- 過濾順序
- 全文搜索
- 匹配查詢
- 多詞查詢
- 組合查詢
- 布爾匹配
- 增加子句
- 控制分析
- 關聯失效
- 多字段搜索
- 多重查詢字符串
- 單一查詢字符串
- 最佳字段
- 最佳字段查詢調優
- 多重匹配查詢
- 最多字段查詢
- 跨字段對象查詢
- 以字段為中心查詢
- 全字段查詢
- 跨字段查詢
- 精確查詢
- 模糊匹配
- Phrase matching
- Slop
- Multi value fields
- Scoring
- Relevance
- Performance
- Shingles
- Partial_Matching
- Postcodes
- Prefix query
- Wildcard Regexp
- Match phrase prefix
- Index time
- Ngram intro
- Search as you type
- Compound words
- Relevance
- Scoring theory
- Practical scoring
- Query time boosting
- Query scoring
- Not quite not
- Ignoring TFIDF
- Function score query
- Popularity
- Boosting filtered subsets
- Random scoring
- Decay functions
- Pluggable similarities
- Conclusion
- Language intro
- Intro
- Using
- Configuring
- Language pitfalls
- One language per doc
- One language per field
- Mixed language fields
- Conclusion
- Identifying words
- Intro
- Standard analyzer
- Standard tokenizer
- ICU plugin
- ICU tokenizer
- Tidying text
- Token normalization
- Intro
- Lowercasing
- Removing diacritics
- Unicode world
- Case folding
- Character folding
- Sorting and collations
- Stemming
- Intro
- Algorithmic stemmers
- Dictionary stemmers
- Hunspell stemmer
- Choosing a stemmer
- Controlling stemming
- Stemming in situ
- Stopwords
- Intro
- Using stopwords
- Stopwords and performance
- Divide and conquer
- Phrase queries
- Common grams
- Relevance
- Synonyms
- Intro
- Using synonyms
- Synonym formats
- Expand contract
- Analysis chain
- Multi word synonyms
- Symbol synonyms
- Fuzzy matching
- Intro
- Fuzziness
- Fuzzy query
- Fuzzy match query
- Scoring fuzziness
- Phonetic matching
- Aggregations
- overview
- circuit breaker fd settings
- filtering
- facets
- docvalues
- eager
- breadth vs depth
- Conclusion
- concepts buckets
- basic example
- add metric
- nested bucket
- extra metrics
- bucket metric list
- histogram
- date histogram
- scope
- filtering
- sorting ordering
- approx intro
- cardinality
- percentiles
- sigterms intro
- sigterms
- fielddata
- analyzed vs not
- 地理坐標點
- 地理坐標點
- 通過地理坐標點過濾
- 地理坐標盒模型過濾器
- 地理距離過濾器
- 緩存地理位置過濾器
- 減少內存占用
- 按距離排序
- Geohashe
- Geohashe
- Geohashe映射
- Geohash單元過濾器
- 地理位置聚合
- 地理位置聚合
- 按距離聚合
- Geohash單元聚合器
- 范圍(邊界)聚合器
- 地理形狀
- 地理形狀
- 映射地理形狀
- 索引地理形狀
- 查詢地理形狀
- 在查詢中使用已索引的形狀
- 地理形狀的過濾與緩存
- 關系
- 關系
- 應用級別的Join操作
- 扁平化你的數據
- Top hits
- Concurrency
- Concurrency solutions
- 嵌套
- 嵌套對象
- 嵌套映射
- 嵌套查詢
- 嵌套排序
- 嵌套集合
- Parent Child
- Parent child
- Indexing parent child
- Has child
- Has parent
- Children agg
- Grandparents
- Practical considerations
- Scaling
- Shard
- Overallocation
- Kagillion shards
- Capacity planning
- Replica shards
- Multiple indices
- Index per timeframe
- Index templates
- Retiring data
- Index per user
- Shared index
- Faking it
- One big user
- Scale is not infinite
- Cluster Admin
- Marvel
- Health
- Node stats
- Other stats
- Deployment
- hardware
- other
- config
- dont touch
- heap
- file descriptors
- conclusion
- cluster settings
- Post Deployment
- dynamic settings
- logging
- indexing perf
- rolling restart
- backup
- restore
- conclusion