Incremental enrichment brings a declarative approach to indexing data. This new feature causes document enrichment to perform at the lowest cost, even while you continue to improve your skills.
Indexers with incremental enrichment can:
- Add documents to the search index from a data search
- Track updates to documents in the data sources
- Update the index with new documents or updates from the data source.
- Drive your documents to eventual consistency with the data source
- Extend change tracking from document changes in the data source to the whole enrichment pipeline
With this feature, the indexer pushes documents to consistency between the data source, indexer and current version of your skillset. Furthermore, the indexer is data source specific, state aware and can be configured to consistency between the data source and index.
This immensely improves the efficiency of the enrichment pipeline. It removes version drift from documents created or updated with different skillsets and removes the potentially expensive choice of re-enriching documents when a skill is added or updated.