A collection in a search engine stores the data ready for crawling. In other words, it holds the database of any source of data that needs to be picked for indexing. It also has all the functionalities that allow you to configure the collection to crawl or index the data source based on your preference.
Data Source Type
A data source type signifies the type of data source you want to crawl. For example, your data source type can be a website, XML feed, external database, or APIs. The data source input will reflect based on the selected source type.
The data source type can be chosen when a new collection is created. New Collection can be added in the ‘All Collection’ tab present under the Collection section.
You can start indexing the data source here by checking the box available. You can proceed without checking this option.
- To Create a New Collection, Go to the Collection tab provided in the side navigation bar.
- Click on All Collections. You will see a button that reads “New Collection”
- Provide a name for the Collection.
- Choose the data source type. We only support websites as a data source for now. We will soon be enabling the support for XML and API push.
- Check the box next to Start Indexing to initiate the indexing process.
- Click on Save to complete the process.
- You can add multiple collections of different data source types within the same engine.
In this panel, you get a consolidated view of all your collections and their details. You can also add/delete a collection from this panel after making an informed decision from the overview. You can also create multiple collections in different data source types within the same engine.