How Crawl and Indexing works in SharePoint?

Remember - For Search to work, SharePoint would first Index the Content Sources.

Main thing is that it can index, it can crawl anything stored in a server, in any format PDF, Zip, word, excel , txt, HTML, RTF, MS-Office etc. But to index the content other than office or Microsoft the index process is little more complicated but interesting. But the initial or the process is same for all. So let’s have a look on it.

1) When the scheduler for the crawl or index run, it will search for every place you have defined or what you called it a content source.

2) When it find a file in that, it will look on it extension. It will check in SharePoint SSP whether the type is defined to be indexed or not.

3) Now when the SharePoint confirms of file type it will look for a software/Ifilter to read this file. Ifilter is a software which will read a file. Every file need its own ifilter.

4) If SharePoint finds a ifilter for it. It will start opening this file and start scanning the file. It will remove certain words that are not required in search or not need to be indexed ex: 1 ,2 numerals etc.

5) After scanning the whole file it will index the content in index file with the pointer of name and location of the file.

6) Once a file is completed with full process. It will start for next file and with the same process as above.

Now to search or index any file of our use like PDF etc. we need to install the ifilter of every such types, which do not come by default. We can also put the images of such file type in SharePoint images file(12 hive) so that in search document come with their images. 

- Blog Credit
Ashish Banga