Wednesday, March 18, 2009

Part 2: Search issues

This is the rest of the story regarding Search issues

Problem: the customer has a server farm (the platform is one SQL, two WFE with MOSS 2007) and NLB running. When accessing one of the servers, the Search Box is missing. This is what the customer reported to me, and where my troubleshooting started.

Summary: Resolving a database schema mismatch after applying Security Update KB 956716

So, I had tried to reset the content index on the WFE in the server farm. But that crashed and I got into a real mess where I restarted the search services, reset the passwords on all search accounts, etc. Nothing helped me so that I could perform a Full crawl again. I got a lot of errors within Search Settings and finally what happened was that when I clicked on Full Crawl it only jumped back to "Idle", so nothing could trigger the Full crawl.

Then I discovered the real issue: on the WFE where Search box was disabled, a patch for SharePoint had been installed. No one knows why that patch was installed, probably it has been downloaded and installed along with some other updates. So, there was a mismatch in patches since the KB956716 only was installed on one server and not on the other (as you can see, it is divided into three instances - strange):

So, I ran the Sharepoint Configuration Wizard. That was successful, and the search box came back! Success I thought and ran a Full Crawl. Everything was working perfectly. The morning after I tested the search again... No results!! Hm, what had happened during the night?? Looked through Event Viewer and there were some error messages saying that there was a database error:

Event Type: ErrorEvent Source: Office Server SearchEvent Category: Gatherer Event ID: 10036Date: 18-03-2009Time: 14:00:07User: N/AComputer: Dxxxx001
Description:A database error occurred.
Source: Microsoft OLE DB Provider for SQL
ServerCode: 4060 occurred 1 time(s)
Description: Cannot open database "SharedServices2_Search_DB" requested by the login. The login failed.

So I installed the kb on the search server as well, but the first time the server hung and we had to restart it. Then I ran the kb installation again and success this time. Restarted the server. Ran the configuration wizard. and it was successful. Started CA and Search settings and tada, there was a new link, Search Administration which is the same link that you get when you install the infrastructure upgrade.Performed a full crawl.... and yes! It started!

Conclusion: always make sure your server farm has a "patch match" :) And always run the Config Wizard after a patch has been installed, then all will work fine.


Aaron Boundy said...

Just wanted to thank you for this post. I experienced the same issue and spent days searching to confirm my suspicion.

Wasn't until I came across your post that described word for word what I was experiencing that I was able to proceed with confidence!!

Thanks again - you have been subscribed!!

Lise said...

Thank you Aaron! I am so happy to hear that!! That is exactly why you run a blog, to help others that are struggling with these issues all the time :=)
Take care,

anthony said...

You own a very interesting blog covering lots of topics I am interested as well. I just added your site to my favorites for being able in the future... Please continue your excellent article writing

Lise said...

Thanks Anthony! I will proceed my blogging again soon..
Take care,